2024-04-25T15:32:47.059 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T15:32:47.060 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_pdonnell@teuthology --description fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227 -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227/orig.config.yaml 2024-04-25T15:32:47.116 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T15:32:47.219 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: pdonnell@redhat.com first_in_suite: false job_id: '7673227' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/quincy ' name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 targets: smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAsE3bZBAWGFLdZIGuH4oDbcYzcmN2PuscP8jNOkAo9EXcakMElLjD28TLGKf0eGtjnNXCwLx+2o7oZxw1gPp/c= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBBXyEf7DAbFDAwRr+lEjRD7WZWBtpyYRpz0s6rqeS8/tDfH9zJH9K5P/M0XtiIvu200Damexr3gMvGGiQHIVd4= tasks: - install: branch: quincy exclude_packages: - ceph-volume - print: '**** done install task...' - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy roleless: true - print: '**** done end installing quincy cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 1 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: - /cephfs/home/pdonnell/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20240425.015853-debug/qa/suites/fs/upgrade/mds_upgrade_sequence/tasks/3-upgrade-mgr-staggered.yaml meta: {} postmerge: [] variables: fail_fs: false teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs false || true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-25T15:32:47.219 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T15:32:47.220 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T15:32:47.220 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T15:32:47.222 INFO:teuthology.task.internal:Checking packages... 2024-04-25T15:32:47.246 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T15:32:47.246 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T15:32:47.246 INFO:teuthology.packaging:ref: None 2024-04-25T15:32:47.246 INFO:teuthology.packaging:tag: None 2024-04-25T15:32:47.246 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T15:32:47.247 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:32:47.247 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T15:32:47.386 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291.g994e2f20 2024-04-25T15:32:47.387 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T15:32:47.394 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T15:32:47.394 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T15:32:47.407 INFO:teuthology.task.internal:Saving configuration 2024-04-25T15:32:47.433 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T15:32:47.439 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T15:32:47.465 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 15:27:54.038051', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAsE3bZBAWGFLdZIGuH4oDbcYzcmN2PuscP8jNOkAo9EXcakMElLjD28TLGKf0eGtjnNXCwLx+2o7oZxw1gPp/c='} 2024-04-25T15:32:47.485 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 15:27:54.036775', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBBXyEf7DAbFDAwRr+lEjRD7WZWBtpyYRpz0s6rqeS8/tDfH9zJH9K5P/M0XtiIvu200Damexr3gMvGGiQHIVd4='} 2024-04-25T15:32:47.485 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T15:32:47.492 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-25T15:32:47.492 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-25T15:32:47.492 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T15:32:47.564 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1ca0e9e0d0>, signals=[15]) 2024-04-25T15:32:47.564 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T15:32:47.570 INFO:teuthology.task.internal:Opening connections... 2024-04-25T15:32:47.570 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:32:47.572 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T15:32:47.649 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T15:32:47.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T15:32:47.740 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T15:32:47.746 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-04-25T15:32:47.765 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-04-25T15:32:47.765 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-04-25T15:32:47.822 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-04-25T15:32:47.822 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-04-25T15:32:47.822 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-04-25T15:32:47.822 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-04-25T15:32:47.822 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-04-25T15:32:47.822 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-04-25T15:32:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T15:32:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-04-25T15:32:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-04-25T15:32:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T15:32:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-04-25T15:32:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T15:32:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T15:32:47.823 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T15:32:47.824 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-04-25T15:32:47.845 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-04-25T15:32:47.863 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-04-25T15:32:47.863 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-04-25T15:32:47.920 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-04-25T15:32:47.921 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-04-25T15:32:47.921 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-04-25T15:32:47.921 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-04-25T15:32:47.921 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-04-25T15:32:47.921 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-04-25T15:32:47.921 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T15:32:47.922 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-04-25T15:32:47.922 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-04-25T15:32:47.922 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T15:32:47.922 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-04-25T15:32:47.922 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T15:32:47.922 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T15:32:47.922 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T15:32:47.922 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-04-25T15:32:47.944 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T15:32:47.954 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T15:32:47.960 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T15:32:47.960 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-04-25T15:32:47.962 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-04-25T15:32:47.979 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T15:32:47.986 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T15:32:47.986 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-04-25T15:32:48.019 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-04-25T15:32:48.118 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T15:32:48.184 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T15:32:48.200 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T15:32:48.200 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T15:32:48.200 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T15:32:48.201 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T15:32:48.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:32:48.217 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-04-25T15:32:48.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:32:48.218 DEBUG:teuthology.orchestra.run.smithi110:> uname -r 2024-04-25T15:32:48.273 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-437.el9.x86_64 2024-04-25T15:32:48.274 INFO:teuthology.task.kernel:Running kernel on smithi193: 5.14.0-437.el9.x86_64 2024-04-25T15:32:48.274 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2024-04-25T15:32:48.275 INFO:teuthology.orchestra.run.smithi110.stdout:5.14.0-437.el9.x86_64 2024-04-25T15:32:48.275 INFO:teuthology.task.kernel:Running kernel on smithi110: 5.14.0-437.el9.x86_64 2024-04-25T15:32:48.275 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum install -y kernel 2024-04-25T15:32:50.662 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 9.6 MB/s | 8.0 MB 00:00 2024-04-25T15:32:51.052 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 7.3 MB/s | 8.0 MB 00:01 2024-04-25T15:32:52.455 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-25T15:32:52.862 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-04-25T15:32:56.606 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T15:32:56.929 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T15:32:58.479 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-25T15:32:58.752 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 25 kB/s | 16 kB 00:00 2024-04-25T15:33:00.106 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-25T15:33:01.014 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 21 MB 00:02 2024-04-25T15:33:05.145 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T15:33:06.005 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T15:33:07.026 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T15:33:07.026 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T15:33:07.118 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:33:07.119 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:33:07.119 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:33:07.224 DEBUG:teuthology.orchestra.run.smithi193:> echo no | sudo yum reinstall kernel || true 2024-04-25T15:33:07.724 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 03:33:05 PM UTC. 2024-04-25T15:33:07.886 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T15:33:07.886 INFO:teuthology.orchestra.run.smithi110.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T15:33:07.887 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:33:07.895 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:33:07.895 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-04-25T15:33:07.895 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:33:07.895 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalling: 2024-04-25T15:33:07.895 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T15:33:07.895 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:33:07.895 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:33:07.896 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:33:07.896 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:33:07.896 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 255 k 2024-04-25T15:33:07.896 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 0 2024-04-25T15:33:07.896 INFO:teuthology.orchestra.run.smithi193.stderr:Operation aborted. 2024-04-25T15:33:07.957 INFO:teuthology.orchestra.run.smithi193.stdout:Is this ok [y/N]: 2024-04-25T15:33:07.957 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum reinstall -y kernel || true 2024-04-25T15:33:07.978 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:33:07.979 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:33:07.979 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:33:08.086 DEBUG:teuthology.orchestra.run.smithi110:> echo no | sudo yum reinstall kernel || true 2024-04-25T15:33:08.452 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 03:33:05 PM UTC. 2024-04-25T15:33:08.569 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:33:08.570 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:33:08.570 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-04-25T15:33:08.570 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:33:08.570 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalling: 2024-04-25T15:33:08.570 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T15:33:08.571 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:33:08.571 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:33:08.571 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:33:08.571 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:33:08.571 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 255 k 2024-04-25T15:33:08.571 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 0 2024-04-25T15:33:08.571 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:33:08.582 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 03:33:05 PM UTC. 2024-04-25T15:33:08.708 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:33:08.718 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:33:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-04-25T15:33:08.718 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:33:08.718 INFO:teuthology.orchestra.run.smithi110.stdout:Reinstalling: 2024-04-25T15:33:08.718 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T15:33:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:33:08.719 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:33:08.719 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:33:08.719 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:33:08.719 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 255 k 2024-04-25T15:33:08.719 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 0 2024-04-25T15:33:08.719 INFO:teuthology.orchestra.run.smithi110.stderr:Operation aborted. 2024-04-25T15:33:08.774 INFO:teuthology.orchestra.run.smithi110.stdout:Is this ok [y/N]: 2024-04-25T15:33:08.775 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum reinstall -y kernel || true 2024-04-25T15:33:08.880 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.8 MB/s | 255 kB 00:00 2024-04-25T15:33:08.880 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:33:08.881 INFO:teuthology.orchestra.run.smithi193.stdout:Total 822 kB/s | 255 kB 00:00 2024-04-25T15:33:08.896 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:33:09.068 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:33:09.068 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:33:09.073 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:33:09.073 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:33:09.217 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:33:09.265 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 03:33:05 PM UTC. 2024-04-25T15:33:09.281 INFO:teuthology.orchestra.run.smithi193.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T15:33:09.281 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T15:33:09.379 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:33:09.380 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:33:09.380 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-04-25T15:33:09.380 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:33:09.380 INFO:teuthology.orchestra.run.smithi110.stdout:Reinstalling: 2024-04-25T15:33:09.381 INFO:teuthology.orchestra.run.smithi110.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T15:33:09.381 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:33:09.381 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:33:09.381 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:33:09.381 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:33:09.381 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 255 k 2024-04-25T15:33:09.381 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 0 2024-04-25T15:33:09.381 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:33:09.666 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.7 MB/s | 255 kB 00:00 2024-04-25T15:33:09.666 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:33:09.666 INFO:teuthology.orchestra.run.smithi110.stdout:Total 892 kB/s | 255 kB 00:00 2024-04-25T15:33:09.687 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:33:09.883 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:33:09.883 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:33:09.888 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:33:09.888 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:33:10.017 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T15:33:10.017 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T15:33:10.034 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:33:10.117 INFO:teuthology.orchestra.run.smithi110.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T15:33:10.117 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T15:33:10.683 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T15:33:10.683 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:33:10.683 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalled: 2024-04-25T15:33:10.683 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T15:33:10.683 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:33:10.683 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:33:10.835 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T15:33:10.836 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T15:33:10.894 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T15:33:10.941 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T15:33:10.941 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T15:33:10.942 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T15:33:10.942 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-25T15:33:10.942 DEBUG:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T15:33:10.967 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2024-04-25T15:33:10.989 DEBUG:teuthology.parallel:result is None 2024-04-25T15:33:11.563 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T15:33:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:33:11.564 INFO:teuthology.orchestra.run.smithi110.stdout:Reinstalled: 2024-04-25T15:33:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T15:33:11.564 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:33:11.564 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:33:11.747 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T15:33:11.833 INFO:teuthology.orchestra.run.smithi110.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T15:33:11.835 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T15:33:11.835 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T15:33:11.835 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-25T15:33:11.835 DEBUG:teuthology.orchestra.run.smithi110:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T15:33:11.861 INFO:teuthology.orchestra.run.smithi110.stdout:ttyS1 2024-04-25T15:33:11.888 DEBUG:teuthology.parallel:result is None 2024-04-25T15:33:11.888 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T15:33:11.895 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T15:33:11.895 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T15:33:11.931 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T15:33:11.948 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T15:33:11.955 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T15:33:11.963 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T15:33:11.963 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T15:33:11.985 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T15:33:12.013 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T15:33:12.020 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T15:33:12.020 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T15:33:12.046 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T15:33:12.076 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T15:33:12.086 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T15:33:12.109 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T15:33:12.121 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T15:33:12.122 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T15:33:12.130 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T15:33:12.130 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T15:33:12.132 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T15:33:12.161 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T15:33:12.170 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T15:33:12.170 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T15:33:12.195 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T15:33:12.217 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T15:33:12.286 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T15:33:12.345 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T15:33:12.421 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T15:33:12.465 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:33:12.465 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T15:33:12.528 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T15:33:12.567 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T15:33:12.625 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T15:33:12.703 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T15:33:12.751 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:33:12.751 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T15:33:12.816 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-04-25T15:33:12.818 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-04-25T15:33:12.851 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T15:33:12.898 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T15:33:13.270 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T15:33:13.279 INFO:teuthology.task.internal:Starting timer... 2024-04-25T15:33:13.279 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T15:33:13.288 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T15:33:13.298 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T15:33:13.298 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-04-25T15:33:13.348 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-04-25T15:33:13.350 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-04-25T15:33:13.418 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-04-25T15:33:13.420 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T15:33:13.421 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-04-25T15:33:13.439 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-04-25T15:33:13.439 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-04-25T15:33:13.458 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-04-25T15:33:13.459 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi110.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-04-25T15:33:13.459 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T15:33:13.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:33:13.504 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T15:33:13.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:33:13.529 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T15:33:13.529 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-04-25T15:33:13.568 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-04-25T15:33:13.596 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T15:33:13.606 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T15:33:13.621 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T15:33:13.622 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi110.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T15:36:43.111 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi110.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-04-25T15:36:43.112 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-04-25T15:36:43.113 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T15:36:43.189 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-04-25T15:36:43.265 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-04-25T15:36:43.265 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-04-25T15:36:43.266 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T15:36:43.343 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-04-25T15:36:43.378 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-04-25T15:36:43.379 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T15:36:43.389 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T15:36:43.390 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T15:36:43.390 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T15:36:43.392 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T15:36:43.393 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T15:36:43.417 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T15:36:43.431 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T15:36:43.447 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T15:36:43.453 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-04-25T15:36:43.460 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T15:36:43.463 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-04-25T15:36:43.476 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T15:36:43.483 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-04-25T15:36:43.489 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T15:36:43.494 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-04-25T15:36:43.507 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T15:36:43.520 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T15:36:43.546 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-04-25T15:36:43.551 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T15:36:43.551 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-04-25T15:36:43.551 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:36:43.551 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:36:43.551 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:36:43.551 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:36:43.574 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-04-25T15:36:43.577 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T15:36:43.577 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-04-25T15:36:43.577 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:36:43.577 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:36:43.577 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:36:43.577 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T15:36:43.578 INFO:teuthology.run_tasks:Running task install... 2024-04-25T15:36:43.587 DEBUG:teuthology.task.install:project ceph 2024-04-25T15:36:43.587 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T15:36:43.587 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T15:36:43.588 INFO:teuthology.task.install:Using flavor: default 2024-04-25T15:36:43.593 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T15:36:43.593 INFO:teuthology.task.install:extra packages: [] 2024-04-25T15:36:43.594 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T15:36:43.594 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T15:36:43.594 INFO:teuthology.packaging:ref: None 2024-04-25T15:36:43.594 INFO:teuthology.packaging:tag: None 2024-04-25T15:36:43.594 INFO:teuthology.packaging:branch: quincy 2024-04-25T15:36:43.594 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:36:43.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-25T15:36:43.597 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'tag': None, 'wait_for_package': False} 2024-04-25T15:36:43.597 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T15:36:43.597 INFO:teuthology.packaging:ref: None 2024-04-25T15:36:43.597 INFO:teuthology.packaging:tag: None 2024-04-25T15:36:43.597 INFO:teuthology.packaging:branch: quincy 2024-04-25T15:36:43.597 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:36:43.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-25T15:36:43.813 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/ 2024-04-25T15:36:43.814 INFO:teuthology.task.install.rpm:Package version is 17.2.7-902.gc61472c8 2024-04-25T15:36:44.054 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/ 2024-04-25T15:36:44.054 INFO:teuthology.task.install.rpm:Package version is 17.2.7-902.gc61472c8 2024-04-25T15:36:44.098 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T15:36:44.098 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:36:44.098 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T15:36:44.124 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-25T15:36:44.124 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T15:36:44.124 INFO:teuthology.packaging:ref: None 2024-04-25T15:36:44.124 INFO:teuthology.packaging:tag: None 2024-04-25T15:36:44.124 INFO:teuthology.packaging:branch: quincy 2024-04-25T15:36:44.124 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:36:44.124 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T15:36:44.190 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T15:36:44.264 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T15:36:44.327 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-04-25T15:36:44.329 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-04-25T15:36:44.334 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/c61472c83ce76869b0fe076c19ed2f4bcc3fb4af/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T15:36:44.334 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:36:44.334 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T15:36:44.360 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-25T15:36:44.360 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T15:36:44.360 INFO:teuthology.packaging:ref: None 2024-04-25T15:36:44.360 INFO:teuthology.packaging:tag: None 2024-04-25T15:36:44.360 INFO:teuthology.packaging:branch: quincy 2024-04-25T15:36:44.360 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:36:44.360 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T15:36:44.426 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T15:36:44.499 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T15:36:44.552 INFO:teuthology.orchestra.run.smithi193.stdout:48 files removed 2024-04-25T15:36:44.562 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-04-25T15:36:44.563 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-04-25T15:36:44.579 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw 2024-04-25T15:36:44.786 INFO:teuthology.orchestra.run.smithi110.stdout:48 files removed 2024-04-25T15:36:44.813 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw 2024-04-25T15:36:45.424 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 132 kB/s | 77 kB 00:00 2024-04-25T15:36:45.555 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 160 kB/s | 77 kB 00:00 2024-04-25T15:36:45.916 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-04-25T15:36:46.014 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 26 kB/s | 11 kB 00:00 2024-04-25T15:36:46.342 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 4.6 kB/s | 1.8 kB 00:00 2024-04-25T15:36:46.506 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 3.9 kB/s | 1.8 kB 00:00 2024-04-25T15:36:47.378 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 8.0 MB 00:01 2024-04-25T15:36:47.584 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 7.7 MB/s | 8.0 MB 00:01 2024-04-25T15:36:49.788 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T15:36:49.902 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-04-25T15:36:54.155 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.2 MB 00:00 2024-04-25T15:36:54.502 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-25T15:36:55.619 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-25T15:36:56.684 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 16 kB/s | 16 kB 00:01 2024-04-25T15:36:57.104 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T15:36:58.638 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T15:37:02.371 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 5.6 kB/s | 1.7 kB 00:00 2024-04-25T15:37:03.813 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 7.4 kB/s | 1.7 kB 00:00 2024-04-25T15:37:04.546 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:37:04.548 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:04.548 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:37:04.548 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:04.548 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:37:04.548 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 14 M 2024-04-25T15:37:04.548 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.1 M 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:17.2.7-902.gc61472c8.el9 ceph 5.0 M 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 22 M 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 637 k 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T15:37:04.549 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 457 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 4.9 M 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 45 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 117 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:17.2.7-902.gc61472c8.el9 ceph 156 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:17.2.7-902.gc61472c8.el9 ceph 320 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 293 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 99 k 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T15:37:04.550 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:04.551 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:37:04.551 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:04.551 INFO:teuthology.orchestra.run.smithi193.stdout:Install 22 Packages 2024-04-25T15:37:04.551 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 2 Packages 2024-04-25T15:37:04.551 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:04.551 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 58 M 2024-04-25T15:37:04.551 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:37:05.380 INFO:teuthology.orchestra.run.smithi193.stdout:(1/24): ceph-base-17.2.7-902.gc61472c8.el9.x86_ 8.9 MB/s | 5.0 MB 00:00 2024-04-25T15:37:05.430 INFO:teuthology.orchestra.run.smithi193.stdout:(2/24): ceph-selinux-17.2.7-902.gc61472c8.el9.x 505 kB/s | 25 kB 00:00 2024-04-25T15:37:05.497 INFO:teuthology.orchestra.run.smithi193.stdout:(3/24): libcephfs2-17.2.7-902.gc61472c8.el9.x86 9.3 MB/s | 637 kB 00:00 2024-04-25T15:37:05.556 INFO:teuthology.orchestra.run.smithi193.stdout:(4/24): libradosstriper1-17.2.7-902.gc61472c8.e 7.7 MB/s | 457 kB 00:00 2024-04-25T15:37:05.856 INFO:teuthology.orchestra.run.smithi193.stdout:(5/24): librgw2-17.2.7-902.gc61472c8.el9.x86_64 16 MB/s | 4.9 MB 00:00 2024-04-25T15:37:05.906 INFO:teuthology.orchestra.run.smithi193.stdout:(6/24): python3-ceph-argparse-17.2.7-902.gc6147 904 kB/s | 45 kB 00:00 2024-04-25T15:37:05.982 INFO:teuthology.orchestra.run.smithi193.stdout:(7/24): python3-ceph-common-17.2.7-902.gc61472c 1.5 MB/s | 117 kB 00:00 2024-04-25T15:37:06.004 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:37:06.006 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:06.006 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:37:06.006 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:06.006 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:37:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 14 M 2024-04-25T15:37:06.007 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-04-25T15:37:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.1 M 2024-04-25T15:37:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-25T15:37:06.007 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-25T15:37:06.007 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:17.2.7-902.gc61472c8.el9 ceph 5.0 M 2024-04-25T15:37:06.008 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 22 M 2024-04-25T15:37:06.008 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-25T15:37:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T15:37:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 637 k 2024-04-25T15:37:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T15:37:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T15:37:06.009 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T15:37:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 457 k 2024-04-25T15:37:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T15:37:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:17.2.7-902.gc61472c8.el9 ceph 4.9 M 2024-04-25T15:37:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T15:37:06.010 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T15:37:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 45 k 2024-04-25T15:37:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:17.2.7-902.gc61472c8.el9 ceph 117 k 2024-04-25T15:37:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:17.2.7-902.gc61472c8.el9 ceph 156 k 2024-04-25T15:37:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T15:37:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:17.2.7-902.gc61472c8.el9 ceph 320 k 2024-04-25T15:37:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 293 k 2024-04-25T15:37:06.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:17.2.7-902.gc61472c8.el9 ceph 99 k 2024-04-25T15:37:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T15:37:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:06.012 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:37:06.012 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:06.012 INFO:teuthology.orchestra.run.smithi110.stdout:Install 22 Packages 2024-04-25T15:37:06.012 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 2 Packages 2024-04-25T15:37:06.012 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:06.013 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 58 M 2024-04-25T15:37:06.013 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:37:06.048 INFO:teuthology.orchestra.run.smithi193.stdout:(8/24): python3-cephfs-17.2.7-902.gc61472c8.el9 2.3 MB/s | 156 kB 00:00 2024-04-25T15:37:06.124 INFO:teuthology.orchestra.run.smithi193.stdout:(9/24): python3-rados-17.2.7-902.gc61472c8.el9. 4.2 MB/s | 320 kB 00:00 2024-04-25T15:37:06.190 INFO:teuthology.orchestra.run.smithi193.stdout:(10/24): python3-rbd-17.2.7-902.gc61472c8.el9.x 4.3 MB/s | 293 kB 00:00 2024-04-25T15:37:06.241 INFO:teuthology.orchestra.run.smithi193.stdout:(11/24): python3-rgw-17.2.7-902.gc61472c8.el9.x 1.9 MB/s | 99 kB 00:00 2024-04-25T15:37:06.366 INFO:teuthology.orchestra.run.smithi193.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-25T15:37:06.400 INFO:teuthology.orchestra.run.smithi193.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2024-04-25T15:37:06.500 INFO:teuthology.orchestra.run.smithi193.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 6.5 MB/s | 662 kB 00:00 2024-04-25T15:37:06.542 INFO:teuthology.orchestra.run.smithi193.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T15:37:06.575 INFO:teuthology.orchestra.run.smithi193.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 42 kB 00:00 2024-04-25T15:37:06.717 INFO:teuthology.orchestra.run.smithi193.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.1 MB/s | 309 kB 00:00 2024-04-25T15:37:06.826 INFO:teuthology.orchestra.run.smithi193.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 450 kB/s | 49 kB 00:00 2024-04-25T15:37:06.943 INFO:teuthology.orchestra.run.smithi193.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 580 kB/s | 67 kB 00:00 2024-04-25T15:37:07.093 INFO:teuthology.orchestra.run.smithi193.stdout:(20/24): ceph-radosgw-17.2.7-902.gc61472c8.el9. 6.2 MB/s | 14 MB 00:02 2024-04-25T15:37:07.335 INFO:teuthology.orchestra.run.smithi193.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 4.0 MB/s | 1.6 MB 00:00 2024-04-25T15:37:07.718 INFO:teuthology.orchestra.run.smithi110.stdout:(1/24): ceph-base-17.2.7-902.gc61472c8.el9.x86_ 4.5 MB/s | 5.0 MB 00:01 2024-04-25T15:37:07.769 INFO:teuthology.orchestra.run.smithi193.stdout:(22/24): ceph-common-17.2.7-902.gc61472c8.el9.x 7.6 MB/s | 22 MB 00:02 2024-04-25T15:37:07.780 INFO:teuthology.orchestra.run.smithi110.stdout:(2/24): ceph-selinux-17.2.7-902.gc61472c8.el9.x 404 kB/s | 25 kB 00:00 2024-04-25T15:37:07.852 INFO:teuthology.orchestra.run.smithi193.stdout:(23/24): librbd1-17.2.7-902.gc61472c8.el9.x86_6 5.7 MB/s | 2.9 MB 00:00 2024-04-25T15:37:07.872 INFO:teuthology.orchestra.run.smithi110.stdout:(3/24): libcephfs2-17.2.7-902.gc61472c8.el9.x86 6.8 MB/s | 637 kB 00:00 2024-04-25T15:37:07.973 INFO:teuthology.orchestra.run.smithi110.stdout:(4/24): libradosstriper1-17.2.7-902.gc61472c8.e 4.5 MB/s | 457 kB 00:00 2024-04-25T15:37:08.448 INFO:teuthology.orchestra.run.smithi110.stdout:(5/24): librgw2-17.2.7-902.gc61472c8.el9.x86_64 10 MB/s | 4.9 MB 00:00 2024-04-25T15:37:08.548 INFO:teuthology.orchestra.run.smithi110.stdout:(6/24): python3-ceph-argparse-17.2.7-902.gc6147 452 kB/s | 45 kB 00:00 2024-04-25T15:37:08.610 INFO:teuthology.orchestra.run.smithi193.stdout:(24/24): librados2-17.2.7-902.gc61472c8.el9.x86 2.1 MB/s | 3.1 MB 00:01 2024-04-25T15:37:08.615 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:08.615 INFO:teuthology.orchestra.run.smithi193.stdout:Total 14 MB/s | 58 MB 00:04 2024-04-25T15:37:08.673 INFO:teuthology.orchestra.run.smithi110.stdout:(7/24): python3-ceph-common-17.2.7-902.gc61472c 940 kB/s | 117 kB 00:00 2024-04-25T15:37:08.674 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:37:08.707 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:37:08.707 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:37:08.774 INFO:teuthology.orchestra.run.smithi110.stdout:(8/24): python3-cephfs-17.2.7-902.gc61472c8.el9 1.5 MB/s | 156 kB 00:00 2024-04-25T15:37:08.866 INFO:teuthology.orchestra.run.smithi110.stdout:(9/24): python3-rados-17.2.7-902.gc61472c8.el9. 3.4 MB/s | 320 kB 00:00 2024-04-25T15:37:08.974 INFO:teuthology.orchestra.run.smithi110.stdout:(10/24): python3-rbd-17.2.7-902.gc61472c8.el9.x 2.6 MB/s | 293 kB 00:00 2024-04-25T15:37:09.075 INFO:teuthology.orchestra.run.smithi110.stdout:(11/24): python3-rgw-17.2.7-902.gc61472c8.el9.x 995 kB/s | 99 kB 00:00 2024-04-25T15:37:09.140 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:37:09.140 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:37:09.342 INFO:teuthology.orchestra.run.smithi110.stdout:(12/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 601 kB/s | 160 kB 00:00 2024-04-25T15:37:09.383 INFO:teuthology.orchestra.run.smithi110.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T15:37:09.475 INFO:teuthology.orchestra.run.smithi110.stdout:(14/24): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-25T15:37:09.517 INFO:teuthology.orchestra.run.smithi110.stdout:(15/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.9 MB/s | 292 kB 00:00 2024-04-25T15:37:09.559 INFO:teuthology.orchestra.run.smithi110.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-04-25T15:37:10.109 INFO:teuthology.orchestra.run.smithi110.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 561 kB/s | 309 kB 00:00 2024-04-25T15:37:10.201 INFO:teuthology.orchestra.run.smithi110.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 529 kB/s | 49 kB 00:00 2024-04-25T15:37:10.302 INFO:teuthology.orchestra.run.smithi110.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 672 kB/s | 67 kB 00:00 2024-04-25T15:37:10.398 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:37:10.560 INFO:teuthology.orchestra.run.smithi110.stdout:(20/24): thrift-0.15.0-2.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2024-04-25T15:37:10.721 INFO:teuthology.orchestra.run.smithi110.stdout:(21/24): ceph-radosgw-17.2.7-902.gc61472c8.el9. 3.4 MB/s | 14 MB 00:04 2024-04-25T15:37:10.740 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-25T15:37:10.854 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T15:37:10.928 INFO:teuthology.orchestra.run.smithi110.stdout:(22/24): ceph-common-17.2.7-902.gc61472c8.el9.x 5.2 MB/s | 22 MB 00:04 2024-04-25T15:37:10.947 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T15:37:11.025 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-25T15:37:11.056 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T15:37:11.112 INFO:teuthology.orchestra.run.smithi110.stdout:(23/24): librados2-17.2.7-902.gc61472c8.el9.x86 5.6 MB/s | 3.1 MB 00:00 2024-04-25T15:37:11.135 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T15:37:11.189 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-25T15:37:11.219 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-25T15:37:11.504 INFO:teuthology.orchestra.run.smithi110.stdout:(24/24): librbd1-17.2.7-902.gc61472c8.el9.x86_6 3.8 MB/s | 2.9 MB 00:00 2024-04-25T15:37:11.508 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:11.508 INFO:teuthology.orchestra.run.smithi110.stdout:Total 10 MB/s | 58 MB 00:05 2024-04-25T15:37:11.567 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:37:11.600 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:37:11.600 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:37:11.650 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-25T15:37:11.676 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-25T15:37:11.738 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-25T15:37:11.767 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-25T15:37:11.809 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 10/26 2024-04-25T15:37:11.847 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-25T15:37:11.916 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-25T15:37:11.945 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T15:37:12.002 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T15:37:12.038 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:37:12.038 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:37:12.043 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-25T15:37:12.072 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-25T15:37:12.105 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-25T15:37:12.397 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-25T15:37:12.442 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-25T15:37:12.531 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-25T15:37:12.605 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 19/26 2024-04-25T15:37:12.636 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 20/26 2024-04-25T15:37:13.294 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:37:13.680 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-04-25T15:37:13.764 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T15:37:13.854 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T15:37:13.934 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-25T15:37:13.958 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T15:37:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T15:37:14.091 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-04-25T15:37:14.120 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-04-25T15:37:14.552 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-04-25T15:37:14.577 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-25T15:37:14.632 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 8/26 2024-04-25T15:37:14.660 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-04-25T15:37:14.703 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 10/26 2024-04-25T15:37:14.740 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-25T15:37:14.802 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-25T15:37:14.830 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T15:37:14.895 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T15:37:14.934 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-04-25T15:37:14.963 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-04-25T15:37:14.996 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-04-25T15:37:15.031 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T15:37:15.066 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T15:37:15.289 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-04-25T15:37:15.326 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-25T15:37:15.424 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 18/26 2024-04-25T15:37:15.498 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 19/26 2024-04-25T15:37:15.528 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 20/26 2024-04-25T15:37:15.574 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T15:37:15.611 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-25T15:37:15.695 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-25T15:37:15.695 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T15:37:15.695 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T15:37:15.695 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:15.728 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T15:37:17.864 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T15:37:17.900 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T15:37:18.402 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 21/26 2024-04-25T15:37:18.445 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-25T15:37:18.524 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 22/26 2024-04-25T15:37:18.524 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T15:37:18.524 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T15:37:18.524 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:18.546 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-04-25T15:37:25.166 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:26.461 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-25T15:37:26.538 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-25T15:37:26.539 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:26.539 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T15:37:26.539 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T15:37:26.539 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T15:37:26.539 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:26.540 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-25T15:37:26.622 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-25T15:37:26.623 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T15:37:27.778 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T15:37:27.778 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 1/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 5/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 6/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 7/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 8/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 9/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 10/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-25T15:37:27.779 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T15:37:27.780 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-25T15:37:27.781 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 25/26 2024-04-25T15:37:27.868 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T15:37:27.868 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-04-25T15:37:27.868 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-04-25T15:37:27.868 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-04-25T15:37:27.868 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-04-25T15:37:27.868 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-04-25T15:37:27.868 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-04-25T15:37:27.869 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-04-25T15:37:27.869 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:28.450 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T15:37:28.450 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:28.450 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-04-25T15:37:28.450 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.450 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T15:37:28.451 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:28.452 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:37:28.798 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-test 2024-04-25T15:37:29.131 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-25T15:37:29.208 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 24/26 2024-04-25T15:37:29.209 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:29.209 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T15:37:29.209 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T15:37:29.209 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T15:37:29.209 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:29.210 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-25T15:37:29.293 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-04-25T15:37:29.294 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T15:37:29.335 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:27 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:37:29.460 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:37:29.460 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:29.460 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:37:29.460 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:17.2.7-902.gc61472c8.el9 ceph 48 M 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:17.2.7-902.gc61472c8.el9 ceph 163 k 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout:Install 4 Packages 2024-04-25T15:37:29.461 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:29.462 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 48 M 2024-04-25T15:37:29.462 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 192 M 2024-04-25T15:37:29.462 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:37:29.971 INFO:teuthology.orchestra.run.smithi193.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.6 MB/s | 305 kB 00:00 2024-04-25T15:37:30.046 INFO:teuthology.orchestra.run.smithi193.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 853 kB/s | 64 kB 00:00 2024-04-25T15:37:30.088 INFO:teuthology.orchestra.run.smithi193.stdout:(3/4): libcephsqlite-17.2.7-902.gc61472c8.el9.x 533 kB/s | 163 kB 00:00 2024-04-25T15:37:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T15:37:30.616 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 1/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 3/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 4/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 5/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 6/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 7/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x 8/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86 9/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 10/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 11/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 12/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 13/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-04-25T15:37:30.617 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:17.2.7-902.gc61472c8.el9.x86_64 23/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-04-25T15:37:30.618 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 25/26 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.348 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.349 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T15:37:31.350 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:31.350 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:37:31.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-test 2024-04-25T15:37:32.170 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:29 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:37:32.293 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:17.2.7-902.gc61472c8.el9 ceph 48 M 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:17.2.7-902.gc61472c8.el9 ceph 163 k 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:32.294 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:37:32.295 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:32.295 INFO:teuthology.orchestra.run.smithi110.stdout:Install 4 Packages 2024-04-25T15:37:32.295 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:32.295 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 48 M 2024-04-25T15:37:32.295 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 192 M 2024-04-25T15:37:32.295 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:37:32.390 INFO:teuthology.orchestra.run.smithi193.stdout:(4/4): ceph-test-17.2.7-902.gc61472c8.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-25T15:37:32.393 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:32.394 INFO:teuthology.orchestra.run.smithi193.stdout:Total 17 MB/s | 48 MB 00:02 2024-04-25T15:37:32.420 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:37:32.439 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:37:32.439 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:37:32.550 INFO:teuthology.orchestra.run.smithi110.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-25T15:37:32.600 INFO:teuthology.orchestra.run.smithi110.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T15:37:32.626 INFO:teuthology.orchestra.run.smithi110.stdout:(3/4): libcephsqlite-17.2.7-902.gc61472c8.el9.x 607 kB/s | 163 kB 00:00 2024-04-25T15:37:32.800 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:37:32.800 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:37:33.314 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:37:33.390 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-25T15:37:33.431 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-25T15:37:33.458 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-25T15:37:34.972 INFO:teuthology.orchestra.run.smithi110.stdout:(4/4): ceph-test-17.2.7-902.gc61472c8.el9.x86_6 18 MB/s | 48 MB 00:02 2024-04-25T15:37:34.975 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:34.975 INFO:teuthology.orchestra.run.smithi110.stdout:Total 18 MB/s | 48 MB 00:02 2024-04-25T15:37:35.001 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:37:35.021 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:37:35.021 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:37:35.382 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:37:35.383 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:37:35.891 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:37:35.956 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-25T15:37:35.996 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-25T15:37:36.024 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-25T15:37:37.811 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-25T15:37:38.180 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-25T15:37:40.376 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 3/4 2024-04-25T15:37:40.445 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-25T15:37:40.445 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 1/4 2024-04-25T15:37:40.445 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2/4 2024-04-25T15:37:40.445 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-25T15:37:40.446 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-25T15:37:40.710 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-25T15:37:40.710 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:40.710 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:37:40.711 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:40.711 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:40.711 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T15:37:40.711 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T15:37:40.711 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:40.711 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:37:41.020 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph 2024-04-25T15:37:41.529 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:39 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:37:41.655 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:17.2.7-902.gc61472c8.el9 ceph 6.5 k 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.1 M 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:17.2.7-902.gc61472c8.el9 ceph 1.4 M 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 240 k 2024-04-25T15:37:41.658 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.9 M 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 15 M 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T15:37:41.659 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T15:37:41.660 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout:Install 33 Packages 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:41.661 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 27 M 2024-04-25T15:37:41.662 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 96 M 2024-04-25T15:37:41.662 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:37:42.609 INFO:teuthology.orchestra.run.smithi193.stdout:(1/33): ceph-17.2.7-902.gc61472c8.el9.x86_64.rp 40 kB/s | 6.5 kB 00:00 2024-04-25T15:37:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 4/4 2024-04-25T15:37:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 1/4 2024-04-25T15:37:42.610 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2/4 2024-04-25T15:37:42.611 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:42.911 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:37:42.990 INFO:teuthology.orchestra.run.smithi193.stdout:(2/33): ceph-mon-17.2.7-902.gc61472c8.el9.x86_6 10 MB/s | 3.9 MB 00:00 2024-04-25T15:37:43.065 INFO:teuthology.orchestra.run.smithi193.stdout:(3/33): ceph-mds-17.2.7-902.gc61472c8.el9.x86_6 3.3 MB/s | 2.1 MB 00:00 2024-04-25T15:37:43.185 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph 2024-04-25T15:37:43.199 INFO:teuthology.orchestra.run.smithi193.stdout:(4/33): ceph-mgr-modules-core-17.2.7-902.gc6147 1.8 MB/s | 240 kB 00:00 2024-04-25T15:37:43.324 INFO:teuthology.orchestra.run.smithi193.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-25T15:37:43.407 INFO:teuthology.orchestra.run.smithi193.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 15 MB/s | 1.2 MB 00:00 2024-04-25T15:37:43.441 INFO:teuthology.orchestra.run.smithi193.stdout:(7/33): ceph-mgr-17.2.7-902.gc61472c8.el9.x86_6 1.4 MB/s | 1.4 MB 00:00 2024-04-25T15:37:43.466 INFO:teuthology.orchestra.run.smithi193.stdout:(8/33): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T15:37:43.491 INFO:teuthology.orchestra.run.smithi193.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-04-25T15:37:43.517 INFO:teuthology.orchestra.run.smithi193.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 2.4 MB/s | 125 kB 00:00 2024-04-25T15:37:43.542 INFO:teuthology.orchestra.run.smithi193.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-04-25T15:37:43.567 INFO:teuthology.orchestra.run.smithi193.stdout:(12/33): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T15:37:43.593 INFO:teuthology.orchestra.run.smithi193.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 695 kB/s | 35 kB 00:00 2024-04-25T15:37:43.618 INFO:teuthology.orchestra.run.smithi193.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T15:37:43.643 INFO:teuthology.orchestra.run.smithi193.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 831 kB/s | 42 kB 00:00 2024-04-25T15:37:43.703 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:40 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:37:43.825 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:17.2.7-902.gc61472c8.el9 ceph 6.5 k 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.1 M 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:17.2.7-902.gc61472c8.el9 ceph 1.4 M 2024-04-25T15:37:43.828 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 240 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:17.2.7-902.gc61472c8.el9 ceph 3.9 M 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 15 M 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T15:37:43.829 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T15:37:43.830 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T15:37:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T15:37:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T15:37:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:43.831 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:37:43.831 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:43.831 INFO:teuthology.orchestra.run.smithi110.stdout:Install 33 Packages 2024-04-25T15:37:43.831 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:43.832 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 27 M 2024-04-25T15:37:43.832 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 96 M 2024-04-25T15:37:43.832 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:37:43.910 INFO:teuthology.orchestra.run.smithi193.stdout:(16/33): ceph-osd-17.2.7-902.gc61472c8.el9.x86_ 17 MB/s | 15 MB 00:00 2024-04-25T15:37:44.077 INFO:teuthology.orchestra.run.smithi193.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 95 kB/s | 43 kB 00:00 2024-04-25T15:37:44.169 INFO:teuthology.orchestra.run.smithi193.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 116 kB/s | 11 kB 00:00 2024-04-25T15:37:44.194 INFO:teuthology.orchestra.run.smithi193.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 312 kB/s | 172 kB 00:00 2024-04-25T15:37:44.253 INFO:teuthology.orchestra.run.smithi193.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 211 kB/s | 18 kB 00:00 2024-04-25T15:37:44.286 INFO:teuthology.orchestra.run.smithi193.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 252 kB/s | 23 kB 00:00 2024-04-25T15:37:44.353 INFO:teuthology.orchestra.run.smithi193.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 194 kB/s | 19 kB 00:00 2024-04-25T15:37:44.378 INFO:teuthology.orchestra.run.smithi193.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 214 kB/s | 20 kB 00:00 2024-04-25T15:37:44.404 INFO:teuthology.orchestra.run.smithi193.stdout:(24/33): python3-cherrypy-18.6.1-2.el9.noarch.r 730 kB/s | 358 kB 00:00 2024-04-25T15:37:44.480 INFO:teuthology.orchestra.run.smithi193.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 786 kB/s | 79 kB 00:00 2024-04-25T15:37:44.489 INFO:teuthology.orchestra.run.smithi110.stdout:(1/33): ceph-17.2.7-902.gc61472c8.el9.x86_64.rp 25 kB/s | 6.5 kB 00:00 2024-04-25T15:37:44.504 INFO:teuthology.orchestra.run.smithi193.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 307 kB/s | 46 kB 00:00 2024-04-25T15:37:44.530 INFO:teuthology.orchestra.run.smithi193.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-25T15:37:44.572 INFO:teuthology.orchestra.run.smithi193.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 178 kB/s | 16 kB 00:00 2024-04-25T15:37:44.597 INFO:teuthology.orchestra.run.smithi193.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 976 kB/s | 90 kB 00:00 2024-04-25T15:37:44.631 INFO:teuthology.orchestra.run.smithi193.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 355 kB/s | 36 kB 00:00 2024-04-25T15:37:44.731 INFO:teuthology.orchestra.run.smithi193.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 200 kB/s | 20 kB 00:00 2024-04-25T15:37:44.739 INFO:teuthology.orchestra.run.smithi110.stdout:(2/33): ceph-mds-17.2.7-902.gc61472c8.el9.x86_6 4.1 MB/s | 2.1 MB 00:00 2024-04-25T15:37:44.756 INFO:teuthology.orchestra.run.smithi193.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2024-04-25T15:37:44.782 INFO:teuthology.orchestra.run.smithi110.stdout:(3/33): ceph-mgr-17.2.7-902.gc61472c8.el9.x86_6 2.6 MB/s | 1.4 MB 00:00 2024-04-25T15:37:44.789 INFO:teuthology.orchestra.run.smithi193.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.2 MB/s | 427 kB 00:00 2024-04-25T15:37:44.799 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:44.800 INFO:teuthology.orchestra.run.smithi193.stdout:Total 8.7 MB/s | 27 MB 00:03 2024-04-25T15:37:44.857 INFO:teuthology.orchestra.run.smithi110.stdout:(4/33): ceph-mgr-modules-core-17.2.7-902.gc6147 3.1 MB/s | 240 kB 00:00 2024-04-25T15:37:44.914 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:37:44.950 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:37:44.950 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:37:44.982 INFO:teuthology.orchestra.run.smithi110.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.0 MB/s | 253 kB 00:00 2024-04-25T15:37:45.058 INFO:teuthology.orchestra.run.smithi110.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 16 MB/s | 1.2 MB 00:00 2024-04-25T15:37:45.082 INFO:teuthology.orchestra.run.smithi110.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 4.2 MB/s | 106 kB 00:00 2024-04-25T15:37:45.116 INFO:teuthology.orchestra.run.smithi110.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 4.0 MB/s | 135 kB 00:00 2024-04-25T15:37:45.158 INFO:teuthology.orchestra.run.smithi110.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-04-25T15:37:45.200 INFO:teuthology.orchestra.run.smithi110.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 5.0 MB/s | 215 kB 00:00 2024-04-25T15:37:45.214 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:37:45.215 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:37:45.250 INFO:teuthology.orchestra.run.smithi110.stdout:(11/33): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T15:37:45.309 INFO:teuthology.orchestra.run.smithi110.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 595 kB/s | 35 kB 00:00 2024-04-25T15:37:45.376 INFO:teuthology.orchestra.run.smithi110.stdout:(13/33): ceph-mon-17.2.7-902.gc61472c8.el9.x86_ 4.4 MB/s | 3.9 MB 00:00 2024-04-25T15:37:45.393 INFO:teuthology.orchestra.run.smithi110.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 609 kB/s | 51 kB 00:00 2024-04-25T15:37:45.410 INFO:teuthology.orchestra.run.smithi110.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2024-04-25T15:37:45.627 INFO:teuthology.orchestra.run.smithi110.stdout:(16/33): ceph-osd-17.2.7-902.gc61472c8.el9.x86_ 17 MB/s | 15 MB 00:00 2024-04-25T15:37:45.652 INFO:teuthology.orchestra.run.smithi110.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 168 kB/s | 43 kB 00:00 2024-04-25T15:37:45.719 INFO:teuthology.orchestra.run.smithi110.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 255 kB/s | 11 kB 00:00 2024-04-25T15:37:45.744 INFO:teuthology.orchestra.run.smithi110.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 514 kB/s | 172 kB 00:00 2024-04-25T15:37:45.770 INFO:teuthology.orchestra.run.smithi110.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 359 kB/s | 18 kB 00:00 2024-04-25T15:37:45.795 INFO:teuthology.orchestra.run.smithi110.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 462 kB/s | 23 kB 00:00 2024-04-25T15:37:45.820 INFO:teuthology.orchestra.run.smithi110.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-04-25T15:37:45.845 INFO:teuthology.orchestra.run.smithi110.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-04-25T15:37:45.871 INFO:teuthology.orchestra.run.smithi110.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-04-25T15:37:45.896 INFO:teuthology.orchestra.run.smithi110.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-04-25T15:37:45.930 INFO:teuthology.orchestra.run.smithi110.stdout:(26/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-04-25T15:37:45.955 INFO:teuthology.orchestra.run.smithi110.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-04-25T15:37:45.980 INFO:teuthology.orchestra.run.smithi110.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-04-25T15:37:46.005 INFO:teuthology.orchestra.run.smithi110.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 711 kB/s | 36 kB 00:00 2024-04-25T15:37:46.031 INFO:teuthology.orchestra.run.smithi110.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-04-25T15:37:46.056 INFO:teuthology.orchestra.run.smithi110.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T15:37:46.081 INFO:teuthology.orchestra.run.smithi110.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-04-25T15:37:46.106 INFO:teuthology.orchestra.run.smithi110.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-04-25T15:37:46.112 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:46.113 INFO:teuthology.orchestra.run.smithi110.stdout:Total 12 MB/s | 27 MB 00:02 2024-04-25T15:37:46.226 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:37:46.263 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:37:46.263 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:37:46.345 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:37:46.400 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-25T15:37:46.441 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-25T15:37:46.474 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-25T15:37:46.528 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:37:46.529 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:37:46.574 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-25T15:37:46.652 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-25T15:37:46.693 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-25T15:37:46.728 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-25T15:37:46.758 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-25T15:37:46.801 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-25T15:37:46.834 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-25T15:37:46.874 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-25T15:37:46.909 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-25T15:37:46.944 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-25T15:37:46.982 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-25T15:37:47.055 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-25T15:37:47.200 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-25T15:37:47.281 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-25T15:37:47.334 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-25T15:37:47.373 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-25T15:37:47.422 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-25T15:37:47.477 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-25T15:37:47.662 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-25T15:37:47.667 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:37:47.719 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-25T15:37:47.743 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-25T15:37:47.775 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-25T15:37:47.796 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-25T15:37:47.807 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-25T15:37:47.885 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-25T15:37:47.900 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-25T15:37:47.926 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-25T15:37:47.980 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-25T15:37:48.027 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-25T15:37:48.041 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-25T15:37:48.062 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-25T15:37:48.092 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-25T15:37:48.126 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-25T15:37:48.160 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-25T15:37:48.188 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 28/33 2024-04-25T15:37:48.201 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-25T15:37:48.212 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-25T15:37:48.235 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-25T15:37:48.236 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-25T15:37:48.236 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:48.237 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T15:37:48.237 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T15:37:48.237 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T15:37:48.237 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:48.270 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-25T15:37:48.308 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-25T15:37:48.382 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-25T15:37:48.537 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-25T15:37:48.617 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-25T15:37:48.669 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-25T15:37:48.708 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-25T15:37:48.757 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-25T15:37:48.813 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-25T15:37:48.998 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-25T15:37:49.054 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-25T15:37:49.131 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-25T15:37:49.220 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-25T15:37:49.261 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-25T15:37:49.376 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-25T15:37:49.524 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 28/33 2024-04-25T15:37:49.546 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-25T15:37:49.547 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-25T15:37:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 29/33 2024-04-25T15:37:49.571 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:49.571 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T15:37:49.571 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T15:37:49.571 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T15:37:49.571 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:49.572 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-25T15:37:49.572 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:49.572 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T15:37:49.572 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T15:37:49.572 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T15:37:49.572 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:49.947 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-25T15:37:49.969 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-25T15:37:49.969 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:49.969 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T15:37:49.970 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T15:37:49.970 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T15:37:49.970 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:50.172 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-25T15:37:50.195 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-25T15:37:50.195 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:50.195 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T15:37:50.195 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T15:37:50.195 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T15:37:50.195 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:50.258 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-25T15:37:50.915 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-25T15:37:50.942 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 30/33 2024-04-25T15:37:50.942 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:50.942 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T15:37:50.942 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T15:37:50.942 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T15:37:50.943 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:51.324 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-25T15:37:51.347 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 31/33 2024-04-25T15:37:51.347 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:51.347 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T15:37:51.348 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T15:37:51.348 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T15:37:51.348 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:51.550 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-25T15:37:51.572 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 32/33 2024-04-25T15:37:51.573 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:51.573 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T15:37:51.573 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T15:37:51.573 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T15:37:51.573 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:51.635 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-25T15:37:51.826 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 1/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 3/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 4/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 5/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 6/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-25T15:37:51.827 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-25T15:37:51.828 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-25T15:37:51.829 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-25T15:37:51.829 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-25T15:37:51.829 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-25T15:37:51.829 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-25T15:37:52.646 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T15:37:52.647 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T15:37:52.648 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T15:37:52.649 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T15:37:52.649 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T15:37:52.649 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T15:37:52.649 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:52.649 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:37:52.988 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-base 2024-04-25T15:37:53.193 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 33/33 2024-04-25T15:37:53.193 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:17.2.7-902.gc61472c8.el9.x86_64 1/33 2024-04-25T15:37:53.193 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2/33 2024-04-25T15:37:53.193 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 3/33 2024-04-25T15:37:53.193 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 4/33 2024-04-25T15:37:53.193 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 5/33 2024-04-25T15:37:53.193 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.n 6/33 2024-04-25T15:37:53.193 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-25T15:37:53.194 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-25T15:37:53.195 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-25T15:37:53.195 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-25T15:37:53.195 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-25T15:37:53.195 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-25T15:37:53.196 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-25T15:37:53.196 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-25T15:37:53.196 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-25T15:37:53.196 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-25T15:37:53.196 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-25T15:37:53.196 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-25T15:37:53.196 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-25T15:37:53.501 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:51 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:37:53.580 INFO:teuthology.orchestra.run.smithi193.stdout:Package ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:37:53.621 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:37:53.621 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:37:53.622 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:37:53.698 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install cephadm 2024-04-25T15:37:54.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T15:37:54.009 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T15:37:54.010 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:54.011 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:37:54.203 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:52 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:37:54.322 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:37:54.322 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:54.322 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 82 k 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 82 k 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 372 k 2024-04-25T15:37:54.323 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:37:54.429 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-base 2024-04-25T15:37:54.613 INFO:teuthology.orchestra.run.smithi193.stdout:cephadm-17.2.7-902.gc61472c8.el9.noarch.rpm 283 kB/s | 82 kB 00:00 2024-04-25T15:37:54.614 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:54.614 INFO:teuthology.orchestra.run.smithi193.stdout:Total 282 kB/s | 82 kB 00:00 2024-04-25T15:37:54.614 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:37:54.619 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:37:54.619 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:37:54.621 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:37:54.621 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:37:54.694 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:37:54.934 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:51 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:37:55.010 INFO:teuthology.orchestra.run.smithi110.stdout:Package ceph-base-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:37:55.052 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:37:55.053 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:37:55.053 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:37:55.119 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T15:37:55.120 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install cephadm 2024-04-25T15:37:55.207 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T15:37:55.606 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T15:37:55.622 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:52 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:37:55.740 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:37:55.741 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:55.741 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:37:55.741 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:55.741 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 82 k 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 82 k 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 372 k 2024-04-25T15:37:55.742 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:37:55.863 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T15:37:55.863 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:55.864 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:37:55.864 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:37:55.864 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:55.864 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:37:56.067 INFO:teuthology.orchestra.run.smithi110.stdout:cephadm-17.2.7-902.gc61472c8.el9.noarch.rpm 253 kB/s | 82 kB 00:00 2024-04-25T15:37:56.067 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:56.068 INFO:teuthology.orchestra.run.smithi110.stdout:Total 252 kB/s | 82 kB 00:00 2024-04-25T15:37:56.068 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:37:56.072 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:37:56.073 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:37:56.075 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:37:56.075 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:37:56.077 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-immutable-object-cache 2024-04-25T15:37:56.149 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:37:56.614 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T15:37:56.615 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:54 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:37:56.693 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T15:37:56.737 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:37:56.737 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:56.737 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-04-25T15:37:56.737 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:56.737 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:37:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-902.gc61472c8.el9 ceph 137 k 2024-04-25T15:37:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:56.738 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:37:56.738 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:56.738 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-04-25T15:37:56.738 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:56.738 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 137 k 2024-04-25T15:37:56.739 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 379 k 2024-04-25T15:37:56.739 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:37:56.983 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T15:37:57.094 INFO:teuthology.orchestra.run.smithi193.stdout:ceph-immutable-object-cache-17.2.7-902.gc61472c 387 kB/s | 137 kB 00:00 2024-04-25T15:37:57.094 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:57.095 INFO:teuthology.orchestra.run.smithi193.stdout:Total 385 kB/s | 137 kB 00:00 2024-04-25T15:37:57.095 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:37:57.102 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:37:57.102 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:37:57.158 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:37:57.158 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:37:57.218 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/1 2024-04-25T15:37:57.218 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:57.218 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:37:57.218 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:37:57.218 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:57.219 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:37:57.306 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:37:57.370 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T15:37:57.395 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T15:37:57.395 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:57.395 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T15:37:57.395 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:57.435 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-immutable-object-cache 2024-04-25T15:37:57.948 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:54 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:37:58.069 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:37:58.070 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-04-25T15:37:58.070 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:58.070 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:37:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-902.gc61472c8.el9 ceph 137 k 2024-04-25T15:37:58.070 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:58.070 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:37:58.071 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:37:58.071 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-04-25T15:37:58.071 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:58.071 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 137 k 2024-04-25T15:37:58.071 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 379 k 2024-04-25T15:37:58.071 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:37:58.276 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T15:37:58.276 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:58.276 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:37:58.276 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:58.276 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:58.276 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:37:58.288 INFO:teuthology.orchestra.run.smithi110.stdout:ceph-immutable-object-cache-17.2.7-902.gc61472c 633 kB/s | 137 kB 00:00 2024-04-25T15:37:58.289 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:37:58.289 INFO:teuthology.orchestra.run.smithi110.stdout:Total 628 kB/s | 137 kB 00:00 2024-04-25T15:37:58.289 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:37:58.297 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:37:58.297 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:37:58.352 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:37:58.353 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:37:58.472 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr 2024-04-25T15:37:58.501 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:37:58.589 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T15:37:58.613 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T15:37:58.613 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:37:58.614 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T15:37:58.614 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:58.987 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:56 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:37:59.068 INFO:teuthology.orchestra.run.smithi193.stdout:Package ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:37:59.109 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:37:59.109 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:37:59.110 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:37:59.249 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-dashboard 2024-04-25T15:37:59.431 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.e 1/1 2024-04-25T15:37:59.431 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:59.432 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:37:59.432 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:37:59.432 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:37:59.432 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:37:59.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr 2024-04-25T15:37:59.760 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:57 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:37:59.880 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:37:59.880 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:59.880 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 3.4 M 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 24 k 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 15 k 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-25T15:37:59.881 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:59.882 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:37:59.882 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:37:59.882 INFO:teuthology.orchestra.run.smithi193.stdout:Install 7 Packages 2024-04-25T15:37:59.882 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:37:59.882 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 3.7 M 2024-04-25T15:37:59.882 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 80 M 2024-04-25T15:37:59.882 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:00.145 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:57 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:00.226 INFO:teuthology.orchestra.run.smithi110.stdout:Package ceph-mgr-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:00.265 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:00.266 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:38:00.266 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:00.281 INFO:teuthology.orchestra.run.smithi193.stdout:(1/7): ceph-grafana-dashboards-17.2.7-902.gc614 119 kB/s | 24 kB 00:00 2024-04-25T15:38:00.358 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-dashboard 2024-04-25T15:38:00.456 INFO:teuthology.orchestra.run.smithi193.stdout:(2/7): ceph-prometheus-alerts-17.2.7-902.gc6147 38 kB/s | 15 kB 00:00 2024-04-25T15:38:00.548 INFO:teuthology.orchestra.run.smithi193.stdout:(3/7): ceph-mgr-dashboard-17.2.7-902.gc61472c8. 7.1 MB/s | 3.4 MB 00:00 2024-04-25T15:38:00.590 INFO:teuthology.orchestra.run.smithi193.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 29 kB/s | 9.0 kB 00:00 2024-04-25T15:38:00.724 INFO:teuthology.orchestra.run.smithi193.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 176 kB/s | 31 kB 00:00 2024-04-25T15:38:00.807 INFO:teuthology.orchestra.run.smithi193.stdout:(6/7): python3-jwt-2.4.0-1.el9.noarch.rpm 117 kB/s | 41 kB 00:00 2024-04-25T15:38:00.874 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:00:57 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:00.994 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:00.994 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:00.994 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 3.4 M 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 24 k 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 15 k 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout:Installing weak dependencies: 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-25T15:38:00.995 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:00.996 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:00.996 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:00.996 INFO:teuthology.orchestra.run.smithi110.stdout:Install 7 Packages 2024-04-25T15:38:00.996 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:00.996 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 3.7 M 2024-04-25T15:38:00.996 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 80 M 2024-04-25T15:38:00.996 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:01.091 INFO:teuthology.orchestra.run.smithi193.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 377 kB/s | 188 kB 00:00 2024-04-25T15:38:01.092 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:01.093 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.0 MB/s | 3.7 MB 00:01 2024-04-25T15:38:01.093 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:01.101 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:01.101 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:01.162 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:01.164 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:01.468 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:01.550 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-25T15:38:01.551 INFO:teuthology.orchestra.run.smithi110.stdout:(1/7): ceph-grafana-dashboards-17.2.7-902.gc614 133 kB/s | 24 kB 00:00 2024-04-25T15:38:01.576 INFO:teuthology.orchestra.run.smithi110.stdout:(2/7): ceph-prometheus-alerts-17.2.7-902.gc6147 70 kB/s | 15 kB 00:00 2024-04-25T15:38:01.584 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-25T15:38:01.631 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-25T15:38:01.643 INFO:teuthology.orchestra.run.smithi110.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 98 kB/s | 9.0 kB 00:00 2024-04-25T15:38:01.668 INFO:teuthology.orchestra.run.smithi110.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-04-25T15:38:01.671 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-25T15:38:01.700 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 5/7 2024-04-25T15:38:01.701 INFO:teuthology.orchestra.run.smithi110.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 527 kB/s | 31 kB 00:00 2024-04-25T15:38:01.768 INFO:teuthology.orchestra.run.smithi110.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 1.8 MB/s | 188 kB 00:00 2024-04-25T15:38:01.852 INFO:teuthology.orchestra.run.smithi110.stdout:(7/7): ceph-mgr-dashboard-17.2.7-902.gc61472c8. 6.9 MB/s | 3.4 MB 00:00 2024-04-25T15:38:01.854 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:01.854 INFO:teuthology.orchestra.run.smithi110.stdout:Total 4.3 MB/s | 3.7 MB 00:00 2024-04-25T15:38:01.854 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:01.865 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:01.865 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:01.924 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:01.924 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:02.223 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:02.295 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-25T15:38:02.329 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-25T15:38:02.384 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-25T15:38:02.424 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-25T15:38:02.453 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 5/7 2024-04-25T15:38:02.696 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 6/7 2024-04-25T15:38:02.728 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-25T15:38:03.451 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 6/7 2024-04-25T15:38:03.482 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-25T15:38:03.815 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-25T15:38:03.815 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 1/7 2024-04-25T15:38:03.815 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2/7 2024-04-25T15:38:03.815 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 3/7 2024-04-25T15:38:03.816 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-25T15:38:03.816 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-25T15:38:03.816 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-25T15:38:04.064 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-25T15:38:04.064 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:04.064 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:04.065 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:04.260 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-25T15:38:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 7/7 2024-04-25T15:38:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.n 1/7 2024-04-25T15:38:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2/7 2024-04-25T15:38:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.no 3/7 2024-04-25T15:38:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-25T15:38:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-25T15:38:04.561 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-25T15:38:04.767 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:02 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T15:38:04.805 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:04.806 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:04.888 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:04.889 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-04-25T15:38:04.889 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:38:04.889 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 7.4 M 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T15:38:04.890 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T15:38:04.891 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T15:38:04.891 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:04.891 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:04.891 INFO:teuthology.orchestra.run.smithi193.stdout:====================================================================================== 2024-04-25T15:38:04.891 INFO:teuthology.orchestra.run.smithi193.stdout:Install 12 Packages 2024-04-25T15:38:04.891 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:04.891 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 38 M 2024-04-25T15:38:04.892 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 209 M 2024-04-25T15:38:04.892 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:05.005 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-25T15:38:05.457 INFO:teuthology.orchestra.run.smithi193.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 195 kB 00:00 2024-04-25T15:38:05.499 INFO:teuthology.orchestra.run.smithi193.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-04-25T15:38:05.515 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:02 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:05.524 INFO:teuthology.orchestra.run.smithi193.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 495 kB/s | 33 kB 00:00 2024-04-25T15:38:05.566 INFO:teuthology.orchestra.run.smithi193.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 447 kB/s | 18 kB 00:00 2024-04-25T15:38:05.632 INFO:teuthology.orchestra.run.smithi193.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 22 MB/s | 3.0 MB 00:00 2024-04-25T15:38:05.635 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 7.4 M 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T15:38:05.637 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout:====================================================================================== 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout:Install 12 Packages 2024-04-25T15:38:05.638 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:05.639 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 38 M 2024-04-25T15:38:05.639 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 209 M 2024-04-25T15:38:05.639 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:05.658 INFO:teuthology.orchestra.run.smithi193.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 393 kB/s | 35 kB 00:00 2024-04-25T15:38:05.700 INFO:teuthology.orchestra.run.smithi193.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 5.9 MB/s | 244 kB 00:00 2024-04-25T15:38:05.800 INFO:teuthology.orchestra.run.smithi193.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 28 MB/s | 4.7 MB 00:00 2024-04-25T15:38:05.884 INFO:teuthology.orchestra.run.smithi193.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-04-25T15:38:05.909 INFO:teuthology.orchestra.run.smithi193.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.3 MB/s | 250 kB 00:00 2024-04-25T15:38:05.944 INFO:teuthology.orchestra.run.smithi110.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-25T15:38:06.001 INFO:teuthology.orchestra.run.smithi193.stdout:(11/12): ceph-mgr-diskprediction-local-17.2.7-9 11 MB/s | 7.4 MB 00:00 2024-04-25T15:38:06.002 INFO:teuthology.orchestra.run.smithi110.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.7 MB/s | 803 kB 00:00 2024-04-25T15:38:06.028 INFO:teuthology.orchestra.run.smithi110.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 396 kB/s | 33 kB 00:00 2024-04-25T15:38:06.195 INFO:teuthology.orchestra.run.smithi110.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 109 kB/s | 18 kB 00:00 2024-04-25T15:38:06.253 INFO:teuthology.orchestra.run.smithi110.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 12 MB/s | 3.0 MB 00:00 2024-04-25T15:38:06.279 INFO:teuthology.orchestra.run.smithi110.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 427 kB/s | 35 kB 00:00 2024-04-25T15:38:06.343 INFO:teuthology.orchestra.run.smithi193.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-04-25T15:38:06.345 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:06.346 INFO:teuthology.orchestra.run.smithi193.stdout:Total 26 MB/s | 38 MB 00:01 2024-04-25T15:38:06.479 INFO:teuthology.orchestra.run.smithi110.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2024-04-25T15:38:06.504 INFO:teuthology.orchestra.run.smithi110.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-04-25T15:38:06.630 INFO:teuthology.orchestra.run.smithi110.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 5.1 MB 00:00 2024-04-25T15:38:06.655 INFO:teuthology.orchestra.run.smithi110.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.6 MB/s | 250 kB 00:00 2024-04-25T15:38:06.699 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:06.727 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:06.727 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:07.030 INFO:teuthology.orchestra.run.smithi110.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-04-25T15:38:07.136 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:07.136 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:07.890 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:07.947 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-25T15:38:07.985 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-25T15:38:08.083 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-25T15:38:08.113 INFO:teuthology.orchestra.run.smithi110.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-9 3.2 MB/s | 7.4 MB 00:02 2024-04-25T15:38:08.116 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:08.116 INFO:teuthology.orchestra.run.smithi110.stdout:Total 15 MB/s | 38 MB 00:02 2024-04-25T15:38:08.126 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-25T15:38:08.303 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-25T15:38:08.332 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-25T15:38:08.409 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-25T15:38:08.468 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:08.494 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:08.494 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:08.896 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:08.897 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:09.048 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-25T15:38:09.153 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-25T15:38:09.652 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:09.718 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-25T15:38:09.761 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-25T15:38:09.861 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-25T15:38:09.905 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-25T15:38:10.084 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-25T15:38:10.112 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-25T15:38:10.188 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-25T15:38:10.374 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T15:38:10.823 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-25T15:38:10.933 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-25T15:38:11.408 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-25T15:38:11.432 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-25T15:38:12.144 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T15:38:13.169 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-25T15:38:13.196 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 1/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T15:38:14.047 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-25T15:38:14.398 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T15:38:14.399 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T15:38:14.400 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T15:38:14.400 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T15:38:14.400 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:14.400 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:14.770 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-rook 2024-04-25T15:38:15.293 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:13 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:15.415 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:15.416 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:15.416 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 47 k 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T15:38:15.417 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout:Install 13 Packages 2024-04-25T15:38:15.418 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:15.419 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 2.2 M 2024-04-25T15:38:15.419 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 26 M 2024-04-25T15:38:15.419 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:15.599 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 12/12 2024-04-25T15:38:15.599 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472 1/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-25T15:38:15.600 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-25T15:38:15.915 INFO:teuthology.orchestra.run.smithi193.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 141 kB/s | 19 kB 00:00 2024-04-25T15:38:15.940 INFO:teuthology.orchestra.run.smithi193.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 159 kB/s | 26 kB 00:00 2024-04-25T15:38:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-25T15:38:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:15.960 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T15:38:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T15:38:15.960 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:15.961 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:15.990 INFO:teuthology.orchestra.run.smithi193.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.9 MB/s | 222 kB 00:00 2024-04-25T15:38:16.015 INFO:teuthology.orchestra.run.smithi193.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T15:38:16.041 INFO:teuthology.orchestra.run.smithi193.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.5 MB/s | 279 kB 00:00 2024-04-25T15:38:16.066 INFO:teuthology.orchestra.run.smithi193.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-04-25T15:38:16.092 INFO:teuthology.orchestra.run.smithi193.stdout:(7/13): ceph-mgr-rook-17.2.7-902.gc61472c8.el9. 152 kB/s | 47 kB 00:00 2024-04-25T15:38:16.200 INFO:teuthology.orchestra.run.smithi193.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 112 kB/s | 15 kB 00:00 2024-04-25T15:38:16.225 INFO:teuthology.orchestra.run.smithi193.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 175 kB/s | 32 kB 00:00 2024-04-25T15:38:16.326 INFO:teuthology.orchestra.run.smithi193.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 879 kB/s | 206 kB 00:00 2024-04-25T15:38:16.351 INFO:teuthology.orchestra.run.smithi193.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 472 kB/s | 59 kB 00:00 2024-04-25T15:38:16.393 INFO:teuthology.orchestra.run.smithi193.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-04-25T15:38:16.434 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-rook 2024-04-25T15:38:16.468 INFO:teuthology.orchestra.run.smithi193.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.8 MB/s | 1.0 MB 00:00 2024-04-25T15:38:16.469 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:16.470 INFO:teuthology.orchestra.run.smithi193.stdout:Total 2.1 MB/s | 2.2 MB 00:01 2024-04-25T15:38:16.531 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:16.546 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:16.546 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:16.628 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:16.629 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:16.949 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:13 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:17.068 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:17.069 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:17.069 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 47 k 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T15:38:17.070 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout:Install 13 Packages 2024-04-25T15:38:17.071 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:17.072 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 2.2 M 2024-04-25T15:38:17.072 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 26 M 2024-04-25T15:38:17.072 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:17.215 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:17.381 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-25T15:38:17.444 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-25T15:38:17.507 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-25T15:38:17.537 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-25T15:38:17.578 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-25T15:38:17.637 INFO:teuthology.orchestra.run.smithi110.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 213 kB/s | 26 kB 00:00 2024-04-25T15:38:17.654 INFO:teuthology.orchestra.run.smithi110.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 140 kB/s | 19 kB 00:00 2024-04-25T15:38:17.679 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-25T15:38:17.704 INFO:teuthology.orchestra.run.smithi110.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 3.3 MB/s | 222 kB 00:00 2024-04-25T15:38:17.729 INFO:teuthology.orchestra.run.smithi110.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-25T15:38:17.755 INFO:teuthology.orchestra.run.smithi110.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.4 MB/s | 279 kB 00:00 2024-04-25T15:38:17.780 INFO:teuthology.orchestra.run.smithi110.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.1 MB/s | 54 kB 00:00 2024-04-25T15:38:17.790 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-25T15:38:17.805 INFO:teuthology.orchestra.run.smithi110.stdout:(7/13): ceph-mgr-rook-17.2.7-902.gc61472c8.el9. 165 kB/s | 47 kB 00:00 2024-04-25T15:38:17.837 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-25T15:38:18.089 INFO:teuthology.orchestra.run.smithi110.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 49 kB/s | 15 kB 00:00 2024-04-25T15:38:18.114 INFO:teuthology.orchestra.run.smithi110.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 90 kB/s | 32 kB 00:00 2024-04-25T15:38:18.264 INFO:teuthology.orchestra.run.smithi110.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 395 kB/s | 59 kB 00:00 2024-04-25T15:38:18.289 INFO:teuthology.orchestra.run.smithi110.stdout:(11/13): python3-google-auth-2.29.0-1.el9.noarc 425 kB/s | 206 kB 00:00 2024-04-25T15:38:18.373 INFO:teuthology.orchestra.run.smithi110.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 826 kB/s | 90 kB 00:00 2024-04-25T15:38:18.457 INFO:teuthology.orchestra.run.smithi110.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.8 MB/s | 1.0 MB 00:00 2024-04-25T15:38:18.461 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:18.461 INFO:teuthology.orchestra.run.smithi110.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-04-25T15:38:18.522 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:18.538 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:18.538 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:18.556 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-25T15:38:18.610 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-25T15:38:18.620 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:18.621 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:18.644 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-25T15:38:18.689 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-25T15:38:18.714 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-25T15:38:19.211 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:19.377 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-25T15:38:19.411 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 1/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-25T15:38:19.412 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-25T15:38:19.433 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-25T15:38:19.488 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-25T15:38:19.518 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-25T15:38:19.559 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-25T15:38:19.651 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-25T15:38:19.729 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-25T15:38:19.729 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:19.729 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:19.729 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:19.729 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T15:38:19.729 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T15:38:19.729 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T15:38:19.729 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:19.730 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:19.753 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-25T15:38:19.811 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-25T15:38:19.972 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-mgr-cephadm 2024-04-25T15:38:20.488 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:18 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:20.521 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-25T15:38:20.575 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-25T15:38:20.609 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-25T15:38:20.609 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:20.610 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:20.610 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:38:20.610 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:20.610 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:20.610 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 123 k 2024-04-25T15:38:20.610 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T15:38:20.610 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T15:38:20.610 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T15:38:20.611 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T15:38:20.611 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T15:38:20.611 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T15:38:20.611 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:20.611 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:20.611 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:20.611 INFO:teuthology.orchestra.run.smithi193.stdout:Install 6 Packages 2024-04-25T15:38:20.611 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:20.612 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 7.0 M 2024-04-25T15:38:20.612 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 33 M 2024-04-25T15:38:20.612 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:20.654 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-25T15:38:20.678 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-25T15:38:21.244 INFO:teuthology.orchestra.run.smithi193.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 999 kB/s | 248 kB 00:00 2024-04-25T15:38:21.269 INFO:teuthology.orchestra.run.smithi193.stdout:(2/6): ceph-mgr-cephadm-17.2.7-902.gc61472c8.el 451 kB/s | 123 kB 00:00 2024-04-25T15:38:21.361 INFO:teuthology.orchestra.run.smithi193.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 636 kB/s | 58 kB 00:00 2024-04-25T15:38:21.445 INFO:teuthology.orchestra.run.smithi193.stdout:(4/6): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-04-25T15:38:21.470 INFO:teuthology.orchestra.run.smithi193.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 471 kB/s | 51 kB 00:00 2024-04-25T15:38:21.519 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 13/13 2024-04-25T15:38:21.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 1/13 2024-04-25T15:38:21.519 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-25T15:38:21.520 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-25T15:38:21.562 INFO:teuthology.orchestra.run.smithi193.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.7 MB/s | 548 kB 00:00 2024-04-25T15:38:21.564 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:21.564 INFO:teuthology.orchestra.run.smithi193.stdout:Total 7.3 MB/s | 7.0 MB 00:00 2024-04-25T15:38:21.634 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:21.645 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:21.645 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:21.740 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:21.740 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T15:38:21.907 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T15:38:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T15:38:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T15:38:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T15:38:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T15:38:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T15:38:21.908 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:21.908 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:22.052 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:22.123 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-mgr-cephadm 2024-04-25T15:38:22.216 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-25T15:38:22.271 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-25T15:38:22.624 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:19 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:22.742 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:17.2.7-902.gc61472c8.el9 ceph-noarch 123 k 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T15:38:22.743 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T15:38:22.744 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T15:38:22.744 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T15:38:22.744 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:22.744 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:22.744 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:22.744 INFO:teuthology.orchestra.run.smithi110.stdout:Install 6 Packages 2024-04-25T15:38:22.744 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:22.744 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 7.0 M 2024-04-25T15:38:22.745 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 33 M 2024-04-25T15:38:22.745 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:22.787 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-25T15:38:22.883 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-25T15:38:22.937 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-25T15:38:22.967 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-25T15:38:23.329 INFO:teuthology.orchestra.run.smithi110.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.5 MB/s | 248 kB 00:00 2024-04-25T15:38:23.479 INFO:teuthology.orchestra.run.smithi110.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-25T15:38:23.605 INFO:teuthology.orchestra.run.smithi110.stdout:(3/6): ceph-mgr-cephadm-17.2.7-902.gc61472c8.el 280 kB/s | 123 kB 00:00 2024-04-25T15:38:23.629 INFO:teuthology.orchestra.run.smithi110.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.8 MB/s | 548 kB 00:00 2024-04-25T15:38:23.655 INFO:teuthology.orchestra.run.smithi110.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 329 kB/s | 58 kB 00:00 2024-04-25T15:38:23.724 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-25T15:38:23.724 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/6 2024-04-25T15:38:23.724 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-25T15:38:23.725 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-25T15:38:23.725 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-25T15:38:23.725 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-25T15:38:23.772 INFO:teuthology.orchestra.run.smithi110.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 305 kB/s | 51 kB 00:00 2024-04-25T15:38:23.773 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:23.774 INFO:teuthology.orchestra.run.smithi110.stdout:Total 6.8 MB/s | 7.0 MB 00:01 2024-04-25T15:38:23.843 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:23.854 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:23.854 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:23.948 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:23.949 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:23.970 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-25T15:38:23.970 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:23.970 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:23.971 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:23.971 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T15:38:23.971 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T15:38:23.971 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T15:38:23.971 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T15:38:23.971 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T15:38:23.971 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:23.971 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:24.218 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-fuse 2024-04-25T15:38:24.249 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:24.411 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-25T15:38:24.465 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-25T15:38:24.727 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:22 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:24.846 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:24.847 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:24.847 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T15:38:24.847 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:24.847 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:24.847 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 814 k 2024-04-25T15:38:24.847 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T15:38:24.847 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T15:38:24.848 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:24.848 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:24.848 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:24.848 INFO:teuthology.orchestra.run.smithi193.stdout:Install 2 Packages 2024-04-25T15:38:24.848 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:24.849 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 894 k 2024-04-25T15:38:24.849 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 2.6 M 2024-04-25T15:38:24.849 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:24.981 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-25T15:38:25.039 INFO:teuthology.orchestra.run.smithi193.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 662 kB/s | 80 kB 00:00 2024-04-25T15:38:25.084 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-25T15:38:25.140 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-25T15:38:25.169 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-25T15:38:25.438 INFO:teuthology.orchestra.run.smithi193.stdout:(2/2): ceph-fuse-17.2.7-902.gc61472c8.el9.x86_6 1.5 MB/s | 814 kB 00:00 2024-04-25T15:38:25.440 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:25.440 INFO:teuthology.orchestra.run.smithi193.stdout:Total 1.5 MB/s | 894 kB 00:00 2024-04-25T15:38:25.452 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:25.473 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:25.473 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:25.549 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:25.550 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:25.730 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:25.859 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-25T15:38:25.928 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-25T15:38:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 6/6 2024-04-25T15:38:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 1/6 2024-04-25T15:38:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-25T15:38:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-25T15:38:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-25T15:38:26.013 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:17.2.7-902.gc61472c8.el9.noarch 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:26.261 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:26.501 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-fuse 2024-04-25T15:38:26.631 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-25T15:38:26.631 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/2 2024-04-25T15:38:26.879 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-25T15:38:26.880 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:26.880 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:26.880 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-25T15:38:26.880 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:26.880 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:27.012 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:24 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 814 k 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T15:38:27.131 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:27.132 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:27.132 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:27.132 INFO:teuthology.orchestra.run.smithi110.stdout:Install 2 Packages 2024-04-25T15:38:27.132 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:27.132 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 894 k 2024-04-25T15:38:27.132 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 2.6 M 2024-04-25T15:38:27.133 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:27.167 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librados-devel 2024-04-25T15:38:27.373 INFO:teuthology.orchestra.run.smithi110.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 655 kB/s | 80 kB 00:00 2024-04-25T15:38:27.590 INFO:teuthology.orchestra.run.smithi110.stdout:(2/2): ceph-fuse-17.2.7-902.gc61472c8.el9.x86_6 2.3 MB/s | 814 kB 00:00 2024-04-25T15:38:27.591 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:27.591 INFO:teuthology.orchestra.run.smithi110.stdout:Total 1.9 MB/s | 894 kB 00:00 2024-04-25T15:38:27.602 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:27.623 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:27.623 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:27.682 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:25 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:27.698 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:27.699 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:27.804 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:27.804 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 126 k 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 126 k 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 448 k 2024-04-25T15:38:27.805 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:27.875 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:28.010 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-25T15:38:28.035 INFO:teuthology.orchestra.run.smithi193.stdout:librados-devel-17.2.7-902.gc61472c8.el9.x86_64. 551 kB/s | 126 kB 00:00 2024-04-25T15:38:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:Total 546 kB/s | 126 kB 00:00 2024-04-25T15:38:28.036 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:28.041 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:28.041 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:28.078 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-25T15:38:28.104 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:28.105 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:28.255 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:28.383 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:28.693 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2/2 2024-04-25T15:38:28.693 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/2 2024-04-25T15:38:28.790 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:28.965 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-25T15:38:28.965 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:28.965 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:28.965 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-25T15:38:28.965 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:28.966 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:29.032 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:29.032 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:29.032 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:29.032 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:29.032 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:29.032 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:29.220 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librados-devel 2024-04-25T15:38:29.266 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs2 2024-04-25T15:38:29.742 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:26 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:29.800 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:27 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:29.865 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:29.865 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:29.865 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-04-25T15:38:29.865 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:29.865 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:29.866 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 126 k 2024-04-25T15:38:29.866 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:29.866 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:29.866 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:29.866 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-04-25T15:38:29.866 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:29.866 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 126 k 2024-04-25T15:38:29.867 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 448 k 2024-04-25T15:38:29.867 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:29.881 INFO:teuthology.orchestra.run.smithi193.stdout:Package libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:29.922 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:29.923 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:38:29.923 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:30.006 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install libcephfs-devel 2024-04-25T15:38:30.187 INFO:teuthology.orchestra.run.smithi110.stdout:librados-devel-17.2.7-902.gc61472c8.el9.x86_64. 395 kB/s | 126 kB 00:00 2024-04-25T15:38:30.187 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:30.187 INFO:teuthology.orchestra.run.smithi110.stdout:Total 393 kB/s | 126 kB 00:00 2024-04-25T15:38:30.188 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:30.193 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:30.193 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:30.255 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:30.256 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:30.420 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:30.515 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:30.517 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:28 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:30.636 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:30.637 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 25 k 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 110 k 2024-04-25T15:38:30.638 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:30.808 INFO:teuthology.orchestra.run.smithi193.stdout:libcephfs-devel-17.2.7-902.gc61472c8.el9.x86_64 147 kB/s | 25 kB 00:00 2024-04-25T15:38:30.809 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:30.809 INFO:teuthology.orchestra.run.smithi193.stdout:Total 146 kB/s | 25 kB 00:00 2024-04-25T15:38:30.809 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:30.812 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:30.812 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:30.814 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:30.814 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:30.901 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:30.907 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:31.011 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:31.143 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:31.144 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:31.144 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:31.144 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:31.144 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:31.144 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:31.319 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:31.402 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install libcephfs2 2024-04-25T15:38:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:31.517 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:31.518 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:31.518 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:31.518 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:31.696 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librados2 2024-04-25T15:38:31.915 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:28 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:31.997 INFO:teuthology.orchestra.run.smithi110.stdout:Package libcephfs2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:32.037 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:32.038 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:38:32.039 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:32.142 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install libcephfs-devel 2024-04-25T15:38:32.205 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:30 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:32.284 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:32.325 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:32.327 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:38:32.327 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:32.398 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install librbd1 2024-04-25T15:38:32.660 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:29 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:32.779 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:32.780 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:32.780 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-04-25T15:38:32.780 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:32.780 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:32.780 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:17.2.7-902.gc61472c8.el9 ceph 25 k 2024-04-25T15:38:32.781 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:32.781 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:32.781 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:32.781 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-04-25T15:38:32.781 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:32.781 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 25 k 2024-04-25T15:38:32.781 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 110 k 2024-04-25T15:38:32.781 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:32.910 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:30 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:32.988 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:33.029 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:33.030 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:38:33.030 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:33.097 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rados 2024-04-25T15:38:33.106 INFO:teuthology.orchestra.run.smithi110.stdout:libcephfs-devel-17.2.7-902.gc61472c8.el9.x86_64 77 kB/s | 25 kB 00:00 2024-04-25T15:38:33.107 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:33.108 INFO:teuthology.orchestra.run.smithi110.stdout:Total 76 kB/s | 25 kB 00:00 2024-04-25T15:38:33.108 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:33.111 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:33.111 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:33.112 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:33.113 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:33.199 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:33.310 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:33.606 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:31 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:33.654 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:33.685 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:33.725 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:33.726 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:38:33.726 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:33.793 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rgw 2024-04-25T15:38:33.896 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:33.896 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:33.896 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:33.896 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:33.896 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:33.896 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:34.090 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librados2 2024-04-25T15:38:34.300 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:32 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:34.379 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:34.419 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:34.420 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:38:34.421 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:34.495 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-cephfs 2024-04-25T15:38:34.594 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:31 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:34.672 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:38:34.714 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:34.783 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install librbd1 2024-04-25T15:38:35.004 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:33 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:35.082 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:35.123 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:35.124 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:38:35.124 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:35.195 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install python3-rbd 2024-04-25T15:38:35.288 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:32 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:35.366 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:35.406 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:35.407 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:38:35.407 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:35.475 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rados 2024-04-25T15:38:35.701 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:33 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:35.779 INFO:teuthology.orchestra.run.smithi193.stdout:Package python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:35.820 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:35.821 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T15:38:35.821 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:35.886 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-fuse 2024-04-25T15:38:35.978 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:32 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:36.056 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rados-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:36.096 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:36.098 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:38:36.098 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:36.167 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rgw 2024-04-25T15:38:36.395 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:34 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:36.515 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:36.515 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:36.515 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 82 k 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 82 k 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 222 k 2024-04-25T15:38:36.516 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:36.670 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:33 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:36.737 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-fuse-17.2.7-902.gc61472c8.el9.x86_64.rpm 372 kB/s | 82 kB 00:00 2024-04-25T15:38:36.737 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:36.737 INFO:teuthology.orchestra.run.smithi193.stdout:Total 370 kB/s | 82 kB 00:00 2024-04-25T15:38:36.737 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:36.744 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:36.744 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:36.748 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rgw-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:36.789 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:36.790 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:38:36.790 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:36.806 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:36.807 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:36.858 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-cephfs 2024-04-25T15:38:36.949 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:37.040 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:37.361 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:34 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:37.401 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:37.440 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-cephfs-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:37.480 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:38:37.481 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:37.551 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install python3-rbd 2024-04-25T15:38:37.624 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:37.624 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:37.624 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:37.624 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:37.625 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:37.625 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:37.861 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-mirror 2024-04-25T15:38:38.056 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:35 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:38.134 INFO:teuthology.orchestra.run.smithi110.stdout:Package python3-rbd-2:17.2.7-902.gc61472c8.el9.x86_64 is already installed. 2024-04-25T15:38:38.177 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:38.178 INFO:teuthology.orchestra.run.smithi110.stdout:Nothing to do. 2024-04-25T15:38:38.178 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:38.250 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-fuse 2024-04-25T15:38:38.383 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:36 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:38.505 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repo Size 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-04-25T15:38:38.506 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:38.507 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 2.9 M 2024-04-25T15:38:38.507 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 12 M 2024-04-25T15:38:38.507 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:38.762 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:35 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:38.883 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:17.2.7-902.gc61472c8.el9 ceph 82 k 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:38.884 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 82 k 2024-04-25T15:38:38.885 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 222 k 2024-04-25T15:38:38.885 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:39.068 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-fuse-17.2.7-902.gc61472c8.el9.x86_64.rpm 448 kB/s | 82 kB 00:00 2024-04-25T15:38:39.068 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:39.069 INFO:teuthology.orchestra.run.smithi110.stdout:Total 444 kB/s | 82 kB 00:00 2024-04-25T15:38:39.069 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:39.075 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:39.075 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:39.137 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:39.137 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:39.279 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:39.379 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:39.380 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-mirror-17.2.7-902.gc61472c8.el9.x86_64.rpm 3.4 MB/s | 2.9 MB 00:00 2024-04-25T15:38:39.381 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:39.382 INFO:teuthology.orchestra.run.smithi193.stdout:Total 3.4 MB/s | 2.9 MB 00:00 2024-04-25T15:38:39.382 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:39.391 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:39.392 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:39.470 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:39.470 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:39.724 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:39.890 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:39.957 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:39.975 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:39.975 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:39.975 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:39.975 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:39.975 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:39.975 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:39.981 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:39.981 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:38:39.981 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T15:38:39.981 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T15:38:39.981 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T15:38:39.981 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:40.243 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-mirror 2024-04-25T15:38:40.755 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:37 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:40.875 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:40.876 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repo Size 2024-04-25T15:38:40.877 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:40.877 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:17.2.7-902.gc61472c8.el9 ceph 2.9 M 2024-04-25T15:38:40.877 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:40.877 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:40.878 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:40.878 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-04-25T15:38:40.878 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:40.878 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 2.9 M 2024-04-25T15:38:40.878 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 12 M 2024-04-25T15:38:40.879 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:41.020 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:41.021 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:41.021 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:41.021 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:41.265 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install rbd-nbd 2024-04-25T15:38:41.387 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-mirror-17.2.7-902.gc61472c8.el9.x86_64.rpm 5.8 MB/s | 2.9 MB 00:00 2024-04-25T15:38:41.388 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:41.388 INFO:teuthology.orchestra.run.smithi110.stdout:Total 5.7 MB/s | 2.9 MB 00:00 2024-04-25T15:38:41.388 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:41.398 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:41.398 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:41.477 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:41.477 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:41.789 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:01:39 ago on Thu 25 Apr 2024 03:37:02 PM UTC. 2024-04-25T15:38:41.891 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:41.913 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 165 k 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout:Install 1 Package 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:41.914 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 165 k 2024-04-25T15:38:41.915 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 477 k 2024-04-25T15:38:41.915 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T15:38:41.955 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:41.978 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:41.978 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T15:38:41.978 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T15:38:41.979 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T15:38:41.979 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T15:38:41.979 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:42.509 INFO:teuthology.orchestra.run.smithi193.stdout:rbd-nbd-17.2.7-902.gc61472c8.el9.x86_64.rpm 277 kB/s | 165 kB 00:00 2024-04-25T15:38:42.510 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:42.510 INFO:teuthology.orchestra.run.smithi193.stdout:Total 277 kB/s | 165 kB 00:00 2024-04-25T15:38:42.510 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T15:38:42.518 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T15:38:42.518 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T15:38:42.582 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T15:38:42.582 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T15:38:42.731 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T15:38:42.830 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:43.044 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:43.044 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:43.044 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:43.232 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:43.291 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install rbd-nbd 2024-04-25T15:38:43.470 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:43.470 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:43.470 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T15:38:43.470 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:43.470 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:38:43.471 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T15:38:43.705 DEBUG:teuthology.parallel:result is None 2024-04-25T15:38:43.817 INFO:teuthology.orchestra.run.smithi110.stdout:Last metadata expiration check: 0:01:40 ago on Thu 25 Apr 2024 03:37:03 PM UTC. 2024-04-25T15:38:43.940 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout: Package Architecture Version Repository Size 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:17.2.7-902.gc61472c8.el9 ceph 165 k 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:================================================================================ 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:Install 1 Package 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 165 k 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:Installed size: 477 k 2024-04-25T15:38:43.941 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-04-25T15:38:44.157 INFO:teuthology.orchestra.run.smithi110.stdout:rbd-nbd-17.2.7-902.gc61472c8.el9.x86_64.rpm 766 kB/s | 165 kB 00:00 2024-04-25T15:38:44.158 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:38:44.158 INFO:teuthology.orchestra.run.smithi110.stdout:Total 762 kB/s | 165 kB 00:00 2024-04-25T15:38:44.158 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-04-25T15:38:44.165 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-04-25T15:38:44.166 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-04-25T15:38:44.228 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-04-25T15:38:44.228 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-04-25T15:38:44.380 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-04-25T15:38:44.477 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:44.881 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 1/1 2024-04-25T15:38:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:45.135 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-04-25T15:38:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:17.2.7-902.gc61472c8.el9.x86_64 2024-04-25T15:38:45.135 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:38:45.135 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-04-25T15:38:45.361 DEBUG:teuthology.parallel:result is None 2024-04-25T15:38:45.361 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T15:38:45.361 INFO:teuthology.packaging:ref: None 2024-04-25T15:38:45.361 INFO:teuthology.packaging:tag: None 2024-04-25T15:38:45.361 INFO:teuthology.packaging:branch: quincy 2024-04-25T15:38:45.361 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:38:45.362 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-25T15:38:45.674 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T15:38:45.695 INFO:teuthology.orchestra.run.smithi110.stdout:17.2.7-902.gc61472c8.el9 2024-04-25T15:38:45.696 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-902.gc61472c8.el9 2024-04-25T15:38:45.696 INFO:teuthology.task.install:The correct ceph version 17.2.7-902.gc61472c8 is installed. 2024-04-25T15:38:45.698 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T15:38:45.698 INFO:teuthology.packaging:ref: None 2024-04-25T15:38:45.698 INFO:teuthology.packaging:tag: None 2024-04-25T15:38:45.698 INFO:teuthology.packaging:branch: quincy 2024-04-25T15:38:45.698 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:38:45.698 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-04-25T15:38:45.915 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T15:38:45.939 INFO:teuthology.orchestra.run.smithi193.stdout:17.2.7-902.gc61472c8.el9 2024-04-25T15:38:45.939 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-902.gc61472c8.el9 2024-04-25T15:38:45.939 INFO:teuthology.task.install:The correct ceph version 17.2.7-902.gc61472c8 is installed. 2024-04-25T15:38:45.941 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T15:38:45.941 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:38:45.941 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T15:38:45.966 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:38:45.966 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T15:38:46.007 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T15:38:46.007 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:38:46.007 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T15:38:46.031 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T15:38:46.095 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:38:46.096 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T15:38:46.122 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T15:38:46.188 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T15:38:46.188 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:38:46.188 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T15:38:46.212 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T15:38:46.275 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:38:46.276 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T15:38:46.303 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T15:38:46.369 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T15:38:46.369 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:38:46.369 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T15:38:46.392 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T15:38:46.454 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:38:46.454 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T15:38:46.479 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T15:38:46.545 INFO:teuthology.run_tasks:Running task print... 2024-04-25T15:38:46.554 INFO:teuthology.task.print:**** done install task... 2024-04-25T15:38:46.554 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T15:38:46.664 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T15:38:46.664 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-04-25T15:38:46.665 INFO:tasks.cephadm:Cluster fsid is e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:38:46.665 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T15:38:46.665 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-25T15:38:46.665 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi110': '172.21.15.110', 'mon.smithi193': '172.21.15.193'} 2024-04-25T15:38:46.665 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T15:38:46.665 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-04-25T15:38:46.689 DEBUG:teuthology.orchestra.run.smithi193:> sudo hostname $(hostname -s) 2024-04-25T15:38:46.714 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-04-25T15:38:46.715 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T15:38:47.064 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 25 15:38 /home/ubuntu/cephtest/cephadm 2024-04-25T15:38:47.065 DEBUG:teuthology.orchestra.run.smithi193:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T15:38:47.279 INFO:teuthology.orchestra.run.smithi193.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Apr 25 15:38 /home/ubuntu/cephtest/cephadm 2024-04-25T15:38:47.279 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T15:38:47.296 DEBUG:teuthology.orchestra.run.smithi193:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T15:38:47.323 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-04-25T15:38:47.324 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-25T15:38:47.339 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-04-25T15:38:48.911 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-25T15:38:49.088 INFO:teuthology.orchestra.run.smithi193.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-25T15:39:15.970 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:39:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-25T15:39:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-25T15:39:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-04-25T15:39:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-25T15:39:15.971 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-04-25T15:39:15.971 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:39:22.458 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T15:39:22.458 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-25T15:39:22.458 INFO:teuthology.orchestra.run.smithi193.stdout: "image_id": "0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43", 2024-04-25T15:39:22.458 INFO:teuthology.orchestra.run.smithi193.stdout: "repo_digests": [ 2024-04-25T15:39:22.458 INFO:teuthology.orchestra.run.smithi193.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17" 2024-04-25T15:39:22.458 INFO:teuthology.orchestra.run.smithi193.stdout: ] 2024-04-25T15:39:22.458 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T15:39:22.478 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-04-25T15:39:22.509 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph 2024-04-25T15:39:22.535 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-04-25T15:39:22.577 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /etc/ceph 2024-04-25T15:39:22.600 INFO:tasks.cephadm:Writing seed config... 2024-04-25T15:39:22.601 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-25T15:39:22.601 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-25T15:39:22.601 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T15:39:22.601 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T15:39:22.601 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T15:39:22.601 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T15:39:22.601 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T15:39:22.601 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T15:39:22.602 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T15:39:22.603 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T15:39:22.604 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T15:39:22.604 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T15:39:22.604 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T15:39:22.604 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T15:39:22.604 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T15:39:22.605 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:39:22.605 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T15:39:22.635 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e79cc6fc-0319-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T15:39:22.635 DEBUG:teuthology.orchestra.run.smithi110:mon.smithi110> sudo journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110.service 2024-04-25T15:39:22.678 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T15:39:22.678 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.110 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:39:22.883 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-04-25T15:39:22.883 INFO:teuthology.orchestra.run.smithi110.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'e79cc6fc-0319-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.110', '--skip-admin-label'] 2024-04-25T15:39:22.910 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T15:39:22.911 INFO:teuthology.orchestra.run.smithi110.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T15:39:22.911 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying podman|docker is present... 2024-04-25T15:39:22.940 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T15:39:22.940 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying lvm2 is present... 2024-04-25T15:39:22.940 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying time synchronization is in place... 2024-04-25T15:39:22.948 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T15:39:22.948 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T15:39:22.954 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T15:39:22.954 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-25T15:39:22.961 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout enabled 2024-04-25T15:39:22.967 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout active 2024-04-25T15:39:22.967 INFO:teuthology.orchestra.run.smithi110.stdout:Unit chronyd.service is enabled and running 2024-04-25T15:39:22.967 INFO:teuthology.orchestra.run.smithi110.stdout:Repeating the final host check... 2024-04-25T15:39:22.991 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T15:39:22.991 INFO:teuthology.orchestra.run.smithi110.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T15:39:22.991 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl is present 2024-04-25T15:39:22.991 INFO:teuthology.orchestra.run.smithi110.stdout:lvcreate is present 2024-04-25T15:39:22.997 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T15:39:22.998 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T15:39:23.004 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T15:39:23.004 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-25T15:39:23.010 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout enabled 2024-04-25T15:39:23.016 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout active 2024-04-25T15:39:23.017 INFO:teuthology.orchestra.run.smithi110.stdout:Unit chronyd.service is enabled and running 2024-04-25T15:39:23.017 INFO:teuthology.orchestra.run.smithi110.stdout:Host looks OK 2024-04-25T15:39:23.017 INFO:teuthology.orchestra.run.smithi110.stdout:Cluster fsid: e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:39:23.017 INFO:teuthology.orchestra.run.smithi110.stdout:Acquiring lock 140008610514256 on /run/cephadm/e79cc6fc-0319-11ef-bc93-c7b262605968.lock 2024-04-25T15:39:23.017 INFO:teuthology.orchestra.run.smithi110.stdout:Lock 140008610514256 acquired on /run/cephadm/e79cc6fc-0319-11ef-bc93-c7b262605968.lock 2024-04-25T15:39:23.017 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying IP 172.21.15.110 port 3300 ... 2024-04-25T15:39:23.017 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying IP 172.21.15.110 port 6789 ... 2024-04-25T15:39:23.018 INFO:teuthology.orchestra.run.smithi110.stdout:Base mon IP(s) is [172.21.15.110:3300, 172.21.15.110:6789], mon addrv is [v2:172.21.15.110:3300,v1:172.21.15.110:6789] 2024-04-25T15:39:23.021 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.110 metric 100 2024-04-25T15:39:23.021 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.110 metric 100 2024-04-25T15:39:23.024 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T15:39:23.024 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T15:39:23.024 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T15:39:23.026 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T15:39:23.026 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T15:39:23.026 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T15:39:23.026 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T15:39:23.026 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout inet6 fe80::e268:46e2:ccbd:e2c8/64 scope link noprefixroute 2024-04-25T15:39:23.027 INFO:teuthology.orchestra.run.smithi110.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T15:39:23.027 INFO:teuthology.orchestra.run.smithi110.stdout:Mon IP `172.21.15.110` is in CIDR network `172.21.0.0/20` 2024-04-25T15:39:23.027 INFO:teuthology.orchestra.run.smithi110.stdout:Mon IP `172.21.15.110` is in CIDR network `172.21.0.0/20` 2024-04-25T15:39:23.027 INFO:teuthology.orchestra.run.smithi110.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T15:39:23.027 INFO:teuthology.orchestra.run.smithi110.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T15:39:23.028 INFO:teuthology.orchestra.run.smithi110.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-25T15:39:23.503 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stdout 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-25T15:39:23.503 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-04-25T15:39:23.503 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T15:39:23.503 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying blob sha256:0e2af62137028b073c6ccdb8d4d7e0ccccc475ce5f6e225cb983e90813813045 2024-04-25T15:39:23.503 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T15:39:23.504 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Copying config sha256:0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 2024-04-25T15:39:23.504 INFO:teuthology.orchestra.run.smithi110.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T15:39:24.474 INFO:teuthology.orchestra.run.smithi110.stdout:ceph: stdout ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-25T15:39:24.474 INFO:teuthology.orchestra.run.smithi110.stdout:Ceph version: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable) 2024-04-25T15:39:24.474 INFO:teuthology.orchestra.run.smithi110.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T15:39:25.363 INFO:teuthology.orchestra.run.smithi110.stdout:stat: stdout 167 167 2024-04-25T15:39:25.364 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial keys... 2024-04-25T15:39:26.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQAteSpmKEbvKBAAv9ikyG+KK936dRKUck7zJg== 2024-04-25T15:39:27.328 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQAueSpm7ghKKxAA+/myp9RgnmGYYmGloIKF4g== 2024-04-25T15:39:28.262 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph-authtool: stdout AQAveSpmLr/wJxAAuFTta1jXse56xS4uIjVJdg== 2024-04-25T15:39:28.262 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial monmap... 2024-04-25T15:39:29.222 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T15:39:29.222 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-04-25T15:39:29.222 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:39:29.222 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T15:39:29.222 INFO:teuthology.orchestra.run.smithi110.stdout:monmaptool for smithi110 [v2:172.21.15.110:3300,v1:172.21.15.110:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T15:39:29.222 INFO:teuthology.orchestra.run.smithi110.stdout:setting min_mon_release = octopus 2024-04-25T15:39:29.223 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: set fsid to e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:39:29.223 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T15:39:29.223 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:39:29.223 INFO:teuthology.orchestra.run.smithi110.stdout:Creating mon... 2024-04-25T15:39:30.196 INFO:teuthology.orchestra.run.smithi110.stdout:create mon.smithi110 on 2024-04-25T15:39:30.399 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T15:39:30.533 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T15:39:30.675 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968.target. 2024-04-25T15:39:30.676 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968.target. 2024-04-25T15:39:30.841 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110 2024-04-25T15:39:30.841 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110.service: Unit ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110.service not loaded. 2024-04-25T15:39:30.999 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968.target.wants/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110.service → /etc/systemd/system/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@.service. 2024-04-25T15:39:31.522 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:31 smithi110 ceph-mon[25959]: mon.smithi110 is new leader, mons smithi110 in quorum (ranks 0) 2024-04-25T15:39:31.523 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:31 smithi110 bash[25939]: 5c5ae0670ef4ba54532b745d8c2099564ef496bd42408dfe621cedc19fa5fd00 2024-04-25T15:39:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T15:39:31.535 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-04-25T15:39:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T15:39:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-25T15:39:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-04-25T15:39:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T15:39:31.543 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mon to start... 2024-04-25T15:39:31.544 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mon... 2024-04-25T15:39:31.778 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:31 smithi110 systemd[1]: Started Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:39:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T15:39:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout id: e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:39:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T15:39:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout services: 2024-04-25T15:39:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi110 (age 0.573249s) 2024-04-25T15:39:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout data: 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.955+0000 7f069c397700 1 Processor -- start 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.955+0000 7f069c397700 1 -- start start 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.955+0000 7f069c397700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.955+0000 7f069c397700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06941031d0 con 0x7f0694102880 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.956+0000 7f069a133700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:32.647 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.956+0000 7f069a133700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58924/0 (socket says 172.21.15.110:58924) 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.956+0000 7f069a133700 1 -- 172.21.15.110:0/2399753885 learned_addr learned my addr 172.21.15.110:0/2399753885 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.956+0000 7f069a133700 1 -- 172.21.15.110:0/2399753885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0694103310 con 0x7f0694102880 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.956+0000 7f069a133700 1 --2- 172.21.15.110:0/2399753885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694102c90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06900098e0 tx=0x7f0690009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f5f0bb494aa193 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.956+0000 7f0699131700 1 -- 172.21.15.110:0/2399753885 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f069000b860 con 0x7f0694102880 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.956+0000 7f0699131700 1 -- 172.21.15.110:0/2399753885 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069000b9c0 con 0x7f0694102880 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.956+0000 7f0699131700 1 -- 172.21.15.110:0/2399753885 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f069000bc90 con 0x7f0694102880 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 -- 172.21.15.110:0/2399753885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 msgr2=0x7f0694102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 --2- 172.21.15.110:0/2399753885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694102c90 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f06900098e0 tx=0x7f0690009250 comp rx=0 tx=0).stop 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 -- 172.21.15.110:0/2399753885 shutdown_connections 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 --2- 172.21.15.110:0/2399753885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694102c90 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 -- 172.21.15.110:0/2399753885 >> 172.21.15.110:0/2399753885 conn(0x7f06940fe060 msgr2=0x7f0694100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 -- 172.21.15.110:0/2399753885 shutdown_connections 2024-04-25T15:39:32.648 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 -- 172.21.15.110:0/2399753885 wait complete. 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 Processor -- start 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 -- start start 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694110e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.957+0000 7f069c397700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06941113a0 con 0x7f0694102880 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f069a133700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694110e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f069a133700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694110e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58930/0 (socket says 172.21.15.110:58930) 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f069a133700 1 -- 172.21.15.110:0/428927137 learned_addr learned my addr 172.21.15.110:0/428927137 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f069a133700 1 -- 172.21.15.110:0/428927137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06900094f0 con 0x7f0694102880 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f069a133700 1 --2- 172.21.15.110:0/428927137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694110e60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0690003080 tx=0x7f0690016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f068affd700 1 -- 172.21.15.110:0/428927137 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f069001c5c0 con 0x7f0694102880 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f068affd700 1 -- 172.21.15.110:0/428927137 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f069001c720 con 0x7f0694102880 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06941115a0 con 0x7f0694102880 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f068affd700 1 -- 172.21.15.110:0/428927137 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f069001ca30 con 0x7f0694102880 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0694111fd0 con 0x7f0694102880 2024-04-25T15:39:32.649 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.958+0000 7f068affd700 1 -- 172.21.15.110:0/428927137 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f069001cc60 con 0x7f0694102880 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.959+0000 7f068affd700 1 -- 172.21.15.110:0/428927137 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f069001ad90 con 0x7f0694102880 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.959+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069404bed0 con 0x7f0694102880 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:31.961+0000 7f068affd700 1 -- 172.21.15.110:0/428927137 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f069001a450 con 0x7f0694102880 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.031+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f0694112ab0 con 0x7f0694102880 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.032+0000 7f068affd700 1 -- 172.21.15.110:0/428927137 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f0690025440 con 0x7f0694102880 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.033+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 msgr2=0x7f0694110e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.033+0000 7f069c397700 1 --2- 172.21.15.110:0/428927137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694110e60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0690003080 tx=0x7f0690016000 comp rx=0 tx=0).stop 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.033+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 shutdown_connections 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.033+0000 7f069c397700 1 --2- 172.21.15.110:0/428927137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0694102880 0x7f0694110e60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.033+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 >> 172.21.15.110:0/428927137 conn(0x7f06940fe060 msgr2=0x7f0694105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.033+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 shutdown_connections 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:32.033+0000 7f069c397700 1 -- 172.21.15.110:0/428927137 wait complete. 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:mon is available 2024-04-25T15:39:32.650 INFO:teuthology.orchestra.run.smithi110.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T15:39:32.755 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:32 smithi110 ceph-mon[25959]: mon.smithi110 is new leader, mons smithi110 in quorum (ranks 0) 2024-04-25T15:39:32.755 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:32 smithi110 ceph-mon[25959]: monmap e1: 1 mons at {smithi110=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-25T15:39:32.755 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:32 smithi110 ceph-mon[25959]: fsmap 2024-04-25T15:39:32.755 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:32 smithi110 ceph-mon[25959]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T15:39:32.755 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:32 smithi110 ceph-mon[25959]: mgrmap e1: no daemons active 2024-04-25T15:39:32.755 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:32 smithi110 ceph-mon[25959]: from='client.? 172.21.15.110:0/428927137' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T15:39:33.670 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:33.670 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T15:39:33.670 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout fsid = e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:39:33.670 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.110:3300,v1:172.21.15.110:6789] 2024-04-25T15:39:33.670 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T15:39:33.670 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T15:39:33.670 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.042+0000 7fd543bf3700 1 Processor -- start 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.042+0000 7fd543bf3700 1 -- start start 2024-04-25T15:39:33.671 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.042+0000 7fd543bf3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c1054e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.042+0000 7fd543bf3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c105a20 con 0x7fd53c076b70 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.042+0000 7fd54198f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c1054e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.042+0000 7fd54198f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c1054e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58938/0 (socket says 172.21.15.110:58938) 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.042+0000 7fd54198f700 1 -- 172.21.15.110:0/1149782156 learned_addr learned my addr 172.21.15.110:0/1149782156 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.043+0000 7fd54198f700 1 -- 172.21.15.110:0/1149782156 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53c105b60 con 0x7fd53c076b70 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.043+0000 7fd54198f700 1 --2- 172.21.15.110:0/1149782156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c1054e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd5380098e0 tx=0x7fd538009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9864c8f59ffa60f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.043+0000 7fd54098d700 1 -- 172.21.15.110:0/1149782156 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd53800b860 con 0x7fd53c076b70 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.043+0000 7fd54098d700 1 -- 172.21.15.110:0/1149782156 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53800b9c0 con 0x7fd53c076b70 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.043+0000 7fd54098d700 1 -- 172.21.15.110:0/1149782156 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd53800bc90 con 0x7fd53c076b70 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.043+0000 7fd543bf3700 1 -- 172.21.15.110:0/1149782156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 msgr2=0x7fd53c1054e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.043+0000 7fd543bf3700 1 --2- 172.21.15.110:0/1149782156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c1054e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd5380098e0 tx=0x7fd538009250 comp rx=0 tx=0).stop 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 -- 172.21.15.110:0/1149782156 shutdown_connections 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 --2- 172.21.15.110:0/1149782156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c1054e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 -- 172.21.15.110:0/1149782156 >> 172.21.15.110:0/1149782156 conn(0x7fd53c0fcef0 msgr2=0x7fd53c0ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:33.672 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 -- 172.21.15.110:0/1149782156 shutdown_connections 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 -- 172.21.15.110:0/1149782156 wait complete. 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 Processor -- start 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 -- start start 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c110ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd543bf3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd53c111410 con 0x7fd53c076b70 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd54198f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c110ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd54198f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c110ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58944/0 (socket says 172.21.15.110:58944) 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.044+0000 7fd54198f700 1 -- 172.21.15.110:0/37555576 learned_addr learned my addr 172.21.15.110:0/37555576 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd54198f700 1 -- 172.21.15.110:0/37555576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5380094f0 con 0x7fd53c076b70 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd54198f700 1 --2- 172.21.15.110:0/37555576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c110ed0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd538009290 tx=0x7fd538016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:33.673 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd5327fc700 1 -- 172.21.15.110:0/37555576 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd53801c5c0 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd5327fc700 1 -- 172.21.15.110:0/37555576 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53801c740 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd53c111610 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd5327fc700 1 -- 172.21.15.110:0/37555576 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd53801ca50 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd53c111ff0 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd5327fc700 1 -- 172.21.15.110:0/37555576 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd53801cbb0 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.045+0000 7fd5327fc700 1 -- 172.21.15.110:0/37555576 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd53801ac30 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.046+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd53c05f5c0 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.048+0000 7fd5327fc700 1 -- 172.21.15.110:0/37555576 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7fd5380243e0 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.114+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd53c10abb0 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.117+0000 7fd5327fc700 1 -- 172.21.15.110:0/37555576 <== mon.0 v2:172.21.15.110:3300/0 7 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fd53801ae90 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.117+0000 7fd5327fc700 1 -- 172.21.15.110:0/37555576 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fd5380364f0 con 0x7fd53c076b70 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.118+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 msgr2=0x7fd53c110ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.118+0000 7fd543bf3700 1 --2- 172.21.15.110:0/37555576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c110ed0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd538009290 tx=0x7fd538016000 comp rx=0 tx=0).stop 2024-04-25T15:39:33.674 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.118+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 shutdown_connections 2024-04-25T15:39:33.675 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.118+0000 7fd543bf3700 1 --2- 172.21.15.110:0/37555576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd53c076b70 0x7fd53c110ed0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:33.675 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.118+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 >> 172.21.15.110:0/37555576 conn(0x7fd53c0fcef0 msgr2=0x7fd53c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:33.675 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.118+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 shutdown_connections 2024-04-25T15:39:33.675 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:33.118+0000 7fd543bf3700 1 -- 172.21.15.110:0/37555576 wait complete. 2024-04-25T15:39:33.675 INFO:teuthology.orchestra.run.smithi110.stdout:Generating new minimal ceph.conf... 2024-04-25T15:39:33.787 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:33 smithi110 ceph-mon[25959]: from='client.? 172.21.15.110:0/37555576' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T15:39:33.787 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:33 smithi110 ceph-mon[25959]: from='client.? 172.21.15.110:0/37555576' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T15:39:34.703 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:34 smithi110 ceph-mon[25959]: from='client.? 172.21.15.110:0/3707364355' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:39:34.710 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6ce46c700 1 Processor -- start 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6ce46c700 1 -- start start 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6ce46c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8076140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6ce46c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8077ff0 con 0x7ff6c8077be0 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8076140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8076140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58948/0 (socket says 172.21.15.110:58948) 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6c7fff700 1 -- 172.21.15.110:0/90378120 learned_addr learned my addr 172.21.15.110:0/90378120 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6c7fff700 1 -- 172.21.15.110:0/90378120 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c8076680 con 0x7ff6c8077be0 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6c7fff700 1 --2- 172.21.15.110:0/90378120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8076140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0098e0 tx=0x7ff6bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397883a3c833b7a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6c6ffd700 1 -- 172.21.15.110:0/90378120 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6bc00b860 con 0x7ff6c8077be0 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6c6ffd700 1 -- 172.21.15.110:0/90378120 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff6bc00be50 con 0x7ff6c8077be0 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.102+0000 7ff6c6ffd700 1 -- 172.21.15.110:0/90378120 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6bc010380 con 0x7ff6c8077be0 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.103+0000 7ff6ce46c700 1 -- 172.21.15.110:0/90378120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 msgr2=0x7ff6c8076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:34.711 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.103+0000 7ff6ce46c700 1 --2- 172.21.15.110:0/90378120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8076140 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc0098e0 tx=0x7ff6bc009250 comp rx=0 tx=0).stop 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.103+0000 7ff6ce46c700 1 -- 172.21.15.110:0/90378120 shutdown_connections 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.103+0000 7ff6ce46c700 1 --2- 172.21.15.110:0/90378120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8076140 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.103+0000 7ff6ce46c700 1 -- 172.21.15.110:0/90378120 >> 172.21.15.110:0/90378120 conn(0x7ff6c80fdff0 msgr2=0x7ff6c8100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.103+0000 7ff6ce46c700 1 -- 172.21.15.110:0/90378120 shutdown_connections 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.103+0000 7ff6ce46c700 1 -- 172.21.15.110:0/90378120 wait complete. 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.103+0000 7ff6ce46c700 1 Processor -- start 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6ce46c700 1 -- start start 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6ce46c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8113860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6ce46c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c8113da0 con 0x7ff6c8077be0 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8113860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6c7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8113860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58950/0 (socket says 172.21.15.110:58950) 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6c7fff700 1 -- 172.21.15.110:0/3707364355 learned_addr learned my addr 172.21.15.110:0/3707364355 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6c7fff700 1 -- 172.21.15.110:0/3707364355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6bc0094f0 con 0x7ff6c8077be0 2024-04-25T15:39:34.712 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6c7fff700 1 --2- 172.21.15.110:0/3707364355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8113860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc006fd0 tx=0x7ff6bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6c4ff9700 1 -- 172.21.15.110:0/3707364355 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6bc0107c0 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6c4ff9700 1 -- 172.21.15.110:0/3707364355 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff6bc010db0 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c8113fa0 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6c4ff9700 1 -- 172.21.15.110:0/3707364355 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff6bc01adc0 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.104+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c8114b10 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.105+0000 7ff6c4ff9700 1 -- 172.21.15.110:0/3707364355 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff6bc019410 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.105+0000 7ff6c4ff9700 1 -- 172.21.15.110:0/3707364355 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff6bc024750 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.105+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c805f5c0 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.108+0000 7ff6c4ff9700 1 -- 172.21.15.110:0/3707364355 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7ff6bc029060 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.174+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff6c804bed0 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.174+0000 7ff6c4ff9700 1 -- 172.21.15.110:0/3707364355 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7ff6c804bed0 con 0x7ff6c8077be0 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.175+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 msgr2=0x7ff6c8113860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.175+0000 7ff6ce46c700 1 --2- 172.21.15.110:0/3707364355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8113860 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc006fd0 tx=0x7ff6bc009250 comp rx=0 tx=0).stop 2024-04-25T15:39:34.713 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.175+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 shutdown_connections 2024-04-25T15:39:34.714 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.175+0000 7ff6ce46c700 1 --2- 172.21.15.110:0/3707364355 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff6c8077be0 0x7ff6c8113860 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:34.714 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.175+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 >> 172.21.15.110:0/3707364355 conn(0x7ff6c80fdff0 msgr2=0x7ff6c80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:34.714 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.175+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 shutdown_connections 2024-04-25T15:39:34.714 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:34.175+0000 7ff6ce46c700 1 -- 172.21.15.110:0/3707364355 wait complete. 2024-04-25T15:39:34.714 INFO:teuthology.orchestra.run.smithi110.stdout:Restarting the monitor... 2024-04-25T15:39:34.991 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:34 smithi110 systemd[1]: Stopping Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:39:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:34 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[25954]: 2024-04-25T15:39:34.989+0000 7f6b307b8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi110 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:39:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:34 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[25954]: 2024-04-25T15:39:34.989+0000 7f6b307b8700 -1 mon.smithi110@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T15:39:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:35 smithi110 podman[26215]: 2024-04-25 15:39:35.153688099 +0000 UTC m=+0.282515005 container died 5c5ae0670ef4ba54532b745d8c2099564ef496bd42408dfe621cedc19fa5fd00 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:39:35.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:35 smithi110 podman[26215]: 2024-04-25 15:39:35.482527743 +0000 UTC m=+0.611354659 container cleanup 5c5ae0670ef4ba54532b745d8c2099564ef496bd42408dfe621cedc19fa5fd00 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T15:39:35.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:35 smithi110 bash[26215]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110 2024-04-25T15:39:35.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:35 smithi110 podman[26229]: 2024-04-25 15:39:35.658603981 +0000 UTC m=+0.499519921 container remove 5c5ae0670ef4ba54532b745d8c2099564ef496bd42408dfe621cedc19fa5fd00 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=quincy-c61472c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:39:35.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:35 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110.service: Deactivated successfully. 2024-04-25T15:39:35.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:35 smithi110 systemd[1]: Stopped Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:39:36.183 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:35 smithi110 systemd[1]: Starting Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:39:36.183 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 podman[26325]: 2024-04-25 15:39:36.109664769 +0000 UTC m=+0.098023195 container create e6fee4ab4b51e376c1e7197c8a7a3f024b4a10a24982a393a9636d06f66c58a5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T15:39:36.183 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 podman[26325]: 2024-04-25 15:39:36.029270025 +0000 UTC m=+0.017628451 image pull 0de407c1b3afb7a571e0cc4b0c826a995ed358d49d5f013ca6bdfad025b55c43 quay.ceph.io/ceph-ci/ceph:quincy 2024-04-25T15:39:36.349 INFO:teuthology.orchestra.run.smithi110.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T15:39:36.463 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 podman[26325]: 2024-04-25 15:39:36.270347178 +0000 UTC m=+0.258705604 container init e6fee4ab4b51e376c1e7197c8a7a3f024b4a10a24982a393a9636d06f66c58a5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T15:39:36.463 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 podman[26325]: 2024-04-25 15:39:36.27327075 +0000 UTC m=+0.261629176 container start e6fee4ab4b51e376c1e7197c8a7a3f024b4a10a24982a393a9636d06f66c58a5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T15:39:36.463 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T15:39:36.463 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable), process ceph-mon, pid 2 2024-04-25T15:39:36.463 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: pidfile_write: ignore empty --pid-file 2024-04-25T15:39:36.463 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: load: jerasure load: lrc 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: RocksDB version: 6.15.5 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Compile date Apr 24 2024 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: DB SUMMARY 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: DB Session ID: DSQH0FF8HQE589ZD9HYU 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: CURRENT file: CURRENT 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi110/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi110/store.db: 000010.log size: 98172 ; 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.error_if_exists: 0 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.create_if_missing: 0 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.paranoid_checks: 1 2024-04-25T15:39:36.464 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T15:39:36.465 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.env: 0x55e59fcacb00 2024-04-25T15:39:36.465 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.fs: Posix File System 2024-04-25T15:39:36.465 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.info_log: 0x55e5a0b25620 2024-04-25T15:39:36.465 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T15:39:36.465 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.statistics: (nil) 2024-04-25T15:39:36.465 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.use_fsync: 0 2024-04-25T15:39:36.465 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_log_file_size: 0 2024-04-25T15:39:36.466 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T15:39:36.466 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T15:39:36.466 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T15:39:36.466 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.allow_fallocate: 1 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.use_direct_reads: 0 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.db_log_dir: 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi110/store.db 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T15:39:36.467 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.write_buffer_manager: 0x55e5a18189c0 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.unordered_write: 0 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T15:39:36.468 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.row_cache: None 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.wal_filter: None 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.preserve_deletes: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.two_write_queues: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.atomic_flush: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.log_readahead_size: 0 2024-04-25T15:39:36.469 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_background_jobs: 2 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_background_compactions: -1 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_subcompactions: 1 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T15:39:36.470 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_open_files: -1 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_background_flushes: -1 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Compression algorithms supported: 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: kZSTD supported: 0 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: kXpressCompression supported: 0 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: kZlibCompression supported: 1 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: kSnappyCompression supported: 1 2024-04-25T15:39:36.471 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: kLZ4Compression supported: 1 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: kBZip2Compression supported: 0 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi110/store.db/MANIFEST-000009 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.merge_operator: 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_filter: None 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e5a0af8f58) 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-04-25T15:39:36.472 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_type: 0 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_block_index_type: 0 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_shortening: 1 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: hash_index_allow_collision: 1 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: checksum: 1 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: no_block_cache: 0 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache: 0x55e5a0b2b090 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_options: 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: capacity : 536870912 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: num_shard_bits : 4 2024-04-25T15:39:36.473 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: strict_capacity_limit : 0 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_compressed: (nil) 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: persistent_cache: (nil) 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_size: 4096 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_size_deviation: 10 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_restart_interval: 16 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_block_restart_interval: 1 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: metadata_block_size: 4096 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: partition_filters: 0 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: use_delta_encoding: 1 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: whole_key_filtering: 1 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: verify_compression: 0 2024-04-25T15:39:36.474 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: format_version: 4 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: enable_index_compression: 1 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_align: 0 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compression: NoCompression 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.num_levels: 7 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T15:39:36.475 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T15:39:36.476 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T15:39:36.477 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.arena_block_size: 4194304 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T15:39:36.478 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.table_properties_collectors: 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.inplace_update_support: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.bloom_locality: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.max_successive_merges: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.ttl: 2592000 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.enable_blob_files: false 2024-04-25T15:39:36.479 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.min_blob_size: 0 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi110/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714059576300180, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714059576301032, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 93378, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 91490, "index_size": 248, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84054, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1714059576, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "716ef407-baec-4d5b-ab88-8f1408e30c08", "db_session_id": "DSQH0FF8HQE589ZD9HYU"}} 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714059576301360, "job": 1, "event": "recovery_finished"} 2024-04-25T15:39:36.480 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-smithi110/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55e5a0abf180 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: DB pointer 0x55e5a0b94000 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** DB Stats ** 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-04-25T15:39:36.481 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: L0 2/0 92.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.9 0.00 0.00 1 0.001 0 0 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Sum 2/0 92.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.9 0.00 0.00 1 0.001 0 0 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.9 0.00 0.00 1 0.001 0 0 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.9 0.00 0.00 1 0.001 0 0 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T15:39:36.482 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative compaction: 0.00 GB write, 29.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval compaction: 0.00 GB write, 29.69 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: L0 2/0 92.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.9 0.00 0.00 1 0.001 0 0 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Sum 2/0 92.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 147.9 0.00 0.00 1 0.001 0 0 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:39:36.483 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 147.9 0.00 0.00 1 0.001 0 0 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative compaction: 0.00 GB write, 29.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: starting mon.smithi110 rank 0 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi110 fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:39:36.484 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110@-1(???) e1 preinit fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110@-1(???).mds e1 new map 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110@-1(???).mds e1 print_map 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: e1 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: legacy client fscid: -1 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: No filesystems configured 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 bash[26325]: e6fee4ab4b51e376c1e7197c8a7a3f024b4a10a24982a393a9636d06f66c58a5 2024-04-25T15:39:36.485 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 systemd[1]: Started Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:39:36.486 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mon.smithi110 is new leader, mons smithi110 in quorum (ranks 0) 2024-04-25T15:39:36.486 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: monmap e1: 1 mons at {smithi110=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-25T15:39:36.486 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: fsmap 2024-04-25T15:39:36.486 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T15:39:36.486 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:36 smithi110 ceph-mon[26344]: mgrmap e1: no daemons active 2024-04-25T15:39:37.422 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.824+0000 7f2d9bb18700 1 Processor -- start 2024-04-25T15:39:37.423 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.824+0000 7f2d9bb18700 1 -- start start 2024-04-25T15:39:37.423 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.824+0000 7f2d9bb18700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d94105410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:37.423 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.824+0000 7f2d9bb18700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d94105950 con 0x7f2d94071310 2024-04-25T15:39:37.423 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.825+0000 7f2d998b4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d94105410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:37.423 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.825+0000 7f2d998b4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d94105410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58958/0 (socket says 172.21.15.110:58958) 2024-04-25T15:39:37.423 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.825+0000 7f2d998b4700 1 -- 172.21.15.110:0/2431886956 learned_addr learned my addr 172.21.15.110:0/2431886956 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:37.423 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.825+0000 7f2d998b4700 1 -- 172.21.15.110:0/2431886956 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d94105a90 con 0x7f2d94071310 2024-04-25T15:39:37.424 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.825+0000 7f2d998b4700 1 --2- 172.21.15.110:0/2431886956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d94105410 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2d880098e0 tx=0x7f2d88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa9ee09a9ef507d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:37.424 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d988b2700 1 -- 172.21.15.110:0/2431886956 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d8800b860 con 0x7f2d94071310 2024-04-25T15:39:37.424 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d988b2700 1 -- 172.21.15.110:0/2431886956 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2d8800be50 con 0x7f2d94071310 2024-04-25T15:39:37.424 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d988b2700 1 -- 172.21.15.110:0/2431886956 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d88010380 con 0x7f2d94071310 2024-04-25T15:39:37.424 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d9bb18700 1 -- 172.21.15.110:0/2431886956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 msgr2=0x7f2d94105410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:37.424 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d9bb18700 1 --2- 172.21.15.110:0/2431886956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d94105410 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f2d880098e0 tx=0x7f2d88009250 comp rx=0 tx=0).stop 2024-04-25T15:39:37.424 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d9bb18700 1 -- 172.21.15.110:0/2431886956 shutdown_connections 2024-04-25T15:39:37.424 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d9bb18700 1 --2- 172.21.15.110:0/2431886956 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d94105410 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d9bb18700 1 -- 172.21.15.110:0/2431886956 >> 172.21.15.110:0/2431886956 conn(0x7f2d940fcec0 msgr2=0x7f2d940ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d9bb18700 1 -- 172.21.15.110:0/2431886956 shutdown_connections 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.826+0000 7f2d9bb18700 1 -- 172.21.15.110:0/2431886956 wait complete. 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.827+0000 7f2d9bb18700 1 Processor -- start 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.827+0000 7f2d9bb18700 1 -- start start 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.827+0000 7f2d9bb18700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d941115e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.827+0000 7f2d9bb18700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d94111b20 con 0x7f2d94071310 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.827+0000 7f2d998b4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d941115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.827+0000 7f2d998b4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d941115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58966/0 (socket says 172.21.15.110:58966) 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.827+0000 7f2d998b4700 1 -- 172.21.15.110:0/3047425224 learned_addr learned my addr 172.21.15.110:0/3047425224 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.827+0000 7f2d998b4700 1 -- 172.21.15.110:0/3047425224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d880094f0 con 0x7f2d94071310 2024-04-25T15:39:37.425 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.828+0000 7f2d998b4700 1 --2- 172.21.15.110:0/3047425224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d941115e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2d88003050 tx=0x7f2d88009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:37.426 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.828+0000 7f2d867fc700 1 -- 172.21.15.110:0/3047425224 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d88010760 con 0x7f2d94071310 2024-04-25T15:39:37.426 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.828+0000 7f2d867fc700 1 -- 172.21.15.110:0/3047425224 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f2d88010d50 con 0x7f2d94071310 2024-04-25T15:39:37.426 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.828+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d94111d20 con 0x7f2d94071310 2024-04-25T15:39:37.426 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.828+0000 7f2d867fc700 1 -- 172.21.15.110:0/3047425224 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2d8801ae20 con 0x7f2d94071310 2024-04-25T15:39:37.426 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.828+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d94112890 con 0x7f2d94071310 2024-04-25T15:39:37.426 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.828+0000 7f2d867fc700 1 -- 172.21.15.110:0/3047425224 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f2d8802e440 con 0x7f2d94071310 2024-04-25T15:39:37.426 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.828+0000 7f2d867fc700 1 -- 172.21.15.110:0/3047425224 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2d88024740 con 0x7f2d94071310 2024-04-25T15:39:37.427 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.829+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9405f5c0 con 0x7f2d94071310 2024-04-25T15:39:37.427 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.831+0000 7f2d867fc700 1 -- 172.21.15.110:0/3047425224 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7f2d88029070 con 0x7f2d94071310 2024-04-25T15:39:37.427 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.896+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f2d94106840 con 0x7f2d94071310 2024-04-25T15:39:37.427 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.898+0000 7f2d867fc700 1 -- 172.21.15.110:0/3047425224 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f2d8801a6a0 con 0x7f2d94071310 2024-04-25T15:39:37.427 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.899+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 msgr2=0x7f2d941115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:37.427 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.899+0000 7f2d9bb18700 1 --2- 172.21.15.110:0/3047425224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d941115e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2d88003050 tx=0x7f2d88009250 comp rx=0 tx=0).stop 2024-04-25T15:39:37.427 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.899+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 shutdown_connections 2024-04-25T15:39:37.427 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.899+0000 7f2d9bb18700 1 --2- 172.21.15.110:0/3047425224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d94071310 0x7f2d941115e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:37.428 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.899+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 >> 172.21.15.110:0/3047425224 conn(0x7f2d940fcec0 msgr2=0x7f2d940fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:37.428 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.899+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 shutdown_connections 2024-04-25T15:39:37.428 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:36.899+0000 7f2d9bb18700 1 -- 172.21.15.110:0/3047425224 wait complete. 2024-04-25T15:39:37.428 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T15:39:37.428 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:39:37.428 INFO:teuthology.orchestra.run.smithi110.stdout:Creating mgr... 2024-04-25T15:39:37.428 INFO:teuthology.orchestra.run.smithi110.stdout:Verifying port 9283 ... 2024-04-25T15:39:37.596 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mgr.smithi110.lvgbkf 2024-04-25T15:39:37.596 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mgr.smithi110.lvgbkf.service: Unit ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mgr.smithi110.lvgbkf.service not loaded. 2024-04-25T15:39:37.739 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968.target.wants/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mgr.smithi110.lvgbkf.service → /etc/systemd/system/ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@.service. 2024-04-25T15:39:38.201 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:37 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3047425224' entity='client.admin' 2024-04-25T15:39:38.317 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T15:39:38.317 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-04-25T15:39:38.325 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T15:39:38.326 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-25T15:39:38.326 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-04-25T15:39:38.326 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T15:39:38.337 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T15:39:38.337 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-04-25T15:39:38.344 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T15:39:38.344 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-25T15:39:38.344 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-04-25T15:39:38.344 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-04-25T15:39:38.344 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr to start... 2024-04-25T15:39:38.344 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr... 2024-04-25T15:39:39.021 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:38 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2194961937' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T15:39:39.354 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:39.354 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-25T15:39:39.355 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "e79cc6fc-0319-11ef-bc93-c7b262605968", 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-04-25T15:39:39.357 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "smithi110" 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T15:39:39.358 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T15:39:39.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T15:39:39.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T15:39:39.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T15:39:39.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:39.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:39.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T15:39:31.461118+0000", 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T15:39:39.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.743+0000 7ff9223a9700 1 Processor -- start 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.743+0000 7ff9223a9700 1 -- start start 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.743+0000 7ff9223a9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c102c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.743+0000 7ff9223a9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c1031c0 con 0x7ff91c102870 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.743+0000 7ff91bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.743+0000 7ff91bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58994/0 (socket says 172.21.15.110:58994) 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.743+0000 7ff91bfff700 1 -- 172.21.15.110:0/3575000399 learned_addr learned my addr 172.21.15.110:0/3575000399 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.743+0000 7ff91bfff700 1 -- 172.21.15.110:0/3575000399 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c103300 con 0x7ff91c102870 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff91bfff700 1 --2- 172.21.15.110:0/3575000399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c102c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0098e0 tx=0x7ff90c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6302a61bf9c605e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff91affd700 1 -- 172.21.15.110:0/3575000399 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff90c00b860 con 0x7ff91c102870 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff91affd700 1 -- 172.21.15.110:0/3575000399 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff90c00be50 con 0x7ff91c102870 2024-04-25T15:39:39.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff91affd700 1 -- 172.21.15.110:0/3575000399 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff90c010380 con 0x7ff91c102870 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff9223a9700 1 -- 172.21.15.110:0/3575000399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 msgr2=0x7ff91c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff9223a9700 1 --2- 172.21.15.110:0/3575000399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c102c80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff90c0098e0 tx=0x7ff90c009250 comp rx=0 tx=0).stop 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff9223a9700 1 -- 172.21.15.110:0/3575000399 shutdown_connections 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff9223a9700 1 --2- 172.21.15.110:0/3575000399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c102c80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.744+0000 7ff9223a9700 1 -- 172.21.15.110:0/3575000399 >> 172.21.15.110:0/3575000399 conn(0x7ff91c0fe030 msgr2=0x7ff91c100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.745+0000 7ff9223a9700 1 -- 172.21.15.110:0/3575000399 shutdown_connections 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.745+0000 7ff9223a9700 1 -- 172.21.15.110:0/3575000399 wait complete. 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.745+0000 7ff9223a9700 1 Processor -- start 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.745+0000 7ff9223a9700 1 -- start start 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.745+0000 7ff9223a9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.745+0000 7ff9223a9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c111ad0 con 0x7ff91c102870 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff91bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff91bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59006/0 (socket says 172.21.15.110:59006) 2024-04-25T15:39:39.370 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff91bfff700 1 -- 172.21.15.110:0/2194961937 learned_addr learned my addr 172.21.15.110:0/2194961937 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff91bfff700 1 -- 172.21.15.110:0/2194961937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0094f0 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff91bfff700 1 --2- 172.21.15.110:0/2194961937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c111590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00b440 tx=0x7ff90c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff918ff9700 1 -- 172.21.15.110:0/2194961937 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff90c010760 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff918ff9700 1 -- 172.21.15.110:0/2194961937 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7ff90c010d50 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff918ff9700 1 -- 172.21.15.110:0/2194961937 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff90c01ae20 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c111cd0 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c112840 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff918ff9700 1 -- 172.21.15.110:0/2194961937 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff90c02e440 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.746+0000 7ff918ff9700 1 -- 172.21.15.110:0/2194961937 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff90c024740 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.747+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c05f5c0 con 0x7ff91c102870 2024-04-25T15:39:39.371 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.749+0000 7ff918ff9700 1 -- 172.21.15.110:0/2194961937 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7ff90c029070 con 0x7ff91c102870 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.818+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff91c04bed0 con 0x7ff91c102870 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.819+0000 7ff918ff9700 1 -- 172.21.15.110:0/2194961937 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7ff90c01a6a0 con 0x7ff91c102870 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.820+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 msgr2=0x7ff91c111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.820+0000 7ff9223a9700 1 --2- 172.21.15.110:0/2194961937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c111590 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff90c00b440 tx=0x7ff90c009250 comp rx=0 tx=0).stop 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.820+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 shutdown_connections 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.820+0000 7ff9223a9700 1 --2- 172.21.15.110:0/2194961937 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff91c102870 0x7ff91c111590 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.820+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 >> 172.21.15.110:0/2194961937 conn(0x7ff91c0fe030 msgr2=0x7ff91c0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.820+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 shutdown_connections 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:38.820+0000 7ff9223a9700 1 -- 172.21.15.110:0/2194961937 wait complete. 2024-04-25T15:39:39.372 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (1/15)... 2024-04-25T15:39:44.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: Activating manager daemon smithi110.lvgbkf 2024-04-25T15:39:44.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: mgrmap e2: smithi110.lvgbkf(active, starting, since 0.00328213s) 2024-04-25T15:39:44.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:39:44.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: Manager daemon smithi110.lvgbkf is now available 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:39:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:44 smithi110 ceph-mon[26344]: from='mgr.14100 172.21.15.110:0/4229797653' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:39:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:45 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2638567524' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T15:39:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:45 smithi110 ceph-mon[26344]: mgrmap e3: smithi110.lvgbkf(active, since 1.00677s) 2024-04-25T15:39:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:47 smithi110 ceph-mon[26344]: mgrmap e4: smithi110.lvgbkf(active, since 2s) 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "e79cc6fc-0319-11ef-bc93-c7b262605968", 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-04-25T15:39:50.300 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "smithi110" 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T15:39:50.301 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T15:39:50.309 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T15:39:50.309 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T15:39:50.309 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T15:39:50.309 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T15:39:50.310 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T15:39:31.461118+0000", 2024-04-25T15:39:50.311 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.269+0000 7efc5524b700 1 Processor -- start 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.269+0000 7efc5524b700 1 -- start start 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.269+0000 7efc5524b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc50101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.269+0000 7efc5524b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50102080 con 0x7efc50101730 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.270+0000 7efc4ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc50101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.270+0000 7efc4ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc50101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49086/0 (socket says 172.21.15.110:49086) 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.270+0000 7efc4ed9d700 1 -- 172.21.15.110:0/3012279528 learned_addr learned my addr 172.21.15.110:0/3012279528 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.270+0000 7efc4ed9d700 1 -- 172.21.15.110:0/3012279528 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc501021c0 con 0x7efc50101730 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.270+0000 7efc4ed9d700 1 --2- 172.21.15.110:0/3012279528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc50101b40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc4000bb80 tx=0x7efc4000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2652ec9b830ca20f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.270+0000 7efc4dd9b700 1 -- 172.21.15.110:0/3012279528 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc4000dd00 con 0x7efc50101730 2024-04-25T15:39:50.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.270+0000 7efc4dd9b700 1 -- 172.21.15.110:0/3012279528 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efc4000de60 con 0x7efc50101730 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.270+0000 7efc4dd9b700 1 -- 172.21.15.110:0/3012279528 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc400188a0 con 0x7efc50101730 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.271+0000 7efc5524b700 1 -- 172.21.15.110:0/3012279528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 msgr2=0x7efc50101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.271+0000 7efc5524b700 1 --2- 172.21.15.110:0/3012279528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc50101b40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7efc4000bb80 tx=0x7efc4000b4f0 comp rx=0 tx=0).stop 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.271+0000 7efc5524b700 1 -- 172.21.15.110:0/3012279528 shutdown_connections 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.271+0000 7efc5524b700 1 --2- 172.21.15.110:0/3012279528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc50101b40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.271+0000 7efc5524b700 1 -- 172.21.15.110:0/3012279528 >> 172.21.15.110:0/3012279528 conn(0x7efc500fcef0 msgr2=0x7efc500ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.271+0000 7efc5524b700 1 -- 172.21.15.110:0/3012279528 shutdown_connections 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.271+0000 7efc5524b700 1 -- 172.21.15.110:0/3012279528 wait complete. 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc5524b700 1 Processor -- start 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc5524b700 1 -- start start 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc5524b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc501137a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc5524b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc50113ce0 con 0x7efc50101730 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc4ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc501137a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:50.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc4ed9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc501137a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49088/0 (socket says 172.21.15.110:49088) 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc4ed9d700 1 -- 172.21.15.110:0/2638567524 learned_addr learned my addr 172.21.15.110:0/2638567524 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc4ed9d700 1 -- 172.21.15.110:0/2638567524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4000b790 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc4ed9d700 1 --2- 172.21.15.110:0/2638567524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc501137a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efc40010fa0 tx=0x7efc4000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc377fe700 1 -- 172.21.15.110:0/2638567524 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc40018e10 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc50113ee0 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc377fe700 1 -- 172.21.15.110:0/2638567524 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7efc40010b80 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc377fe700 1 -- 172.21.15.110:0/2638567524 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efc40021550 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.272+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc50114a50 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.273+0000 7efc377fe700 1 -- 172.21.15.110:0/2638567524 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 2) v1 ==== 43132+0+0 (secure 0 0 0) 0x7efc400217d0 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.273+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5005f5c0 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.273+0000 7efc377fe700 1 -- 172.21.15.110:0/2638567524 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7efc40014070 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.275+0000 7efc377fe700 1 -- 172.21.15.110:0/2638567524 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71547 (secure 0 0 0) 0x7efc4000ed90 con 0x7efc50101730 2024-04-25T15:39:50.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.343+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efc5004bed0 con 0x7efc50101730 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.344+0000 7efc377fe700 1 -- 172.21.15.110:0/2638567524 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7efc4000e360 con 0x7efc50101730 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.345+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 msgr2=0x7efc501137a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.345+0000 7efc5524b700 1 --2- 172.21.15.110:0/2638567524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc501137a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efc40010fa0 tx=0x7efc4000b4f0 comp rx=0 tx=0).stop 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.345+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 shutdown_connections 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.345+0000 7efc5524b700 1 --2- 172.21.15.110:0/2638567524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc50101730 0x7efc501137a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.345+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 >> 172.21.15.110:0/2638567524 conn(0x7efc500fcef0 msgr2=0x7efc50105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.345+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 shutdown_connections 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:45.345+0000 7efc5524b700 1 -- 172.21.15.110:0/2638567524 wait complete. 2024-04-25T15:39:50.315 INFO:teuthology.orchestra.run.smithi110.stdout:mgr not available, waiting (2/15)... 2024-04-25T15:39:53.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:53 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2710877951' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsid": "e79cc6fc-0319-11ef-bc93-c7b262605968", 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:53.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 0 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "smithi110" 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "quorum_age": 16, 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:53.565 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T15:39:53.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T15:39:53.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T15:39:53.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T15:39:53.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T15:39:53.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T15:39:53.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:53.566 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T15:39:53.574 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ], 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T15:39:53.575 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T15:39:31.461118+0000", 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout }, 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.697+0000 7f1fdcea1700 1 Processor -- start 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.697+0000 7f1fdcea1700 1 -- start start 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.698+0000 7f1fdcea1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.698+0000 7f1fdcea1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8103200 con 0x7f1fd81028b0 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.698+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.698+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51448/0 (socket says 172.21.15.110:51448) 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.698+0000 7f1fd659c700 1 -- 172.21.15.110:0/3873120139 learned_addr learned my addr 172.21.15.110:0/3873120139 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.698+0000 7f1fd659c700 1 -- 172.21.15.110:0/3873120139 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8103340 con 0x7f1fd81028b0 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.698+0000 7f1fd659c700 1 --2- 172.21.15.110:0/3873120139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8102cc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009e50 tx=0x7f1fd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a511bc29ca72c04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:53.576 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.698+0000 7f1fd559a700 1 -- 172.21.15.110:0/3873120139 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fd00092e0 con 0x7f1fd81028b0 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.699+0000 7f1fd559a700 1 -- 172.21.15.110:0/3873120139 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1fd0012430 con 0x7f1fd81028b0 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.699+0000 7f1fd559a700 1 -- 172.21.15.110:0/3873120139 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fd00075c0 con 0x7f1fd81028b0 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.699+0000 7f1fdcea1700 1 -- 172.21.15.110:0/3873120139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 msgr2=0x7f1fd8102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.699+0000 7f1fdcea1700 1 --2- 172.21.15.110:0/3873120139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8102cc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009e50 tx=0x7f1fd0009840 comp rx=0 tx=0).stop 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.699+0000 7f1fdcea1700 1 -- 172.21.15.110:0/3873120139 shutdown_connections 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.699+0000 7f1fdcea1700 1 --2- 172.21.15.110:0/3873120139 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8102cc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.699+0000 7f1fdcea1700 1 -- 172.21.15.110:0/3873120139 >> 172.21.15.110:0/3873120139 conn(0x7f1fd80fdff0 msgr2=0x7f1fd8100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.699+0000 7f1fdcea1700 1 -- 172.21.15.110:0/3873120139 shutdown_connections 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.700+0000 7f1fdcea1700 1 -- 172.21.15.110:0/3873120139 wait complete. 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.700+0000 7f1fdcea1700 1 Processor -- start 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.700+0000 7f1fdcea1700 1 -- start start 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.700+0000 7f1fdcea1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.700+0000 7f1fdcea1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8078050 con 0x7f1fd81028b0 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.700+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:53.577 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.700+0000 7f1fd659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51458/0 (socket says 172.21.15.110:51458) 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.700+0000 7f1fd659c700 1 -- 172.21.15.110:0/2710877951 learned_addr learned my addr 172.21.15.110:0/2710877951 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.701+0000 7f1fd659c700 1 -- 172.21.15.110:0/2710877951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd0009ae0 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.701+0000 7f1fd659c700 1 --2- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8077b10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009ab0 tx=0x7f1fd0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.701+0000 7f1fc6ffd700 1 -- 172.21.15.110:0/2710877951 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fd00092e0 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.701+0000 7f1fc6ffd700 1 -- 172.21.15.110:0/2710877951 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f1fd0007d10 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.701+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd8076120 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.701+0000 7f1fc6ffd700 1 -- 172.21.15.110:0/2710877951 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f1fd001c990 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.701+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd8076610 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.702+0000 7f1fc6ffd700 1 -- 172.21.15.110:0/2710877951 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f1fd0019070 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.702+0000 7f1fc6ffd700 1 --2- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f1fc0036800 0x7f1fc0038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.702+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd804bed0 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.702+0000 7f1fc6ffd700 1 -- 172.21.15.110:0/2710877951 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1fd0049e80 con 0x7f1fd81028b0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.703+0000 7f1fd5d9b700 1 --2- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f1fc0036800 0x7f1fc0038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:53.578 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.703+0000 7f1fd5d9b700 1 --2- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f1fc0036800 0x7f1fc0038cb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009990 tx=0x7f1fcc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.708+0000 7f1fc6ffd700 1 -- 172.21.15.110:0/2710877951 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1fd0017080 con 0x7f1fd81028b0 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.977+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1fd8076ca0 con 0x7f1fd81028b0 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.978+0000 7f1fc6ffd700 1 -- 172.21.15.110:0/2710877951 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1248 (secure 0 0 0) 0x7f1fd0020020 con 0x7f1fd81028b0 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f1fc0036800 msgr2=0x7f1fc0038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 --2- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f1fc0036800 0x7f1fc0038cb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc009990 tx=0x7f1fcc006e40 comp rx=0 tx=0).stop 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 msgr2=0x7f1fd8077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 --2- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8077b10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009ab0 tx=0x7f1fd0009840 comp rx=0 tx=0).stop 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 shutdown_connections 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 --2- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f1fc0036800 0x7f1fc0038cb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 --2- 172.21.15.110:0/2710877951 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fd81028b0 0x7f1fd8077b10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 >> 172.21.15.110:0/2710877951 conn(0x7f1fd80fdff0 msgr2=0x7f1fd80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 shutdown_connections 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:52.980+0000 7f1fdcea1700 1 -- 172.21.15.110:0/2710877951 wait complete. 2024-04-25T15:39:53.579 INFO:teuthology.orchestra.run.smithi110.stdout:mgr is available 2024-04-25T15:39:54.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:54 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/228842016' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout fsid = e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T15:39:55.417 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.972+0000 7f7bf8eb6700 1 Processor -- start 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.972+0000 7f7bf8eb6700 1 -- start start 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.973+0000 7f7bf8eb6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.973+0000 7f7bf8eb6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4079090 con 0x7f7bf4078c80 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.973+0000 7f7bf259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.973+0000 7f7bf259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51470/0 (socket says 172.21.15.110:51470) 2024-04-25T15:39:55.418 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.973+0000 7f7bf259c700 1 -- 172.21.15.110:0/2108529752 learned_addr learned my addr 172.21.15.110:0/2108529752 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.973+0000 7f7bf259c700 1 -- 172.21.15.110:0/2108529752 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf4077770 con 0x7f7bf4078c80 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.973+0000 7f7bf259c700 1 --2- 172.21.15.110:0/2108529752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4077230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7be40098e0 tx=0x7f7be4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e30710ca3f1da3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.973+0000 7f7bf159a700 1 -- 172.21.15.110:0/2108529752 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7be400b860 con 0x7f7bf4078c80 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf159a700 1 -- 172.21.15.110:0/2108529752 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7be400be50 con 0x7f7bf4078c80 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf159a700 1 -- 172.21.15.110:0/2108529752 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7be4010380 con 0x7f7bf4078c80 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/2108529752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 msgr2=0x7f7bf4077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf8eb6700 1 --2- 172.21.15.110:0/2108529752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4077230 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7be40098e0 tx=0x7f7be4009250 comp rx=0 tx=0).stop 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/2108529752 shutdown_connections 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf8eb6700 1 --2- 172.21.15.110:0/2108529752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4077230 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/2108529752 >> 172.21.15.110:0/2108529752 conn(0x7f7bf40fe060 msgr2=0x7f7bf4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/2108529752 shutdown_connections 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.974+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/2108529752 wait complete. 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf8eb6700 1 Processor -- start 2024-04-25T15:39:55.419 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf8eb6700 1 -- start start 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf8eb6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4113810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf8eb6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf4113d50 con 0x7f7bf4078c80 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51486/0 (socket says 172.21.15.110:51486) 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf259c700 1 -- 172.21.15.110:0/228842016 learned_addr learned my addr 172.21.15.110:0/228842016 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf259c700 1 -- 172.21.15.110:0/228842016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be40094f0 con 0x7f7bf4078c80 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.975+0000 7f7bf259c700 1 --2- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4113810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7be4003050 tx=0x7f7be4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.976+0000 7f7be2ffd700 1 -- 172.21.15.110:0/228842016 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7be4010760 con 0x7f7bf4078c80 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.976+0000 7f7be2ffd700 1 -- 172.21.15.110:0/228842016 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f7be4010d50 con 0x7f7bf4078c80 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.976+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf4113f50 con 0x7f7bf4078c80 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.976+0000 7f7be2ffd700 1 -- 172.21.15.110:0/228842016 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7be401ae20 con 0x7f7bf4078c80 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.976+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf4114ac0 con 0x7f7bf4078c80 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.976+0000 7f7be2ffd700 1 -- 172.21.15.110:0/228842016 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f7be402e440 con 0x7f7bf4078c80 2024-04-25T15:39:55.420 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.977+0000 7f7be2ffd700 1 --2- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f7bdc036850 0x7f7bdc038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.977+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf405f5c0 con 0x7f7bf4078c80 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.977+0000 7f7be2ffd700 1 -- 172.21.15.110:0/228842016 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f7be404a5a0 con 0x7f7bf4078c80 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.977+0000 7f7bf1d9b700 1 --2- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f7bdc036850 0x7f7bdc038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.977+0000 7f7bf1d9b700 1 --2- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f7bdc036850 0x7f7bdc038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009990 tx=0x7f7be8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:53.981+0000 7f7be2ffd700 1 -- 172.21.15.110:0/228842016 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7be40108c0 con 0x7f7bf4078c80 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.164+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f7bf404bed0 con 0x7f7bf4078c80 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.165+0000 7f7be2ffd700 1 -- 172.21.15.110:0/228842016 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f7be4018350 con 0x7f7bf4078c80 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f7bdc036850 msgr2=0x7f7bdc038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 --2- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f7bdc036850 0x7f7bdc038d00 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009990 tx=0x7f7be8006e40 comp rx=0 tx=0).stop 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 msgr2=0x7f7bf4113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 --2- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4113810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7be4003050 tx=0x7f7be4009250 comp rx=0 tx=0).stop 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 shutdown_connections 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 --2- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f7bdc036850 0x7f7bdc038d00 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 --2- 172.21.15.110:0/228842016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bf4078c80 0x7f7bf4113810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:55.421 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 >> 172.21.15.110:0/228842016 conn(0x7f7bf40fe060 msgr2=0x7f7bf40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:55.422 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 shutdown_connections 2024-04-25T15:39:55.422 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:54.167+0000 7f7bf8eb6700 1 -- 172.21.15.110:0/228842016 wait complete. 2024-04-25T15:39:55.422 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling cephadm module... 2024-04-25T15:39:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:56 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/409572769' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T15:39:56.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.805+0000 7f69cf508700 1 Processor -- start 2024-04-25T15:39:56.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.805+0000 7f69cf508700 1 -- start start 2024-04-25T15:39:56.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.805+0000 7f69cf508700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c8101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:56.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.805+0000 7f69cf508700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c8102080 con 0x7f69c8101730 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.805+0000 7f69cd2a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c8101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.805+0000 7f69cd2a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c8101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51492/0 (socket says 172.21.15.110:51492) 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.805+0000 7f69cd2a4700 1 -- 172.21.15.110:0/3762879169 learned_addr learned my addr 172.21.15.110:0/3762879169 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69cd2a4700 1 -- 172.21.15.110:0/3762879169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c81021c0 con 0x7f69c8101730 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69cd2a4700 1 --2- 172.21.15.110:0/3762879169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c8101b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69c40098e0 tx=0x7f69c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9150a27bc476cc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69bffff700 1 -- 172.21.15.110:0/3762879169 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f69c400b860 con 0x7f69c8101730 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69bffff700 1 -- 172.21.15.110:0/3762879169 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f69c400be50 con 0x7f69c8101730 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69bffff700 1 -- 172.21.15.110:0/3762879169 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f69c4010380 con 0x7f69c8101730 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69cf508700 1 -- 172.21.15.110:0/3762879169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 msgr2=0x7f69c8101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69cf508700 1 --2- 172.21.15.110:0/3762879169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c8101b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f69c40098e0 tx=0x7f69c4009250 comp rx=0 tx=0).stop 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69cf508700 1 -- 172.21.15.110:0/3762879169 shutdown_connections 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69cf508700 1 --2- 172.21.15.110:0/3762879169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c8101b40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:56.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.806+0000 7f69cf508700 1 -- 172.21.15.110:0/3762879169 >> 172.21.15.110:0/3762879169 conn(0x7f69c80fcef0 msgr2=0x7f69c80ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cf508700 1 -- 172.21.15.110:0/3762879169 shutdown_connections 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cf508700 1 -- 172.21.15.110:0/3762879169 wait complete. 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cf508700 1 Processor -- start 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cf508700 1 -- start start 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cf508700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c81115c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cf508700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c8111b00 con 0x7f69c8101730 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cd2a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c81115c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cd2a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c81115c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51502/0 (socket says 172.21.15.110:51502) 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.807+0000 7f69cd2a4700 1 -- 172.21.15.110:0/409572769 learned_addr learned my addr 172.21.15.110:0/409572769 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.808+0000 7f69cd2a4700 1 -- 172.21.15.110:0/409572769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40094f0 con 0x7f69c8101730 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.808+0000 7f69cd2a4700 1 --2- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c81115c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f69c4000c00 tx=0x7f69c4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.808+0000 7f69bdffb700 1 -- 172.21.15.110:0/409572769 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f69c40107c0 con 0x7f69c8101730 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.808+0000 7f69bdffb700 1 -- 172.21.15.110:0/409572769 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f69c4010db0 con 0x7f69c8101730 2024-04-25T15:39:56.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.808+0000 7f69bdffb700 1 -- 172.21.15.110:0/409572769 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f69c401ad00 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.808+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c8111d00 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.808+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c8112870 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.809+0000 7f69bdffb700 1 -- 172.21.15.110:0/409572769 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 4) v1 ==== 43365+0+0 (secure 0 0 0) 0x7f69c401a530 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.809+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c805f5c0 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.809+0000 7f69bdffb700 1 --2- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f69b4036850 0x7f69b4038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.809+0000 7f69bdffb700 1 -- 172.21.15.110:0/409572769 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f69c404a580 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.809+0000 7f69ccaa3700 1 --2- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f69b4036850 0x7f69b4038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.810+0000 7f69ccaa3700 1 --2- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f69b4036850 0x7f69b4038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009990 tx=0x7f69b8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:55.814+0000 7f69bdffb700 1 -- 172.21.15.110:0/409572769 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f69c4010920 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.051+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f69c804bed0 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.104+0000 7f69bdffb700 1 -- 172.21.15.110:0/409572769 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f69c40193c0 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.105+0000 7f69bdffb700 1 -- 172.21.15.110:0/409572769 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f69c4044820 con 0x7f69c8101730 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f69b4036850 msgr2=0x7f69b4038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 --2- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f69b4036850 0x7f69b4038d00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f69b8009990 tx=0x7f69b8006e40 comp rx=0 tx=0).stop 2024-04-25T15:39:56.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 msgr2=0x7f69c81115c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:56.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 --2- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c81115c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f69c4000c00 tx=0x7f69c4009250 comp rx=0 tx=0).stop 2024-04-25T15:39:56.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 shutdown_connections 2024-04-25T15:39:56.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 --2- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f69b4036850 0x7f69b4038d00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:56.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 --2- 172.21.15.110:0/409572769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69c8101730 0x7f69c81115c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:56.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 >> 172.21.15.110:0/409572769 conn(0x7f69c80fcef0 msgr2=0x7f69c80fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:56.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 shutdown_connections 2024-04-25T15:39:56.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:56.107+0000 7f69cf508700 1 -- 172.21.15.110:0/409572769 wait complete. 2024-04-25T15:39:57.333 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:57 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/409572769' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T15:39:57.333 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:57 smithi110 ceph-mon[26344]: mgrmap e5: smithi110.lvgbkf(active, since 11s) 2024-04-25T15:39:57.889 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-25T15:39:57.889 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T15:39:57.889 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T15:39:57.889 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "active_name": "smithi110.lvgbkf", 2024-04-25T15:39:57.889 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.046+0000 7fba20508700 1 Processor -- start 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.046+0000 7fba20508700 1 -- start start 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.046+0000 7fba20508700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18101010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.046+0000 7fba20508700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba18101550 con 0x7fba18100c00 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.047+0000 7fba1e2a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18101010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.047+0000 7fba1e2a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18101010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51526/0 (socket says 172.21.15.110:51526) 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.047+0000 7fba1e2a4700 1 -- 172.21.15.110:0/2322998432 learned_addr learned my addr 172.21.15.110:0/2322998432 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.047+0000 7fba1e2a4700 1 -- 172.21.15.110:0/2322998432 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba18101690 con 0x7fba18100c00 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.047+0000 7fba1e2a4700 1 --2- 172.21.15.110:0/2322998432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18101010 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fba080098e0 tx=0x7fba08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e442014e41c306d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.047+0000 7fba1d2a2700 1 -- 172.21.15.110:0/2322998432 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba0800b860 con 0x7fba18100c00 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.047+0000 7fba1d2a2700 1 -- 172.21.15.110:0/2322998432 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fba0800be50 con 0x7fba18100c00 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.047+0000 7fba1d2a2700 1 -- 172.21.15.110:0/2322998432 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba08010380 con 0x7fba18100c00 2024-04-25T15:39:57.890 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.048+0000 7fba20508700 1 -- 172.21.15.110:0/2322998432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 msgr2=0x7fba18101010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.048+0000 7fba20508700 1 --2- 172.21.15.110:0/2322998432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18101010 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fba080098e0 tx=0x7fba08009250 comp rx=0 tx=0).stop 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.048+0000 7fba20508700 1 -- 172.21.15.110:0/2322998432 shutdown_connections 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.048+0000 7fba20508700 1 --2- 172.21.15.110:0/2322998432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18101010 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.048+0000 7fba20508700 1 -- 172.21.15.110:0/2322998432 >> 172.21.15.110:0/2322998432 conn(0x7fba180fc6f0 msgr2=0x7fba180feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.048+0000 7fba20508700 1 -- 172.21.15.110:0/2322998432 shutdown_connections 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.048+0000 7fba20508700 1 -- 172.21.15.110:0/2322998432 wait complete. 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.049+0000 7fba20508700 1 Processor -- start 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.049+0000 7fba20508700 1 -- start start 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.049+0000 7fba20508700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18111590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.049+0000 7fba20508700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba18111ad0 con 0x7fba18100c00 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.049+0000 7fba1e2a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18111590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.049+0000 7fba1e2a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18111590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51540/0 (socket says 172.21.15.110:51540) 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.049+0000 7fba1e2a4700 1 -- 172.21.15.110:0/769354157 learned_addr learned my addr 172.21.15.110:0/769354157 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:39:57.891 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.049+0000 7fba1e2a4700 1 -- 172.21.15.110:0/769354157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba080094f0 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.050+0000 7fba1e2a4700 1 --2- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18111590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fba08000c00 tx=0x7fba08009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.050+0000 7fba0effd700 1 -- 172.21.15.110:0/769354157 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba080107c0 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.050+0000 7fba0effd700 1 -- 172.21.15.110:0/769354157 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fba08010db0 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.050+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba18111cd0 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.050+0000 7fba0effd700 1 -- 172.21.15.110:0/769354157 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba0801ad00 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.050+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba18112840 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.051+0000 7fba0effd700 1 -- 172.21.15.110:0/769354157 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fba080193c0 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.051+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1805f5c0 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.051+0000 7fba0effd700 1 --2- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7fba04036850 0x7fba04038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.051+0000 7fba1daa3700 1 -- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7fba04036850 msgr2=0x7fba04038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1682815503 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.051+0000 7fba0effd700 1 -- 172.21.15.110:0/769354157 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fba0804a660 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.051+0000 7fba1daa3700 1 --2- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7fba04036850 0x7fba04038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.055+0000 7fba0effd700 1 -- 172.21.15.110:0/769354157 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fba08022700 con 0x7fba18100c00 2024-04-25T15:39:57.892 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.252+0000 7fba1daa3700 1 -- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7fba04036850 msgr2=0x7fba04038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1682815503 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.252+0000 7fba1daa3700 1 --2- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7fba04036850 0x7fba04038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.332+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fba18112b80 con 0x7fba18100c00 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.334+0000 7fba0effd700 1 -- 172.21.15.110:0/769354157 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7fba08022340 con 0x7fba18100c00 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.337+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7fba04036850 msgr2=0x7fba04038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.337+0000 7fba20508700 1 --2- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7fba04036850 0x7fba04038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.337+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 msgr2=0x7fba18111590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.337+0000 7fba20508700 1 --2- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18111590 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fba08000c00 tx=0x7fba08009250 comp rx=0 tx=0).stop 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.337+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 shutdown_connections 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.337+0000 7fba20508700 1 --2- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7fba04036850 0x7fba04038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.337+0000 7fba20508700 1 --2- 172.21.15.110:0/769354157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fba18100c00 0x7fba18111590 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.338+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 >> 172.21.15.110:0/769354157 conn(0x7fba180fc6f0 msgr2=0x7fba180fd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.338+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 shutdown_connections 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:57.338+0000 7fba20508700 1 -- 172.21.15.110:0/769354157 wait complete. 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for the mgr to restart... 2024-04-25T15:39:57.893 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr epoch 5... 2024-04-25T15:39:58.262 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:39:58 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/769354157' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T15:40:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: Active manager daemon smithi110.lvgbkf restarted 2024-04-25T15:40:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: Activating manager daemon smithi110.lvgbkf 2024-04-25T15:40:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T15:40:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: mgrmap e6: smithi110.lvgbkf(active, starting, since 0.00316359s) 2024-04-25T15:40:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:40:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: Manager daemon smithi110.lvgbkf is now available 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:40:02.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:02 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:40:03.819 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:03 smithi110 ceph-mon[26344]: Found migration_current of "None". Setting to last migration. 2024-04-25T15:40:03.819 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:03 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:03.819 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:03 smithi110 ceph-mon[26344]: mgrmap e7: smithi110.lvgbkf(active, since 1.00467s) 2024-04-25T15:40:03.819 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:03 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:40:04.096 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-25T15:40:04.097 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T15:40:04.097 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T15:40:04.097 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-25T15:40:04.097 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.303+0000 7f057e94f700 1 Processor -- start 2024-04-25T15:40:04.097 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.303+0000 7f057e94f700 1 -- start start 2024-04-25T15:40:04.097 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.303+0000 7f057e94f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578102c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:04.097 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.303+0000 7f057e94f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05781031a0 con 0x7f0578102850 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.303+0000 7f0577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.304+0000 7f0577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51550/0 (socket says 172.21.15.110:51550) 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.304+0000 7f0577fff700 1 -- 172.21.15.110:0/782776440 learned_addr learned my addr 172.21.15.110:0/782776440 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.304+0000 7f0577fff700 1 -- 172.21.15.110:0/782776440 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05781032e0 con 0x7f0578102850 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.304+0000 7f0577fff700 1 --2- 172.21.15.110:0/782776440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578102c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f056c0098e0 tx=0x7f056c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318a54019af9c30c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.304+0000 7f0576ffd700 1 -- 172.21.15.110:0/782776440 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f056c00b860 con 0x7f0578102850 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.304+0000 7f0576ffd700 1 -- 172.21.15.110:0/782776440 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f056c00be50 con 0x7f0578102850 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.304+0000 7f0576ffd700 1 -- 172.21.15.110:0/782776440 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f056c010380 con 0x7f0578102850 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.305+0000 7f057e94f700 1 -- 172.21.15.110:0/782776440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 msgr2=0x7f0578102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.305+0000 7f057e94f700 1 --2- 172.21.15.110:0/782776440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578102c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f056c0098e0 tx=0x7f056c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.305+0000 7f057e94f700 1 -- 172.21.15.110:0/782776440 shutdown_connections 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.305+0000 7f057e94f700 1 --2- 172.21.15.110:0/782776440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578102c60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:04.098 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.305+0000 7f057e94f700 1 -- 172.21.15.110:0/782776440 >> 172.21.15.110:0/782776440 conn(0x7f05780fdf90 msgr2=0x7f05781003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.305+0000 7f057e94f700 1 -- 172.21.15.110:0/782776440 shutdown_connections 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.305+0000 7f057e94f700 1 -- 172.21.15.110:0/782776440 wait complete. 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.305+0000 7f057e94f700 1 Processor -- start 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f057e94f700 1 -- start start 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f057e94f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578077b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f057e94f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0578078050 con 0x7f0578102850 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f0577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578077b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f0577fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578077b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51552/0 (socket says 172.21.15.110:51552) 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f0577fff700 1 -- 172.21.15.110:0/3477311103 learned_addr learned my addr 172.21.15.110:0/3477311103 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f0577fff700 1 -- 172.21.15.110:0/3477311103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056c0094f0 con 0x7f0578102850 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f0577fff700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578077b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f056c000c00 tx=0x7f056c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f056c0107c0 con 0x7f0578102850 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f056c010db0 con 0x7f0578102850 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.306+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05780760c0 con 0x7f0578102850 2024-04-25T15:40:04.099 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.307+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f056c01ad00 con 0x7f0578102850 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.307+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05780765b0 con 0x7f0578102850 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.308+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 5) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f056c01a530 con 0x7f0578102850 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.308+0000 7f0574ff9700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 0x7f0560038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.308+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 --> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0560039410 con 0x7f0560036850 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.308+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f056c04a610 con 0x7f0578102850 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.308+0000 7f05777fe700 1 -- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 msgr2=0x7f0560038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1682815503 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.308+0000 7f05777fe700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 0x7f0560038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.509+0000 7f05777fe700 1 -- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 msgr2=0x7f0560038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1682815503 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.509+0000 7f05777fe700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 0x7f0560038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.910+0000 7f05777fe700 1 -- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 msgr2=0x7f0560038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1682815503 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:58.910+0000 7f05777fe700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 0x7f0560038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:59.711+0000 7f05777fe700 1 -- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 msgr2=0x7f0560038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1682815503 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:39:59.711+0000 7f05777fe700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 0x7f0560038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T15:40:04.100 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:01.312+0000 7f05777fe700 1 -- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 msgr2=0x7f0560038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1682815503 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:01.312+0000 7f05777fe700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 0x7f0560038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:02.514+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mgrmap(e 6) v1 ==== 43143+0+0 (secure 0 0 0) 0x7f056c016ce0 con 0x7f0578102850 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:02.514+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 msgr2=0x7f0560038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:02.514+0000 7f0574ff9700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 0x7f0560038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.515+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7f056c020d20 con 0x7f0578102850 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.515+0000 7f0574ff9700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0560039ff0 0x7f056003c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.515+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0560039410 con 0x7f0560039ff0 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.516+0000 7f05777fe700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0560039ff0 0x7f056003c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.516+0000 7f05777fe700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0560039ff0 0x7f056003c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f05680039d0 tx=0x7f0568009350 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.517+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f0560039410 con 0x7f0560039ff0 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f057804bed0 con 0x7f0560039ff0 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f0574ff9700 1 -- 172.21.15.110:0/3477311103 <== mgr.14118 v2:172.21.15.110:6800/1116386591 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f057804bed0 con 0x7f0560039ff0 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0560039ff0 msgr2=0x7f056003c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0560039ff0 0x7f056003c3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f05680039d0 tx=0x7f0568009350 comp rx=0 tx=0).stop 2024-04-25T15:40:04.101 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 msgr2=0x7f0578077b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578077b10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f056c000c00 tx=0x7f056c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 shutdown_connections 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0560039ff0 0x7f056003c3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0578102850 0x7f0578077b10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 --2- 172.21.15.110:0/3477311103 >> [v2:172.21.15.110:6800/1682815503,v1:172.21.15.110:6801/1682815503] conn(0x7f0560036850 0x7f0560038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.523+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 >> 172.21.15.110:0/3477311103 conn(0x7f05780fdf90 msgr2=0x7f05780fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.524+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 shutdown_connections 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:03.524+0000 7f057e94f700 1 -- 172.21.15.110:0/3477311103 wait complete. 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:mgr epoch 5 is available 2024-04-25T15:40:04.102 INFO:teuthology.orchestra.run.smithi110.stdout:Setting orchestrator backend to cephadm... 2024-04-25T15:40:04.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:04 smithi110 ceph-mon[26344]: [25/Apr/2024:15:40:03] ENGINE Bus STARTING 2024-04-25T15:40:04.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:04 smithi110 ceph-mon[26344]: [25/Apr/2024:15:40:03] ENGINE Serving on https://172.21.15.110:7150 2024-04-25T15:40:04.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:04 smithi110 ceph-mon[26344]: [25/Apr/2024:15:40:03] ENGINE Bus STARTED 2024-04-25T15:40:04.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:04 smithi110 ceph-mon[26344]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T15:40:04.687 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:04 smithi110 ceph-mon[26344]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T15:40:05.251 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.477+0000 7fea72334700 1 Processor -- start 2024-04-25T15:40:05.251 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.477+0000 7fea72334700 1 -- start start 2024-04-25T15:40:05.251 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.477+0000 7fea72334700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:05.251 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.477+0000 7fea72334700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c1031d0 con 0x7fea6c102880 2024-04-25T15:40:05.251 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.477+0000 7fea6bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:05.251 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.477+0000 7fea6bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34172/0 (socket says 172.21.15.110:34172) 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.477+0000 7fea6bfff700 1 -- 172.21.15.110:0/3530915863 learned_addr learned my addr 172.21.15.110:0/3530915863 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.478+0000 7fea6bfff700 1 -- 172.21.15.110:0/3530915863 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c103310 con 0x7fea6c102880 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.478+0000 7fea6bfff700 1 --2- 172.21.15.110:0/3530915863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c102c90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0098e0 tx=0x7fea5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10775df5d7ce86b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.478+0000 7fea6affd700 1 -- 172.21.15.110:0/3530915863 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea5c00b860 con 0x7fea6c102880 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.478+0000 7fea6affd700 1 -- 172.21.15.110:0/3530915863 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fea5c00be50 con 0x7fea6c102880 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.478+0000 7fea6affd700 1 -- 172.21.15.110:0/3530915863 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea5c010380 con 0x7fea6c102880 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 -- 172.21.15.110:0/3530915863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 msgr2=0x7fea6c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 --2- 172.21.15.110:0/3530915863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c102c90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fea5c0098e0 tx=0x7fea5c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 -- 172.21.15.110:0/3530915863 shutdown_connections 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 --2- 172.21.15.110:0/3530915863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c102c90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 -- 172.21.15.110:0/3530915863 >> 172.21.15.110:0/3530915863 conn(0x7fea6c0fe060 msgr2=0x7fea6c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 -- 172.21.15.110:0/3530915863 shutdown_connections 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 -- 172.21.15.110:0/3530915863 wait complete. 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 Processor -- start 2024-04-25T15:40:05.252 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.479+0000 7fea72334700 1 -- start start 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea72334700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c1115f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea72334700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea6c111b30 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea6bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c1115f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea6bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c1115f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34174/0 (socket says 172.21.15.110:34174) 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea6bfff700 1 -- 172.21.15.110:0/463821359 learned_addr learned my addr 172.21.15.110:0/463821359 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea6bfff700 1 -- 172.21.15.110:0/463821359 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5c0094f0 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea6bfff700 1 --2- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c1115f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000c00 tx=0x7fea5c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea68ff9700 1 -- 172.21.15.110:0/463821359 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea5c0107c0 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea6c111d30 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea68ff9700 1 -- 172.21.15.110:0/463821359 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7fea5c010db0 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea68ff9700 1 -- 172.21.15.110:0/463821359 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea5c01ad00 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.480+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea6c1128a0 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.481+0000 7fea68ff9700 1 -- 172.21.15.110:0/463821359 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 7) v1 ==== 43270+0+0 (secure 0 0 0) 0x7fea5c0193c0 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.481+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6c05f5c0 con 0x7fea6c102880 2024-04-25T15:40:05.253 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.481+0000 7fea68ff9700 1 --2- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fea54036720 0x7fea54038bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.481+0000 7fea68ff9700 1 -- 172.21.15.110:0/463821359 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fea5c04a4d0 con 0x7fea6c102880 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.481+0000 7fea6b7fe700 1 --2- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fea54036720 0x7fea54038bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.482+0000 7fea6b7fe700 1 --2- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fea54036720 0x7fea54038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea60009990 tx=0x7fea60006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.486+0000 7fea68ff9700 1 -- 172.21.15.110:0/463821359 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fea5c022930 con 0x7fea6c102880 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.569+0000 7fea68ff9700 1 -- 172.21.15.110:0/463821359 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fea5c01a530 con 0x7fea6c102880 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.686+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fea6c0ff200 con 0x7fea54036720 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.695+0000 7fea68ff9700 1 -- 172.21.15.110:0/463821359 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fea6c0ff200 con 0x7fea54036720 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fea54036720 msgr2=0x7fea54038bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 --2- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fea54036720 0x7fea54038bd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fea60009990 tx=0x7fea60006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 msgr2=0x7fea6c1115f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 --2- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c1115f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000c00 tx=0x7fea5c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 shutdown_connections 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 --2- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fea54036720 0x7fea54038bd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 --2- 172.21.15.110:0/463821359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea6c102880 0x7fea6c1115f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:05.254 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 >> 172.21.15.110:0/463821359 conn(0x7fea6c0fe060 msgr2=0x7fea6c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:05.255 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 shutdown_connections 2024-04-25T15:40:05.255 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:04.697+0000 7fea72334700 1 -- 172.21.15.110:0/463821359 wait complete. 2024-04-25T15:40:05.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:05 smithi110 ceph-mon[26344]: mgrmap e8: smithi110.lvgbkf(active, since 2s) 2024-04-25T15:40:05.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:05 smithi110 ceph-mon[26344]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:05.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:05 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:05.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:05 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.680+0000 7f043eb8e700 1 Processor -- start 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.680+0000 7f043eb8e700 1 -- start start 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.680+0000 7f043eb8e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438105440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.680+0000 7f043eb8e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0438105980 con 0x7f0438076c30 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.680+0000 7f043c92a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438105440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.680+0000 7f043c92a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438105440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34180/0 (socket says 172.21.15.110:34180) 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.680+0000 7f043c92a700 1 -- 172.21.15.110:0/2462819983 learned_addr learned my addr 172.21.15.110:0/2462819983 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:06.366 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043c92a700 1 -- 172.21.15.110:0/2462819983 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0438105ac0 con 0x7f0438076c30 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043c92a700 1 --2- 172.21.15.110:0/2462819983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438105440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f042c0098e0 tx=0x7f042c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51540e8585bbdd1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f04377fe700 1 -- 172.21.15.110:0/2462819983 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f042c00b860 con 0x7f0438076c30 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f04377fe700 1 -- 172.21.15.110:0/2462819983 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f042c00be50 con 0x7f0438076c30 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f04377fe700 1 -- 172.21.15.110:0/2462819983 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f042c010380 con 0x7f0438076c30 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043eb8e700 1 -- 172.21.15.110:0/2462819983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 msgr2=0x7f0438105440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043eb8e700 1 --2- 172.21.15.110:0/2462819983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438105440 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f042c0098e0 tx=0x7f042c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043eb8e700 1 -- 172.21.15.110:0/2462819983 shutdown_connections 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043eb8e700 1 --2- 172.21.15.110:0/2462819983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438105440 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043eb8e700 1 -- 172.21.15.110:0/2462819983 >> 172.21.15.110:0/2462819983 conn(0x7f04380fcef0 msgr2=0x7f04380ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043eb8e700 1 -- 172.21.15.110:0/2462819983 shutdown_connections 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.681+0000 7f043eb8e700 1 -- 172.21.15.110:0/2462819983 wait complete. 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.682+0000 7f043eb8e700 1 Processor -- start 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.682+0000 7f043eb8e700 1 -- start start 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.682+0000 7f043eb8e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.682+0000 7f043eb8e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0438111b40 con 0x7f0438076c30 2024-04-25T15:40:06.367 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.682+0000 7f043c92a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.682+0000 7f043c92a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34194/0 (socket says 172.21.15.110:34194) 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.682+0000 7f043c92a700 1 -- 172.21.15.110:0/921273553 learned_addr learned my addr 172.21.15.110:0/921273553 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.683+0000 7f043c92a700 1 -- 172.21.15.110:0/921273553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f042c0094f0 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.683+0000 7f043c92a700 1 --2- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438111600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f042c006b00 tx=0x7f042c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.683+0000 7f04357fa700 1 -- 172.21.15.110:0/921273553 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f042c0107c0 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.683+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0438111d40 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.683+0000 7f04357fa700 1 -- 172.21.15.110:0/921273553 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f042c010db0 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.683+0000 7f04357fa700 1 -- 172.21.15.110:0/921273553 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f042c01acf0 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.683+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04381128b0 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.684+0000 7f04357fa700 1 -- 172.21.15.110:0/921273553 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f042c0193c0 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.684+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f043805f5c0 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.684+0000 7f04357fa700 1 --2- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f04280368a0 0x7f0428038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.684+0000 7f04357fa700 1 -- 172.21.15.110:0/921273553 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f042c04a780 con 0x7f0438076c30 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.684+0000 7f0437fff700 1 --2- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f04280368a0 0x7f0428038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:06.368 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.684+0000 7f0437fff700 1 --2- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f04280368a0 0x7f0428038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0424009990 tx=0x7f0424006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.689+0000 7f04357fa700 1 -- 172.21.15.110:0/921273553 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f042c022930 con 0x7f0438076c30 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.877+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f04380fe100 con 0x7f04280368a0 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.878+0000 7f04357fa700 1 -- 172.21.15.110:0/921273553 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f04380fe100 con 0x7f04280368a0 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f04280368a0 msgr2=0x7f0428038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 --2- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f04280368a0 0x7f0428038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0424009990 tx=0x7f0424006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 msgr2=0x7f0438111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 --2- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438111600 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f042c006b00 tx=0x7f042c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 shutdown_connections 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 --2- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f04280368a0 0x7f0428038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 --2- 172.21.15.110:0/921273553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0438076c30 0x7f0438111600 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 >> 172.21.15.110:0/921273553 conn(0x7f04380fcef0 msgr2=0x7f04380fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 shutdown_connections 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:05.880+0000 7f043eb8e700 1 -- 172.21.15.110:0/921273553 wait complete. 2024-04-25T15:40:06.369 INFO:teuthology.orchestra.run.smithi110.stdout:Generating ssh key... 2024-04-25T15:40:06.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:06 smithi110 ceph-mon[26344]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:07.547 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.744+0000 7f46c8a0f700 1 Processor -- start 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.744+0000 7f46c8a0f700 1 -- start start 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.744+0000 7f46c8a0f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.744+0000 7f46c8a0f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4070d90 con 0x7f46c4102ea0 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.745+0000 7f46c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.745+0000 7f46c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34210/0 (socket says 172.21.15.110:34210) 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.745+0000 7f46c259c700 1 -- 172.21.15.110:0/3076329182 learned_addr learned my addr 172.21.15.110:0/3076329182 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.745+0000 7f46c259c700 1 -- 172.21.15.110:0/3076329182 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c41037f0 con 0x7f46c4102ea0 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.745+0000 7f46c259c700 1 --2- 172.21.15.110:0/3076329182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41032b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009e50 tx=0x7f46bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59515f9675a2022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.745+0000 7f46c159a700 1 -- 172.21.15.110:0/3076329182 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46bc0092e0 con 0x7f46c4102ea0 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.745+0000 7f46c159a700 1 -- 172.21.15.110:0/3076329182 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f46bc012430 con 0x7f46c4102ea0 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.745+0000 7f46c159a700 1 -- 172.21.15.110:0/3076329182 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46bc0075c0 con 0x7f46c4102ea0 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 -- 172.21.15.110:0/3076329182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 msgr2=0x7f46c41032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:07.548 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 --2- 172.21.15.110:0/3076329182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41032b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009e50 tx=0x7f46bc009840 comp rx=0 tx=0).stop 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 -- 172.21.15.110:0/3076329182 shutdown_connections 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 --2- 172.21.15.110:0/3076329182 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41032b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 -- 172.21.15.110:0/3076329182 >> 172.21.15.110:0/3076329182 conn(0x7f46c40fc6f0 msgr2=0x7f46c40feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 -- 172.21.15.110:0/3076329182 shutdown_connections 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 -- 172.21.15.110:0/3076329182 wait complete. 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 Processor -- start 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.746+0000 7f46c8a0f700 1 -- start start 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c8a0f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41137e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c8a0f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4113d20 con 0x7f46c4102ea0 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41137e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41137e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34222/0 (socket says 172.21.15.110:34222) 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c259c700 1 -- 172.21.15.110:0/1451332053 learned_addr learned my addr 172.21.15.110:0/1451332053 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c259c700 1 -- 172.21.15.110:0/1451332053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46bc009ae0 con 0x7f46c4102ea0 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c259c700 1 --2- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41137e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009fd0 tx=0x7f46bc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:07.549 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46baffd700 1 -- 172.21.15.110:0/1451332053 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46bc0092e0 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46baffd700 1 -- 172.21.15.110:0/1451332053 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f46bc007cc0 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c4113f20 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46baffd700 1 -- 172.21.15.110:0/1451332053 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f46bc01c9a0 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.747+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c4114a90 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.748+0000 7f46baffd700 1 -- 172.21.15.110:0/1451332053 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f46bc019070 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.748+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c405f5c0 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.748+0000 7f46baffd700 1 --2- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f46b0036850 0x7f46b0038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.748+0000 7f46baffd700 1 -- 172.21.15.110:0/1451332053 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f46bc049ea0 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.748+0000 7f46c1d9b700 1 --2- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f46b0036850 0x7f46b0038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.749+0000 7f46c1d9b700 1 --2- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f46b0036850 0x7f46b0038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009990 tx=0x7f46ac006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.753+0000 7f46baffd700 1 -- 172.21.15.110:0/1451332053 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f46bc00be50 con 0x7f46c4102ea0 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:06.937+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f46c40fd920 con 0x7f46b0036850 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.063+0000 7f46baffd700 1 -- 172.21.15.110:0/1451332053 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f46c40fd920 con 0x7f46b0036850 2024-04-25T15:40:07.550 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f46b0036850 msgr2=0x7f46b0038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 --2- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f46b0036850 0x7f46b0038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009990 tx=0x7f46ac006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 msgr2=0x7f46c41137e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 --2- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41137e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009fd0 tx=0x7f46bc009840 comp rx=0 tx=0).stop 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 shutdown_connections 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 --2- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f46b0036850 0x7f46b0038d00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 --2- 172.21.15.110:0/1451332053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f46c4102ea0 0x7f46c41137e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 >> 172.21.15.110:0/1451332053 conn(0x7f46c40fc6f0 msgr2=0x7f46c40fd210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 shutdown_connections 2024-04-25T15:40:07.551 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.066+0000 7f46c8a0f700 1 -- 172.21.15.110:0/1451332053 wait complete. 2024-04-25T15:40:08.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:08 smithi110 ceph-mon[26344]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:08.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:08 smithi110 ceph-mon[26344]: Generating ssh key... 2024-04-25T15:40:08.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:08 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:08.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:08 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:08.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGU63FKYnRaYjybGBNU8fAHoXvTTvDZ0pHE+UIRTeZBFtHxqo/2lr8bfFrx3Usdi9dV2h+ThSRwIz1eGYf2BD8uGbsFJkD4sMQ0ZHJMsuB7pIuAbVDVVus1+TnXvoNfA6vKYETKNfrwk2gX3OEPZwKGSELr+/r2Pqq1nd2oc5p993b7PQQ1/dN8L8KEDGvTAPfHEqNvyo5kydeDdCoTYU+qn6G/O2pYSgQJyM8Y1ha7p5r928y3imOXoagWzSbdhlUa4LaeV1Orhz53kWxGfffkAXonM8hWaBMlmZEt7YPqX4JwX+uwiPwJupg3RjbXuk9QO3e5bfaQIZ5cMyljkwgliuVGsmk7D08yhMmuk1FoposoNLeT7I8+9maSYSUN6VakCOIRSgnkZwlADjTk6ExsPmAQDzxx2+ciBUq6nQC6Vvw9zLXNFMS/Q/CSw6Fu6+tcRZ49pBuRMpzAiHN8CYVqK/BsrJQlSAFJFUJ8bz9Dmnz53FPMjTypCb8PHXubTs= ceph-e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:40:08.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.980+0000 7f95b602c700 1 Processor -- start 2024-04-25T15:40:08.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.980+0000 7f95b602c700 1 -- start start 2024-04-25T15:40:08.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.980+0000 7f95b602c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b0102150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:08.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.980+0000 7f95b602c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0102690 con 0x7f95b0101d40 2024-04-25T15:40:08.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.980+0000 7f95af7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b0102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:08.722 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.980+0000 7f95af7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b0102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34232/0 (socket says 172.21.15.110:34232) 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.980+0000 7f95af7fe700 1 -- 172.21.15.110:0/434145601 learned_addr learned my addr 172.21.15.110:0/434145601 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.980+0000 7f95af7fe700 1 -- 172.21.15.110:0/434145601 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b01027d0 con 0x7f95b0101d40 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95af7fe700 1 --2- 172.21.15.110:0/434145601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b0102150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95a000bb80 tx=0x7f95a000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3471c403606f7d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95ae7fc700 1 -- 172.21.15.110:0/434145601 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95a000dd00 con 0x7f95b0101d40 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95ae7fc700 1 -- 172.21.15.110:0/434145601 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f95a000de60 con 0x7f95b0101d40 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95ae7fc700 1 -- 172.21.15.110:0/434145601 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95a00188a0 con 0x7f95b0101d40 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95b602c700 1 -- 172.21.15.110:0/434145601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 msgr2=0x7f95b0102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95b602c700 1 --2- 172.21.15.110:0/434145601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b0102150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f95a000bb80 tx=0x7f95a000b4f0 comp rx=0 tx=0).stop 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95b602c700 1 -- 172.21.15.110:0/434145601 shutdown_connections 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95b602c700 1 --2- 172.21.15.110:0/434145601 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b0102150 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95b602c700 1 -- 172.21.15.110:0/434145601 >> 172.21.15.110:0/434145601 conn(0x7f95b00fc6f0 msgr2=0x7f95b00feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95b602c700 1 -- 172.21.15.110:0/434145601 shutdown_connections 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.981+0000 7f95b602c700 1 -- 172.21.15.110:0/434145601 wait complete. 2024-04-25T15:40:08.723 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.982+0000 7f95b602c700 1 Processor -- start 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.982+0000 7f95b602c700 1 -- start start 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.982+0000 7f95b602c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b010f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.982+0000 7f95b602c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b010f870 con 0x7f95b0101d40 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.982+0000 7f95af7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b010f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.982+0000 7f95af7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b010f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34242/0 (socket says 172.21.15.110:34242) 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.982+0000 7f95af7fe700 1 -- 172.21.15.110:0/835546175 learned_addr learned my addr 172.21.15.110:0/835546175 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.983+0000 7f95af7fe700 1 -- 172.21.15.110:0/835546175 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a000b790 con 0x7f95b0101d40 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.983+0000 7f95af7fe700 1 --2- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b010f330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95a000d650 tx=0x7f95a000b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.983+0000 7f9597fff700 1 -- 172.21.15.110:0/835546175 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95a0010b50 con 0x7f95b0101d40 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.983+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b010fa70 con 0x7f95b0101d40 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.983+0000 7f9597fff700 1 -- 172.21.15.110:0/835546175 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f95a001d050 con 0x7f95b0101d40 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.983+0000 7f9597fff700 1 -- 172.21.15.110:0/835546175 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f95a0022590 con 0x7f95b0101d40 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.983+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b01105e0 con 0x7f95b0101d40 2024-04-25T15:40:08.724 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.984+0000 7f9597fff700 1 -- 172.21.15.110:0/835546175 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f95a0022790 con 0x7f95b0101d40 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.984+0000 7f9597fff700 1 --2- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9598036830 0x7f9598038ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.984+0000 7f9597fff700 1 -- 172.21.15.110:0/835546175 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f95a0014070 con 0x7f95b0101d40 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.984+0000 7f95aeffd700 1 --2- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9598036830 0x7f9598038ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.984+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b005f5c0 con 0x7f95b0101d40 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.985+0000 7f95aeffd700 1 --2- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9598036830 0x7f9598038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009990 tx=0x7f95a4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:07.989+0000 7f9597fff700 1 -- 172.21.15.110:0/835546175 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f95a0021d90 con 0x7f95b0101d40 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.176+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f95b01108e0 con 0x7f9598036830 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.177+0000 7f9597fff700 1 -- 172.21.15.110:0/835546175 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f95b01108e0 con 0x7f9598036830 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9598036830 msgr2=0x7f9598038ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 --2- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9598036830 0x7f9598038ce0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009990 tx=0x7f95a4006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 msgr2=0x7f95b010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 --2- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b010f330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f95a000d650 tx=0x7f95a000b4f0 comp rx=0 tx=0).stop 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 shutdown_connections 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 --2- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9598036830 0x7f9598038ce0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 --2- 172.21.15.110:0/835546175 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95b0101d40 0x7f95b010f330 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:08.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 >> 172.21.15.110:0/835546175 conn(0x7f95b00fc6f0 msgr2=0x7f95b0106120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:08.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 shutdown_connections 2024-04-25T15:40:08.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:08.179+0000 7f95b602c700 1 -- 172.21.15.110:0/835546175 wait complete. 2024-04-25T15:40:08.726 INFO:teuthology.orchestra.run.smithi110.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T15:40:08.726 INFO:teuthology.orchestra.run.smithi110.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T15:40:08.726 INFO:teuthology.orchestra.run.smithi110.stdout:Adding host smithi110... 2024-04-25T15:40:09.404 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:09 smithi110 ceph-mon[26344]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:10.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:10 smithi110 ceph-mon[26344]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "addr": "172.21.15.110", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:10.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:10 smithi110 ceph-mon[26344]: Deploying cephadm binary to smithi110 2024-04-25T15:40:11.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Added host 'smithi110' with addr '172.21.15.110' 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.152+0000 7f9b7a36a700 1 Processor -- start 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.152+0000 7f9b7a36a700 1 -- start start 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.152+0000 7f9b7a36a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74101b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.152+0000 7f9b7a36a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b74102060 con 0x7f9b74101710 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.152+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.152+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34246/0 (socket says 172.21.15.110:34246) 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.152+0000 7f9b73fff700 1 -- 172.21.15.110:0/3283430179 learned_addr learned my addr 172.21.15.110:0/3283430179 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b73fff700 1 -- 172.21.15.110:0/3283430179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b741021a0 con 0x7f9b74101710 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b73fff700 1 --2- 172.21.15.110:0/3283430179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74101b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9b680098e0 tx=0x7f9b68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f69c5d114ce7978 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b72ffd700 1 -- 172.21.15.110:0/3283430179 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b6800b860 con 0x7f9b74101710 2024-04-25T15:40:11.730 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b72ffd700 1 -- 172.21.15.110:0/3283430179 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9b6800be50 con 0x7f9b74101710 2024-04-25T15:40:11.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b72ffd700 1 -- 172.21.15.110:0/3283430179 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b68010380 con 0x7f9b74101710 2024-04-25T15:40:11.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b7a36a700 1 -- 172.21.15.110:0/3283430179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 msgr2=0x7f9b74101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b7a36a700 1 --2- 172.21.15.110:0/3283430179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74101b20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9b680098e0 tx=0x7f9b68009250 comp rx=0 tx=0).stop 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b7a36a700 1 -- 172.21.15.110:0/3283430179 shutdown_connections 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b7a36a700 1 --2- 172.21.15.110:0/3283430179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74101b20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.153+0000 7f9b7a36a700 1 -- 172.21.15.110:0/3283430179 >> 172.21.15.110:0/3283430179 conn(0x7f9b740fced0 msgr2=0x7f9b740ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.154+0000 7f9b7a36a700 1 -- 172.21.15.110:0/3283430179 shutdown_connections 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.154+0000 7f9b7a36a700 1 -- 172.21.15.110:0/3283430179 wait complete. 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.154+0000 7f9b7a36a700 1 Processor -- start 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.154+0000 7f9b7a36a700 1 -- start start 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b7a36a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74076b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b7a36a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b741142a0 con 0x7f9b74101710 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74076b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b73fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74076b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34262/0 (socket says 172.21.15.110:34262) 2024-04-25T15:40:11.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b73fff700 1 -- 172.21.15.110:0/2236335057 learned_addr learned my addr 172.21.15.110:0/2236335057 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b73fff700 1 -- 172.21.15.110:0/2236335057 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b680094f0 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b73fff700 1 --2- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74076b00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800b440 tx=0x7f9b68009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b70ff9700 1 -- 172.21.15.110:0/2236335057 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b68010760 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b70ff9700 1 -- 172.21.15.110:0/2236335057 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f9b68010d50 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b741143e0 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b70ff9700 1 -- 172.21.15.110:0/2236335057 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9b6801ae20 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.155+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b74114870 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.156+0000 7f9b70ff9700 1 -- 172.21.15.110:0/2236335057 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f9b6802e440 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.156+0000 7f9b70ff9700 1 --2- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9b5c0368a0 0x7f9b5c038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.156+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7404bed0 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.156+0000 7f9b70ff9700 1 -- 172.21.15.110:0/2236335057 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9b6804a570 con 0x7f9b74101710 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.156+0000 7f9b737fe700 1 --2- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9b5c0368a0 0x7f9b5c038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:11.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.157+0000 7f9b737fe700 1 --2- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9b5c0368a0 0x7f9b5c038d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009990 tx=0x7f9b64006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.161+0000 7f9b70ff9700 1 -- 172.21.15.110:0/2236335057 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9b68018530 con 0x7f9b74101710 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:09.350+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi110", "addr": "172.21.15.110", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b740fe020 con 0x7f9b5c0368a0 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.185+0000 7f9b70ff9700 1 -- 172.21.15.110:0/2236335057 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9b740fe020 con 0x7f9b5c0368a0 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.187+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9b5c0368a0 msgr2=0x7f9b5c038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.187+0000 7f9b7a36a700 1 --2- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9b5c0368a0 0x7f9b5c038d50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9b64009990 tx=0x7f9b64006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.188+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 msgr2=0x7f9b74076b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.188+0000 7f9b7a36a700 1 --2- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74076b00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9b6800b440 tx=0x7f9b68009250 comp rx=0 tx=0).stop 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.188+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 shutdown_connections 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.188+0000 7f9b7a36a700 1 --2- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9b5c0368a0 0x7f9b5c038d50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.188+0000 7f9b7a36a700 1 --2- 172.21.15.110:0/2236335057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b74101710 0x7f9b74076b00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.188+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 >> 172.21.15.110:0/2236335057 conn(0x7f9b740fced0 msgr2=0x7f9b740fd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:11.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.188+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 shutdown_connections 2024-04-25T15:40:11.737 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:11.188+0000 7f9b7a36a700 1 -- 172.21.15.110:0/2236335057 wait complete. 2024-04-25T15:40:11.737 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying mon service with default placement... 2024-04-25T15:40:12.348 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:12 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:12.348 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:12 smithi110 ceph-mon[26344]: Added host smithi110 2024-04-25T15:40:12.348 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:12 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:40:13.139 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.113+0000 7f99a184a700 1 Processor -- start 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.113+0000 7f99a184a700 1 -- start start 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.113+0000 7f99a184a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.113+0000 7f99a184a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c103160 con 0x7f999c102810 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.113+0000 7f999affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.113+0000 7f999affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44742/0 (socket says 172.21.15.110:44742) 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.113+0000 7f999affd700 1 -- 172.21.15.110:0/1928647417 learned_addr learned my addr 172.21.15.110:0/1928647417 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.114+0000 7f999affd700 1 -- 172.21.15.110:0/1928647417 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f999c1032a0 con 0x7f999c102810 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.114+0000 7f999affd700 1 --2- 172.21.15.110:0/1928647417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c102c20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f998c0098e0 tx=0x7f998c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4200fe9c9694f90a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.114+0000 7f999a7fc700 1 -- 172.21.15.110:0/1928647417 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f998c00b860 con 0x7f999c102810 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.114+0000 7f999a7fc700 1 -- 172.21.15.110:0/1928647417 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f998c00be50 con 0x7f999c102810 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.114+0000 7f999a7fc700 1 -- 172.21.15.110:0/1928647417 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f998c010380 con 0x7f999c102810 2024-04-25T15:40:13.140 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 -- 172.21.15.110:0/1928647417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 msgr2=0x7f999c102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 --2- 172.21.15.110:0/1928647417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c102c20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f998c0098e0 tx=0x7f998c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 -- 172.21.15.110:0/1928647417 shutdown_connections 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 --2- 172.21.15.110:0/1928647417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c102c20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 -- 172.21.15.110:0/1928647417 >> 172.21.15.110:0/1928647417 conn(0x7f999c0fdfd0 msgr2=0x7f999c1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 -- 172.21.15.110:0/1928647417 shutdown_connections 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 -- 172.21.15.110:0/1928647417 wait complete. 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 Processor -- start 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.115+0000 7f99a184a700 1 -- start start 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f99a184a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c1116a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f99a184a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999c111be0 con 0x7f999c102810 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f999affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c1116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f999affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c1116a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44746/0 (socket says 172.21.15.110:44746) 2024-04-25T15:40:13.141 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f999affd700 1 -- 172.21.15.110:0/1408937486 learned_addr learned my addr 172.21.15.110:0/1408937486 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f999affd700 1 -- 172.21.15.110:0/1408937486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c0094f0 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f999affd700 1 --2- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c1116a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f998c00ba90 tx=0x7f998c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f9993fff700 1 -- 172.21.15.110:0/1408937486 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f998c0107c0 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f9993fff700 1 -- 172.21.15.110:0/1408937486 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f998c010db0 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f9993fff700 1 -- 172.21.15.110:0/1408937486 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f998c01add0 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999c111de0 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.116+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999c112950 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.117+0000 7f9993fff700 1 -- 172.21.15.110:0/1408937486 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f998c02e440 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.117+0000 7f9993fff700 1 --2- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9988036850 0x7f9988038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.117+0000 7f9993fff700 1 -- 172.21.15.110:0/1408937486 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f998c04a7f0 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.117+0000 7f99927fc700 1 --2- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9988036850 0x7f9988038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.118+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f999c05f5c0 con 0x7f999c102810 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.118+0000 7f99927fc700 1 --2- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9988036850 0x7f9988038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9984009990 tx=0x7f9984006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:13.142 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.122+0000 7f9993fff700 1 -- 172.21.15.110:0/1408937486 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f998c018940 con 0x7f999c102810 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.330+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f999c0ff200 con 0x7f9988036850 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.334+0000 7f9993fff700 1 -- 172.21.15.110:0/1408937486 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f999c0ff200 con 0x7f9988036850 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.336+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9988036850 msgr2=0x7f9988038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.336+0000 7f99a184a700 1 --2- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9988036850 0x7f9988038d00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9984009990 tx=0x7f9984006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.337+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 msgr2=0x7f999c1116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.337+0000 7f99a184a700 1 --2- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c1116a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f998c00ba90 tx=0x7f998c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.337+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 shutdown_connections 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.337+0000 7f99a184a700 1 --2- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f9988036850 0x7f9988038d00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.337+0000 7f99a184a700 1 --2- 172.21.15.110:0/1408937486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f999c102810 0x7f999c1116a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.337+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 >> 172.21.15.110:0/1408937486 conn(0x7f999c0fdfd0 msgr2=0x7f999c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.337+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 shutdown_connections 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:12.337+0000 7f99a184a700 1 -- 172.21.15.110:0/1408937486 wait complete. 2024-04-25T15:40:13.143 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying mgr service with default placement... 2024-04-25T15:40:13.398 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:13 smithi110 ceph-mon[26344]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:13.398 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:13 smithi110 ceph-mon[26344]: Saving service mon spec with placement count:5 2024-04-25T15:40:13.398 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:13 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:14 smithi110 ceph-mon[26344]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:14 smithi110 ceph-mon[26344]: Saving service mgr spec with placement count:2 2024-04-25T15:40:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:14 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:15.169 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T15:40:15.169 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.625+0000 7f21d0dd7700 1 Processor -- start 2024-04-25T15:40:15.169 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.625+0000 7f21d0dd7700 1 -- start start 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21d0dd7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc077230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21d0dd7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc079090 con 0x7f21cc078c80 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21ca59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc077230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21ca59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc077230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44754/0 (socket says 172.21.15.110:44754) 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21ca59c700 1 -- 172.21.15.110:0/3913468515 learned_addr learned my addr 172.21.15.110:0/3913468515 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21ca59c700 1 -- 172.21.15.110:0/3913468515 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc077770 con 0x7f21cc078c80 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21ca59c700 1 --2- 172.21.15.110:0/3913468515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc077230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21c00098e0 tx=0x7f21c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaf209cbc539bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21c959a700 1 -- 172.21.15.110:0/3913468515 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21c000b860 con 0x7f21cc078c80 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21c959a700 1 -- 172.21.15.110:0/3913468515 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f21c000be50 con 0x7f21cc078c80 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.626+0000 7f21c959a700 1 -- 172.21.15.110:0/3913468515 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21c0010380 con 0x7f21cc078c80 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.627+0000 7f21d0dd7700 1 -- 172.21.15.110:0/3913468515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 msgr2=0x7f21cc077230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.627+0000 7f21d0dd7700 1 --2- 172.21.15.110:0/3913468515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc077230 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f21c00098e0 tx=0x7f21c0009250 comp rx=0 tx=0).stop 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.627+0000 7f21d0dd7700 1 -- 172.21.15.110:0/3913468515 shutdown_connections 2024-04-25T15:40:15.170 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.627+0000 7f21d0dd7700 1 --2- 172.21.15.110:0/3913468515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc077230 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.627+0000 7f21d0dd7700 1 -- 172.21.15.110:0/3913468515 >> 172.21.15.110:0/3913468515 conn(0x7f21cc0fe010 msgr2=0x7f21cc100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.627+0000 7f21d0dd7700 1 -- 172.21.15.110:0/3913468515 shutdown_connections 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.627+0000 7f21d0dd7700 1 -- 172.21.15.110:0/3913468515 wait complete. 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21d0dd7700 1 Processor -- start 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21d0dd7700 1 -- start start 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21d0dd7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc1137b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21d0dd7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc113cf0 con 0x7f21cc078c80 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21ca59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc1137b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21ca59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc1137b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44762/0 (socket says 172.21.15.110:44762) 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21ca59c700 1 -- 172.21.15.110:0/234997766 learned_addr learned my addr 172.21.15.110:0/234997766 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21ca59c700 1 -- 172.21.15.110:0/234997766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c00094f0 con 0x7f21cc078c80 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21ca59c700 1 --2- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc1137b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006b00 tx=0x7f21c0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21baffd700 1 -- 172.21.15.110:0/234997766 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21c00107c0 con 0x7f21cc078c80 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21baffd700 1 -- 172.21.15.110:0/234997766 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1044+0+0 (secure 0 0 0) 0x7f21c0010db0 con 0x7f21cc078c80 2024-04-25T15:40:15.171 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21baffd700 1 -- 172.21.15.110:0/234997766 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21c001acf0 con 0x7f21cc078c80 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.628+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc113ef0 con 0x7f21cc078c80 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.629+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc114a60 con 0x7f21cc078c80 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.629+0000 7f21baffd700 1 -- 172.21.15.110:0/234997766 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f21c0010920 con 0x7f21cc078c80 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.629+0000 7f21baffd700 1 --2- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f21b40368a0 0x7f21b4038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.630+0000 7f21baffd700 1 -- 172.21.15.110:0/234997766 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f21c004a590 con 0x7f21cc078c80 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.630+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc05f5c0 con 0x7f21cc078c80 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.630+0000 7f21c9d9b700 1 --2- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f21b40368a0 0x7f21b4038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.630+0000 7f21c9d9b700 1 --2- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f21b40368a0 0x7f21b4038d50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009990 tx=0x7f21bc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.634+0000 7f21baffd700 1 -- 172.21.15.110:0/234997766 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f21c001a530 con 0x7f21cc078c80 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.833+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f21cc0ff200 con 0x7f21b40368a0 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.837+0000 7f21baffd700 1 -- 172.21.15.110:0/234997766 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f21cc0ff200 con 0x7f21b40368a0 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.839+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f21b40368a0 msgr2=0x7f21b4038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.839+0000 7f21d0dd7700 1 --2- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f21b40368a0 0x7f21b4038d50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f21bc009990 tx=0x7f21bc006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.839+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 msgr2=0x7f21cc1137b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:15.172 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.839+0000 7f21d0dd7700 1 --2- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc1137b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f21c0006b00 tx=0x7f21c0009250 comp rx=0 tx=0).stop 2024-04-25T15:40:15.173 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.839+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 shutdown_connections 2024-04-25T15:40:15.173 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.839+0000 7f21d0dd7700 1 --2- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f21b40368a0 0x7f21b4038d50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:15.173 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.839+0000 7f21d0dd7700 1 --2- 172.21.15.110:0/234997766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f21cc078c80 0x7f21cc1137b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:15.173 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.839+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 >> 172.21.15.110:0/234997766 conn(0x7f21cc0fe010 msgr2=0x7f21cc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:15.173 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.840+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 shutdown_connections 2024-04-25T15:40:15.173 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:13.840+0000 7f21d0dd7700 1 -- 172.21.15.110:0/234997766 wait complete. 2024-04-25T15:40:15.173 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying crash service with default placement... 2024-04-25T15:40:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:15 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:15 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:15 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:16 smithi110 ceph-mon[26344]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:16 smithi110 ceph-mon[26344]: Saving service crash spec with placement * 2024-04-25T15:40:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:16 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:17.173 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.610+0000 7fe69fc7f700 1 Processor -- start 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.610+0000 7fe69fc7f700 1 -- start start 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.610+0000 7fe69fc7f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698077250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.610+0000 7fe69fc7f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6980790b0 con 0x7fe698078ca0 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.611+0000 7fe69da1b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698077250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.611+0000 7fe69da1b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698077250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44766/0 (socket says 172.21.15.110:44766) 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.611+0000 7fe69da1b700 1 -- 172.21.15.110:0/1405569650 learned_addr learned my addr 172.21.15.110:0/1405569650 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.611+0000 7fe69da1b700 1 -- 172.21.15.110:0/1405569650 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe698077790 con 0x7fe698078ca0 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.611+0000 7fe69da1b700 1 --2- 172.21.15.110:0/1405569650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698077250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6940098e0 tx=0x7fe694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97db8c283e31a485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.611+0000 7fe69ca19700 1 -- 172.21.15.110:0/1405569650 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe69400b860 con 0x7fe698078ca0 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.611+0000 7fe69ca19700 1 -- 172.21.15.110:0/1405569650 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe69400be90 con 0x7fe698078ca0 2024-04-25T15:40:17.174 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.611+0000 7fe69ca19700 1 -- 172.21.15.110:0/1405569650 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe6940103c0 con 0x7fe698078ca0 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.612+0000 7fe69fc7f700 1 -- 172.21.15.110:0/1405569650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 msgr2=0x7fe698077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.612+0000 7fe69fc7f700 1 --2- 172.21.15.110:0/1405569650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698077250 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe6940098e0 tx=0x7fe694009250 comp rx=0 tx=0).stop 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.612+0000 7fe69fc7f700 1 -- 172.21.15.110:0/1405569650 shutdown_connections 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.612+0000 7fe69fc7f700 1 --2- 172.21.15.110:0/1405569650 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698077250 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.612+0000 7fe69fc7f700 1 -- 172.21.15.110:0/1405569650 >> 172.21.15.110:0/1405569650 conn(0x7fe6980fe010 msgr2=0x7fe698100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.613+0000 7fe69fc7f700 1 -- 172.21.15.110:0/1405569650 shutdown_connections 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.613+0000 7fe69fc7f700 1 -- 172.21.15.110:0/1405569650 wait complete. 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.613+0000 7fe69fc7f700 1 Processor -- start 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.613+0000 7fe69fc7f700 1 -- start start 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.613+0000 7fe69fc7f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698117c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.613+0000 7fe69fc7f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe698118190 con 0x7fe698078ca0 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe69da1b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698117c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe69da1b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698117c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44770/0 (socket says 172.21.15.110:44770) 2024-04-25T15:40:17.175 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe69da1b700 1 -- 172.21.15.110:0/3746498777 learned_addr learned my addr 172.21.15.110:0/3746498777 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe69da1b700 1 -- 172.21.15.110:0/3746498777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6940094f0 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe69da1b700 1 --2- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698117c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe68e7fc700 1 -- 172.21.15.110:0/3746498777 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe694010630 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe68e7fc700 1 -- 172.21.15.110:0/3746498777 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe694010c60 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe698118390 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe68e7fc700 1 -- 172.21.15.110:0/3746498777 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe69401ad40 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.614+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe698118f00 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.615+0000 7fe68e7fc700 1 -- 172.21.15.110:0/3746498777 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe6940193c0 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.615+0000 7fe68e7fc700 1 --2- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe6840368a0 0x7fe684038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.615+0000 7fe68e7fc700 1 -- 172.21.15.110:0/3746498777 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe694049ff0 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.615+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69805f5c0 con 0x7fe698078ca0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.616+0000 7fe69d21a700 1 --2- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe6840368a0 0x7fe684038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.616+0000 7fe69d21a700 1 --2- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe6840368a0 0x7fe684038d50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe688009990 tx=0x7fe688006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:17.176 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.623+0000 7fe68e7fc700 1 -- 172.21.15.110:0/3746498777 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe694020070 con 0x7fe698078ca0 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.829+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7fe698119a80 con 0x7fe6840368a0 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.833+0000 7fe68e7fc700 1 -- 172.21.15.110:0/3746498777 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fe698119a80 con 0x7fe6840368a0 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe6840368a0 msgr2=0x7fe684038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 --2- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe6840368a0 0x7fe684038d50 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe688009990 tx=0x7fe688006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 msgr2=0x7fe698117c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 --2- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698117c50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe694006fd0 tx=0x7fe694009250 comp rx=0 tx=0).stop 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 shutdown_connections 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 --2- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe6840368a0 0x7fe684038d50 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 --2- 172.21.15.110:0/3746498777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe698078ca0 0x7fe698117c50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 >> 172.21.15.110:0/3746498777 conn(0x7fe6980fe010 msgr2=0x7fe69810efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 shutdown_connections 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:15.835+0000 7fe69fc7f700 1 -- 172.21.15.110:0/3746498777 wait complete. 2024-04-25T15:40:17.177 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying prometheus service with default placement... 2024-04-25T15:40:18.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-25T15:40:18.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.659+0000 7fe08e22f700 1 Processor -- start 2024-04-25T15:40:18.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.659+0000 7fe08e22f700 1 -- start start 2024-04-25T15:40:18.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.660+0000 7fe08e22f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:18.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.660+0000 7fe08e22f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0881059b0 con 0x7fe0880712d0 2024-04-25T15:40:18.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.660+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:18.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.660+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44778/0 (socket says 172.21.15.110:44778) 2024-04-25T15:40:18.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.660+0000 7fe0877fe700 1 -- 172.21.15.110:0/2806333463 learned_addr learned my addr 172.21.15.110:0/2806333463 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.660+0000 7fe0877fe700 1 -- 172.21.15.110:0/2806333463 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe088105af0 con 0x7fe0880712d0 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.660+0000 7fe0877fe700 1 --2- 172.21.15.110:0/2806333463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088105470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe0780098e0 tx=0x7fe078009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211581e391af9b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.660+0000 7fe0867fc700 1 -- 172.21.15.110:0/2806333463 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe07800b860 con 0x7fe0880712d0 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.661+0000 7fe0867fc700 1 -- 172.21.15.110:0/2806333463 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe07800be90 con 0x7fe0880712d0 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.661+0000 7fe0867fc700 1 -- 172.21.15.110:0/2806333463 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0780103c0 con 0x7fe0880712d0 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.661+0000 7fe08e22f700 1 -- 172.21.15.110:0/2806333463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 msgr2=0x7fe088105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.661+0000 7fe08e22f700 1 --2- 172.21.15.110:0/2806333463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088105470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe0780098e0 tx=0x7fe078009250 comp rx=0 tx=0).stop 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.661+0000 7fe08e22f700 1 -- 172.21.15.110:0/2806333463 shutdown_connections 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.661+0000 7fe08e22f700 1 --2- 172.21.15.110:0/2806333463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088105470 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.661+0000 7fe08e22f700 1 -- 172.21.15.110:0/2806333463 >> 172.21.15.110:0/2806333463 conn(0x7fe0880fcea0 msgr2=0x7fe0880ff2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.662+0000 7fe08e22f700 1 -- 172.21.15.110:0/2806333463 shutdown_connections 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.662+0000 7fe08e22f700 1 -- 172.21.15.110:0/2806333463 wait complete. 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.662+0000 7fe08e22f700 1 Processor -- start 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.662+0000 7fe08e22f700 1 -- start start 2024-04-25T15:40:18.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.662+0000 7fe08e22f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088111670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.662+0000 7fe08e22f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe088111bb0 con 0x7fe0880712d0 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe0877fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088111670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44780/0 (socket says 172.21.15.110:44780) 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe0877fe700 1 -- 172.21.15.110:0/4215910746 learned_addr learned my addr 172.21.15.110:0/4215910746 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe0877fe700 1 -- 172.21.15.110:0/4215910746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780094f0 con 0x7fe0880712d0 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe0877fe700 1 --2- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088111670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0780105b0 tx=0x7fe078009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe06ffff700 1 -- 172.21.15.110:0/4215910746 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe078010840 con 0x7fe0880712d0 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe06ffff700 1 -- 172.21.15.110:0/4215910746 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe078010e70 con 0x7fe0880712d0 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe088111db0 con 0x7fe0880712d0 2024-04-25T15:40:18.473 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe06ffff700 1 -- 172.21.15.110:0/4215910746 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe0780243f0 con 0x7fe0880712d0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.663+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe088112920 con 0x7fe0880712d0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.664+0000 7fe06ffff700 1 -- 172.21.15.110:0/4215910746 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe078024550 con 0x7fe0880712d0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.664+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08805f5c0 con 0x7fe0880712d0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.664+0000 7fe06ffff700 1 --2- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe0700368a0 0x7fe070038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.665+0000 7fe06ffff700 1 -- 172.21.15.110:0/4215910746 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe07804a560 con 0x7fe0880712d0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.665+0000 7fe086ffd700 1 --2- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe0700368a0 0x7fe070038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.665+0000 7fe086ffd700 1 --2- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe0700368a0 0x7fe070038d50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009990 tx=0x7fe07c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.669+0000 7fe06ffff700 1 -- 172.21.15.110:0/4215910746 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe078018940 con 0x7fe0880712d0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.885+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0880fe120 con 0x7fe0700368a0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.889+0000 7fe06ffff700 1 -- 172.21.15.110:0/4215910746 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7fe0880fe120 con 0x7fe0700368a0 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.892+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe0700368a0 msgr2=0x7fe070038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.892+0000 7fe08e22f700 1 --2- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe0700368a0 0x7fe070038d50 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe07c009990 tx=0x7fe07c006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.892+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 msgr2=0x7fe088111670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:18.474 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.892+0000 7fe08e22f700 1 --2- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088111670 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe0780105b0 tx=0x7fe078009250 comp rx=0 tx=0).stop 2024-04-25T15:40:18.475 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.892+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 shutdown_connections 2024-04-25T15:40:18.475 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.892+0000 7fe08e22f700 1 --2- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe0700368a0 0x7fe070038d50 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:18.475 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.892+0000 7fe08e22f700 1 --2- 172.21.15.110:0/4215910746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0880712d0 0x7fe088111670 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:18.475 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.892+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 >> 172.21.15.110:0/4215910746 conn(0x7fe0880fcea0 msgr2=0x7fe0880fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:18.475 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.893+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 shutdown_connections 2024-04-25T15:40:18.475 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:17.893+0000 7fe08e22f700 1 -- 172.21.15.110:0/4215910746 wait complete. 2024-04-25T15:40:18.475 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying grafana service with default placement... 2024-04-25T15:40:18.727 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:18 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:18.727 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:18 smithi110 ceph-mon[26344]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:18.727 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:18 smithi110 ceph-mon[26344]: Saving service prometheus spec with placement count:1 2024-04-25T15:40:18.727 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:18 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:19.732 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-25T15:40:19.732 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.947+0000 7fddd42be700 1 Processor -- start 2024-04-25T15:40:19.732 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.947+0000 7fddd42be700 1 -- start start 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.947+0000 7fddd42be700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.947+0000 7fddd42be700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc103160 con 0x7fddcc102810 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.947+0000 7fddd205a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.947+0000 7fddd205a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44792/0 (socket says 172.21.15.110:44792) 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.947+0000 7fddd205a700 1 -- 172.21.15.110:0/3493808469 learned_addr learned my addr 172.21.15.110:0/3493808469 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.947+0000 7fddd205a700 1 -- 172.21.15.110:0/3493808469 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddcc1032a0 con 0x7fddcc102810 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.948+0000 7fddd205a700 1 --2- 172.21.15.110:0/3493808469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc102c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fddc80098e0 tx=0x7fddc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b071958d220958 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.948+0000 7fddd1058700 1 -- 172.21.15.110:0/3493808469 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddc800b860 con 0x7fddcc102810 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.948+0000 7fddd1058700 1 -- 172.21.15.110:0/3493808469 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddc800be90 con 0x7fddcc102810 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.948+0000 7fddd1058700 1 -- 172.21.15.110:0/3493808469 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddc80103c0 con 0x7fddcc102810 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.948+0000 7fddd42be700 1 -- 172.21.15.110:0/3493808469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 msgr2=0x7fddcc102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.948+0000 7fddd42be700 1 --2- 172.21.15.110:0/3493808469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc102c20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fddc80098e0 tx=0x7fddc8009250 comp rx=0 tx=0).stop 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 -- 172.21.15.110:0/3493808469 shutdown_connections 2024-04-25T15:40:19.733 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 --2- 172.21.15.110:0/3493808469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc102c20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 -- 172.21.15.110:0/3493808469 >> 172.21.15.110:0/3493808469 conn(0x7fddcc0fdfd0 msgr2=0x7fddcc1003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 -- 172.21.15.110:0/3493808469 shutdown_connections 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 -- 172.21.15.110:0/3493808469 wait complete. 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 Processor -- start 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 -- start start 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc1116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd42be700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddcc111c00 con 0x7fddcc102810 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd205a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc1116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd205a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc1116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44802/0 (socket says 172.21.15.110:44802) 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.949+0000 7fddd205a700 1 -- 172.21.15.110:0/1754055212 learned_addr learned my addr 172.21.15.110:0/1754055212 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:19.734 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.950+0000 7fddd205a700 1 -- 172.21.15.110:0/1754055212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80094f0 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.950+0000 7fddd205a700 1 --2- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc1116c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fddc8006b00 tx=0x7fddc8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.950+0000 7fddc2ffd700 1 -- 172.21.15.110:0/1754055212 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddc8010840 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.950+0000 7fddc2ffd700 1 -- 172.21.15.110:0/1754055212 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddc8010e70 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.950+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddcc111e00 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.950+0000 7fddc2ffd700 1 -- 172.21.15.110:0/1754055212 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fddc801ae70 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.950+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddcc112970 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.951+0000 7fddc2ffd700 1 -- 172.21.15.110:0/1754055212 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fddc8024480 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.951+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddcc05f5c0 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.951+0000 7fddc2ffd700 1 --2- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fddb80368a0 0x7fddb8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.951+0000 7fddc2ffd700 1 -- 172.21.15.110:0/1754055212 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fddc804ac00 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.951+0000 7fddd1859700 1 --2- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fddb80368a0 0x7fddb8038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.951+0000 7fddd1859700 1 --2- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fddb80368a0 0x7fddb8038d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009990 tx=0x7fddbc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:18.956+0000 7fddc2ffd700 1 -- 172.21.15.110:0/1754055212 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fddc802c570 con 0x7fddcc102810 2024-04-25T15:40:19.735 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.158+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7fddcc0ff230 con 0x7fddb80368a0 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.163+0000 7fddc2ffd700 1 -- 172.21.15.110:0/1754055212 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fddcc0ff230 con 0x7fddb80368a0 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.164+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fddb80368a0 msgr2=0x7fddb8038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.164+0000 7fddd42be700 1 --2- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fddb80368a0 0x7fddb8038d50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fddbc009990 tx=0x7fddbc006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.164+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 msgr2=0x7fddcc1116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.164+0000 7fddd42be700 1 --2- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc1116c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fddc8006b00 tx=0x7fddc8009250 comp rx=0 tx=0).stop 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.165+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 shutdown_connections 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.165+0000 7fddd42be700 1 --2- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fddb80368a0 0x7fddb8038d50 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.165+0000 7fddd42be700 1 --2- 172.21.15.110:0/1754055212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fddcc102810 0x7fddcc1116c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.165+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 >> 172.21.15.110:0/1754055212 conn(0x7fddcc0fdfd0 msgr2=0x7fddcc0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.165+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 shutdown_connections 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:19.165+0000 7fddd42be700 1 -- 172.21.15.110:0/1754055212 wait complete. 2024-04-25T15:40:19.736 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying node-exporter service with default placement... 2024-04-25T15:40:20.329 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:20 smithi110 ceph-mon[26344]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:20.329 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:20 smithi110 ceph-mon[26344]: Saving service grafana spec with placement count:1 2024-04-25T15:40:20.329 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:20 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:21.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-25T15:40:21.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.424+0000 7f7357afd700 1 Processor -- start 2024-04-25T15:40:21.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.424+0000 7f7357afd700 1 -- start start 2024-04-25T15:40:21.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.424+0000 7f7357afd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350102110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:21.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.424+0000 7f7357afd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7350102650 con 0x7f7350101d00 2024-04-25T15:40:21.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.424+0000 7f7355899700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350102110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:21.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.424+0000 7f7355899700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350102110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55214/0 (socket says 172.21.15.110:55214) 2024-04-25T15:40:21.468 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.424+0000 7f7355899700 1 -- 172.21.15.110:0/2880893347 learned_addr learned my addr 172.21.15.110:0/2880893347 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.425+0000 7f7355899700 1 -- 172.21.15.110:0/2880893347 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7350102790 con 0x7f7350101d00 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.425+0000 7f7355899700 1 --2- 172.21.15.110:0/2880893347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350102110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f734c0098e0 tx=0x7f734c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6df5a3c37bd01a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.425+0000 7f7354897700 1 -- 172.21.15.110:0/2880893347 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f734c00b860 con 0x7f7350101d00 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.425+0000 7f7354897700 1 -- 172.21.15.110:0/2880893347 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f734c00be90 con 0x7f7350101d00 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.425+0000 7f7354897700 1 -- 172.21.15.110:0/2880893347 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f734c0103c0 con 0x7f7350101d00 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.425+0000 7f7357afd700 1 -- 172.21.15.110:0/2880893347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 msgr2=0x7f7350102110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.425+0000 7f7357afd700 1 --2- 172.21.15.110:0/2880893347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350102110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f734c0098e0 tx=0x7f734c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 -- 172.21.15.110:0/2880893347 shutdown_connections 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 --2- 172.21.15.110:0/2880893347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350102110 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:21.469 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 -- 172.21.15.110:0/2880893347 >> 172.21.15.110:0/2880893347 conn(0x7f73500fc6f0 msgr2=0x7f73500feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 -- 172.21.15.110:0/2880893347 shutdown_connections 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 -- 172.21.15.110:0/2880893347 wait complete. 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 Processor -- start 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 -- start start 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350101520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7357afd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7350114300 con 0x7f7350101d00 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7355899700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350101520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7355899700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350101520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55222/0 (socket says 172.21.15.110:55222) 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.426+0000 7f7355899700 1 -- 172.21.15.110:0/720229143 learned_addr learned my addr 172.21.15.110:0/720229143 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.427+0000 7f7355899700 1 -- 172.21.15.110:0/720229143 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c0094f0 con 0x7f7350101d00 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.427+0000 7f7355899700 1 --2- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350101520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f734c0090b0 tx=0x7f734c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.427+0000 7f73467fc700 1 -- 172.21.15.110:0/720229143 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f734c010840 con 0x7f7350101d00 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.427+0000 7f73467fc700 1 -- 172.21.15.110:0/720229143 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f734c010e70 con 0x7f7350101d00 2024-04-25T15:40:21.470 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.427+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7350114440 con 0x7f7350101d00 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.427+0000 7f73467fc700 1 -- 172.21.15.110:0/720229143 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f734c024e40 con 0x7f7350101d00 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.427+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73501148d0 con 0x7f7350101d00 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.428+0000 7f73467fc700 1 -- 172.21.15.110:0/720229143 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f734c01a510 con 0x7f7350101d00 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.428+0000 7f73467fc700 1 --2- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f733c036800 0x7f733c038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.428+0000 7f73467fc700 1 -- 172.21.15.110:0/720229143 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f734c04a7f0 con 0x7f7350101d00 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.428+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735004bed0 con 0x7f7350101d00 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.428+0000 7f7355098700 1 --2- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f733c036800 0x7f733c038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.428+0000 7f7355098700 1 --2- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f733c036800 0x7f733c038cb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7340009990 tx=0x7f7340006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.433+0000 7f73467fc700 1 -- 172.21.15.110:0/720229143 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f734c018940 con 0x7f7350101d00 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.634+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f73500fd8e0 con 0x7f733c036800 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.638+0000 7f73467fc700 1 -- 172.21.15.110:0/720229143 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f73500fd8e0 con 0x7f733c036800 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f733c036800 msgr2=0x7f733c038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 --2- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f733c036800 0x7f733c038cb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f7340009990 tx=0x7f7340006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 msgr2=0x7f7350101520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:21.471 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 --2- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350101520 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f734c0090b0 tx=0x7f734c009250 comp rx=0 tx=0).stop 2024-04-25T15:40:21.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 shutdown_connections 2024-04-25T15:40:21.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 --2- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f733c036800 0x7f733c038cb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:21.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 --2- 172.21.15.110:0/720229143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7350101d00 0x7f7350101520 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:21.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 >> 172.21.15.110:0/720229143 conn(0x7f73500fc6f0 msgr2=0x7f73500fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:21.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 shutdown_connections 2024-04-25T15:40:21.472 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:20.640+0000 7f7357afd700 1 -- 172.21.15.110:0/720229143 wait complete. 2024-04-25T15:40:21.472 INFO:teuthology.orchestra.run.smithi110.stdout:Deploying alertmanager service with default placement... 2024-04-25T15:40:21.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:21 smithi110 ceph-mon[26344]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:21.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:21 smithi110 ceph-mon[26344]: Saving service node-exporter spec with placement * 2024-04-25T15:40:21.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:21 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.866+0000 7fe3e8f4d700 1 Processor -- start 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.866+0000 7fe3e8f4d700 1 -- start start 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.867+0000 7fe3e8f4d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e4102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.867+0000 7fe3e8f4d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4103160 con 0x7fe3e4102810 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.867+0000 7fe3e259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e4102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.867+0000 7fe3e259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e4102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55226/0 (socket says 172.21.15.110:55226) 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.867+0000 7fe3e259c700 1 -- 172.21.15.110:0/723778451 learned_addr learned my addr 172.21.15.110:0/723778451 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.867+0000 7fe3e259c700 1 -- 172.21.15.110:0/723778451 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e41032a0 con 0x7fe3e4102810 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.867+0000 7fe3e259c700 1 --2- 172.21.15.110:0/723778451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e4102c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40098e0 tx=0x7fe3d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d1ede7c96edd28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.867+0000 7fe3e159a700 1 -- 172.21.15.110:0/723778451 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3d400b860 con 0x7fe3e4102810 2024-04-25T15:40:22.666 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e159a700 1 -- 172.21.15.110:0/723778451 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3d400be90 con 0x7fe3e4102810 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e159a700 1 -- 172.21.15.110:0/723778451 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3d40103c0 con 0x7fe3e4102810 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/723778451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 msgr2=0x7fe3e4102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e8f4d700 1 --2- 172.21.15.110:0/723778451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e4102c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe3d40098e0 tx=0x7fe3d4009250 comp rx=0 tx=0).stop 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/723778451 shutdown_connections 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e8f4d700 1 --2- 172.21.15.110:0/723778451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e4102c20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/723778451 >> 172.21.15.110:0/723778451 conn(0x7fe3e40fdfd0 msgr2=0x7fe3e41003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/723778451 shutdown_connections 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.868+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/723778451 wait complete. 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e8f4d700 1 Processor -- start 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e8f4d700 1 -- start start 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e8f4d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e41116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e8f4d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e4111c00 con 0x7fe3e4102810 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e41116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e41116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55238/0 (socket says 172.21.15.110:55238) 2024-04-25T15:40:22.667 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e259c700 1 -- 172.21.15.110:0/4053698502 learned_addr learned my addr 172.21.15.110:0/4053698502 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e259c700 1 -- 172.21.15.110:0/4053698502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d40094f0 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3e259c700 1 --2- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e41116c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4006b00 tx=0x7fe3d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.869+0000 7fe3d2ffd700 1 -- 172.21.15.110:0/4053698502 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3d4010840 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.870+0000 7fe3d2ffd700 1 -- 172.21.15.110:0/4053698502 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3d4010e70 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.870+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e4111e00 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.870+0000 7fe3d2ffd700 1 -- 172.21.15.110:0/4053698502 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe3d401ae70 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.870+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e4112970 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.870+0000 7fe3d2ffd700 1 -- 172.21.15.110:0/4053698502 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7fe3d40244d0 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.870+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e405f5c0 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.870+0000 7fe3d2ffd700 1 --2- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe3cc036850 0x7fe3cc038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.871+0000 7fe3d2ffd700 1 -- 172.21.15.110:0/4053698502 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fe3d404ab70 con 0x7fe3e4102810 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.871+0000 7fe3e1d9b700 1 --2- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe3cc036850 0x7fe3cc038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.871+0000 7fe3e1d9b700 1 --2- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe3cc036850 0x7fe3cc038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009990 tx=0x7fe3d8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:22.668 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:21.875+0000 7fe3d2ffd700 1 -- 172.21.15.110:0/4053698502 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe3d402e440 con 0x7fe3e4102810 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.072+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3e40ff230 con 0x7fe3cc036850 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.076+0000 7fe3d2ffd700 1 -- 172.21.15.110:0/4053698502 <== mgr.14118 v2:172.21.15.110:6800/1116386591 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7fe3e40ff230 con 0x7fe3cc036850 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe3cc036850 msgr2=0x7fe3cc038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 --2- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe3cc036850 0x7fe3cc038d00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8009990 tx=0x7fe3d8006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 msgr2=0x7fe3e41116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 --2- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e41116c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4006b00 tx=0x7fe3d4009250 comp rx=0 tx=0).stop 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 shutdown_connections 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 --2- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fe3cc036850 0x7fe3cc038d00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 --2- 172.21.15.110:0/4053698502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e4102810 0x7fe3e41116c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 >> 172.21.15.110:0/4053698502 conn(0x7fe3e40fdfd0 msgr2=0x7fe3e40feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 shutdown_connections 2024-04-25T15:40:22.669 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:22.079+0000 7fe3e8f4d700 1 -- 172.21.15.110:0/4053698502 wait complete. 2024-04-25T15:40:23.312 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:23 smithi110 ceph-mon[26344]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:23.312 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:23 smithi110 ceph-mon[26344]: Saving service alertmanager spec with placement count:1 2024-04-25T15:40:23.312 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:23 smithi110 ceph-mon[26344]: from='mgr.14118 172.21.15.110:0/3021288117' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:23.855 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff8633a0700 1 Processor -- start 2024-04-25T15:40:23.855 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff8633a0700 1 -- start start 2024-04-25T15:40:23.855 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff8633a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c105470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff8633a0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c1059b0 con 0x7ff85c0712b0 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff86113c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c105470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff86113c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c105470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55254/0 (socket says 172.21.15.110:55254) 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff86113c700 1 -- 172.21.15.110:0/1311322069 learned_addr learned my addr 172.21.15.110:0/1311322069 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff86113c700 1 -- 172.21.15.110:0/1311322069 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85c105af0 con 0x7ff85c0712b0 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.112+0000 7ff86113c700 1 --2- 172.21.15.110:0/1311322069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c105470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8580098e0 tx=0x7ff858009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b07e42c5a0c1e56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff853fff700 1 -- 172.21.15.110:0/1311322069 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff85800b860 con 0x7ff85c0712b0 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff853fff700 1 -- 172.21.15.110:0/1311322069 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff85800be90 con 0x7ff85c0712b0 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff853fff700 1 -- 172.21.15.110:0/1311322069 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8580103c0 con 0x7ff85c0712b0 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff8633a0700 1 -- 172.21.15.110:0/1311322069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 msgr2=0x7ff85c105470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff8633a0700 1 --2- 172.21.15.110:0/1311322069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c105470 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff8580098e0 tx=0x7ff858009250 comp rx=0 tx=0).stop 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff8633a0700 1 -- 172.21.15.110:0/1311322069 shutdown_connections 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff8633a0700 1 --2- 172.21.15.110:0/1311322069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c105470 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:23.856 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff8633a0700 1 -- 172.21.15.110:0/1311322069 >> 172.21.15.110:0/1311322069 conn(0x7ff85c0fcec0 msgr2=0x7ff85c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.113+0000 7ff8633a0700 1 -- 172.21.15.110:0/1311322069 shutdown_connections 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.114+0000 7ff8633a0700 1 -- 172.21.15.110:0/1311322069 wait complete. 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.114+0000 7ff8633a0700 1 Processor -- start 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.114+0000 7ff8633a0700 1 -- start start 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.114+0000 7ff8633a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c1137d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.114+0000 7ff8633a0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85c113d10 con 0x7ff85c0712b0 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.114+0000 7ff86113c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c1137d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.114+0000 7ff86113c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c1137d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55260/0 (socket says 172.21.15.110:55260) 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.114+0000 7ff86113c700 1 -- 172.21.15.110:0/389524823 learned_addr learned my addr 172.21.15.110:0/389524823 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.115+0000 7ff86113c700 1 -- 172.21.15.110:0/389524823 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8580094f0 con 0x7ff85c0712b0 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.115+0000 7ff86113c700 1 --2- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c1137d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8580094c0 tx=0x7ff858009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.115+0000 7ff851ffb700 1 -- 172.21.15.110:0/389524823 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff858010890 con 0x7ff85c0712b0 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.115+0000 7ff851ffb700 1 -- 172.21.15.110:0/389524823 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff858010ec0 con 0x7ff85c0712b0 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.115+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85c113f10 con 0x7ff85c0712b0 2024-04-25T15:40:23.857 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.115+0000 7ff851ffb700 1 -- 172.21.15.110:0/389524823 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff8580243f0 con 0x7ff85c0712b0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.115+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff85c114a80 con 0x7ff85c0712b0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.116+0000 7ff851ffb700 1 -- 172.21.15.110:0/389524823 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7ff85802d3f0 con 0x7ff85c0712b0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.116+0000 7ff851ffb700 1 --2- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7ff8480368a0 0x7ff848038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.116+0000 7ff851ffb700 1 -- 172.21.15.110:0/389524823 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7ff85801a7e0 con 0x7ff85c0712b0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.116+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85c05f5c0 con 0x7ff85c0712b0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.116+0000 7ff86093b700 1 --2- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7ff8480368a0 0x7ff848038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.117+0000 7ff86093b700 1 --2- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7ff8480368a0 0x7ff848038d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009990 tx=0x7ff84c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.121+0000 7ff851ffb700 1 -- 172.21.15.110:0/389524823 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff85802b8e0 con 0x7ff85c0712b0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.312+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ff85c04bed0 con 0x7ff85c0712b0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.315+0000 7ff851ffb700 1 -- 172.21.15.110:0/389524823 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff85802c350 con 0x7ff85c0712b0 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.317+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7ff8480368a0 msgr2=0x7ff848038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 --2- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7ff8480368a0 0x7ff848038d50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009990 tx=0x7ff84c006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 msgr2=0x7ff85c1137d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:23.858 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 --2- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c1137d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff8580094c0 tx=0x7ff858009250 comp rx=0 tx=0).stop 2024-04-25T15:40:23.859 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 shutdown_connections 2024-04-25T15:40:23.859 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 --2- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7ff8480368a0 0x7ff848038d50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:23.859 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 --2- 172.21.15.110:0/389524823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff85c0712b0 0x7ff85c1137d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:23.859 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 >> 172.21.15.110:0/389524823 conn(0x7ff85c0fcec0 msgr2=0x7ff85c0fd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:23.859 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 shutdown_connections 2024-04-25T15:40:23.859 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:23.318+0000 7ff8633a0700 1 -- 172.21.15.110:0/389524823 wait complete. 2024-04-25T15:40:24.241 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:24 smithi110 ceph-mon[26344]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:40:24.242 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:24 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/389524823' entity='client.admin' 2024-04-25T15:40:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.282+0000 7f0074d3f700 1 Processor -- start 2024-04-25T15:40:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.282+0000 7f0074d3f700 1 -- start start 2024-04-25T15:40:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.282+0000 7f0074d3f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070077210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.282+0000 7f0074d3f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0070079070 con 0x7f0070078c60 2024-04-25T15:40:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.282+0000 7f006e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070077210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.282+0000 7f006e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070077210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55262/0 (socket says 172.21.15.110:55262) 2024-04-25T15:40:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.282+0000 7f006e59c700 1 -- 172.21.15.110:0/4008301502 learned_addr learned my addr 172.21.15.110:0/4008301502 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:25.070 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.283+0000 7f006e59c700 1 -- 172.21.15.110:0/4008301502 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0070077750 con 0x7f0070078c60 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.283+0000 7f006e59c700 1 --2- 172.21.15.110:0/4008301502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070077210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00600098e0 tx=0x7f0060009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f698f2818dfa54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.283+0000 7f006d59a700 1 -- 172.21.15.110:0/4008301502 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f006000b860 con 0x7f0070078c60 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.283+0000 7f006d59a700 1 -- 172.21.15.110:0/4008301502 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f006000be90 con 0x7f0070078c60 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.283+0000 7f006d59a700 1 -- 172.21.15.110:0/4008301502 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00600103c0 con 0x7f0070078c60 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 -- 172.21.15.110:0/4008301502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 msgr2=0x7f0070077210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 --2- 172.21.15.110:0/4008301502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070077210 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f00600098e0 tx=0x7f0060009250 comp rx=0 tx=0).stop 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 -- 172.21.15.110:0/4008301502 shutdown_connections 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 --2- 172.21.15.110:0/4008301502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070077210 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 -- 172.21.15.110:0/4008301502 >> 172.21.15.110:0/4008301502 conn(0x7f00700fe060 msgr2=0x7f0070100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 -- 172.21.15.110:0/4008301502 shutdown_connections 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 -- 172.21.15.110:0/4008301502 wait complete. 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 Processor -- start 2024-04-25T15:40:25.071 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 -- start start 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070113840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f0074d3f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0070113d80 con 0x7f0070078c60 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.284+0000 7f006e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070113840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f006e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070113840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55268/0 (socket says 172.21.15.110:55268) 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f006e59c700 1 -- 172.21.15.110:0/1827775589 learned_addr learned my addr 172.21.15.110:0/1827775589 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f006e59c700 1 -- 172.21.15.110:0/1827775589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00600094f0 con 0x7f0070078c60 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f006e59c700 1 --2- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070113840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0060006d40 tx=0x7f0060009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f005effd700 1 -- 172.21.15.110:0/1827775589 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0060010840 con 0x7f0070078c60 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f005effd700 1 -- 172.21.15.110:0/1827775589 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0060010e70 con 0x7f0070078c60 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0070113f80 con 0x7f0070078c60 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f005effd700 1 -- 172.21.15.110:0/1827775589 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0060024ea0 con 0x7f0070078c60 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.285+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0070114af0 con 0x7f0070078c60 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.286+0000 7f005effd700 1 -- 172.21.15.110:0/1827775589 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f006001a4e0 con 0x7f0070078c60 2024-04-25T15:40:25.072 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.286+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007005f5c0 con 0x7f0070078c60 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.286+0000 7f005effd700 1 --2- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0058036850 0x7f0058038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.286+0000 7f005effd700 1 -- 172.21.15.110:0/1827775589 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f006004a800 con 0x7f0070078c60 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.286+0000 7f006dd9b700 1 --2- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0058036850 0x7f0058038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.287+0000 7f006dd9b700 1 --2- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0058036850 0x7f0058038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0064009990 tx=0x7f0064006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.291+0000 7f005effd700 1 -- 172.21.15.110:0/1827775589 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0060023d60 con 0x7f0070078c60 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.475+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f007004bed0 con 0x7f0070078c60 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.478+0000 7f005effd700 1 -- 172.21.15.110:0/1827775589 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f0060023b80 con 0x7f0070078c60 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0058036850 msgr2=0x7f0058038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 --2- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0058036850 0x7f0058038d00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0064009990 tx=0x7f0064006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 msgr2=0x7f0070113840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 --2- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070113840 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0060006d40 tx=0x7f0060009250 comp rx=0 tx=0).stop 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 shutdown_connections 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 --2- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f0058036850 0x7f0058038d00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:25.073 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 --2- 172.21.15.110:0/1827775589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0070078c60 0x7f0070113840 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:25.074 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 >> 172.21.15.110:0/1827775589 conn(0x7f00700fe060 msgr2=0x7f00700febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:25.074 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 shutdown_connections 2024-04-25T15:40:25.074 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:24.480+0000 7f0074d3f700 1 -- 172.21.15.110:0/1827775589 wait complete. 2024-04-25T15:40:25.074 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling the dashboard module... 2024-04-25T15:40:25.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:25 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1827775589' entity='client.admin' 2024-04-25T15:40:25.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:25 smithi110 ceph-mon[26344]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:40:26.810 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:26 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2218773687' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T15:40:27.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.514+0000 7f6fae94b700 1 Processor -- start 2024-04-25T15:40:27.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.514+0000 7f6fae94b700 1 -- start start 2024-04-25T15:40:27.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.514+0000 7f6fae94b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:27.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.514+0000 7f6fae94b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa81031d0 con 0x7f6fa8102880 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.514+0000 7f6fa7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.514+0000 7f6fa7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55278/0 (socket says 172.21.15.110:55278) 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.514+0000 7f6fa7fff700 1 -- 172.21.15.110:0/3630854714 learned_addr learned my addr 172.21.15.110:0/3630854714 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.514+0000 7f6fa7fff700 1 -- 172.21.15.110:0/3630854714 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fa8103310 con 0x7f6fa8102880 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fa7fff700 1 --2- 172.21.15.110:0/3630854714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8102c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6f980098e0 tx=0x7f6f98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc95333d5c97121 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fa6ffd700 1 -- 172.21.15.110:0/3630854714 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f9800b860 con 0x7f6fa8102880 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fa6ffd700 1 -- 172.21.15.110:0/3630854714 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f9800be90 con 0x7f6fa8102880 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fa6ffd700 1 -- 172.21.15.110:0/3630854714 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f980103c0 con 0x7f6fa8102880 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fae94b700 1 -- 172.21.15.110:0/3630854714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 msgr2=0x7f6fa8102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fae94b700 1 --2- 172.21.15.110:0/3630854714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8102c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6f980098e0 tx=0x7f6f98009250 comp rx=0 tx=0).stop 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fae94b700 1 -- 172.21.15.110:0/3630854714 shutdown_connections 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fae94b700 1 --2- 172.21.15.110:0/3630854714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8102c90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fae94b700 1 -- 172.21.15.110:0/3630854714 >> 172.21.15.110:0/3630854714 conn(0x7f6fa80fe060 msgr2=0x7f6fa8100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:27.120 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fae94b700 1 -- 172.21.15.110:0/3630854714 shutdown_connections 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.515+0000 7f6fae94b700 1 -- 172.21.15.110:0/3630854714 wait complete. 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.516+0000 7f6fae94b700 1 Processor -- start 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.516+0000 7f6fae94b700 1 -- start start 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.516+0000 7f6fae94b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.516+0000 7f6fae94b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fa8079160 con 0x7f6fa8102880 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.516+0000 7f6fa7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.516+0000 7f6fa7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55286/0 (socket says 172.21.15.110:55286) 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.516+0000 7f6fa7fff700 1 -- 172.21.15.110:0/2218773687 learned_addr learned my addr 172.21.15.110:0/2218773687 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.517+0000 7f6fa7fff700 1 -- 172.21.15.110:0/2218773687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f980094f0 con 0x7f6fa8102880 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.517+0000 7f6fa7fff700 1 --2- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8078c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6f980090b0 tx=0x7f6f98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.517+0000 7f6fa4ff9700 1 -- 172.21.15.110:0/2218773687 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f98010840 con 0x7f6fa8102880 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.517+0000 7f6fa4ff9700 1 -- 172.21.15.110:0/2218773687 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f98010e70 con 0x7f6fa8102880 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.517+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fa80771d0 con 0x7f6fa8102880 2024-04-25T15:40:27.121 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.517+0000 7f6fa4ff9700 1 -- 172.21.15.110:0/2218773687 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6f98024e90 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.517+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fa80776c0 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.518+0000 7f6fa4ff9700 1 -- 172.21.15.110:0/2218773687 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 8) v1 ==== 43376+0+0 (secure 0 0 0) 0x7f6f9801a510 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.518+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fa804bed0 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.518+0000 7f6fa4ff9700 1 --2- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f6f90036800 0x7f6f90038cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.518+0000 7f6fa4ff9700 1 -- 172.21.15.110:0/2218773687 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6f9804a7c0 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.518+0000 7f6fa77fe700 1 --2- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f6f90036800 0x7f6f90038cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.518+0000 7f6fa77fe700 1 --2- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f6f90036800 0x7f6f90038cb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009990 tx=0x7f6f9c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.523+0000 7f6fa4ff9700 1 -- 172.21.15.110:0/2218773687 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6f98018940 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:25.760+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f6fa8077e10 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.484+0000 7f6fa4ff9700 1 -- 172.21.15.110:0/2218773687 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f6f98024430 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.486+0000 7f6fa4ff9700 1 -- 172.21.15.110:0/2218773687 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f6f9801ac50 con 0x7f6fa8102880 2024-04-25T15:40:27.122 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.489+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f6f90036800 msgr2=0x7f6f90038cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.489+0000 7f6fae94b700 1 --2- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f6f90036800 0x7f6f90038cb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c009990 tx=0x7f6f9c006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.489+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 msgr2=0x7f6fa8078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.489+0000 7f6fae94b700 1 --2- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8078c20 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6f980090b0 tx=0x7f6f98009250 comp rx=0 tx=0).stop 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.490+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 shutdown_connections 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.490+0000 7f6fae94b700 1 --2- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f6f90036800 0x7f6f90038cb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.490+0000 7f6fae94b700 1 --2- 172.21.15.110:0/2218773687 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fa8102880 0x7f6fa8078c20 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.490+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 >> 172.21.15.110:0/2218773687 conn(0x7f6fa80fe060 msgr2=0x7f6fa80feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.490+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 shutdown_connections 2024-04-25T15:40:27.123 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:26.490+0000 7f6fae94b700 1 -- 172.21.15.110:0/2218773687 wait complete. 2024-04-25T15:40:27.761 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:27 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2218773687' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T15:40:27.761 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:27 smithi110 ceph-mon[26344]: mgrmap e9: smithi110.lvgbkf(active, since 23s) 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "active_name": "smithi110.lvgbkf", 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.554+0000 7fb3b653f700 1 Processor -- start 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3b653f700 1 -- start start 2024-04-25T15:40:28.443 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3b653f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0107890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3b653f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b0103280 con 0x7fb3b0071290 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55310/0 (socket says 172.21.15.110:55310) 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3affff700 1 -- 172.21.15.110:0/3544765217 learned_addr learned my addr 172.21.15.110:0/3544765217 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3affff700 1 -- 172.21.15.110:0/3544765217 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b0107dd0 con 0x7fb3b0071290 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3affff700 1 --2- 172.21.15.110:0/3544765217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0107890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00098e0 tx=0x7fb3a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=407f7bcc613475b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3aeffd700 1 -- 172.21.15.110:0/3544765217 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3a000b860 con 0x7fb3b0071290 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3aeffd700 1 -- 172.21.15.110:0/3544765217 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3a000be90 con 0x7fb3b0071290 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.555+0000 7fb3aeffd700 1 -- 172.21.15.110:0/3544765217 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3a00103c0 con 0x7fb3b0071290 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.556+0000 7fb3b653f700 1 -- 172.21.15.110:0/3544765217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 msgr2=0x7fb3b0107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:28.444 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.556+0000 7fb3b653f700 1 --2- 172.21.15.110:0/3544765217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0107890 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb3a00098e0 tx=0x7fb3a0009250 comp rx=0 tx=0).stop 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.556+0000 7fb3b653f700 1 -- 172.21.15.110:0/3544765217 shutdown_connections 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.556+0000 7fb3b653f700 1 --2- 172.21.15.110:0/3544765217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0107890 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.556+0000 7fb3b653f700 1 -- 172.21.15.110:0/3544765217 >> 172.21.15.110:0/3544765217 conn(0x7fb3b00fcec0 msgr2=0x7fb3b00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.556+0000 7fb3b653f700 1 -- 172.21.15.110:0/3544765217 shutdown_connections 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.556+0000 7fb3b653f700 1 -- 172.21.15.110:0/3544765217 wait complete. 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.557+0000 7fb3b653f700 1 Processor -- start 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.557+0000 7fb3b653f700 1 -- start start 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.557+0000 7fb3b653f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0113800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.557+0000 7fb3b653f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b0113d40 con 0x7fb3b0071290 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.557+0000 7fb3affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0113800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.557+0000 7fb3affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0113800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55324/0 (socket says 172.21.15.110:55324) 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.557+0000 7fb3affff700 1 -- 172.21.15.110:0/1353378358 learned_addr learned my addr 172.21.15.110:0/1353378358 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:28.445 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.557+0000 7fb3affff700 1 -- 172.21.15.110:0/1353378358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a00094f0 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.558+0000 7fb3affff700 1 --2- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0113800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0006d40 tx=0x7fb3a0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.558+0000 7fb3acff9700 1 -- 172.21.15.110:0/1353378358 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3a0010840 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.558+0000 7fb3acff9700 1 -- 172.21.15.110:0/1353378358 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3a0010e70 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.558+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b0113f40 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.558+0000 7fb3acff9700 1 -- 172.21.15.110:0/1353378358 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3a0024ea0 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.558+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b0114ab0 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.559+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b005f5c0 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.559+0000 7fb3acff9700 1 -- 172.21.15.110:0/1353378358 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7fb3a001a510 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.559+0000 7fb3acff9700 1 --2- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fb398036850 0x7fb398038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.559+0000 7fb3acff9700 1 -- 172.21.15.110:0/1353378358 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fb3a004a7d0 con 0x7fb3b0071290 2024-04-25T15:40:28.446 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.559+0000 7fb3af7fe700 1 -- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fb398036850 msgr2=0x7fb398038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1116386591 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.559+0000 7fb3af7fe700 1 --2- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fb398036850 0x7fb398038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.564+0000 7fb3acff9700 1 -- 172.21.15.110:0/1353378358 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb3a0018940 con 0x7fb3b0071290 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.760+0000 7fb3af7fe700 1 -- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fb398036850 msgr2=0x7fb398038d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1116386591 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.760+0000 7fb3af7fe700 1 --2- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fb398036850 0x7fb398038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.846+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fb3b0114df0 con 0x7fb3b0071290 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.846+0000 7fb3acff9700 1 -- 172.21.15.110:0/1353378358 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fb3a0018350 con 0x7fb3b0071290 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fb398036850 msgr2=0x7fb398038d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 --2- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fb398036850 0x7fb398038d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 msgr2=0x7fb3b0113800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 --2- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0113800 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb3a0006d40 tx=0x7fb3a0009250 comp rx=0 tx=0).stop 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 shutdown_connections 2024-04-25T15:40:28.447 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 --2- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7fb398036850 0x7fb398038d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:28.448 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 --2- 172.21.15.110:0/1353378358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3b0071290 0x7fb3b0113800 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:28.448 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 >> 172.21.15.110:0/1353378358 conn(0x7fb3b00fcec0 msgr2=0x7fb3b00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:28.448 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 shutdown_connections 2024-04-25T15:40:28.448 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:27.848+0000 7fb3b653f700 1 -- 172.21.15.110:0/1353378358 wait complete. 2024-04-25T15:40:28.448 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for the mgr to restart... 2024-04-25T15:40:28.448 INFO:teuthology.orchestra.run.smithi110.stdout:Waiting for mgr epoch 9... 2024-04-25T15:40:28.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:28 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1353378358' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T15:40:33.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: Active manager daemon smithi110.lvgbkf restarted 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: Activating manager daemon smithi110.lvgbkf 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: mgrmap e10: smithi110.lvgbkf(active, starting, since 0.00332903s) 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: Manager daemon smithi110.lvgbkf is now available 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:40:33.050 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:40:33.051 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:40:34.359 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:34 smithi110 ceph-mon[26344]: [25/Apr/2024:15:40:33] ENGINE Bus STARTING 2024-04-25T15:40:34.360 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:34 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:34.360 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:34 smithi110 ceph-mon[26344]: mgrmap e11: smithi110.lvgbkf(active, since 1.00592s) 2024-04-25T15:40:34.629 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout { 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout } 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.853+0000 7f009a483700 1 Processor -- start 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.853+0000 7f009a483700 1 -- start start 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f009a483700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f0094102c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f009a483700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094103160 con 0x7f0094102810 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f0093fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f0094102c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:34.637 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f0093fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f0094102c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55334/0 (socket says 172.21.15.110:55334) 2024-04-25T15:40:34.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f0093fff700 1 -- 172.21.15.110:0/1811941949 learned_addr learned my addr 172.21.15.110:0/1811941949 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:34.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f0093fff700 1 -- 172.21.15.110:0/1811941949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00941032a0 con 0x7f0094102810 2024-04-25T15:40:34.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f0093fff700 1 --2- 172.21.15.110:0/1811941949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f0094102c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00880098e0 tx=0x7f0088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f258773ad29972c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:34.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f0092ffd700 1 -- 172.21.15.110:0/1811941949 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f008800b860 con 0x7f0094102810 2024-04-25T15:40:34.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f0092ffd700 1 -- 172.21.15.110:0/1811941949 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f008800be90 con 0x7f0094102810 2024-04-25T15:40:34.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.854+0000 7f0092ffd700 1 -- 172.21.15.110:0/1811941949 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00880103c0 con 0x7f0094102810 2024-04-25T15:40:34.638 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.855+0000 7f009a483700 1 -- 172.21.15.110:0/1811941949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 msgr2=0x7f0094102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.855+0000 7f009a483700 1 --2- 172.21.15.110:0/1811941949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f0094102c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00880098e0 tx=0x7f0088009250 comp rx=0 tx=0).stop 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.855+0000 7f009a483700 1 -- 172.21.15.110:0/1811941949 shutdown_connections 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.855+0000 7f009a483700 1 --2- 172.21.15.110:0/1811941949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f0094102c20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.855+0000 7f009a483700 1 -- 172.21.15.110:0/1811941949 >> 172.21.15.110:0/1811941949 conn(0x7f00940fdfd0 msgr2=0x7f00941003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.855+0000 7f009a483700 1 -- 172.21.15.110:0/1811941949 shutdown_connections 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.855+0000 7f009a483700 1 -- 172.21.15.110:0/1811941949 wait complete. 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f009a483700 1 Processor -- start 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f009a483700 1 -- start start 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f009a483700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f00941116c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f009a483700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0094111c00 con 0x7f0094102810 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f0093fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f00941116c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:34.639 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f0093fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f00941116c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55344/0 (socket says 172.21.15.110:55344) 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f0093fff700 1 -- 172.21.15.110:0/1945674941 learned_addr learned my addr 172.21.15.110:0/1945674941 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f0093fff700 1 -- 172.21.15.110:0/1945674941 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00880094f0 con 0x7f0094102810 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.856+0000 7f0093fff700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f00941116c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0088011c10 tx=0x7f0088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.857+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0088010890 con 0x7f0094102810 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.857+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0088010ec0 con 0x7f0094102810 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.857+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0094111e00 con 0x7f0094102810 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.857+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f00880243f0 con 0x7f0094102810 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.857+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0094112970 con 0x7f0094102810 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.857+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 9) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f008802d3f0 con 0x7f0094102810 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.858+0000 7f0090ff9700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 0x7f007c038da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.858+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 --> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f009405f5c0 con 0x7f007c0368f0 2024-04-25T15:40:34.640 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.858+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f008804b2a0 con 0x7f0094102810 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.858+0000 7f00937fe700 1 -- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 msgr2=0x7f007c038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1116386591 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:28.858+0000 7f00937fe700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 0x7f007c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:29.059+0000 7f00937fe700 1 -- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 msgr2=0x7f007c038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1116386591 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:29.059+0000 7f00937fe700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 0x7f007c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:29.460+0000 7f00937fe700 1 -- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 msgr2=0x7f007c038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1116386591 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:29.460+0000 7f00937fe700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 0x7f007c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:30.261+0000 7f00937fe700 1 -- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 msgr2=0x7f007c038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1116386591 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:30.261+0000 7f00937fe700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 0x7f007c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:31.862+0000 7f00937fe700 1 -- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 msgr2=0x7f007c038da0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1116386591 2024-04-25T15:40:34.641 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:31.862+0000 7f00937fe700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 0x7f007c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:32.961+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mgrmap(e 10) v1 ==== 43156+0+0 (secure 0 0 0) 0x7f008802d740 con 0x7f0094102810 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:32.961+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 msgr2=0x7f007c038da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:32.961+0000 7f0090ff9700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 0x7f007c038da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.963+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f008804a850 con 0x7f0094102810 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.963+0000 7f0090ff9700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f007c039ff0 0x7f007c03c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.963+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 --> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f009405f5c0 con 0x7f007c039ff0 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.964+0000 7f00937fe700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f007c039ff0 0x7f007c03c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.964+0000 7f00937fe700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f007c039ff0 0x7f007c03c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f00840039d0 tx=0x7f0084009350 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.965+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mgr.14160 v2:172.21.15.110:6800/3209979562 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f009405f5c0 con 0x7f007c039ff0 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 --> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0094112290 con 0x7f007c039ff0 2024-04-25T15:40:34.642 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f0090ff9700 1 -- 172.21.15.110:0/1945674941 <== mgr.14160 v2:172.21.15.110:6800/3209979562 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f0094112290 con 0x7f007c039ff0 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f007c039ff0 msgr2=0x7f007c03c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f007c039ff0 0x7f007c03c3d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f00840039d0 tx=0x7f0084009350 comp rx=0 tx=0).stop 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 msgr2=0x7f00941116c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f00941116c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0088011c10 tx=0x7f0088009250 comp rx=0 tx=0).stop 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 shutdown_connections 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0094102810 0x7f00941116c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f007c039ff0 0x7f007c03c3d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 --2- 172.21.15.110:0/1945674941 >> [v2:172.21.15.110:6800/1116386591,v1:172.21.15.110:6801/1116386591] conn(0x7f007c0368f0 0x7f007c038da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 >> 172.21.15.110:0/1945674941 conn(0x7f00940fdfd0 msgr2=0x7f00940feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 shutdown_connections 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:33.971+0000 7f009a483700 1 -- 172.21.15.110:0/1945674941 wait complete. 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:mgr epoch 9 is available 2024-04-25T15:40:34.643 INFO:teuthology.orchestra.run.smithi110.stdout:Generating a dashboard self-signed certificate... 2024-04-25T15:40:35.236 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:35 smithi110 ceph-mon[26344]: [25/Apr/2024:15:40:33] ENGINE Serving on https://172.21.15.110:7150 2024-04-25T15:40:35.236 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:35 smithi110 ceph-mon[26344]: [25/Apr/2024:15:40:33] ENGINE Bus STARTED 2024-04-25T15:40:35.236 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:35 smithi110 ceph-mon[26344]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T15:40:35.236 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:35 smithi110 ceph-mon[26344]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T15:40:35.236 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:35 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:36.115 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.018+0000 7f3ace397700 1 Processor -- start 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ace397700 1 -- start start 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ace397700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac81053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ace397700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8105920 con 0x7f3ac8076c50 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ac7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac81053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ac7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac81053e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40298/0 (socket says 172.21.15.110:40298) 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ac7fff700 1 -- 172.21.15.110:0/1436375510 learned_addr learned my addr 172.21.15.110:0/1436375510 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ac7fff700 1 -- 172.21.15.110:0/1436375510 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ac8105a60 con 0x7f3ac8076c50 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ac7fff700 1 --2- 172.21.15.110:0/1436375510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac81053e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80098e0 tx=0x7f3ab8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2137266ad29a3fa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.019+0000 7f3ac6ffd700 1 -- 172.21.15.110:0/1436375510 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ab800b860 con 0x7f3ac8076c50 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ac6ffd700 1 -- 172.21.15.110:0/1436375510 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ab800be90 con 0x7f3ac8076c50 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ac6ffd700 1 -- 172.21.15.110:0/1436375510 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ab80103c0 con 0x7f3ac8076c50 2024-04-25T15:40:36.116 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ace397700 1 -- 172.21.15.110:0/1436375510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 msgr2=0x7f3ac81053e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ace397700 1 --2- 172.21.15.110:0/1436375510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac81053e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80098e0 tx=0x7f3ab8009250 comp rx=0 tx=0).stop 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ace397700 1 -- 172.21.15.110:0/1436375510 shutdown_connections 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ace397700 1 --2- 172.21.15.110:0/1436375510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac81053e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ace397700 1 -- 172.21.15.110:0/1436375510 >> 172.21.15.110:0/1436375510 conn(0x7f3ac80fced0 msgr2=0x7f3ac80ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ace397700 1 -- 172.21.15.110:0/1436375510 shutdown_connections 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.020+0000 7f3ace397700 1 -- 172.21.15.110:0/1436375510 wait complete. 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.021+0000 7f3ace397700 1 Processor -- start 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.021+0000 7f3ace397700 1 -- start start 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.021+0000 7f3ace397700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac8102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.021+0000 7f3ace397700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ac8103270 con 0x7f3ac8076c50 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.021+0000 7f3ac7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac8102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.021+0000 7f3ac7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac8102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40304/0 (socket says 172.21.15.110:40304) 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.021+0000 7f3ac7fff700 1 -- 172.21.15.110:0/2681302046 learned_addr learned my addr 172.21.15.110:0/2681302046 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.022+0000 7f3ac7fff700 1 -- 172.21.15.110:0/2681302046 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab80094f0 con 0x7f3ac8076c50 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.022+0000 7f3ac7fff700 1 --2- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac8102d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80094c0 tx=0x7f3ab8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:36.117 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.022+0000 7f3ac4ff9700 1 -- 172.21.15.110:0/2681302046 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ab8010890 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.022+0000 7f3ac4ff9700 1 -- 172.21.15.110:0/2681302046 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ab8010ec0 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.022+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ac81012e0 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.022+0000 7f3ac4ff9700 1 -- 172.21.15.110:0/2681302046 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3ab80243f0 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.022+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ac81017d0 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.023+0000 7f3ac4ff9700 1 -- 172.21.15.110:0/2681302046 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 11) v1 ==== 43283+0+0 (secure 0 0 0) 0x7f3ab802d3f0 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.023+0000 7f3ac4ff9700 1 --2- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3ab0036770 0x7f3ab0038c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.023+0000 7f3ac4ff9700 1 -- 172.21.15.110:0/2681302046 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3ab804b290 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.023+0000 7f3ac77fe700 1 --2- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3ab0036770 0x7f3ab0038c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.023+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ac804bed0 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.024+0000 7f3ac77fe700 1 --2- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3ab0036770 0x7f3ab0038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009990 tx=0x7f3abc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.028+0000 7f3ac4ff9700 1 -- 172.21.15.110:0/2681302046 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3ab802b8e0 con 0x7f3ac8076c50 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.235+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 --> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ac8101e60 con 0x7f3ab0036770 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.540+0000 7f3ac4ff9700 1 -- 172.21.15.110:0/2681302046 <== mgr.14160 v2:172.21.15.110:6800/3209979562 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f3ac8101e60 con 0x7f3ab0036770 2024-04-25T15:40:36.118 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3ab0036770 msgr2=0x7f3ab0038c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 --2- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3ab0036770 0x7f3ab0038c20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3abc009990 tx=0x7f3abc006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 msgr2=0x7f3ac8102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 --2- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac8102d30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3ab80094c0 tx=0x7f3ab8009250 comp rx=0 tx=0).stop 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 shutdown_connections 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 --2- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3ab0036770 0x7f3ab0038c20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 --2- 172.21.15.110:0/2681302046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3ac8076c50 0x7f3ac8102d30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 >> 172.21.15.110:0/2681302046 conn(0x7f3ac80fced0 msgr2=0x7f3ac80fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 shutdown_connections 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:35.542+0000 7f3ace397700 1 -- 172.21.15.110:0/2681302046 wait complete. 2024-04-25T15:40:36.119 INFO:teuthology.orchestra.run.smithi110.stdout:Creating initial admin user... 2024-04-25T15:40:36.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:36 smithi110 ceph-mon[26344]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:36.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:36 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:36.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:36 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:36.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:36 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:36.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:36 smithi110 ceph-mon[26344]: mgrmap e12: smithi110.lvgbkf(active, since 2s) 2024-04-25T15:40:38.071 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:37 smithi110 ceph-mon[26344]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:38.071 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:37 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:38.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$i222kLpz.LhY5Bx7ZmTl0ezpLG5qGUjKNvyehhX3d/K3PeF/x8T9.", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714059636, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T15:40:38.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.550+0000 7f0e0e480700 1 Processor -- start 2024-04-25T15:40:38.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.550+0000 7f0e0e480700 1 -- start start 2024-04-25T15:40:38.312 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.550+0000 7f0e0e480700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08102150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.550+0000 7f0e0e480700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08102690 con 0x7f0e08101d40 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.550+0000 7f0e07fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08102150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.551+0000 7f0e07fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08102150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40308/0 (socket says 172.21.15.110:40308) 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.551+0000 7f0e07fff700 1 -- 172.21.15.110:0/383563649 learned_addr learned my addr 172.21.15.110:0/383563649 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.551+0000 7f0e07fff700 1 -- 172.21.15.110:0/383563649 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e081027d0 con 0x7f0e08101d40 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.551+0000 7f0e07fff700 1 --2- 172.21.15.110:0/383563649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08102150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0df80098e0 tx=0x7f0df8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e14451c1161f17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.551+0000 7f0e06ffd700 1 -- 172.21.15.110:0/383563649 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0df800b860 con 0x7f0e08101d40 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.551+0000 7f0e06ffd700 1 -- 172.21.15.110:0/383563649 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0df800be90 con 0x7f0e08101d40 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.551+0000 7f0e06ffd700 1 -- 172.21.15.110:0/383563649 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0df80103c0 con 0x7f0e08101d40 2024-04-25T15:40:38.313 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.552+0000 7f0e0e480700 1 -- 172.21.15.110:0/383563649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 msgr2=0x7f0e08102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.552+0000 7f0e0e480700 1 --2- 172.21.15.110:0/383563649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08102150 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0df80098e0 tx=0x7f0df8009250 comp rx=0 tx=0).stop 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.552+0000 7f0e0e480700 1 -- 172.21.15.110:0/383563649 shutdown_connections 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.552+0000 7f0e0e480700 1 --2- 172.21.15.110:0/383563649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08102150 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.552+0000 7f0e0e480700 1 -- 172.21.15.110:0/383563649 >> 172.21.15.110:0/383563649 conn(0x7f0e080fc6f0 msgr2=0x7f0e080feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.552+0000 7f0e0e480700 1 -- 172.21.15.110:0/383563649 shutdown_connections 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.552+0000 7f0e0e480700 1 -- 172.21.15.110:0/383563649 wait complete. 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.552+0000 7f0e0e480700 1 Processor -- start 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e0e480700 1 -- start start 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e0e480700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e0e480700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08114300 con 0x7f0e08101d40 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e07fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e07fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40322/0 (socket says 172.21.15.110:40322) 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e07fff700 1 -- 172.21.15.110:0/2161384581 learned_addr learned my addr 172.21.15.110:0/2161384581 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:38.314 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e07fff700 1 -- 172.21.15.110:0/2161384581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df80094f0 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e07fff700 1 --2- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08101540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0df8011680 tx=0x7f0df8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e04ff9700 1 -- 172.21.15.110:0/2161384581 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0df8010890 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e04ff9700 1 -- 172.21.15.110:0/2161384581 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0df8010ec0 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e04ff9700 1 -- 172.21.15.110:0/2161384581 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0df80243f0 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e08114440 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.553+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e081148d0 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.554+0000 7f0e04ff9700 1 -- 172.21.15.110:0/2161384581 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f0df802d3f0 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.554+0000 7f0e04ff9700 1 --2- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f0df00368a0 0x7f0df0038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.554+0000 7f0e04ff9700 1 -- 172.21.15.110:0/2161384581 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0df804a3a0 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.554+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0804bed0 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.554+0000 7f0e077fe700 1 --2- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f0df00368a0 0x7f0df0038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.555+0000 7f0e077fe700 1 --2- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f0df00368a0 0x7f0df0038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009990 tx=0x7f0dfc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.559+0000 7f0e04ff9700 1 -- 172.21.15.110:0/2161384581 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0df802b8e0 con 0x7f0e08101d40 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.774+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 --> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0e0805f3b0 con 0x7f0df00368a0 2024-04-25T15:40:38.315 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.997+0000 7f0e04ff9700 1 -- 172.21.15.110:0/2161384581 <== mgr.14160 v2:172.21.15.110:6800/3209979562 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f0e0805f3b0 con 0x7f0df00368a0 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.999+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f0df00368a0 msgr2=0x7f0df0038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.999+0000 7f0e0e480700 1 --2- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f0df00368a0 0x7f0df0038d50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc009990 tx=0x7f0dfc006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.999+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 msgr2=0x7f0e08101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.999+0000 7f0e0e480700 1 --2- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08101540 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0df8011680 tx=0x7f0df8009250 comp rx=0 tx=0).stop 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.999+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 shutdown_connections 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.999+0000 7f0e0e480700 1 --2- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f0df00368a0 0x7f0df0038d50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:36.999+0000 7f0e0e480700 1 --2- 172.21.15.110:0/2161384581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e08101d40 0x7f0e08101540 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:37.000+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 >> 172.21.15.110:0/2161384581 conn(0x7f0e080fc6f0 msgr2=0x7f0e080fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:37.000+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 shutdown_connections 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:37.000+0000 7f0e0e480700 1 -- 172.21.15.110:0/2161384581 wait complete. 2024-04-25T15:40:38.316 INFO:teuthology.orchestra.run.smithi110.stdout:Fetching dashboard port number... 2024-04-25T15:40:39.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:38 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/488445364' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T15:40:39.560 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.762+0000 7f68c7ad0700 1 Processor -- start 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.762+0000 7f68c7ad0700 1 -- start start 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.762+0000 7f68c7ad0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.762+0000 7f68c7ad0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c01031d0 con 0x7f68c0102880 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.762+0000 7f68c586c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.762+0000 7f68c586c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40324/0 (socket says 172.21.15.110:40324) 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.762+0000 7f68c586c700 1 -- 172.21.15.110:0/1607639756 learned_addr learned my addr 172.21.15.110:0/1607639756 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.762+0000 7f68c586c700 1 -- 172.21.15.110:0/1607639756 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c0103310 con 0x7f68c0102880 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c586c700 1 --2- 172.21.15.110:0/1607639756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0102c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68b40098e0 tx=0x7f68b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0ad925a844a92f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c486a700 1 -- 172.21.15.110:0/1607639756 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b400b860 con 0x7f68c0102880 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c486a700 1 -- 172.21.15.110:0/1607639756 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b400be90 con 0x7f68c0102880 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c486a700 1 -- 172.21.15.110:0/1607639756 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b40103c0 con 0x7f68c0102880 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c7ad0700 1 -- 172.21.15.110:0/1607639756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 msgr2=0x7f68c0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:39.561 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c7ad0700 1 --2- 172.21.15.110:0/1607639756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0102c90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f68b40098e0 tx=0x7f68b4009250 comp rx=0 tx=0).stop 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c7ad0700 1 -- 172.21.15.110:0/1607639756 shutdown_connections 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c7ad0700 1 --2- 172.21.15.110:0/1607639756 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0102c90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c7ad0700 1 -- 172.21.15.110:0/1607639756 >> 172.21.15.110:0/1607639756 conn(0x7f68c00fe060 msgr2=0x7f68c0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.763+0000 7f68c7ad0700 1 -- 172.21.15.110:0/1607639756 shutdown_connections 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.764+0000 7f68c7ad0700 1 -- 172.21.15.110:0/1607639756 wait complete. 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.764+0000 7f68c7ad0700 1 Processor -- start 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.764+0000 7f68c7ad0700 1 -- start start 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.764+0000 7f68c7ad0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0078c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.764+0000 7f68c7ad0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c0079160 con 0x7f68c0102880 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.764+0000 7f68c586c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0078c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.764+0000 7f68c586c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0078c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40338/0 (socket says 172.21.15.110:40338) 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.764+0000 7f68c586c700 1 -- 172.21.15.110:0/488445364 learned_addr learned my addr 172.21.15.110:0/488445364 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.765+0000 7f68c586c700 1 -- 172.21.15.110:0/488445364 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b40094f0 con 0x7f68c0102880 2024-04-25T15:40:39.562 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.765+0000 7f68c586c700 1 --2- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0078c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68b40090b0 tx=0x7f68b4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.765+0000 7f68b27fc700 1 -- 172.21.15.110:0/488445364 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b4010840 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.765+0000 7f68b27fc700 1 -- 172.21.15.110:0/488445364 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68b4010e70 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.765+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c00771d0 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.765+0000 7f68b27fc700 1 -- 172.21.15.110:0/488445364 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f68b4024e40 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.765+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c00776c0 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.766+0000 7f68b27fc700 1 -- 172.21.15.110:0/488445364 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 12) v1 ==== 43389+0+0 (secure 0 0 0) 0x7f68b401a530 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.766+0000 7f68b27fc700 1 --2- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f68ac036850 0x7f68ac038d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.766+0000 7f68b27fc700 1 -- 172.21.15.110:0/488445364 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f68b404a9d0 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.766+0000 7f68c506b700 1 --2- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f68ac036850 0x7f68ac038d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.766+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c004bed0 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.766+0000 7f68c506b700 1 --2- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f68ac036850 0x7f68ac038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009990 tx=0x7f68bc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.771+0000 7f68b27fc700 1 -- 172.21.15.110:0/488445364 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f68b4018940 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.955+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f68c0104620 con 0x7f68c0102880 2024-04-25T15:40:39.563 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.955+0000 7f68b27fc700 1 -- 172.21.15.110:0/488445364 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f68b4049020 con 0x7f68c0102880 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f68ac036850 msgr2=0x7f68ac038d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 --2- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f68ac036850 0x7f68ac038d00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009990 tx=0x7f68bc006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 msgr2=0x7f68c0078c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 --2- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0078c20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f68b40090b0 tx=0x7f68b4009250 comp rx=0 tx=0).stop 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 shutdown_connections 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 --2- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f68ac036850 0x7f68ac038d00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 --2- 172.21.15.110:0/488445364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68c0102880 0x7f68c0078c20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 >> 172.21.15.110:0/488445364 conn(0x7f68c00fe060 msgr2=0x7f68c00feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 shutdown_connections 2024-04-25T15:40:39.564 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:38.957+0000 7f68c7ad0700 1 -- 172.21.15.110:0/488445364 wait complete. 2024-04-25T15:40:39.567 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T15:40:39.567 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout disabled 2024-04-25T15:40:39.575 INFO:teuthology.orchestra.run.smithi110.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T15:40:39.575 INFO:teuthology.orchestra.run.smithi110.stdout:systemctl: stdout inactive 2024-04-25T15:40:39.575 INFO:teuthology.orchestra.run.smithi110.stdout:firewalld.service is not enabled 2024-04-25T15:40:39.575 INFO:teuthology.orchestra.run.smithi110.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T15:40:39.576 INFO:teuthology.orchestra.run.smithi110.stdout:Ceph Dashboard is now available at: 2024-04-25T15:40:39.576 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:39.576 INFO:teuthology.orchestra.run.smithi110.stdout: URL: https://smithi110:8443/ 2024-04-25T15:40:39.576 INFO:teuthology.orchestra.run.smithi110.stdout: User: admin 2024-04-25T15:40:39.577 INFO:teuthology.orchestra.run.smithi110.stdout: Password: sp6z0qcwaf 2024-04-25T15:40:39.577 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:39.577 INFO:teuthology.orchestra.run.smithi110.stdout:Saving cluster configuration to /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config directory 2024-04-25T15:40:39.577 INFO:teuthology.orchestra.run.smithi110.stdout:Enabling autotune for osd_memory_target 2024-04-25T15:40:40.164 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:40 smithi110 ceph-mon[26344]: mgrmap e13: smithi110.lvgbkf(active, since 6s) 2024-04-25T15:40:40.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.968+0000 7fb56d3ad700 1 Processor -- start 2024-04-25T15:40:40.725 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.968+0000 7fb56d3ad700 1 -- start start 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.968+0000 7fb56d3ad700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568101b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.968+0000 7fb56d3ad700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb568102060 con 0x7fb568101710 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.968+0000 7fb566ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568101b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.968+0000 7fb566ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568101b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36860/0 (socket says 172.21.15.110:36860) 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.968+0000 7fb566ffd700 1 -- 172.21.15.110:0/2959495275 learned_addr learned my addr 172.21.15.110:0/2959495275 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.969+0000 7fb566ffd700 1 -- 172.21.15.110:0/2959495275 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5681021a0 con 0x7fb568101710 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.969+0000 7fb566ffd700 1 --2- 172.21.15.110:0/2959495275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568101b20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5580098e0 tx=0x7fb558009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179c1603b262c307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.969+0000 7fb5667fc700 1 -- 172.21.15.110:0/2959495275 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55800b860 con 0x7fb568101710 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.969+0000 7fb5667fc700 1 -- 172.21.15.110:0/2959495275 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb55800be90 con 0x7fb568101710 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.969+0000 7fb5667fc700 1 -- 172.21.15.110:0/2959495275 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5580103c0 con 0x7fb568101710 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.969+0000 7fb56d3ad700 1 -- 172.21.15.110:0/2959495275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 msgr2=0x7fb568101b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:40.726 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.969+0000 7fb56d3ad700 1 --2- 172.21.15.110:0/2959495275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568101b20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb5580098e0 tx=0x7fb558009250 comp rx=0 tx=0).stop 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 -- 172.21.15.110:0/2959495275 shutdown_connections 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 --2- 172.21.15.110:0/2959495275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568101b20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 -- 172.21.15.110:0/2959495275 >> 172.21.15.110:0/2959495275 conn(0x7fb5680fced0 msgr2=0x7fb5680ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 -- 172.21.15.110:0/2959495275 shutdown_connections 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 -- 172.21.15.110:0/2959495275 wait complete. 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 Processor -- start 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 -- start start 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568076b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb56d3ad700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5681142b0 con 0x7fb568101710 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb566ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568076b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb566ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568076b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36876/0 (socket says 172.21.15.110:36876) 2024-04-25T15:40:40.727 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.970+0000 7fb566ffd700 1 -- 172.21.15.110:0/477984666 learned_addr learned my addr 172.21.15.110:0/477984666 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.971+0000 7fb566ffd700 1 -- 172.21.15.110:0/477984666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5580094f0 con 0x7fb568101710 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.971+0000 7fb566ffd700 1 --2- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568076b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb558011680 tx=0x7fb558009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.971+0000 7fb55f7fe700 1 -- 172.21.15.110:0/477984666 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb558010890 con 0x7fb568101710 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.971+0000 7fb55f7fe700 1 -- 172.21.15.110:0/477984666 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb558010ec0 con 0x7fb568101710 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.971+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5681143f0 con 0x7fb568101710 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.971+0000 7fb55f7fe700 1 -- 172.21.15.110:0/477984666 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb5580243f0 con 0x7fb568101710 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.971+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5681148e0 con 0x7fb568101710 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.972+0000 7fb55f7fe700 1 -- 172.21.15.110:0/477984666 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fb55802d3f0 con 0x7fb568101710 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.972+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb56804bed0 con 0x7fb568101710 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.972+0000 7fb55f7fe700 1 --2- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb554036940 0x7fb554038df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:40.728 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.972+0000 7fb55f7fe700 1 -- 172.21.15.110:0/477984666 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb55804a760 con 0x7fb568101710 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.972+0000 7fb55ffff700 1 --2- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb554036940 0x7fb554038df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.972+0000 7fb55ffff700 1 --2- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb554036940 0x7fb554038df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb550009990 tx=0x7fb550006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:39.976+0000 7fb55f7fe700 1 -- 172.21.15.110:0/477984666 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb55802b8e0 con 0x7fb568101710 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.163+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fb568114b20 con 0x7fb568101710 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.163+0000 7fb55f7fe700 1 -- 172.21.15.110:0/477984666 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fb558006d70 con 0x7fb568101710 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb554036940 msgr2=0x7fb554038df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 --2- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb554036940 0x7fb554038df0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb550009990 tx=0x7fb550006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 msgr2=0x7fb568076b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 --2- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568076b60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb558011680 tx=0x7fb558009250 comp rx=0 tx=0).stop 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 shutdown_connections 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 --2- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb554036940 0x7fb554038df0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 --2- 172.21.15.110:0/477984666 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb568101710 0x7fb568076b60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 >> 172.21.15.110:0/477984666 conn(0x7fb5680fced0 msgr2=0x7fb5680fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 shutdown_connections 2024-04-25T15:40:40.729 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:40.165+0000 7fb56d3ad700 1 -- 172.21.15.110:0/477984666 wait complete. 2024-04-25T15:40:42.006 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.147+0000 7f3a4b2ad700 1 Processor -- start 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.148+0000 7f3a4b2ad700 1 -- start start 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.148+0000 7f3a4b2ad700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a44101b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.148+0000 7f3a4b2ad700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a44102080 con 0x7f3a44101730 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.148+0000 7f3a49049700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a44101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.148+0000 7f3a49049700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a44101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36882/0 (socket says 172.21.15.110:36882) 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.148+0000 7f3a49049700 1 -- 172.21.15.110:0/50037547 learned_addr learned my addr 172.21.15.110:0/50037547 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.148+0000 7f3a49049700 1 -- 172.21.15.110:0/50037547 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a441021c0 con 0x7f3a44101730 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.148+0000 7f3a49049700 1 --2- 172.21.15.110:0/50037547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a44101b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00d0d0 tx=0x7f3a3c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eec2e602c02e5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a43fff700 1 -- 172.21.15.110:0/50037547 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a3c00ddf0 con 0x7f3a44101730 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a43fff700 1 -- 172.21.15.110:0/50037547 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a3c009610 con 0x7f3a44101730 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a43fff700 1 -- 172.21.15.110:0/50037547 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a3c0146c0 con 0x7f3a44101730 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/50037547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 msgr2=0x7f3a44101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a4b2ad700 1 --2- 172.21.15.110:0/50037547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a44101b40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00d0d0 tx=0x7f3a3c00ba50 comp rx=0 tx=0).stop 2024-04-25T15:40:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/50037547 shutdown_connections 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a4b2ad700 1 --2- 172.21.15.110:0/50037547 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a44101b40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/50037547 >> 172.21.15.110:0/50037547 conn(0x7f3a440fcef0 msgr2=0x7f3a440ff300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/50037547 shutdown_connections 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.149+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/50037547 wait complete. 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a4b2ad700 1 Processor -- start 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a4b2ad700 1 -- start start 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a4b2ad700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a4410f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a4b2ad700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a4410f9b0 con 0x7f3a44101730 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a49049700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a4410f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a49049700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a4410f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36896/0 (socket says 172.21.15.110:36896) 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a49049700 1 -- 172.21.15.110:0/2720283887 learned_addr learned my addr 172.21.15.110:0/2720283887 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a49049700 1 -- 172.21.15.110:0/2720283887 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c00bcf0 con 0x7f3a44101730 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.150+0000 7f3a49049700 1 --2- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a4410f470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009e20 tx=0x7f3a3c00ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.151+0000 7f3a41ffb700 1 -- 172.21.15.110:0/2720283887 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a3c00ddf0 con 0x7f3a44101730 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.151+0000 7f3a41ffb700 1 -- 172.21.15.110:0/2720283887 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a3c014e00 con 0x7f3a44101730 2024-04-25T15:40:42.008 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.151+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a4410fbb0 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.151+0000 7f3a41ffb700 1 -- 172.21.15.110:0/2720283887 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3a3c01eaf0 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.151+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a44110720 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.152+0000 7f3a41ffb700 1 -- 172.21.15.110:0/2720283887 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f3a3c01ece0 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.152+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a4405f5c0 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.152+0000 7f3a41ffb700 1 --2- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3a2c036990 0x7f3a2c038e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.152+0000 7f3a41ffb700 1 -- 172.21.15.110:0/2720283887 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f3a3c04c300 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.152+0000 7f3a48848700 1 --2- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3a2c036990 0x7f3a2c038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.152+0000 7f3a48848700 1 --2- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3a2c036990 0x7f3a2c038e40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009990 tx=0x7f3a38006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.156+0000 7f3a41ffb700 1 -- 172.21.15.110:0/2720283887 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3a3c013bd0 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.444+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f3a4404bed0 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.447+0000 7f3a41ffb700 1 -- 172.21.15.110:0/2720283887 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v33)=0 set mgr/dashboard/cluster/status v33) v1 ==== 153+0+0 (secure 0 0 0) 0x7f3a3c019070 con 0x7f3a44101730 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3a2c036990 msgr2=0x7f3a2c038e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 --2- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3a2c036990 0x7f3a2c038e40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3a38009990 tx=0x7f3a38006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 msgr2=0x7f3a4410f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:42.009 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 --2- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a4410f470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009e20 tx=0x7f3a3c00ba50 comp rx=0 tx=0).stop 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 shutdown_connections 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 --2- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f3a2c036990 0x7f3a2c038e40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 --2- 172.21.15.110:0/2720283887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a44101730 0x7f3a4410f470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 >> 172.21.15.110:0/2720283887 conn(0x7f3a440fcef0 msgr2=0x7f3a44105c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 shutdown_connections 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr 2024-04-25T15:40:41.449+0000 7f3a4b2ad700 1 -- 172.21.15.110:0/2720283887 wait complete. 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:42.010 INFO:teuthology.orchestra.run.smithi110.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout: ceph telemetry on 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout:For more information see: 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:42.011 INFO:teuthology.orchestra.run.smithi110.stdout:Bootstrap complete. 2024-04-25T15:40:42.040 INFO:tasks.cephadm:Fetching config... 2024-04-25T15:40:42.041 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:40:42.041 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T15:40:42.056 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T15:40:42.057 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:40:42.057 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T15:40:42.112 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T15:40:42.112 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:40:42.112 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/keyring of=/dev/stdout 2024-04-25T15:40:42.176 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T15:40:42.176 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:40:42.176 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T15:40:42.232 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T15:40:42.232 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGU63FKYnRaYjybGBNU8fAHoXvTTvDZ0pHE+UIRTeZBFtHxqo/2lr8bfFrx3Usdi9dV2h+ThSRwIz1eGYf2BD8uGbsFJkD4sMQ0ZHJMsuB7pIuAbVDVVus1+TnXvoNfA6vKYETKNfrwk2gX3OEPZwKGSELr+/r2Pqq1nd2oc5p993b7PQQ1/dN8L8KEDGvTAPfHEqNvyo5kydeDdCoTYU+qn6G/O2pYSgQJyM8Y1ha7p5r928y3imOXoagWzSbdhlUa4LaeV1Orhz53kWxGfffkAXonM8hWaBMlmZEt7YPqX4JwX+uwiPwJupg3RjbXuk9QO3e5bfaQIZ5cMyljkwgliuVGsmk7D08yhMmuk1FoposoNLeT7I8+9maSYSUN6VakCOIRSgnkZwlADjTk6ExsPmAQDzxx2+ciBUq6nQC6Vvw9zLXNFMS/Q/CSw6Fu6+tcRZ49pBuRMpzAiHN8CYVqK/BsrJQlSAFJFUJ8bz9Dmnz53FPMjTypCb8PHXubTs= ceph-e79cc6fc-0319-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T15:40:42.304 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGU63FKYnRaYjybGBNU8fAHoXvTTvDZ0pHE+UIRTeZBFtHxqo/2lr8bfFrx3Usdi9dV2h+ThSRwIz1eGYf2BD8uGbsFJkD4sMQ0ZHJMsuB7pIuAbVDVVus1+TnXvoNfA6vKYETKNfrwk2gX3OEPZwKGSELr+/r2Pqq1nd2oc5p993b7PQQ1/dN8L8KEDGvTAPfHEqNvyo5kydeDdCoTYU+qn6G/O2pYSgQJyM8Y1ha7p5r928y3imOXoagWzSbdhlUa4LaeV1Orhz53kWxGfffkAXonM8hWaBMlmZEt7YPqX4JwX+uwiPwJupg3RjbXuk9QO3e5bfaQIZ5cMyljkwgliuVGsmk7D08yhMmuk1FoposoNLeT7I8+9maSYSUN6VakCOIRSgnkZwlADjTk6ExsPmAQDzxx2+ciBUq6nQC6Vvw9zLXNFMS/Q/CSw6Fu6+tcRZ49pBuRMpzAiHN8CYVqK/BsrJQlSAFJFUJ8bz9Dmnz53FPMjTypCb8PHXubTs= ceph-e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:40:42.314 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGU63FKYnRaYjybGBNU8fAHoXvTTvDZ0pHE+UIRTeZBFtHxqo/2lr8bfFrx3Usdi9dV2h+ThSRwIz1eGYf2BD8uGbsFJkD4sMQ0ZHJMsuB7pIuAbVDVVus1+TnXvoNfA6vKYETKNfrwk2gX3OEPZwKGSELr+/r2Pqq1nd2oc5p993b7PQQ1/dN8L8KEDGvTAPfHEqNvyo5kydeDdCoTYU+qn6G/O2pYSgQJyM8Y1ha7p5r928y3imOXoagWzSbdhlUa4LaeV1Orhz53kWxGfffkAXonM8hWaBMlmZEt7YPqX4JwX+uwiPwJupg3RjbXuk9QO3e5bfaQIZ5cMyljkwgliuVGsmk7D08yhMmuk1FoposoNLeT7I8+9maSYSUN6VakCOIRSgnkZwlADjTk6ExsPmAQDzxx2+ciBUq6nQC6Vvw9zLXNFMS/Q/CSw6Fu6+tcRZ49pBuRMpzAiHN8CYVqK/BsrJQlSAFJFUJ8bz9Dmnz53FPMjTypCb8PHXubTs= ceph-e79cc6fc-0319-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T15:40:42.354 INFO:teuthology.orchestra.run.smithi193.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDGU63FKYnRaYjybGBNU8fAHoXvTTvDZ0pHE+UIRTeZBFtHxqo/2lr8bfFrx3Usdi9dV2h+ThSRwIz1eGYf2BD8uGbsFJkD4sMQ0ZHJMsuB7pIuAbVDVVus1+TnXvoNfA6vKYETKNfrwk2gX3OEPZwKGSELr+/r2Pqq1nd2oc5p993b7PQQ1/dN8L8KEDGvTAPfHEqNvyo5kydeDdCoTYU+qn6G/O2pYSgQJyM8Y1ha7p5r928y3imOXoagWzSbdhlUa4LaeV1Orhz53kWxGfffkAXonM8hWaBMlmZEt7YPqX4JwX+uwiPwJupg3RjbXuk9QO3e5bfaQIZ5cMyljkwgliuVGsmk7D08yhMmuk1FoposoNLeT7I8+9maSYSUN6VakCOIRSgnkZwlADjTk6ExsPmAQDzxx2+ciBUq6nQC6Vvw9zLXNFMS/Q/CSw6Fu6+tcRZ49pBuRMpzAiHN8CYVqK/BsrJQlSAFJFUJ8bz9Dmnz53FPMjTypCb8PHXubTs= ceph-e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:40:42.365 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T15:40:42.631 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:40:42.753 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:42 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2720283887' entity='client.admin' 2024-04-25T15:40:44.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.837+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3444637334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 msgr2=0x7f6ef00fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:44.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.837+0000 7f6ef4a0f700 1 --2- 172.21.15.110:0/3444637334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 0x7f6ef00fd430 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00048c0 tx=0x7f6ee0009e30 comp rx=0 tx=0).stop 2024-04-25T15:40:44.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.837+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3444637334 shutdown_connections 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.837+0000 7f6ef4a0f700 1 --2- 172.21.15.110:0/3444637334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 0x7f6ef00fd430 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.837+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3444637334 >> 172.21.15.110:0/3444637334 conn(0x7f6ef00f70a0 msgr2=0x7f6ef00f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.837+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3444637334 shutdown_connections 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.837+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3444637334 wait complete. 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6ef4a0f700 1 Processor -- start 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6ef4a0f700 1 -- start start 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6ef4a0f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 0x7f6ef010bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6ef4a0f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef010c4f0 con 0x7f6ef00fd020 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6eee59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 0x7f6ef010bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6eee59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 0x7f6ef010bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36920/0 (socket says 172.21.15.110:36920) 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6eee59c700 1 -- 172.21.15.110:0/3420144704 learned_addr learned my addr 172.21.15.110:0/3420144704 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6eee59c700 1 -- 172.21.15.110:0/3420144704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee0005040 con 0x7f6ef00fd020 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.838+0000 7f6eee59c700 1 --2- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 0x7f6ef010bfb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00096b0 tx=0x7f6ee0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:44.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.839+0000 7f6edeffd700 1 -- 172.21.15.110:0/3420144704 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ee000bed0 con 0x7f6ef00fd020 2024-04-25T15:40:44.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.839+0000 7f6edeffd700 1 -- 172.21.15.110:0/3420144704 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ee000b5c0 con 0x7f6ef00fd020 2024-04-25T15:40:44.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.839+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef010c6f0 con 0x7f6ef00fd020 2024-04-25T15:40:44.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.839+0000 7f6edeffd700 1 -- 172.21.15.110:0/3420144704 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6ee001c5b0 con 0x7f6ef00fd020 2024-04-25T15:40:44.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.839+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef010d260 con 0x7f6ef00fd020 2024-04-25T15:40:44.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.839+0000 7f6edeffd700 1 -- 172.21.15.110:0/3420144704 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f6ee001c820 con 0x7f6ef00fd020 2024-04-25T15:40:44.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.840+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef0101e40 con 0x7f6ef00fd020 2024-04-25T15:40:44.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.840+0000 7f6edeffd700 1 --2- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f6ed8036880 0x7f6ed8038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:44.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.840+0000 7f6edeffd700 1 -- 172.21.15.110:0/3420144704 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f6ee0049be0 con 0x7f6ef00fd020 2024-04-25T15:40:44.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.840+0000 7f6eedd9b700 1 --2- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f6ed8036880 0x7f6ed8038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:44.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.840+0000 7f6eedd9b700 1 --2- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f6ed8036880 0x7f6ed8038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009990 tx=0x7f6ee4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:44.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:44.844+0000 7f6edeffd700 1 -- 172.21.15.110:0/3420144704 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6ee0023300 con 0x7f6ef00fd020 2024-04-25T15:40:45.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.035+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f6ef010dc70 con 0x7f6ef00fd020 2024-04-25T15:40:45.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.039+0000 7f6edeffd700 1 -- 172.21.15.110:0/3420144704 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f6ee0011b10 con 0x7f6ef00fd020 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f6ed8036880 msgr2=0x7f6ed8038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 --2- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f6ed8036880 0x7f6ed8038d30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009990 tx=0x7f6ee4006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 msgr2=0x7f6ef010bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 --2- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 0x7f6ef010bfb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00096b0 tx=0x7f6ee0009e30 comp rx=0 tx=0).stop 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 shutdown_connections 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 --2- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f6ed8036880 0x7f6ed8038d30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 --2- 172.21.15.110:0/3420144704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ef00fd020 0x7f6ef010bfb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 >> 172.21.15.110:0/3420144704 conn(0x7f6ef00f70a0 msgr2=0x7f6ef0102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:45.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 shutdown_connections 2024-04-25T15:40:45.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:45.041+0000 7f6ef4a0f700 1 -- 172.21.15.110:0/3420144704 wait complete. 2024-04-25T15:40:45.798 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T15:40:45.798 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: Deploying daemon crash.smithi110 on smithi110 2024-04-25T15:40:45.854 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:45 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3420144704' entity='client.admin' 2024-04-25T15:40:46.612 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:40:48.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.827+0000 7fb8214fe700 1 -- 172.21.15.110:0/2086295463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 msgr2=0x7fb81c0fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:48.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.827+0000 7fb8214fe700 1 --2- 172.21.15.110:0/2086295463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 0x7fb81c0fce40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb80c0048c0 tx=0x7fb80c009e30 comp rx=0 tx=0).stop 2024-04-25T15:40:48.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.827+0000 7fb8214fe700 1 -- 172.21.15.110:0/2086295463 shutdown_connections 2024-04-25T15:40:48.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.827+0000 7fb8214fe700 1 --2- 172.21.15.110:0/2086295463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 0x7fb81c0fce40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:48.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.827+0000 7fb8214fe700 1 -- 172.21.15.110:0/2086295463 >> 172.21.15.110:0/2086295463 conn(0x7fb81c0f8170 msgr2=0x7fb81c0fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:48.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.827+0000 7fb8214fe700 1 -- 172.21.15.110:0/2086295463 shutdown_connections 2024-04-25T15:40:48.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.827+0000 7fb8214fe700 1 -- 172.21.15.110:0/2086295463 wait complete. 2024-04-25T15:40:48.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.828+0000 7fb8214fe700 1 Processor -- start 2024-04-25T15:40:48.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.828+0000 7fb8214fe700 1 -- start start 2024-04-25T15:40:48.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.828+0000 7fb8214fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 0x7fb81c10d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:48.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.828+0000 7fb8214fe700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c10d5c0 con 0x7fb81c0fca30 2024-04-25T15:40:48.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.828+0000 7fb81affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 0x7fb81c10d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:48.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.828+0000 7fb81affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 0x7fb81c10d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:36946/0 (socket says 172.21.15.110:36946) 2024-04-25T15:40:48.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.828+0000 7fb81affd700 1 -- 172.21.15.110:0/2632223955 learned_addr learned my addr 172.21.15.110:0/2632223955 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:48.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.828+0000 7fb81affd700 1 -- 172.21.15.110:0/2632223955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80c005040 con 0x7fb81c0fca30 2024-04-25T15:40:48.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.829+0000 7fb81affd700 1 --2- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 0x7fb81c10d080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009680 tx=0x7fb80c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:48.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.829+0000 7fb8137fe700 1 -- 172.21.15.110:0/2632223955 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb80c00b5c0 con 0x7fb81c0fca30 2024-04-25T15:40:48.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.829+0000 7fb8137fe700 1 -- 172.21.15.110:0/2632223955 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb80c00be70 con 0x7fb81c0fca30 2024-04-25T15:40:48.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.829+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb81c10d7c0 con 0x7fb81c0fca30 2024-04-25T15:40:48.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.829+0000 7fb8137fe700 1 -- 172.21.15.110:0/2632223955 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb80c0135a0 con 0x7fb81c0fca30 2024-04-25T15:40:48.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.829+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb81c10e330 con 0x7fb81c0fca30 2024-04-25T15:40:48.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.830+0000 7fb8137fe700 1 -- 172.21.15.110:0/2632223955 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fb80c013860 con 0x7fb81c0fca30 2024-04-25T15:40:48.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.830+0000 7fb8137fe700 1 --2- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb808036880 0x7fb808038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:48.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.830+0000 7fb8137fe700 1 -- 172.21.15.110:0/2632223955 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fb80c049540 con 0x7fb81c0fca30 2024-04-25T15:40:48.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.830+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb81c102f40 con 0x7fb81c0fca30 2024-04-25T15:40:48.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.830+0000 7fb81a7fc700 1 --2- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb808036880 0x7fb808038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:48.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.830+0000 7fb81a7fc700 1 --2- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb808036880 0x7fb808038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb804009990 tx=0x7fb804006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:48.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:48.835+0000 7fb8137fe700 1 -- 172.21.15.110:0/2632223955 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb80c013bb0 con 0x7fb81c0fca30 2024-04-25T15:40:49.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.033+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 --> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fb81c102920 con 0x7fb808036880 2024-04-25T15:40:49.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.036+0000 7fb8137fe700 1 -- 172.21.15.110:0/2632223955 <== mgr.14160 v2:172.21.15.110:6800/3209979562 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fb804009590 con 0x7fb808036880 2024-04-25T15:40:49.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.038+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb808036880 msgr2=0x7fb808038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:49.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.038+0000 7fb8214fe700 1 --2- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb808036880 0x7fb808038d30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb804009990 tx=0x7fb804006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:49.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.038+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 msgr2=0x7fb81c10d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:49.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.038+0000 7fb8214fe700 1 --2- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 0x7fb81c10d080 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb80c009680 tx=0x7fb80c009e30 comp rx=0 tx=0).stop 2024-04-25T15:40:49.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.038+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 shutdown_connections 2024-04-25T15:40:49.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.038+0000 7fb8214fe700 1 --2- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fb808036880 0x7fb808038d30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:49.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.038+0000 7fb8214fe700 1 --2- 172.21.15.110:0/2632223955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb81c0fca30 0x7fb81c10d080 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:49.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.038+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 >> 172.21.15.110:0/2632223955 conn(0x7fb81c0f8170 msgr2=0x7fb81c103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:49.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.039+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 shutdown_connections 2024-04-25T15:40:49.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:49.039+0000 7fb8214fe700 1 -- 172.21.15.110:0/2632223955 wait complete. 2024-04-25T15:40:49.970 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi193 2024-04-25T15:40:49.971 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:40:49.971 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.conf 2024-04-25T15:40:49.988 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:40:49.989 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:40:50.046 INFO:tasks.cephadm:Adding host smithi193 to orchestrator... 2024-04-25T15:40:50.046 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch host add smithi193 2024-04-25T15:40:50.268 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:50 smithi110 ceph-mon[26344]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:50.269 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:50 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:50.516 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:40:51.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:51 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:51.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:51 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:51.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:51 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:51.595 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:51 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:51.596 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:51 smithi110 ceph-mon[26344]: Deploying daemon node-exporter.smithi110 on smithi110 2024-04-25T15:40:52.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.108+0000 7f49d3654700 1 -- 172.21.15.110:0/2094392767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 msgr2=0x7f49cc0ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:52.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.108+0000 7f49d3654700 1 --2- 172.21.15.110:0/2094392767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 0x7f49cc0ff690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49c80048c0 tx=0x7f49c8009e30 comp rx=0 tx=0).stop 2024-04-25T15:40:52.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.108+0000 7f49d3654700 1 -- 172.21.15.110:0/2094392767 shutdown_connections 2024-04-25T15:40:52.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.108+0000 7f49d3654700 1 --2- 172.21.15.110:0/2094392767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 0x7f49cc0ff690 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:52.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.108+0000 7f49d3654700 1 -- 172.21.15.110:0/2094392767 >> 172.21.15.110:0/2094392767 conn(0x7f49cc0f70a0 msgr2=0x7f49cc0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:52.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.108+0000 7f49d3654700 1 -- 172.21.15.110:0/2094392767 shutdown_connections 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.108+0000 7f49d3654700 1 -- 172.21.15.110:0/2094392767 wait complete. 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.109+0000 7f49d3654700 1 Processor -- start 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.109+0000 7f49d3654700 1 -- start start 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.109+0000 7f49d3654700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 0x7f49cc10d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.109+0000 7f49d3654700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc10d5f0 con 0x7f49cc065cd0 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.109+0000 7f49d13f0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 0x7f49cc10d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.109+0000 7f49d13f0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 0x7f49cc10d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53252/0 (socket says 172.21.15.110:53252) 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.109+0000 7f49d13f0700 1 -- 172.21.15.110:0/2625824682 learned_addr learned my addr 172.21.15.110:0/2625824682 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.110+0000 7f49d13f0700 1 -- 172.21.15.110:0/2625824682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c8005040 con 0x7f49cc065cd0 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.110+0000 7f49d13f0700 1 --2- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 0x7f49cc10d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009680 tx=0x7f49c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.110+0000 7f49c1ffb700 1 -- 172.21.15.110:0/2625824682 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49c800bed0 con 0x7f49cc065cd0 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.110+0000 7f49c1ffb700 1 -- 172.21.15.110:0/2625824682 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49c800b5c0 con 0x7f49cc065cd0 2024-04-25T15:40:52.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.110+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc10d7f0 con 0x7f49cc065cd0 2024-04-25T15:40:52.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.110+0000 7f49c1ffb700 1 -- 172.21.15.110:0/2625824682 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49c801c5b0 con 0x7f49cc065cd0 2024-04-25T15:40:52.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.110+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc10e360 con 0x7f49cc065cd0 2024-04-25T15:40:52.113 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.111+0000 7f49c1ffb700 1 -- 172.21.15.110:0/2625824682 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f49c801c710 con 0x7f49cc065cd0 2024-04-25T15:40:52.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.111+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc102f40 con 0x7f49cc065cd0 2024-04-25T15:40:52.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.111+0000 7f49c1ffb700 1 --2- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f49b80368d0 0x7f49b8038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:52.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.111+0000 7f49c1ffb700 1 -- 172.21.15.110:0/2625824682 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f49c8049940 con 0x7f49cc065cd0 2024-04-25T15:40:52.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.112+0000 7f49d0bef700 1 --2- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f49b80368d0 0x7f49b8038d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:52.114 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.112+0000 7f49d0bef700 1 --2- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f49b80368d0 0x7f49b8038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009990 tx=0x7f49bc006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:52.117 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.116+0000 7f49c1ffb700 1 -- 172.21.15.110:0/2625824682 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f49c8025400 con 0x7f49cc065cd0 2024-04-25T15:40:52.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:52.322+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 --> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}) v1 -- 0x7f49cc10e5a0 con 0x7f49b80368d0 2024-04-25T15:40:52.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:52 smithi110 ceph-mon[26344]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:40:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:Added host 'smithi193' with addr '172.21.15.193' 2024-04-25T15:40:54.310 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.308+0000 7f49c1ffb700 1 -- 172.21.15.110:0/2625824682 <== mgr.14160 v2:172.21.15.110:6800/3209979562 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f49cc10e5a0 con 0x7f49b80368d0 2024-04-25T15:40:54.310 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:54 smithi110 ceph-mon[26344]: Deploying cephadm binary to smithi193 2024-04-25T15:40:54.310 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:54 smithi110 ceph-mon[26344]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:40:54.310 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:54 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f49b80368d0 msgr2=0x7f49b8038d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 --2- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f49b80368d0 0x7f49b8038d80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009990 tx=0x7f49bc006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 msgr2=0x7f49cc10d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 --2- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 0x7f49cc10d0b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f49c8009680 tx=0x7f49c8009e30 comp rx=0 tx=0).stop 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 shutdown_connections 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 --2- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f49b80368d0 0x7f49b8038d80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 --2- 172.21.15.110:0/2625824682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49cc065cd0 0x7f49cc10d0b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 >> 172.21.15.110:0/2625824682 conn(0x7f49cc0f70a0 msgr2=0x7f49cc103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 shutdown_connections 2024-04-25T15:40:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:54.311+0000 7f49d3654700 1 -- 172.21.15.110:0/2625824682 wait complete. 2024-04-25T15:40:55.262 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T15:40:55.504 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:55 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:55.504 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:55 smithi110 ceph-mon[26344]: Added host smithi193 2024-04-25T15:40:55.707 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:40:56.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:56 smithi110 ceph-mon[26344]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:40:56.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:56 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:56.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:56 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:56.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:56 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:56.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:56 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:57.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.279+0000 7f25e6299700 1 -- 172.21.15.110:0/2493081144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 msgr2=0x7f25e00fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:57.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.279+0000 7f25e6299700 1 --2- 172.21.15.110:0/2493081144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 0x7f25e00fbd50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f25d00048c0 tx=0x7f25d0009e30 comp rx=0 tx=0).stop 2024-04-25T15:40:57.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.279+0000 7f25e6299700 1 -- 172.21.15.110:0/2493081144 shutdown_connections 2024-04-25T15:40:57.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.279+0000 7f25e6299700 1 --2- 172.21.15.110:0/2493081144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 0x7f25e00fbd50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.279+0000 7f25e6299700 1 -- 172.21.15.110:0/2493081144 >> 172.21.15.110:0/2493081144 conn(0x7f25e00f70a0 msgr2=0x7f25e00f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.279+0000 7f25e6299700 1 -- 172.21.15.110:0/2493081144 shutdown_connections 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.279+0000 7f25e6299700 1 -- 172.21.15.110:0/2493081144 wait complete. 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.280+0000 7f25e6299700 1 Processor -- start 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.280+0000 7f25e6299700 1 -- start start 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.280+0000 7f25e6299700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 0x7f25e010d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.280+0000 7f25e6299700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e010d5b0 con 0x7f25e00fb940 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.280+0000 7f25dffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 0x7f25e010d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.280+0000 7f25dffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 0x7f25e010d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53266/0 (socket says 172.21.15.110:53266) 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.280+0000 7f25dffff700 1 -- 172.21.15.110:0/1581411676 learned_addr learned my addr 172.21.15.110:0/1581411676 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.280+0000 7f25dffff700 1 -- 172.21.15.110:0/1581411676 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d0005040 con 0x7f25e00fb940 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.281+0000 7f25dffff700 1 --2- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 0x7f25e010d070 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009680 tx=0x7f25d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:57.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.281+0000 7f25dcff9700 1 -- 172.21.15.110:0/1581411676 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25d000b5c0 con 0x7f25e00fb940 2024-04-25T15:40:57.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.281+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e010d7b0 con 0x7f25e00fb940 2024-04-25T15:40:57.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.281+0000 7f25dcff9700 1 -- 172.21.15.110:0/1581411676 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25d000be70 con 0x7f25e00fb940 2024-04-25T15:40:57.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.281+0000 7f25dcff9700 1 -- 172.21.15.110:0/1581411676 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25d00135a0 con 0x7f25e00fb940 2024-04-25T15:40:57.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.281+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e010e320 con 0x7f25e00fb940 2024-04-25T15:40:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.282+0000 7f25dcff9700 1 -- 172.21.15.110:0/1581411676 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f25d0013860 con 0x7f25e00fb940 2024-04-25T15:40:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.282+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e0101e50 con 0x7f25e00fb940 2024-04-25T15:40:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.282+0000 7f25dcff9700 1 --2- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f25cc036880 0x7f25cc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:40:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.282+0000 7f25dcff9700 1 -- 172.21.15.110:0/1581411676 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f25d0049330 con 0x7f25e00fb940 2024-04-25T15:40:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.282+0000 7f25df7fe700 1 --2- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f25cc036880 0x7f25cc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:40:57.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.282+0000 7f25df7fe700 1 --2- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f25cc036880 0x7f25cc038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009990 tx=0x7f25c8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:40:57.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.286+0000 7f25dcff9700 1 -- 172.21.15.110:0/1581411676 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f25d0011960 con 0x7f25e00fb940 2024-04-25T15:40:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:57 smithi110 ceph-mon[26344]: Deploying daemon alertmanager.smithi110 on smithi110 2024-04-25T15:40:57.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.506+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 --> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25e005f660 con 0x7f25cc036880 2024-04-25T15:40:57.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.507+0000 7f25dcff9700 1 -- 172.21.15.110:0/1581411676 <== mgr.14160 v2:172.21.15.110:6800/3209979562 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f25e005f660 con 0x7f25cc036880 2024-04-25T15:40:57.508 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:40:57.508 INFO:teuthology.orchestra.run.smithi110.stdout:[{"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}, {"addr": "172.21.15.193", "hostname": "smithi193", "labels": [], "status": ""}] 2024-04-25T15:40:57.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f25cc036880 msgr2=0x7f25cc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:57.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 --2- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f25cc036880 0x7f25cc038d30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f25c8009990 tx=0x7f25c8006e40 comp rx=0 tx=0).stop 2024-04-25T15:40:57.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 msgr2=0x7f25e010d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:40:57.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 --2- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 0x7f25e010d070 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009680 tx=0x7f25d0009e30 comp rx=0 tx=0).stop 2024-04-25T15:40:57.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 shutdown_connections 2024-04-25T15:40:57.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 --2- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f25cc036880 0x7f25cc038d30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:57.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 --2- 172.21.15.110:0/1581411676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25e00fb940 0x7f25e010d070 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:40:57.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 >> 172.21.15.110:0/1581411676 conn(0x7f25e00f70a0 msgr2=0x7f25e0065050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:40:57.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 shutdown_connections 2024-04-25T15:40:57.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:40:57.509+0000 7f25e6299700 1 -- 172.21.15.110:0/1581411676 wait complete. 2024-04-25T15:40:58.566 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:58 smithi110 ceph-mon[26344]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:40:58.567 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:58 smithi110 ceph-mon[26344]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:40:58.567 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:40:58 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:40:58.568 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T15:40:58.568 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T15:40:58.814 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:41:00.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:00 smithi110 ceph-mon[26344]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:01.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.403+0000 7f51eabb3700 1 -- 172.21.15.110:0/1744172510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 msgr2=0x7f51e40fe490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:01.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.403+0000 7f51eabb3700 1 --2- 172.21.15.110:0/1744172510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 0x7f51e40fe490 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f51dc009c60 tx=0x7f51dc004d10 comp rx=0 tx=0).stop 2024-04-25T15:41:01.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.403+0000 7f51eabb3700 1 -- 172.21.15.110:0/1744172510 shutdown_connections 2024-04-25T15:41:01.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.403+0000 7f51eabb3700 1 --2- 172.21.15.110:0/1744172510 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 0x7f51e40fe490 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.403+0000 7f51eabb3700 1 -- 172.21.15.110:0/1744172510 >> 172.21.15.110:0/1744172510 conn(0x7f51e40f6820 msgr2=0x7f51e40f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.403+0000 7f51eabb3700 1 -- 172.21.15.110:0/1744172510 shutdown_connections 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.403+0000 7f51eabb3700 1 -- 172.21.15.110:0/1744172510 wait complete. 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.403+0000 7f51eabb3700 1 Processor -- start 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51eabb3700 1 -- start start 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51eabb3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 0x7f51e410d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51eabb3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e410d5e0 con 0x7f51e40fe080 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51e894f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 0x7f51e410d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51e894f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 0x7f51e410d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52684/0 (socket says 172.21.15.110:52684) 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51e894f700 1 -- 172.21.15.110:0/2347269559 learned_addr learned my addr 172.21.15.110:0/2347269559 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51e894f700 1 -- 172.21.15.110:0/2347269559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc009940 con 0x7f51e40fe080 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51e894f700 1 --2- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 0x7f51e410d0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f51dc004e60 tx=0x7f51dc004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51e17fa700 1 -- 172.21.15.110:0/2347269559 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51dc012410 con 0x7f51e40fe080 2024-04-25T15:41:01.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.404+0000 7f51e17fa700 1 -- 172.21.15.110:0/2347269559 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51dc012a40 con 0x7f51e40fe080 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.405+0000 7f51e17fa700 1 -- 172.21.15.110:0/2347269559 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51dc01ba40 con 0x7f51e40fe080 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.405+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e410d7e0 con 0x7f51e40fe080 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.405+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e410e350 con 0x7f51e40fe080 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.405+0000 7f51e17fa700 1 -- 172.21.15.110:0/2347269559 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f51dc019040 con 0x7f51e40fe080 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.405+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e4102f50 con 0x7f51e40fe080 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.406+0000 7f51e17fa700 1 --2- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f51d0036880 0x7f51d0038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.406+0000 7f51e17fa700 1 -- 172.21.15.110:0/2347269559 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f51dc049da0 con 0x7f51e40fe080 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.406+0000 7f51e3fff700 1 --2- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f51d0036880 0x7f51d0038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.406+0000 7f51e3fff700 1 --2- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f51d0036880 0x7f51d0038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009990 tx=0x7f51d4006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:01.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.410+0000 7f51e17fa700 1 -- 172.21.15.110:0/2347269559 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f51dc0234f0 con 0x7f51e40fe080 2024-04-25T15:41:01.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:01.620+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f51e410ed60 con 0x7f51e40fe080 2024-04-25T15:41:02.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.317+0000 7f51e17fa700 1 -- 172.21.15.110:0/2347269559 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f51dc017020 con 0x7f51e40fe080 2024-04-25T15:41:02.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f51d0036880 msgr2=0x7f51d0038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:02.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 --2- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f51d0036880 0x7f51d0038d30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009990 tx=0x7f51d4006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:02.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 msgr2=0x7f51e410d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:02.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 --2- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 0x7f51e410d0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f51dc004e60 tx=0x7f51dc004d10 comp rx=0 tx=0).stop 2024-04-25T15:41:02.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 shutdown_connections 2024-04-25T15:41:02.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 --2- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f51d0036880 0x7f51d0038d30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:02.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 --2- 172.21.15.110:0/2347269559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51e40fe080 0x7f51e410d0a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:02.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 >> 172.21.15.110:0/2347269559 conn(0x7f51e40f6820 msgr2=0x7f51e4103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:02.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 shutdown_connections 2024-04-25T15:41:02.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:41:02.320+0000 7f51eabb3700 1 -- 172.21.15.110:0/2347269559 wait complete. 2024-04-25T15:41:02.322 INFO:teuthology.orchestra.run.smithi110.stderr:adjusted tunables profile to default 2024-04-25T15:41:02.590 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:02 smithi110 ceph-mon[26344]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:02.591 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:02 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2347269559' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T15:41:03.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:03 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2347269559' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T15:41:03.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:03 smithi110 ceph-mon[26344]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T15:41:03.822 INFO:tasks.cephadm:Adding mon.smithi110 on smithi110 2024-04-25T15:41:03.822 INFO:tasks.cephadm:Adding mon.smithi193 on smithi193 2024-04-25T15:41:03.822 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi110:172.21.15.110=smithi110;smithi193:172.21.15.193=smithi193' 2024-04-25T15:41:04.069 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:04.116 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:04.521 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:04 smithi110 ceph-mon[26344]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:05.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.641+0000 7fea30523700 1 -- 172.21.15.193:0/1895388941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 msgr2=0x7fea280ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:05.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.641+0000 7fea30523700 1 --2- 172.21.15.193:0/1895388941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 0x7fea280ff6f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fea20009c60 tx=0x7fea20004d10 comp rx=0 tx=0).stop 2024-04-25T15:41:05.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.641+0000 7fea30523700 1 -- 172.21.15.193:0/1895388941 shutdown_connections 2024-04-25T15:41:05.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.641+0000 7fea30523700 1 --2- 172.21.15.193:0/1895388941 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 0x7fea280ff6f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:05.647 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.641+0000 7fea30523700 1 -- 172.21.15.193:0/1895388941 >> 172.21.15.193:0/1895388941 conn(0x7fea280f70a0 msgr2=0x7fea280f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.641+0000 7fea30523700 1 -- 172.21.15.193:0/1895388941 shutdown_connections 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.641+0000 7fea30523700 1 -- 172.21.15.193:0/1895388941 wait complete. 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.642+0000 7fea30523700 1 Processor -- start 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.642+0000 7fea30523700 1 -- start start 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.642+0000 7fea30523700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 0x7fea2810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.642+0000 7fea30523700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2810d5d0 con 0x7fea28065c70 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.642+0000 7fea2e2bf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 0x7fea2810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.642+0000 7fea2e2bf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 0x7fea2810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:49000/0 (socket says 172.21.15.193:49000) 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.642+0000 7fea2e2bf700 1 -- 172.21.15.193:0/2539167193 learned_addr learned my addr 172.21.15.193:0/2539167193 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.643+0000 7fea2e2bf700 1 -- 172.21.15.193:0/2539167193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea20009940 con 0x7fea28065c70 2024-04-25T15:41:05.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.643+0000 7fea2e2bf700 1 --2- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 0x7fea2810d090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea200097b0 tx=0x7fea20004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.643+0000 7fea26ffd700 1 -- 172.21.15.193:0/2539167193 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea20012410 con 0x7fea28065c70 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.643+0000 7fea26ffd700 1 -- 172.21.15.193:0/2539167193 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea20012a40 con 0x7fea28065c70 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.643+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2810d7d0 con 0x7fea28065c70 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.643+0000 7fea26ffd700 1 -- 172.21.15.193:0/2539167193 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fea2001b980 con 0x7fea28065c70 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.643+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2810e340 con 0x7fea28065c70 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.644+0000 7fea26ffd700 1 -- 172.21.15.193:0/2539167193 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fea20019040 con 0x7fea28065c70 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.644+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea28102f40 con 0x7fea28065c70 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.644+0000 7fea26ffd700 1 --2- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fea10036880 0x7fea10038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.644+0000 7fea26ffd700 1 -- 172.21.15.193:0/2539167193 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fea20049d60 con 0x7fea28065c70 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.644+0000 7fea2dabe700 1 --2- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fea10036880 0x7fea10038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:05.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.645+0000 7fea2dabe700 1 --2- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fea10036880 0x7fea10038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009990 tx=0x7fea1c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:05.650 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.649+0000 7fea26ffd700 1 -- 172.21.15.193:0/2539167193 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fea20022b50 con 0x7fea28065c70 2024-04-25T15:41:05.841 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.841+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 --> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi110:172.21.15.110=smithi110;smithi193:172.21.15.193=smithi193", "target": ["mon-mgr", ""]}) v1 -- 0x7fea2810e630 con 0x7fea10036880 2024-04-25T15:41:05.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.849+0000 7fea26ffd700 1 -- 172.21.15.193:0/2539167193 <== mgr.14160 v2:172.21.15.110:6800/3209979562 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fea2810e630 con 0x7fea10036880 2024-04-25T15:41:05.850 INFO:teuthology.orchestra.run.smithi193.stdout:Scheduled mon update... 2024-04-25T15:41:05.851 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fea10036880 msgr2=0x7fea10038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 --2- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fea10036880 0x7fea10038d30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009990 tx=0x7fea1c006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 msgr2=0x7fea2810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 --2- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 0x7fea2810d090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fea200097b0 tx=0x7fea20004d10 comp rx=0 tx=0).stop 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 shutdown_connections 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 --2- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fea10036880 0x7fea10038d30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 --2- 172.21.15.193:0/2539167193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea28065c70 0x7fea2810d090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 >> 172.21.15.193:0/2539167193 conn(0x7fea280f70a0 msgr2=0x7fea28103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 shutdown_connections 2024-04-25T15:41:05.852 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:05.851+0000 7fea30523700 1 -- 172.21.15.193:0/2539167193 wait complete. 2024-04-25T15:41:06.345 DEBUG:teuthology.orchestra.run.smithi193:mon.smithi193> sudo journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi193.service 2024-04-25T15:41:06.348 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:06.348 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:06.439 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:06 smithi110 ceph-mon[26344]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:06.439 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:06 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:06.622 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:06.672 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='client.14192 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi110:172.21.15.110=smithi110;smithi193:172.21.15.193=smithi193", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:41:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: Saving service mon spec with placement smithi110:172.21.15.110=smithi110;smithi193:172.21.15.193=smithi193;count:2 2024-04-25T15:41:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-25T15:41:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:07 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:08.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.001+0000 7fd72de52700 1 -- 172.21.15.193:0/3366241142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 msgr2=0x7fd7280fd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:08.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.001+0000 7fd72de52700 1 --2- 172.21.15.193:0/3366241142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 0x7fd7280fd330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd720009c60 tx=0x7fd720004d10 comp rx=0 tx=0).stop 2024-04-25T15:41:08.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.001+0000 7fd72de52700 1 -- 172.21.15.193:0/3366241142 shutdown_connections 2024-04-25T15:41:08.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.001+0000 7fd72de52700 1 --2- 172.21.15.193:0/3366241142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 0x7fd7280fd330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:08.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.001+0000 7fd72de52700 1 -- 172.21.15.193:0/3366241142 >> 172.21.15.193:0/3366241142 conn(0x7fd7280f6850 msgr2=0x7fd7280f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:08.003 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.001+0000 7fd72de52700 1 -- 172.21.15.193:0/3366241142 shutdown_connections 2024-04-25T15:41:08.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.001+0000 7fd72de52700 1 -- 172.21.15.193:0/3366241142 wait complete. 2024-04-25T15:41:08.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.002+0000 7fd72de52700 1 Processor -- start 2024-04-25T15:41:08.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.002+0000 7fd72de52700 1 -- start start 2024-04-25T15:41:08.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.002+0000 7fd72de52700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 0x7fd72810d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:08.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.002+0000 7fd72de52700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72810d600 con 0x7fd7280fcf20 2024-04-25T15:41:08.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.002+0000 7fd7277fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 0x7fd72810d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:08.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.002+0000 7fd7277fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 0x7fd72810d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:41550/0 (socket says 172.21.15.193:41550) 2024-04-25T15:41:08.004 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.002+0000 7fd7277fe700 1 -- 172.21.15.193:0/233242682 learned_addr learned my addr 172.21.15.193:0/233242682 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:08.005 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.003+0000 7fd7277fe700 1 -- 172.21.15.193:0/233242682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd720009940 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.003+0000 7fd7277fe700 1 --2- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 0x7fd72810d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097b0 tx=0x7fd720004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.003+0000 7fd71ffff700 1 -- 172.21.15.193:0/233242682 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd720012410 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.003+0000 7fd71ffff700 1 -- 172.21.15.193:0/233242682 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd720012a40 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.003+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72810d800 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.003+0000 7fd71ffff700 1 -- 172.21.15.193:0/233242682 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd72001b980 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.003+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72810e370 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.004+0000 7fd71ffff700 1 -- 172.21.15.193:0/233242682 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7fd720019040 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.004+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd728101e50 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.004+0000 7fd71ffff700 1 --2- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fd714036880 0x7fd714038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.004+0000 7fd71ffff700 1 -- 172.21.15.193:0/233242682 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fd720049d60 con 0x7fd7280fcf20 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.004+0000 7fd726ffd700 1 --2- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fd714036880 0x7fd714038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:08.006 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.005+0000 7fd726ffd700 1 --2- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fd714036880 0x7fd714038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd710009990 tx=0x7fd710006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:08.010 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.009+0000 7fd71ffff700 1 -- 172.21.15.193:0/233242682 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd720022b50 con 0x7fd7280fcf20 2024-04-25T15:41:08.277 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.276+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd72810e5b0 con 0x7fd7280fcf20 2024-04-25T15:41:08.277 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.277+0000 7fd71ffff700 1 -- 172.21.15.193:0/233242682 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fd720017090 con 0x7fd7280fcf20 2024-04-25T15:41:08.278 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:08.278 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:08.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.278+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fd714036880 msgr2=0x7fd714038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:08.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.278+0000 7fd72de52700 1 --2- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fd714036880 0x7fd714038d30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd710009990 tx=0x7fd710006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:08.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.279+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 msgr2=0x7fd72810d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:08.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.279+0000 7fd72de52700 1 --2- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 0x7fd72810d0c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd7200097b0 tx=0x7fd720004d10 comp rx=0 tx=0).stop 2024-04-25T15:41:08.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.279+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 shutdown_connections 2024-04-25T15:41:08.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.279+0000 7fd72de52700 1 --2- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7fd714036880 0x7fd714038d30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:08.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.279+0000 7fd72de52700 1 --2- 172.21.15.193:0/233242682 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd7280fcf20 0x7fd72810d0c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:08.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.279+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 >> 172.21.15.193:0/233242682 conn(0x7fd7280f6850 msgr2=0x7fd7280fab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:08.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.279+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 shutdown_connections 2024-04-25T15:41:08.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:08.279+0000 7fd72de52700 1 -- 172.21.15.193:0/233242682 wait complete. 2024-04-25T15:41:08.280 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:08 smithi110 ceph-mon[26344]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-25T15:41:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:08 smithi110 ceph-mon[26344]: Deploying daemon grafana.smithi110 on smithi110 2024-04-25T15:41:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:08 smithi110 ceph-mon[26344]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:08 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:08 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/233242682' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:09.864 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:09.864 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:10.122 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:10.173 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:10.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:10 smithi110 ceph-mon[26344]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 -- 172.21.15.193:0/1547868779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 msgr2=0x7f41c40fd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 --2- 172.21.15.193:0/1547868779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 0x7f41c40fd420 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f41b40048c0 tx=0x7f41b4009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 -- 172.21.15.193:0/1547868779 shutdown_connections 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 --2- 172.21.15.193:0/1547868779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 0x7f41c40fd420 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 -- 172.21.15.193:0/1547868779 >> 172.21.15.193:0/1547868779 conn(0x7f41c40f70a0 msgr2=0x7f41c40f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 -- 172.21.15.193:0/1547868779 shutdown_connections 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 -- 172.21.15.193:0/1547868779 wait complete. 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 Processor -- start 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 -- start start 2024-04-25T15:41:11.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 0x7f41c41136b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.626+0000 7f41c8976700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c4113bf0 con 0x7f41c40669a0 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 0x7f41c41136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 0x7f41c41136b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:41556/0 (socket says 172.21.15.193:41556) 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41c259c700 1 -- 172.21.15.193:0/3637624852 learned_addr learned my addr 172.21.15.193:0/3637624852 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41c259c700 1 -- 172.21.15.193:0/3637624852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b4005040 con 0x7f41c40669a0 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41c259c700 1 --2- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 0x7f41c41136b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009680 tx=0x7f41b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41b2ffd700 1 -- 172.21.15.193:0/3637624852 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41b400bed0 con 0x7f41c40669a0 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41b2ffd700 1 -- 172.21.15.193:0/3637624852 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41b400b5c0 con 0x7f41c40669a0 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41b2ffd700 1 -- 172.21.15.193:0/3637624852 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41b401c5b0 con 0x7f41c40669a0 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41c4113df0 con 0x7f41c40669a0 2024-04-25T15:41:11.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.627+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c4114960 con 0x7f41c40669a0 2024-04-25T15:41:11.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.628+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c4100d30 con 0x7f41c40669a0 2024-04-25T15:41:11.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.628+0000 7f41b2ffd700 1 -- 172.21.15.193:0/3637624852 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f41b4013420 con 0x7f41c40669a0 2024-04-25T15:41:11.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.628+0000 7f41b2ffd700 1 --2- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f41ac036880 0x7f41ac038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:11.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.628+0000 7f41b2ffd700 1 -- 172.21.15.193:0/3637624852 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f41b4049a00 con 0x7f41c40669a0 2024-04-25T15:41:11.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.628+0000 7f41c1d9b700 1 --2- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f41ac036880 0x7f41ac038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:11.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.629+0000 7f41c1d9b700 1 --2- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f41ac036880 0x7f41ac038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009990 tx=0x7f41b8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:11.633 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.633+0000 7f41b2ffd700 1 -- 172.21.15.193:0/3637624852 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f41b4013a80 con 0x7f41c40669a0 2024-04-25T15:41:11.894 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.893+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f41c4115430 con 0x7f41c40669a0 2024-04-25T15:41:11.894 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.893+0000 7f41b2ffd700 1 -- 172.21.15.193:0/3637624852 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f41b4011370 con 0x7f41c40669a0 2024-04-25T15:41:11.894 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:11.895 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:11.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.895+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f41ac036880 msgr2=0x7f41ac038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:11.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.895+0000 7f41c8976700 1 --2- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f41ac036880 0x7f41ac038d30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009990 tx=0x7f41b8006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:11.896 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.895+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 msgr2=0x7f41c41136b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:11.897 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.895+0000 7f41c8976700 1 --2- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 0x7f41c41136b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f41b4009680 tx=0x7f41b4009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:11.897 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.895+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 shutdown_connections 2024-04-25T15:41:11.897 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.895+0000 7f41c8976700 1 --2- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f41ac036880 0x7f41ac038d30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:11.897 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.895+0000 7f41c8976700 1 --2- 172.21.15.193:0/3637624852 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41c40669a0 0x7f41c41136b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:11.897 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.895+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 >> 172.21.15.193:0/3637624852 conn(0x7f41c40f70a0 msgr2=0x7f41c410a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:11.897 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.896+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 shutdown_connections 2024-04-25T15:41:11.897 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:11.896+0000 7f41c8976700 1 -- 172.21.15.193:0/3637624852 wait complete. 2024-04-25T15:41:11.897 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:12.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:12 smithi110 ceph-mon[26344]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:12.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:12 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/3637624852' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:13.501 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:13.502 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:13.750 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:13.805 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:14.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:14 smithi110 ceph-mon[26344]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.075+0000 7f5f9daec700 1 -- 172.21.15.193:0/1744947362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 msgr2=0x7f5f98070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.075+0000 7f5f9daec700 1 --2- 172.21.15.193:0/1744947362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 0x7f5f98070350 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f880048c0 tx=0x7f5f88009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.075+0000 7f5f9daec700 1 -- 172.21.15.193:0/1744947362 shutdown_connections 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.075+0000 7f5f9daec700 1 --2- 172.21.15.193:0/1744947362 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 0x7f5f98070350 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.075+0000 7f5f9daec700 1 -- 172.21.15.193:0/1744947362 >> 172.21.15.193:0/1744947362 conn(0x7f5f980f8170 msgr2=0x7f5f980fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.075+0000 7f5f9daec700 1 -- 172.21.15.193:0/1744947362 shutdown_connections 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.075+0000 7f5f9daec700 1 -- 172.21.15.193:0/1744947362 wait complete. 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.075+0000 7f5f9daec700 1 Processor -- start 2024-04-25T15:41:15.076 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f9daec700 1 -- start start 2024-04-25T15:41:15.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f9daec700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 0x7f5f9810f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:15.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f9daec700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9810f820 con 0x7f5f98071e40 2024-04-25T15:41:15.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f977fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 0x7f5f9810f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:15.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f977fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 0x7f5f9810f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:41580/0 (socket says 172.21.15.193:41580) 2024-04-25T15:41:15.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f977fe700 1 -- 172.21.15.193:0/2456187318 learned_addr learned my addr 172.21.15.193:0/2456187318 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:15.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f977fe700 1 -- 172.21.15.193:0/2456187318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f88005040 con 0x7f5f98071e40 2024-04-25T15:41:15.078 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f977fe700 1 --2- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 0x7f5f9810f2e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800b900 tx=0x7f5f88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f7ffff700 1 -- 172.21.15.193:0/2456187318 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5f8800b5c0 con 0x7f5f98071e40 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9810fa20 con 0x7f5f98071e40 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f7ffff700 1 -- 172.21.15.193:0/2456187318 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f8800be70 con 0x7f5f98071e40 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.076+0000 7f5f7ffff700 1 -- 172.21.15.193:0/2456187318 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5f8801c590 con 0x7f5f98071e40 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.077+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f98110590 con 0x7f5f98071e40 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.077+0000 7f5f7ffff700 1 -- 172.21.15.193:0/2456187318 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f5f88013420 con 0x7f5f98071e40 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.077+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f98105150 con 0x7f5f98071e40 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.077+0000 7f5f7ffff700 1 --2- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5f80036880 0x7f5f80038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.078+0000 7f5f7ffff700 1 -- 172.21.15.193:0/2456187318 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5f88049400 con 0x7f5f98071e40 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.078+0000 7f5f96ffd700 1 --2- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5f80036880 0x7f5f80038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:15.079 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.078+0000 7f5f96ffd700 1 --2- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5f80036880 0x7f5f80038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009990 tx=0x7f5f8c006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:15.083 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.082+0000 7f5f7ffff700 1 -- 172.21.15.193:0/2456187318 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5f88011550 con 0x7f5f98071e40 2024-04-25T15:41:15.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.344+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5f9805f5c0 con 0x7f5f98071e40 2024-04-25T15:41:15.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.344+0000 7f5f7ffff700 1 -- 172.21.15.193:0/2456187318 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f5f88021020 con 0x7f5f98071e40 2024-04-25T15:41:15.345 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:15.345 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:15.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.346+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5f80036880 msgr2=0x7f5f80038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:15.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.346+0000 7f5f9daec700 1 --2- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5f80036880 0x7f5f80038d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009990 tx=0x7f5f8c006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:15.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.346+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 msgr2=0x7f5f9810f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:15.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.346+0000 7f5f9daec700 1 --2- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 0x7f5f9810f2e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800b900 tx=0x7f5f88009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:15.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.346+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 shutdown_connections 2024-04-25T15:41:15.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.346+0000 7f5f9daec700 1 --2- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5f80036880 0x7f5f80038d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:15.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.346+0000 7f5f9daec700 1 --2- 172.21.15.193:0/2456187318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f98071e40 0x7f5f9810f2e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:15.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.346+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 >> 172.21.15.193:0/2456187318 conn(0x7f5f980f8170 msgr2=0x7f5f98105d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:15.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.347+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 shutdown_connections 2024-04-25T15:41:15.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:15.347+0000 7f5f9daec700 1 -- 172.21.15.193:0/2456187318 wait complete. 2024-04-25T15:41:15.348 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:16.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:16 smithi110 ceph-mon[26344]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:16.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:16 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/2456187318' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:16.946 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:16.946 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:17.194 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:17.241 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.445+0000 7f55a8aff700 1 -- 172.21.15.193:0/3132526624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 msgr2=0x7f55a40ff6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.445+0000 7f55a8aff700 1 --2- 172.21.15.193:0/3132526624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 0x7f55a40ff6c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f55980048c0 tx=0x7f5598009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.445+0000 7f55a8aff700 1 -- 172.21.15.193:0/3132526624 shutdown_connections 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.445+0000 7f55a8aff700 1 --2- 172.21.15.193:0/3132526624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 0x7f55a40ff6c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.445+0000 7f55a8aff700 1 -- 172.21.15.193:0/3132526624 >> 172.21.15.193:0/3132526624 conn(0x7f55a40f7070 msgr2=0x7f55a40f94c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.445+0000 7f55a8aff700 1 -- 172.21.15.193:0/3132526624 shutdown_connections 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.445+0000 7f55a8aff700 1 -- 172.21.15.193:0/3132526624 wait complete. 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.446+0000 7f55a8aff700 1 Processor -- start 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.446+0000 7f55a8aff700 1 -- start start 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.446+0000 7f55a8aff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 0x7f55a40fd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.446+0000 7f55a8aff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40fd550 con 0x7f55a4066900 2024-04-25T15:41:18.448 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.446+0000 7f55a259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 0x7f55a40fd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.446+0000 7f55a259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 0x7f55a40fd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:36734/0 (socket says 172.21.15.193:36734) 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.446+0000 7f55a259c700 1 -- 172.21.15.193:0/1027372871 learned_addr learned my addr 172.21.15.193:0/1027372871 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.446+0000 7f55a259c700 1 -- 172.21.15.193:0/1027372871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598005040 con 0x7f55a4066900 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.447+0000 7f55a259c700 1 --2- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 0x7f55a40fd010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f559800b880 tx=0x7f5598009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.447+0000 7f5592ffd700 1 -- 172.21.15.193:0/1027372871 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f559800bbc0 con 0x7f55a4066900 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.447+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a40fb620 con 0x7f55a4066900 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.447+0000 7f5592ffd700 1 -- 172.21.15.193:0/1027372871 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f559800bd20 con 0x7f55a4066900 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.447+0000 7f5592ffd700 1 -- 172.21.15.193:0/1027372871 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f559801c690 con 0x7f55a4066900 2024-04-25T15:41:18.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.447+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a40fbb10 con 0x7f55a4066900 2024-04-25T15:41:18.450 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.448+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a405f5c0 con 0x7f55a4066900 2024-04-25T15:41:18.450 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.448+0000 7f5592ffd700 1 -- 172.21.15.193:0/1027372871 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f5598013420 con 0x7f55a4066900 2024-04-25T15:41:18.450 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.448+0000 7f5592ffd700 1 --2- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f558c036880 0x7f558c038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:18.450 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.448+0000 7f5592ffd700 1 -- 172.21.15.193:0/1027372871 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f5598049b00 con 0x7f55a4066900 2024-04-25T15:41:18.450 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.448+0000 7f55a1d9b700 1 --2- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f558c036880 0x7f558c038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:18.450 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.449+0000 7f55a1d9b700 1 --2- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f558c036880 0x7f558c038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5594009990 tx=0x7f5594006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:18.453 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.453+0000 7f5592ffd700 1 -- 172.21.15.193:0/1027372871 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5598011940 con 0x7f55a4066900 2024-04-25T15:41:18.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:18 smithi110 ceph-mon[26344]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:18.710 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.709+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f55a404bed0 con 0x7f55a4066900 2024-04-25T15:41:18.711 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.710+0000 7f5592ffd700 1 -- 172.21.15.193:0/1027372871 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f5598011370 con 0x7f55a4066900 2024-04-25T15:41:18.711 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:18.711 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f558c036880 msgr2=0x7f558c038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 --2- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f558c036880 0x7f558c038d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5594009990 tx=0x7f5594006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 msgr2=0x7f55a40fd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 --2- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 0x7f55a40fd010 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f559800b880 tx=0x7f5598009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 shutdown_connections 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 --2- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f558c036880 0x7f558c038d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 --2- 172.21.15.193:0/1027372871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55a4066900 0x7f55a40fd010 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 >> 172.21.15.193:0/1027372871 conn(0x7f55a40f7070 msgr2=0x7f55a4103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 shutdown_connections 2024-04-25T15:41:18.713 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:18.712+0000 7f55a8aff700 1 -- 172.21.15.193:0/1027372871 wait complete. 2024-04-25T15:41:18.714 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:19.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:19 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/1027372871' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:20.269 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:20.269 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:20.512 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:20.541 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:20 smithi110 ceph-mon[26344]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:20.559 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:21.900 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 -- 172.21.15.193:0/2256219954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 msgr2=0x7f7af40ff610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 --2- 172.21.15.193:0/2256219954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 0x7f7af40ff610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40048c0 tx=0x7f7ae4009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 -- 172.21.15.193:0/2256219954 shutdown_connections 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 --2- 172.21.15.193:0/2256219954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 0x7f7af40ff610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 -- 172.21.15.193:0/2256219954 >> 172.21.15.193:0/2256219954 conn(0x7f7af40f70a0 msgr2=0x7f7af40f94b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 -- 172.21.15.193:0/2256219954 shutdown_connections 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 -- 172.21.15.193:0/2256219954 wait complete. 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 Processor -- start 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.899+0000 7f7af9223700 1 -- start start 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af9223700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 0x7f7af410d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:21.901 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af9223700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af410d610 con 0x7f7af40669e0 2024-04-25T15:41:21.902 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af2d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 0x7f7af410d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:21.902 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af2d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 0x7f7af410d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:36754/0 (socket says 172.21.15.193:36754) 2024-04-25T15:41:21.902 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af2d9d700 1 -- 172.21.15.193:0/3730836895 learned_addr learned my addr 172.21.15.193:0/3730836895 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:21.902 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af2d9d700 1 -- 172.21.15.193:0/3730836895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae4005040 con 0x7f7af40669e0 2024-04-25T15:41:21.902 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af2d9d700 1 --2- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 0x7f7af410d0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009680 tx=0x7f7ae4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:21.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7adb7fe700 1 -- 172.21.15.193:0/3730836895 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae400bed0 con 0x7f7af40669e0 2024-04-25T15:41:21.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7adb7fe700 1 -- 172.21.15.193:0/3730836895 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ae400b5c0 con 0x7f7af40669e0 2024-04-25T15:41:21.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af410d810 con 0x7f7af40669e0 2024-04-25T15:41:21.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af410e380 con 0x7f7af40669e0 2024-04-25T15:41:21.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.900+0000 7f7adb7fe700 1 -- 172.21.15.193:0/3730836895 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae4025560 con 0x7f7af40669e0 2024-04-25T15:41:21.903 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.901+0000 7f7adb7fe700 1 -- 172.21.15.193:0/3730836895 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f7ae40256c0 con 0x7f7af40669e0 2024-04-25T15:41:21.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.901+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af4102f20 con 0x7f7af40669e0 2024-04-25T15:41:21.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.901+0000 7f7adb7fe700 1 --2- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f7adc036880 0x7f7adc038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:21.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.901+0000 7f7adb7fe700 1 -- 172.21.15.193:0/3730836895 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f7ae4049bf0 con 0x7f7af40669e0 2024-04-25T15:41:21.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.901+0000 7f7af259c700 1 --2- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f7adc036880 0x7f7adc038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:21.904 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.902+0000 7f7af259c700 1 --2- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f7adc036880 0x7f7adc038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009990 tx=0x7f7ae8006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:21.907 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:21.906+0000 7f7adb7fe700 1 -- 172.21.15.193:0/3730836895 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7ae401c3f0 con 0x7f7af40669e0 2024-04-25T15:41:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:21 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:21 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:21 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:21 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:21 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:21 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:21 smithi110 ceph-mon[26344]: Deploying daemon prometheus.smithi110 on smithi110 2024-04-25T15:41:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:21 smithi110 ceph-mon[26344]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:22.169 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.168+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7af410ed90 con 0x7f7af40669e0 2024-04-25T15:41:22.169 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.169+0000 7f7adb7fe700 1 -- 172.21.15.193:0/3730836895 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f7ae4011b60 con 0x7f7af40669e0 2024-04-25T15:41:22.169 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:22.170 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:22.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f7adc036880 msgr2=0x7f7adc038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:22.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 --2- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f7adc036880 0x7f7adc038d30 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009990 tx=0x7f7ae8006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 msgr2=0x7f7af410d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 --2- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 0x7f7af410d0d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009680 tx=0x7f7ae4009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 shutdown_connections 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 --2- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f7adc036880 0x7f7adc038d30 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 --2- 172.21.15.193:0/3730836895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7af40669e0 0x7f7af410d0d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 >> 172.21.15.193:0/3730836895 conn(0x7f7af40f70a0 msgr2=0x7f7af40f78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 shutdown_connections 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:22.171+0000 7f7af9223700 1 -- 172.21.15.193:0/3730836895 wait complete. 2024-04-25T15:41:22.172 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:23.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:22 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/3730836895' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:23.757 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:23.758 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:24.003 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:24.049 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:24.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:24 smithi110 ceph-mon[26344]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:24.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:24 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:25.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.340+0000 7f513c52b700 1 -- 172.21.15.193:0/1162267830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 msgr2=0x7f51340fd430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:25.342 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.340+0000 7f513c52b700 1 --2- 172.21.15.193:0/1162267830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 0x7f51340fd430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f51300048c0 tx=0x7f5130009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.340+0000 7f513c52b700 1 -- 172.21.15.193:0/1162267830 shutdown_connections 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.340+0000 7f513c52b700 1 --2- 172.21.15.193:0/1162267830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 0x7f51340fd430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.340+0000 7f513c52b700 1 -- 172.21.15.193:0/1162267830 >> 172.21.15.193:0/1162267830 conn(0x7f51340f70a0 msgr2=0x7f51340f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.340+0000 7f513c52b700 1 -- 172.21.15.193:0/1162267830 shutdown_connections 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.340+0000 7f513c52b700 1 -- 172.21.15.193:0/1162267830 wait complete. 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.340+0000 7f513c52b700 1 Processor -- start 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513c52b700 1 -- start start 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513c52b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 0x7f5134066460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513c52b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f513410fdb0 con 0x7f51340fd020 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513a2c7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 0x7f5134066460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513a2c7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 0x7f5134066460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:36766/0 (socket says 172.21.15.193:36766) 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513a2c7700 1 -- 172.21.15.193:0/1926842888 learned_addr learned my addr 172.21.15.193:0/1926842888 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:25.343 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513a2c7700 1 -- 172.21.15.193:0/1926842888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130005040 con 0x7f51340fd020 2024-04-25T15:41:25.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513a2c7700 1 --2- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 0x7f5134066460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5130009680 tx=0x7f5130009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:25.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f512affd700 1 -- 172.21.15.193:0/1926842888 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f513000b5c0 con 0x7f51340fd020 2024-04-25T15:41:25.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f513410fef0 con 0x7f51340fd020 2024-04-25T15:41:25.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.341+0000 7f512affd700 1 -- 172.21.15.193:0/1926842888 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f513000be70 con 0x7f51340fd020 2024-04-25T15:41:25.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.342+0000 7f512affd700 1 -- 172.21.15.193:0/1926842888 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51300135a0 con 0x7f51340fd020 2024-04-25T15:41:25.344 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.342+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51341103e0 con 0x7f51340fd020 2024-04-25T15:41:25.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.342+0000 7f512affd700 1 -- 172.21.15.193:0/1926842888 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f5130013860 con 0x7f51340fd020 2024-04-25T15:41:25.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.342+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51341058f0 con 0x7f51340fd020 2024-04-25T15:41:25.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.343+0000 7f512affd700 1 --2- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5120036880 0x7f5120038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:25.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.343+0000 7f512affd700 1 -- 172.21.15.193:0/1926842888 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f51300495b0 con 0x7f51340fd020 2024-04-25T15:41:25.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.343+0000 7f5139ac6700 1 --2- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5120036880 0x7f5120038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:25.345 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.343+0000 7f5139ac6700 1 --2- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5120036880 0x7f5120038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5124009990 tx=0x7f5124006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:25.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.347+0000 7f512affd700 1 -- 172.21.15.193:0/1926842888 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5130011960 con 0x7f51340fd020 2024-04-25T15:41:25.620 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.619+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f513405f5c0 con 0x7f51340fd020 2024-04-25T15:41:25.620 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.619+0000 7f512affd700 1 -- 172.21.15.193:0/1926842888 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f5130011370 con 0x7f51340fd020 2024-04-25T15:41:25.620 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:25.620 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5120036880 msgr2=0x7f5120038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 --2- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5120036880 0x7f5120038d30 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5124009990 tx=0x7f5124006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 msgr2=0x7f5134066460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 --2- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 0x7f5134066460 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5130009680 tx=0x7f5130009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 shutdown_connections 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 --2- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f5120036880 0x7f5120038d30 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 --2- 172.21.15.193:0/1926842888 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f51340fd020 0x7f5134066460 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 >> 172.21.15.193:0/1926842888 conn(0x7f51340f70a0 msgr2=0x7f5134102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:25.622 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 shutdown_connections 2024-04-25T15:41:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:25.621+0000 7f513c52b700 1 -- 172.21.15.193:0/1926842888 wait complete. 2024-04-25T15:41:25.623 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:26.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:26 smithi110 ceph-mon[26344]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:26.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:26 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/1926842888' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:27.156 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:27.157 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:27.407 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:27.455 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:28.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:28 smithi110 ceph-mon[26344]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:28.963 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 -- 172.21.15.193:0/2732933346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 msgr2=0x7f58cc0fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 --2- 172.21.15.193:0/2732933346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 0x7f58cc0fbd50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0048c0 tx=0x7f58bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 -- 172.21.15.193:0/2732933346 shutdown_connections 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 --2- 172.21.15.193:0/2732933346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 0x7f58cc0fbd50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 -- 172.21.15.193:0/2732933346 >> 172.21.15.193:0/2732933346 conn(0x7f58cc0f70a0 msgr2=0x7f58cc0f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 -- 172.21.15.193:0/2732933346 shutdown_connections 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 -- 172.21.15.193:0/2732933346 wait complete. 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 Processor -- start 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 -- start start 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 0x7f58cc10bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.957+0000 7f58d138d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58cc10c4d0 con 0x7f58cc0fb940 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58caffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 0x7f58cc10bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:28.964 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58caffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 0x7f58cc10bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:39654/0 (socket says 172.21.15.193:39654) 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58caffd700 1 -- 172.21.15.193:0/1693306861 learned_addr learned my addr 172.21.15.193:0/1693306861 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58caffd700 1 -- 172.21.15.193:0/1693306861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc005040 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58caffd700 1 --2- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 0x7f58cc10bf90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0096b0 tx=0x7f58bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58b37fe700 1 -- 172.21.15.193:0/1693306861 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58bc00bed0 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58cc10c6d0 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58b37fe700 1 -- 172.21.15.193:0/1693306861 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58bc00b5c0 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58b37fe700 1 -- 172.21.15.193:0/1693306861 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f58bc01c5b0 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.958+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58cc10d240 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.959+0000 7f58b37fe700 1 -- 172.21.15.193:0/1693306861 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f58bc01c820 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.959+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58cc101e50 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.959+0000 7f58b37fe700 1 --2- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f58b4036880 0x7f58b4038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.959+0000 7f58b37fe700 1 -- 172.21.15.193:0/1693306861 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f58bc049c90 con 0x7f58cc0fb940 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.959+0000 7f58ca7fc700 1 --2- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f58b4036880 0x7f58b4038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:28.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.960+0000 7f58ca7fc700 1 --2- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f58b4036880 0x7f58b4038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009990 tx=0x7f58c0006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:28.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:28.965+0000 7f58b37fe700 1 -- 172.21.15.193:0/1693306861 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f58bc023300 con 0x7f58cc0fb940 2024-04-25T15:41:29.226 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.224+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f58cc10dc50 con 0x7f58cc0fb940 2024-04-25T15:41:29.226 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.225+0000 7f58b37fe700 1 -- 172.21.15.193:0/1693306861 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f58bc011b60 con 0x7f58cc0fb940 2024-04-25T15:41:29.227 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:29.227 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:29.229 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f58b4036880 msgr2=0x7f58b4038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:29.229 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 --2- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f58b4036880 0x7f58b4038d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f58c0009990 tx=0x7f58c0006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:29.229 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 msgr2=0x7f58cc10bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:29.229 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 --2- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 0x7f58cc10bf90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0096b0 tx=0x7f58bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:29.229 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 shutdown_connections 2024-04-25T15:41:29.229 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 --2- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f58b4036880 0x7f58b4038d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:29.230 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 --2- 172.21.15.193:0/1693306861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58cc0fb940 0x7f58cc10bf90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:29.230 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 >> 172.21.15.193:0/1693306861 conn(0x7f58cc0f70a0 msgr2=0x7f58cc102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:29.230 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 shutdown_connections 2024-04-25T15:41:29.230 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:29.227+0000 7f58d138d700 1 -- 172.21.15.193:0/1693306861 wait complete. 2024-04-25T15:41:29.230 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:30.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:30 smithi110 ceph-mon[26344]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:30.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:30 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/1693306861' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:30.750 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:30.750 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:30.996 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:31.044 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:32.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 -- 172.21.15.193:0/1349412278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 msgr2=0x7f15180fce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:32.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 --2- 172.21.15.193:0/1349412278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 0x7f15180fce40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f150c0048c0 tx=0x7f150c009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:32.288 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 -- 172.21.15.193:0/1349412278 shutdown_connections 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 --2- 172.21.15.193:0/1349412278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 0x7f15180fce40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 -- 172.21.15.193:0/1349412278 >> 172.21.15.193:0/1349412278 conn(0x7f15180f8170 msgr2=0x7f15180fa5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 -- 172.21.15.193:0/1349412278 shutdown_connections 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 -- 172.21.15.193:0/1349412278 wait complete. 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 Processor -- start 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.287+0000 7f151f15a700 1 -- start start 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f151f15a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 0x7f151810d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f151f15a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151810d5f0 con 0x7f15180fca30 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f151cef6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 0x7f151810d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f151cef6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 0x7f151810d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:39672/0 (socket says 172.21.15.193:39672) 2024-04-25T15:41:32.289 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f151cef6700 1 -- 172.21.15.193:0/3919818620 learned_addr learned my addr 172.21.15.193:0/3919818620 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f151cef6700 1 -- 172.21.15.193:0/3919818620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f150c005040 con 0x7f15180fca30 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f151cef6700 1 --2- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 0x7f151810d0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f150c0096b0 tx=0x7f150c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f15157fa700 1 -- 172.21.15.193:0/3919818620 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f150c00bed0 con 0x7f15180fca30 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151810d7f0 con 0x7f15180fca30 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.288+0000 7f15157fa700 1 -- 172.21.15.193:0/3919818620 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f150c00b5c0 con 0x7f15180fca30 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.289+0000 7f15157fa700 1 -- 172.21.15.193:0/3919818620 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f150c01c5b0 con 0x7f15180fca30 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.289+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151810e360 con 0x7f15180fca30 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.289+0000 7f15157fa700 1 -- 172.21.15.193:0/3919818620 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) v1 ==== 43433+0+0 (secure 0 0 0) 0x7f150c01c820 con 0x7f15180fca30 2024-04-25T15:41:32.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.289+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1518102f40 con 0x7f15180fca30 2024-04-25T15:41:32.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.289+0000 7f15157fa700 1 --2- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f1508036880 0x7f1508038d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:32.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.289+0000 7f15157fa700 1 -- 172.21.15.193:0/3919818620 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f150c049d00 con 0x7f15180fca30 2024-04-25T15:41:32.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.290+0000 7f1517fff700 1 --2- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f1508036880 0x7f1508038d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:32.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.290+0000 7f1517fff700 1 --2- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f1508036880 0x7f1508038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1504009990 tx=0x7f1504006e40 comp rx=0 tx=0).ready entity=mgr.14160 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:32.297 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.297+0000 7f15157fa700 1 -- 172.21.15.193:0/3919818620 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f150c023300 con 0x7f15180fca30 2024-04-25T15:41:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:32 smithi110 ceph-mon[26344]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:32.558 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.558+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f151810ee30 con 0x7f15180fca30 2024-04-25T15:41:32.559 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.558+0000 7f15157fa700 1 -- 172.21.15.193:0/3919818620 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f150c011b60 con 0x7f15180fca30 2024-04-25T15:41:32.559 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:32.559 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:32.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.560+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f1508036880 msgr2=0x7f1508038d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:32.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.560+0000 7f151f15a700 1 --2- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f1508036880 0x7f1508038d30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f1504009990 tx=0x7f1504006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:32.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.560+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 msgr2=0x7f151810d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:32.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.560+0000 7f151f15a700 1 --2- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 0x7f151810d0b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f150c0096b0 tx=0x7f150c009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:32.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.561+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 shutdown_connections 2024-04-25T15:41:32.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.561+0000 7f151f15a700 1 --2- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f1508036880 0x7f1508038d30 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:32.562 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.561+0000 7f151f15a700 1 --2- 172.21.15.193:0/3919818620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15180fca30 0x7f151810d0b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:32.562 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.561+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 >> 172.21.15.193:0/3919818620 conn(0x7f15180f8170 msgr2=0x7f1518103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:32.562 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.561+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 shutdown_connections 2024-04-25T15:41:32.562 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:32.561+0000 7f151f15a700 1 -- 172.21.15.193:0/3919818620 wait complete. 2024-04-25T15:41:32.562 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:33.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:33 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/3919818620' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:33.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:33.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:33.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:33.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:33 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-25T15:41:34.180 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:34.181 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:34.197 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:34 smithi110 ceph-mon[26344]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:41:34.197 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:34 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:34.198 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:34 smithi110 ceph-mon[26344]: from='mgr.14160 172.21.15.110:0/3927255674' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-25T15:41:34.198 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:34 smithi110 ceph-mon[26344]: mgrmap e14: smithi110.lvgbkf(active, since 60s) 2024-04-25T15:41:34.427 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:34.475 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:35.819 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.817+0000 7f844d57d700 1 -- 172.21.15.193:0/3845833016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 msgr2=0x7f84480ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:35.819 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.817+0000 7f844d57d700 1 --2- 172.21.15.193:0/3845833016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 0x7f84480ff630 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f84380048c0 tx=0x7f8438009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:35.819 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.817+0000 7f844d57d700 1 -- 172.21.15.193:0/3845833016 shutdown_connections 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.817+0000 7f844d57d700 1 --2- 172.21.15.193:0/3845833016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 0x7f84480ff630 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.817+0000 7f844d57d700 1 -- 172.21.15.193:0/3845833016 >> 172.21.15.193:0/3845833016 conn(0x7f84480f70a0 msgr2=0x7f84480f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.818+0000 7f844d57d700 1 -- 172.21.15.193:0/3845833016 shutdown_connections 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.818+0000 7f844d57d700 1 -- 172.21.15.193:0/3845833016 wait complete. 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.818+0000 7f844d57d700 1 Processor -- start 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.818+0000 7f844d57d700 1 -- start start 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.818+0000 7f844d57d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 0x7f844810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.818+0000 7f844d57d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844810d5d0 con 0x7f8448065d30 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.818+0000 7f8446ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 0x7f844810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f8446ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 0x7f844810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:39686/0 (socket says 172.21.15.193:39686) 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f8446ffd700 1 -- 172.21.15.193:0/1351506913 learned_addr learned my addr 172.21.15.193:0/1351506913 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f8446ffd700 1 -- 172.21.15.193:0/1351506913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8438005040 con 0x7f8448065d30 2024-04-25T15:41:35.820 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f8446ffd700 1 --2- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 0x7f844810d090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8438009680 tx=0x7f8438009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f842f7fe700 1 -- 172.21.15.193:0/1351506913 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f843800bed0 con 0x7f8448065d30 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844810d7d0 con 0x7f8448065d30 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f842f7fe700 1 -- 172.21.15.193:0/1351506913 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f843800b5c0 con 0x7f8448065d30 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f842f7fe700 1 -- 172.21.15.193:0/1351506913 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f843801c5b0 con 0x7f8448065d30 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.819+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844810e340 con 0x7f8448065d30 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.820+0000 7f842f7fe700 1 -- 172.21.15.193:0/1351506913 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7f8438013420 con 0x7f8448065d30 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.820+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8448102f40 con 0x7f8448065d30 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.820+0000 7f842f7fe700 1 --2- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f84300368d0 0x7f8430038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.820+0000 7f842f7fe700 1 -- 172.21.15.193:0/1351506913 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f84380499c0 con 0x7f8448065d30 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.820+0000 7f84467fc700 1 -- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f84300368d0 msgr2=0x7f8430038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/3209979562 2024-04-25T15:41:35.822 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.821+0000 7f84467fc700 1 --2- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f84300368d0 0x7f8430038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:41:35.829 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:35.829+0000 7f842f7fe700 1 -- 172.21.15.193:0/1351506913 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8438013a80 con 0x7f8448065d30 2024-04-25T15:41:36.022 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.022+0000 7f84467fc700 1 -- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f84300368d0 msgr2=0x7f8430038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/3209979562 2024-04-25T15:41:36.023 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.022+0000 7f84467fc700 1 --2- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f84300368d0 0x7f8430038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:41:36.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.091+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f844810ed50 con 0x7f8448065d30 2024-04-25T15:41:36.092 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.092+0000 7f842f7fe700 1 -- 172.21.15.193:0/1351506913 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f8438011370 con 0x7f8448065d30 2024-04-25T15:41:36.093 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:36.093 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:36.094 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f84300368d0 msgr2=0x7f8430038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 --2- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f84300368d0 0x7f8430038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 msgr2=0x7f844810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 --2- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 0x7f844810d090 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8438009680 tx=0x7f8438009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 shutdown_connections 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 --2- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f84300368d0 0x7f8430038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 --2- 172.21.15.193:0/1351506913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8448065d30 0x7f844810d090 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 >> 172.21.15.193:0/1351506913 conn(0x7f84480f70a0 msgr2=0x7f8448103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 shutdown_connections 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:36.094+0000 7f844d57d700 1 -- 172.21.15.193:0/1351506913 wait complete. 2024-04-25T15:41:36.095 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:36.347 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:36 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/1351506913' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:37.644 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:37.644 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:37.891 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:37.947 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:39.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.290+0000 7f833f04f700 1 -- 172.21.15.193:0/3161923952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 msgr2=0x7f83380fce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:39.291 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.290+0000 7f833f04f700 1 --2- 172.21.15.193:0/3161923952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 0x7f83380fce50 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8330009c60 tx=0x7f8330004d10 comp rx=0 tx=0).stop 2024-04-25T15:41:39.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.290+0000 7f833f04f700 1 -- 172.21.15.193:0/3161923952 shutdown_connections 2024-04-25T15:41:39.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.290+0000 7f833f04f700 1 --2- 172.21.15.193:0/3161923952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 0x7f83380fce50 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:39.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.290+0000 7f833f04f700 1 -- 172.21.15.193:0/3161923952 >> 172.21.15.193:0/3161923952 conn(0x7f83380f81c0 msgr2=0x7f83380fa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:39.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.290+0000 7f833f04f700 1 -- 172.21.15.193:0/3161923952 shutdown_connections 2024-04-25T15:41:39.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.290+0000 7f833f04f700 1 -- 172.21.15.193:0/3161923952 wait complete. 2024-04-25T15:41:39.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.291+0000 7f833f04f700 1 Processor -- start 2024-04-25T15:41:39.292 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.291+0000 7f833f04f700 1 -- start start 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.291+0000 7f833f04f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 0x7f833810d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.291+0000 7f833f04f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833810d5d0 con 0x7f83380fca40 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.291+0000 7f833cdeb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 0x7f833810d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.291+0000 7f833cdeb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 0x7f833810d090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:43812/0 (socket says 172.21.15.193:43812) 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.291+0000 7f833cdeb700 1 -- 172.21.15.193:0/4141805039 learned_addr learned my addr 172.21.15.193:0/4141805039 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.292+0000 7f833cdeb700 1 -- 172.21.15.193:0/4141805039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8330009940 con 0x7f83380fca40 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.292+0000 7f833cdeb700 1 --2- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 0x7f833810d090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8330009500 tx=0x7f8330004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.292+0000 7f83357fa700 1 -- 172.21.15.193:0/4141805039 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8330007c80 con 0x7f83380fca40 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.292+0000 7f83357fa700 1 -- 172.21.15.193:0/4141805039 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f833001d910 con 0x7f83380fca40 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.292+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833810d7d0 con 0x7f83380fca40 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.292+0000 7f83357fa700 1 -- 172.21.15.193:0/4141805039 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f83300129d0 con 0x7f83380fca40 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.292+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833810e340 con 0x7f83380fca40 2024-04-25T15:41:39.294 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.293+0000 7f83357fa700 1 -- 172.21.15.193:0/4141805039 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 14) v1 ==== 43447+0+0 (secure 0 0 0) 0x7f8330019040 con 0x7f83380fca40 2024-04-25T15:41:39.295 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.293+0000 7f83357fa700 1 --2- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f83240368d0 0x7f8324038d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:39.295 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.293+0000 7f83357fa700 1 -- 172.21.15.193:0/4141805039 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f833004a350 con 0x7f83380fca40 2024-04-25T15:41:39.295 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.293+0000 7f8337fff700 1 -- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f83240368d0 msgr2=0x7f8324038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/3209979562 2024-04-25T15:41:39.295 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.293+0000 7f8337fff700 1 --2- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f83240368d0 0x7f8324038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:41:39.295 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.293+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8338102f50 con 0x7f83380fca40 2024-04-25T15:41:39.298 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.298+0000 7f83357fa700 1 -- 172.21.15.193:0/4141805039 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8330017080 con 0x7f83380fca40 2024-04-25T15:41:39.494 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.494+0000 7f8337fff700 1 -- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f83240368d0 msgr2=0x7f8324038d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/3209979562 2024-04-25T15:41:39.495 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.494+0000 7f8337fff700 1 --2- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f83240368d0 0x7f8324038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:41:39.560 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.559+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f833805f5c0 con 0x7f83380fca40 2024-04-25T15:41:39.561 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.560+0000 7f83357fa700 1 -- 172.21.15.193:0/4141805039 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f8330021020 con 0x7f83380fca40 2024-04-25T15:41:39.561 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:39.561 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:39.562 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f83240368d0 msgr2=0x7f8324038d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 --2- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f83240368d0 0x7f8324038d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 msgr2=0x7f833810d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 --2- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 0x7f833810d090 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8330009500 tx=0x7f8330004d10 comp rx=0 tx=0).stop 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 shutdown_connections 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 --2- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:6800/3209979562,v1:172.21.15.110:6801/3209979562] conn(0x7f83240368d0 0x7f8324038d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 --2- 172.21.15.193:0/4141805039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83380fca40 0x7f833810d090 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 >> 172.21.15.193:0/4141805039 conn(0x7f83380f81c0 msgr2=0x7f8338103b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 shutdown_connections 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:39.562+0000 7f833f04f700 1 -- 172.21.15.193:0/4141805039 wait complete. 2024-04-25T15:41:39.563 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:39 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/4141805039' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: Active manager daemon smithi110.lvgbkf restarted 2024-04-25T15:41:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: Activating manager daemon smithi110.lvgbkf 2024-04-25T15:41:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: osdmap e5: 0 total, 0 up, 0 in 2024-04-25T15:41:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: mgrmap e15: smithi110.lvgbkf(active, starting, since 0.00282526s) 2024-04-25T15:41:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:41:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:41:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:41:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:41:40.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:41:40.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: Manager daemon smithi110.lvgbkf is now available 2024-04-25T15:41:40.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:41:40.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:41:40.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:40.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:41:40.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:41:40.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:41:41.150 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:41.151 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:41.393 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:41.442 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-25T15:41:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:41 smithi110 ceph-mon[26344]: [25/Apr/2024:15:41:40] ENGINE Bus STARTING 2024-04-25T15:41:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:41 smithi110 ceph-mon[26344]: [25/Apr/2024:15:41:41] ENGINE Serving on https://172.21.15.110:7150 2024-04-25T15:41:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:41 smithi110 ceph-mon[26344]: [25/Apr/2024:15:41:41] ENGINE Bus STARTED 2024-04-25T15:41:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:42.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:41 smithi110 ceph-mon[26344]: mgrmap e16: smithi110.lvgbkf(active, since 1.00506s) 2024-04-25T15:41:42.782 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.780+0000 7fcff74a1700 1 -- 172.21.15.193:0/82686611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 msgr2=0x7fcff00fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:42.782 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.780+0000 7fcff74a1700 1 --2- 172.21.15.193:0/82686611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 0x7fcff00fbd50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0048c0 tx=0x7fcfec009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:42.782 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.780+0000 7fcff74a1700 1 -- 172.21.15.193:0/82686611 shutdown_connections 2024-04-25T15:41:42.782 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.780+0000 7fcff74a1700 1 --2- 172.21.15.193:0/82686611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 0x7fcff00fbd50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:42.782 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.780+0000 7fcff74a1700 1 -- 172.21.15.193:0/82686611 >> 172.21.15.193:0/82686611 conn(0x7fcff00f70a0 msgr2=0x7fcff00f94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:42.782 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.780+0000 7fcff74a1700 1 -- 172.21.15.193:0/82686611 shutdown_connections 2024-04-25T15:41:42.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.780+0000 7fcff74a1700 1 -- 172.21.15.193:0/82686611 wait complete. 2024-04-25T15:41:42.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.781+0000 7fcff74a1700 1 Processor -- start 2024-04-25T15:41:42.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.781+0000 7fcff74a1700 1 -- start start 2024-04-25T15:41:42.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.781+0000 7fcff74a1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 0x7fcff00664a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:42.783 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.781+0000 7fcff74a1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff010fd50 con 0x7fcff00fb940 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.781+0000 7fcff523d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 0x7fcff00664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.781+0000 7fcff523d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 0x7fcff00664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:43826/0 (socket says 172.21.15.193:43826) 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.781+0000 7fcff523d700 1 -- 172.21.15.193:0/1792418141 learned_addr learned my addr 172.21.15.193:0/1792418141 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.782+0000 7fcff523d700 1 -- 172.21.15.193:0/1792418141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec005040 con 0x7fcff00fb940 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.782+0000 7fcff523d700 1 --2- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 0x7fcff00664a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009680 tx=0x7fcfec009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.782+0000 7fcfe5ffb700 1 -- 172.21.15.193:0/1792418141 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcfec00b5c0 con 0x7fcff00fb940 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.782+0000 7fcfe5ffb700 1 -- 172.21.15.193:0/1792418141 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcfec00be70 con 0x7fcff00fb940 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.782+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff010fef0 con 0x7fcff00fb940 2024-04-25T15:41:42.785 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.782+0000 7fcfe5ffb700 1 -- 172.21.15.193:0/1792418141 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcfec0135a0 con 0x7fcff00fb940 2024-04-25T15:41:42.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.782+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff0110380 con 0x7fcff00fb940 2024-04-25T15:41:42.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.783+0000 7fcfe5ffb700 1 -- 172.21.15.193:0/1792418141 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) v1 ==== 43297+0+0 (secure 0 0 0) 0x7fcfec013860 con 0x7fcff00fb940 2024-04-25T15:41:42.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.783+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff0105900 con 0x7fcff00fb940 2024-04-25T15:41:42.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.783+0000 7fcfe5ffb700 1 --2- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcfdc036710 0x7fcfdc038bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:42.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.783+0000 7fcfe5ffb700 1 -- 172.21.15.193:0/1792418141 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fcfec01c6b0 con 0x7fcff00fb940 2024-04-25T15:41:42.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.784+0000 7fcff4a3c700 1 --2- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcfdc036710 0x7fcfdc038bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:42.786 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.784+0000 7fcff4a3c700 1 --2- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcfdc036710 0x7fcfdc038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009990 tx=0x7fcfe0006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:42.789 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.788+0000 7fcfe5ffb700 1 -- 172.21.15.193:0/1792418141 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fcfec011960 con 0x7fcff00fb940 2024-04-25T15:41:42.844 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:42.843+0000 7fcfe5ffb700 1 -- 172.21.15.193:0/1792418141 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fcfec004d30 con 0x7fcff00fb940 2024-04-25T15:41:43.065 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.065+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcff005f5c0 con 0x7fcff00fb940 2024-04-25T15:41:43.066 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.065+0000 7fcfe5ffb700 1 -- 172.21.15.193:0/1792418141 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fcfec011960 con 0x7fcff00fb940 2024-04-25T15:41:43.066 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:43.066 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:43.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.067+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcfdc036710 msgr2=0x7fcfdc038bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:43.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.067+0000 7fcff74a1700 1 --2- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcfdc036710 0x7fcfdc038bc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009990 tx=0x7fcfe0006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:43.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.067+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 msgr2=0x7fcff00664a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:43.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.067+0000 7fcff74a1700 1 --2- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 0x7fcff00664a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009680 tx=0x7fcfec009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:43.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.067+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 shutdown_connections 2024-04-25T15:41:43.068 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.067+0000 7fcff74a1700 1 --2- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcfdc036710 0x7fcfdc038bc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:43.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.067+0000 7fcff74a1700 1 --2- 172.21.15.193:0/1792418141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcff00fb940 0x7fcff00664a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:43.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.067+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 >> 172.21.15.193:0/1792418141 conn(0x7fcff00f70a0 msgr2=0x7fcff0102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:43.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.068+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 shutdown_connections 2024-04-25T15:41:43.069 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:43.068+0000 7fcff74a1700 1 -- 172.21.15.193:0/1792418141 wait complete. 2024-04-25T15:41:43.069 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:43 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:43 smithi110 ceph-mon[26344]: mgrmap e17: smithi110.lvgbkf(active, since 2s) 2024-04-25T15:41:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:43 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/1792418141' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:45.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:45 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:45.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:45 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:45.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:45 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:45.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:45 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:45.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:45 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:41:45.936 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:45.937 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:46.184 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:41:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd/host:smithi193", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:41:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:41:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:41:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-25T15:41:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:41:47.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.765+0000 7fbc40c50700 1 -- 172.21.15.193:0/3715116353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 msgr2=0x7fbc3c102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:47.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.765+0000 7fbc40c50700 1 --2- 172.21.15.193:0/3715116353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 0x7fbc3c102e70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbc300048c0 tx=0x7fbc30009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:47.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.765+0000 7fbc40c50700 1 -- 172.21.15.193:0/3715116353 shutdown_connections 2024-04-25T15:41:47.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.765+0000 7fbc40c50700 1 --2- 172.21.15.193:0/3715116353 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 0x7fbc3c102e70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:47.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.765+0000 7fbc40c50700 1 -- 172.21.15.193:0/3715116353 >> 172.21.15.193:0/3715116353 conn(0x7fbc3c0f85c0 msgr2=0x7fbc3c0fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:47.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.765+0000 7fbc40c50700 1 -- 172.21.15.193:0/3715116353 shutdown_connections 2024-04-25T15:41:47.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.765+0000 7fbc40c50700 1 -- 172.21.15.193:0/3715116353 wait complete. 2024-04-25T15:41:47.768 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.766+0000 7fbc40c50700 1 Processor -- start 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.766+0000 7fbc40c50700 1 -- start start 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.766+0000 7fbc40c50700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 0x7fbc3c113100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.766+0000 7fbc40c50700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3c113640 con 0x7fbc3c102aa0 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.766+0000 7fbc3a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 0x7fbc3c113100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.766+0000 7fbc3a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 0x7fbc3c113100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:47408/0 (socket says 172.21.15.193:47408) 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.766+0000 7fbc3a59c700 1 -- 172.21.15.193:0/1264746011 learned_addr learned my addr 172.21.15.193:0/1264746011 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.767+0000 7fbc3a59c700 1 -- 172.21.15.193:0/1264746011 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc30005040 con 0x7fbc3c102aa0 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.767+0000 7fbc3a59c700 1 --2- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 0x7fbc3c113100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000b820 tx=0x7fbc30009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.767+0000 7fbc2affd700 1 -- 172.21.15.193:0/1264746011 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc300093c0 con 0x7fbc3c102aa0 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.767+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc3c113840 con 0x7fbc3c102aa0 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.767+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc3c113d30 con 0x7fbc3c102aa0 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.767+0000 7fbc2affd700 1 -- 172.21.15.193:0/1264746011 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc30013420 con 0x7fbc3c102aa0 2024-04-25T15:41:47.769 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.767+0000 7fbc2affd700 1 -- 172.21.15.193:0/1264746011 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbc300255c0 con 0x7fbc3c102aa0 2024-04-25T15:41:47.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.768+0000 7fbc2affd700 1 -- 172.21.15.193:0/1264746011 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fbc3001c3f0 con 0x7fbc3c102aa0 2024-04-25T15:41:47.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.768+0000 7fbc2affd700 1 --2- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbc24036920 0x7fbc24038dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:47.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.768+0000 7fbc2affd700 1 -- 172.21.15.193:0/1264746011 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fbc30049b90 con 0x7fbc3c102aa0 2024-04-25T15:41:47.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.768+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc1c0052b0 con 0x7fbc3c102aa0 2024-04-25T15:41:47.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.768+0000 7fbc39d9b700 1 --2- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbc24036920 0x7fbc24038dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:47.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.769+0000 7fbc39d9b700 1 --2- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbc24036920 0x7fbc24038dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009990 tx=0x7fbc2c006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:47.774 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:47.773+0000 7fbc2affd700 1 -- 172.21.15.193:0/1264746011 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbc3001a6b0 con 0x7fbc3c102aa0 2024-04-25T15:41:48.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.043+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fbc1c005560 con 0x7fbc3c102aa0 2024-04-25T15:41:48.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.044+0000 7fbc2affd700 1 -- 172.21.15.193:0/1264746011 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fbc30011370 con 0x7fbc3c102aa0 2024-04-25T15:41:48.045 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:48.045 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:48.048 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbc24036920 msgr2=0x7fbc24038dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 --2- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbc24036920 0x7fbc24038dd0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c009990 tx=0x7fbc2c006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 msgr2=0x7fbc3c113100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 --2- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 0x7fbc3c113100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc3000b820 tx=0x7fbc30009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 shutdown_connections 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 --2- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbc24036920 0x7fbc24038dd0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 --2- 172.21.15.193:0/1264746011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc3c102aa0 0x7fbc3c113100 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 >> 172.21.15.193:0/1264746011 conn(0x7fbc3c0f85c0 msgr2=0x7fbc3c0f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 shutdown_connections 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:48.048+0000 7fbc40c50700 1 -- 172.21.15.193:0/1264746011 wait complete. 2024-04-25T15:41:48.049 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:47 smithi110 ceph-mon[26344]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:41:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:47 smithi110 ceph-mon[26344]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:41:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:47 smithi110 ceph-mon[26344]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:41:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:47 smithi110 ceph-mon[26344]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:41:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:47 smithi110 ceph-mon[26344]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:41:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:47 smithi110 ceph-mon[26344]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:41:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:47 smithi110 ceph-mon[26344]: Deploying daemon crash.smithi193 on smithi193 2024-04-25T15:41:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:48 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/1264746011' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:50.445 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:50.445 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:50.763 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:41:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:51 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:51 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:51 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:51 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:51 smithi110 ceph-mon[26344]: Deploying daemon node-exporter.smithi193 on smithi193 2024-04-25T15:41:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:51 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:52.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.299+0000 7f9e5e898700 1 -- 172.21.15.193:0/4096274142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 msgr2=0x7f9e58101da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:52.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.299+0000 7f9e5e898700 1 --2- 172.21.15.193:0/4096274142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 0x7f9e58101da0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c0048c0 tx=0x7f9e4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:52.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 -- 172.21.15.193:0/4096274142 shutdown_connections 2024-04-25T15:41:52.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 --2- 172.21.15.193:0/4096274142 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 0x7f9e58101da0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:52.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 -- 172.21.15.193:0/4096274142 >> 172.21.15.193:0/4096274142 conn(0x7f9e58073090 msgr2=0x7f9e58073490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:52.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 -- 172.21.15.193:0/4096274142 shutdown_connections 2024-04-25T15:41:52.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 -- 172.21.15.193:0/4096274142 wait complete. 2024-04-25T15:41:52.302 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 Processor -- start 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 -- start start 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 0x7f9e5810bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.300+0000 7f9e5e898700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e580fbfc0 con 0x7f9e581019d0 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e57fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 0x7f9e5810bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e57fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 0x7f9e5810bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:47438/0 (socket says 172.21.15.193:47438) 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e57fff700 1 -- 172.21.15.193:0/3957108000 learned_addr learned my addr 172.21.15.193:0/3957108000 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e57fff700 1 -- 172.21.15.193:0/3957108000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c005040 con 0x7f9e581019d0 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e57fff700 1 --2- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 0x7f9e5810bef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00b820 tx=0x7f9e4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e54ff9700 1 -- 172.21.15.193:0/3957108000 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e4c0093f0 con 0x7f9e581019d0 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5810c490 con 0x7f9e581019d0 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e54ff9700 1 -- 172.21.15.193:0/3957108000 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e4c013420 con 0x7f9e581019d0 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5810c900 con 0x7f9e581019d0 2024-04-25T15:41:52.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.301+0000 7f9e54ff9700 1 -- 172.21.15.193:0/3957108000 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9e4c025560 con 0x7f9e581019d0 2024-04-25T15:41:52.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.302+0000 7f9e54ff9700 1 -- 172.21.15.193:0/3957108000 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f9e4c009550 con 0x7f9e581019d0 2024-04-25T15:41:52.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.302+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e581039e0 con 0x7f9e581019d0 2024-04-25T15:41:52.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.302+0000 7f9e54ff9700 1 --2- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9e400369c0 0x7f9e40038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:52.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.303+0000 7f9e54ff9700 1 -- 172.21.15.193:0/3957108000 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f9e4c049b40 con 0x7f9e581019d0 2024-04-25T15:41:52.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.303+0000 7f9e577fe700 1 --2- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9e400369c0 0x7f9e40038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:52.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.303+0000 7f9e577fe700 1 --2- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9e400369c0 0x7f9e40038e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009990 tx=0x7f9e48006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:52.308 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.307+0000 7f9e54ff9700 1 -- 172.21.15.193:0/3957108000 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9e4c011d00 con 0x7f9e581019d0 2024-04-25T15:41:52.581 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.580+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9e580fd150 con 0x7f9e581019d0 2024-04-25T15:41:52.582 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.581+0000 7f9e54ff9700 1 -- 172.21.15.193:0/3957108000 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f9e4c011b20 con 0x7f9e581019d0 2024-04-25T15:41:52.582 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:52.582 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9e400369c0 msgr2=0x7f9e40038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 --2- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9e400369c0 0x7f9e40038e70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009990 tx=0x7f9e48006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 msgr2=0x7f9e5810bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 --2- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 0x7f9e5810bef0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9e4c00b820 tx=0x7f9e4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 shutdown_connections 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 --2- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9e400369c0 0x7f9e40038e70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 --2- 172.21.15.193:0/3957108000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9e581019d0 0x7f9e5810bef0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 >> 172.21.15.193:0/3957108000 conn(0x7f9e58073090 msgr2=0x7f9e581026e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:52.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 shutdown_connections 2024-04-25T15:41:52.585 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:52.583+0000 7f9e5e898700 1 -- 172.21.15.193:0/3957108000 wait complete. 2024-04-25T15:41:52.585 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:53.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:52 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/3957108000' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:55.027 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:55.028 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:41:55.283 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:41:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:56.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:56.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:41:56.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T15:41:56.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:41:56.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:41:56.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: Deploying daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:41:56.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:41:56.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.302+0000 7fb3c7405700 1 -- 172.21.15.193:0/2512666176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 msgr2=0x7fb3c0102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.302+0000 7fb3c7405700 1 --2- 172.21.15.193:0/2512666176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 0x7fb3c0102e80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc0048c0 tx=0x7fb3bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.302+0000 7fb3c7405700 1 -- 172.21.15.193:0/2512666176 shutdown_connections 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.302+0000 7fb3c7405700 1 --2- 172.21.15.193:0/2512666176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 0x7fb3c0102e80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.302+0000 7fb3c7405700 1 -- 172.21.15.193:0/2512666176 >> 172.21.15.193:0/2512666176 conn(0x7fb3c00f85b0 msgr2=0x7fb3c00fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.302+0000 7fb3c7405700 1 -- 172.21.15.193:0/2512666176 shutdown_connections 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.302+0000 7fb3c7405700 1 -- 172.21.15.193:0/2512666176 wait complete. 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.303+0000 7fb3c7405700 1 Processor -- start 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.303+0000 7fb3c7405700 1 -- start start 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.303+0000 7fb3c7405700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 0x7fb3c0113130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:57.304 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.303+0000 7fb3c7405700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0113670 con 0x7fb3c0102ab0 2024-04-25T15:41:57.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.303+0000 7fb3c51a1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 0x7fb3c0113130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:57.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.303+0000 7fb3c51a1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 0x7fb3c0113130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:47446/0 (socket says 172.21.15.193:47446) 2024-04-25T15:41:57.305 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.303+0000 7fb3c51a1700 1 -- 172.21.15.193:0/420487641 learned_addr learned my addr 172.21.15.193:0/420487641 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.304+0000 7fb3c51a1700 1 -- 172.21.15.193:0/420487641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3bc005040 con 0x7fb3c0102ab0 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.304+0000 7fb3c51a1700 1 --2- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 0x7fb3c0113130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00b7f0 tx=0x7fb3bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.304+0000 7fb3b5ffb700 1 -- 172.21.15.193:0/420487641 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3bc009370 con 0x7fb3c0102ab0 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.304+0000 7fb3b5ffb700 1 -- 172.21.15.193:0/420487641 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb3bc013420 con 0x7fb3c0102ab0 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.304+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c0113870 con 0x7fb3c0102ab0 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.304+0000 7fb3b5ffb700 1 -- 172.21.15.193:0/420487641 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb3bc01c590 con 0x7fb3c0102ab0 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.304+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c0113d60 con 0x7fb3c0102ab0 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.305+0000 7fb3b5ffb700 1 -- 172.21.15.193:0/420487641 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7fb3bc0094d0 con 0x7fb3c0102ab0 2024-04-25T15:41:57.306 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.305+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c00fe1b0 con 0x7fb3c0102ab0 2024-04-25T15:41:57.307 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.305+0000 7fb3b5ffb700 1 --2- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb3ac0369c0 0x7fb3ac038e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:41:57.307 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.305+0000 7fb3b5ffb700 1 -- 172.21.15.193:0/420487641 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fb3bc049440 con 0x7fb3c0102ab0 2024-04-25T15:41:57.307 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.305+0000 7fb3c49a0700 1 --2- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb3ac0369c0 0x7fb3ac038e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:41:57.307 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.306+0000 7fb3c49a0700 1 --2- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb3ac0369c0 0x7fb3ac038e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009990 tx=0x7fb3b0006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:41:57.310 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.310+0000 7fb3b5ffb700 1 -- 172.21.15.193:0/420487641 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb3bc013590 con 0x7fb3c0102ab0 2024-04-25T15:41:57.581 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.581+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb3c005f5c0 con 0x7fb3c0102ab0 2024-04-25T15:41:57.582 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.581+0000 7fb3b5ffb700 1 -- 172.21.15.193:0/420487641 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7fb3bc021030 con 0x7fb3c0102ab0 2024-04-25T15:41:57.582 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:41:57.583 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:41:57.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb3ac0369c0 msgr2=0x7fb3ac038e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:57.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 --2- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb3ac0369c0 0x7fb3ac038e70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0009990 tx=0x7fb3b0006e40 comp rx=0 tx=0).stop 2024-04-25T15:41:57.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 msgr2=0x7fb3c0113130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:41:57.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 --2- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 0x7fb3c0113130 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc00b7f0 tx=0x7fb3bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:41:57.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 shutdown_connections 2024-04-25T15:41:57.584 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 --2- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb3ac0369c0 0x7fb3ac038e70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:57.585 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 --2- 172.21.15.193:0/420487641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3c0102ab0 0x7fb3c0113130 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:41:57.585 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 >> 172.21.15.193:0/420487641 conn(0x7fb3c00f85b0 msgr2=0x7fb3c00f9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:41:57.585 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 shutdown_connections 2024-04-25T15:41:57.585 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:41:57.583+0000 7fb3c7405700 1 -- 172.21.15.193:0/420487641 wait complete. 2024-04-25T15:41:57.585 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:41:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:41:57 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/420487641' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:41:59.602 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:41:59.602 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:42:00.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:00 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:00.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:00 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:00.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:00 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:00.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:00 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:00.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:00 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:42:00.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:00 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:00.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:00 smithi110 ceph-mon[26344]: Deploying daemon mon.smithi193 on smithi193 2024-04-25T15:42:01.126 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:42:01.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:01 smithi110 ceph-mon[26344]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:01.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:01 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:02.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.920+0000 7f87f0406700 1 -- 172.21.15.193:0/2690054013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 msgr2=0x7f87e8102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:02.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.920+0000 7f87f0406700 1 --2- 172.21.15.193:0/2690054013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 0x7f87e8102f80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f87e40048c0 tx=0x7f87e4009e30 comp rx=0 tx=0).stop 2024-04-25T15:42:02.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 -- 172.21.15.193:0/2690054013 shutdown_connections 2024-04-25T15:42:02.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 --2- 172.21.15.193:0/2690054013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 0x7f87e8102f80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:02.923 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 -- 172.21.15.193:0/2690054013 >> 172.21.15.193:0/2690054013 conn(0x7f87e80f85c0 msgr2=0x7f87e80fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 -- 172.21.15.193:0/2690054013 shutdown_connections 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 -- 172.21.15.193:0/2690054013 wait complete. 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 Processor -- start 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 -- start start 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 0x7f87e81131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.921+0000 7f87f0406700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e8113720 con 0x7f87e8102bb0 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87ee1a2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 0x7f87e81131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87ee1a2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 0x7f87e81131e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.193:37622/0 (socket says 172.21.15.193:37622) 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87ee1a2700 1 -- 172.21.15.193:0/3514212230 learned_addr learned my addr 172.21.15.193:0/3514212230 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87ee1a2700 1 -- 172.21.15.193:0/3514212230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e4005040 con 0x7f87e8102bb0 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87ee1a2700 1 --2- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 0x7f87e81131e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b820 tx=0x7f87e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87deffd700 1 -- 172.21.15.193:0/3514212230 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f87e40093c0 con 0x7f87e8102bb0 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87deffd700 1 -- 172.21.15.193:0/3514212230 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f87e4013420 con 0x7f87e8102bb0 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e8113920 con 0x7f87e8102bb0 2024-04-25T15:42:02.924 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87deffd700 1 -- 172.21.15.193:0/3514212230 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f87e401c5b0 con 0x7f87e8102bb0 2024-04-25T15:42:02.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.922+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e8113e10 con 0x7f87e8102bb0 2024-04-25T15:42:02.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.923+0000 7f87deffd700 1 -- 172.21.15.193:0/3514212230 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 17) v1 ==== 43491+0+0 (secure 0 0 0) 0x7f87e401c7f0 con 0x7f87e8102bb0 2024-04-25T15:42:02.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.923+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e805f5c0 con 0x7f87e8102bb0 2024-04-25T15:42:02.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.923+0000 7f87deffd700 1 --2- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f87d4036970 0x7f87d4038e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:02.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.923+0000 7f87deffd700 1 -- 172.21.15.193:0/3514212230 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f87e4049e60 con 0x7f87e8102bb0 2024-04-25T15:42:02.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.924+0000 7f87ed9a1700 1 --2- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f87d4036970 0x7f87d4038e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:02.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.924+0000 7f87ed9a1700 1 --2- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f87d4036970 0x7f87d4038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009990 tx=0x7f87d8006e40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:42:02.929 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:02.928+0000 7f87deffd700 1 -- 172.21.15.193:0/3514212230 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f87e4023300 con 0x7f87e8102bb0 2024-04-25T15:42:03.206 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.205+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f87e804bed0 con 0x7f87e8102bb0 2024-04-25T15:42:03.207 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.206+0000 7f87deffd700 1 -- 172.21.15.193:0/3514212230 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+743 (secure 0 0 0) 0x7f87e4012e40 con 0x7f87e8102bb0 2024-04-25T15:42:03.207 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:42:03.207 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":1,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:39:28.650237Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f87d4036970 msgr2=0x7f87d4038e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 --2- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f87d4036970 0x7f87d4038e20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f87d8009990 tx=0x7f87d8006e40 comp rx=0 tx=0).stop 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 msgr2=0x7f87e81131e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 --2- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 0x7f87e81131e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f87e400b820 tx=0x7f87e4009e30 comp rx=0 tx=0).stop 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 shutdown_connections 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 --2- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f87d4036970 0x7f87d4038e20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 --2- 172.21.15.193:0/3514212230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f87e8102bb0 0x7f87e81131e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 >> 172.21.15.193:0/3514212230 conn(0x7f87e80f85c0 msgr2=0x7f87e80f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 shutdown_connections 2024-04-25T15:42:03.209 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:03.208+0000 7f87f0406700 1 -- 172.21.15.193:0/3514212230 wait complete. 2024-04-25T15:42:03.210 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 1 2024-04-25T15:42:03.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:03 smithi110 ceph-mon[26344]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:03.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:03 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/3514212230' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:42:04.977 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-25T15:42:04.977 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T15:42:05.244 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi193/config 2024-04-25T15:42:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:42:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: mon.smithi110 calling monitor election 2024-04-25T15:42:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:42:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:42:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:42:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: mon.smithi193 calling monitor election 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/699116875' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: mon.smithi110 is new leader, mons smithi110,smithi193 in quorum (ranks 0,1) 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: monmap e2: 2 mons at {smithi110=[v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0],smithi193=[v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: fsmap 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: osdmap e5: 0 total, 0 up, 0 in 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: mgrmap e17: smithi110.lvgbkf(active, since 28s) 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: overall HEALTH_OK 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: Standby manager daemon smithi193.mgoteh started 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/699116875' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:42:08.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:08.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/699116875' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:42:08.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/699116875' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:42:08.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:08.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:08.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:08.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:08.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:08 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:09.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:09 smithi110 ceph-mon[26344]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:09.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:09 smithi110 ceph-mon[26344]: mgrmap e18: smithi110.lvgbkf(active, since 28s), standbys: smithi193.mgoteh 2024-04-25T15:42:09.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:09 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mgoteh", "id": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:42:09.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:09 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:42:09.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:09 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:42:09.665 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.664+0000 7f793ef9e700 1 -- 172.21.15.193:0/3615597782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 msgr2=0x7f7928005a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:09.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.664+0000 7f793ef9e700 1 --2- 172.21.15.193:0/3615597782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 0x7f7928005a00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f79380655d0 tx=0x7f792c00b040 comp rx=0 tx=0).stop 2024-04-25T15:42:09.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.664+0000 7f793ef9e700 1 -- 172.21.15.193:0/3615597782 shutdown_connections 2024-04-25T15:42:09.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.664+0000 7f793ef9e700 1 --2- 172.21.15.193:0/3615597782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 0x7f7928005a00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:09.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.664+0000 7f793ef9e700 1 -- 172.21.15.193:0/3615597782 >> 172.21.15.193:0/3615597782 conn(0x7f7938072080 msgr2=0x7f7938072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:42:09.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.665+0000 7f793ef9e700 1 -- 172.21.15.193:0/3615597782 shutdown_connections 2024-04-25T15:42:09.666 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.665+0000 7f793ef9e700 1 -- 172.21.15.193:0/3615597782 wait complete. 2024-04-25T15:42:09.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:09 smithi193 ceph-mon[33747]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:09.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:09 smithi193 ceph-mon[33747]: mgrmap e18: smithi110.lvgbkf(active, since 28s), standbys: smithi193.mgoteh 2024-04-25T15:42:09.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:09 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mgoteh", "id": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:42:09.666 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:09 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:42:09.667 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:09 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:42:09.668 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.665+0000 7f793ef9e700 1 Processor -- start 2024-04-25T15:42:09.668 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.665+0000 7f793ef9e700 1 -- start start 2024-04-25T15:42:09.668 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f793ef9e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 0x7f7938113210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f793ef9e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 0x7f793810d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f793ef9e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7938113d60 con 0x7f7928003580 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f793ef9e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793810d6b0 con 0x7f7938113750 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f7937fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 0x7f793810d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f7937fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 0x7f793810d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:35052/0 (socket says 172.21.15.193:35052) 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f7937fff700 1 -- 172.21.15.193:0/827706044 learned_addr learned my addr 172.21.15.193:0/827706044 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f793cd3a700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 0x7f7938113210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f7937fff700 1 -- 172.21.15.193:0/827706044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 msgr2=0x7f793810d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f7937fff700 1 -- 172.21.15.193:0/827706044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 msgr2=0x7f793810d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f7937fff700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 0x7f793810d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f7937fff700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 0x7f793810d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f793cd3a700 1 -- 172.21.15.193:0/827706044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 msgr2=0x7f793810d170 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f793cd3a700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 0x7f793810d170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.666+0000 7f793cd3a700 1 -- 172.21.15.193:0/827706044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c005040 con 0x7f7928003580 2024-04-25T15:42:09.669 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.667+0000 7f793cd3a700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 0x7f7938113210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f792c0040d0 tx=0x7f792c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:42:09.670 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.667+0000 7f79357fa700 1 -- 172.21.15.193:0/827706044 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f792c015d90 con 0x7f7928003580 2024-04-25T15:42:09.670 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.667+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f793810d930 con 0x7f7928003580 2024-04-25T15:42:09.670 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.667+0000 7f79357fa700 1 -- 172.21.15.193:0/827706044 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f792c0097f0 con 0x7f7928003580 2024-04-25T15:42:09.670 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.667+0000 7f79357fa700 1 -- 172.21.15.193:0/827706044 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f792c0216d0 con 0x7f7928003580 2024-04-25T15:42:09.670 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.667+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f793810de70 con 0x7f7928003580 2024-04-25T15:42:09.670 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.668+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79381073a0 con 0x7f7928003580 2024-04-25T15:42:09.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.668+0000 7f79357fa700 1 -- 172.21.15.193:0/827706044 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f792c016af0 con 0x7f7928003580 2024-04-25T15:42:09.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.669+0000 7f79357fa700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f791c069320 0x7f791c06b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:09.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.669+0000 7f79357fa700 1 -- 172.21.15.193:0/827706044 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f792c087a10 con 0x7f7928003580 2024-04-25T15:42:09.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.669+0000 7f7937fff700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f791c069320 0x7f791c06b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:09.671 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.669+0000 7f7937fff700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f791c069320 0x7f791c06b7d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7924006fd0 tx=0x7f7924008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:42:09.674 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.674+0000 7f79357fa700 1 -- 172.21.15.193:0/827706044 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f792c058830 con 0x7f7928003580 2024-04-25T15:42:09.943 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.942+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f79381049d0 con 0x7f7928003580 2024-04-25T15:42:09.943 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.942+0000 7f79357fa700 1 -- 172.21.15.193:0/827706044 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1021 (secure 0 0 0) 0x7f792c01c410 con 0x7f7928003580 2024-04-25T15:42:09.943 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T15:42:09.943 INFO:teuthology.orchestra.run.smithi193.stdout:{"epoch":2,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","modified":"2024-04-25T15:42:03.287721Z","created":"2024-04-25T15:39:28.650237Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"smithi110","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi193","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:3300","nonce":0},{"type":"v1","addr":"172.21.15.193:6789","nonce":0}]},"addr":"172.21.15.193:6789/0","public_addr":"172.21.15.193:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T15:42:09.945 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f791c069320 msgr2=0x7f791c06b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:09.945 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f791c069320 0x7f791c06b7d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7924006fd0 tx=0x7f7924008040 comp rx=0 tx=0).stop 2024-04-25T15:42:09.945 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 msgr2=0x7f7938113210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:09.945 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 0x7f7938113210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f792c0040d0 tx=0x7f792c00b040 comp rx=0 tx=0).stop 2024-04-25T15:42:09.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 shutdown_connections 2024-04-25T15:42:09.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7928003580 0x7f7938113210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:09.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f791c069320 0x7f791c06b7d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:09.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 --2- 172.21.15.193:0/827706044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7938113750 0x7f793810d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:09.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 >> 172.21.15.193:0/827706044 conn(0x7f7938072080 msgr2=0x7f79380fa4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:42:09.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 shutdown_connections 2024-04-25T15:42:09.946 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:42:09.945+0000 7f793ef9e700 1 -- 172.21.15.193:0/827706044 wait complete. 2024-04-25T15:42:09.946 INFO:teuthology.orchestra.run.smithi193.stderr:dumped monmap epoch 2 2024-04-25T15:42:10.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:10 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/827706044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:42:10.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:10 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:42:10.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:10 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:10.644 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:10 smithi193 ceph-mon[33747]: from='client.? 172.21.15.193:0/827706044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T15:42:10.644 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:10 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:42:10.644 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:10 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:10.668 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T15:42:10.668 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T15:42:10.947 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:42:12.042 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:12.042 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.042 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.042 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:12.042 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:42:12.042 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:42:12.042 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:42:12.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:42:12.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:42:12.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:42:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:12 smithi110 ceph-mon[26344]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:42:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:12 smithi110 ceph-mon[26344]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:42:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:12 smithi110 ceph-mon[26344]: Reconfiguring mon.smithi110 (unknown last config time)... 2024-04-25T15:42:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:12 smithi110 ceph-mon[26344]: Reconfiguring daemon mon.smithi110 on smithi110 2024-04-25T15:42:13.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.089+0000 7fd575c6a700 1 -- 172.21.15.110:0/2006496135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 msgr2=0x7fd5700fde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:13.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.089+0000 7fd575c6a700 1 --2- 172.21.15.110:0/2006496135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 0x7fd5700fde20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd5600048c0 tx=0x7fd560009e30 comp rx=0 tx=0).stop 2024-04-25T15:42:13.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.089+0000 7fd575c6a700 1 -- 172.21.15.110:0/2006496135 shutdown_connections 2024-04-25T15:42:13.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.089+0000 7fd575c6a700 1 --2- 172.21.15.110:0/2006496135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 0x7fd5700fde20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:13.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.089+0000 7fd575c6a700 1 -- 172.21.15.110:0/2006496135 >> 172.21.15.110:0/2006496135 conn(0x7fd5700f74e0 msgr2=0x7fd5700f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:42:13.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.089+0000 7fd575c6a700 1 -- 172.21.15.110:0/2006496135 shutdown_connections 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.089+0000 7fd575c6a700 1 -- 172.21.15.110:0/2006496135 wait complete. 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd575c6a700 1 Processor -- start 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd575c6a700 1 -- start start 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd575c6a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 0x7fd5701132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd575c6a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570113800 0x7fd57010d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd575c6a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd570113e10 con 0x7fd5700fda50 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd575c6a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd57010d760 con 0x7fd570113800 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd56f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 0x7fd5701132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd56f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 0x7fd5701132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39842/0 (socket says 172.21.15.110:39842) 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.090+0000 7fd56f7fe700 1 -- 172.21.15.110:0/650575718 learned_addr learned my addr 172.21.15.110:0/650575718 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd56f7fe700 1 -- 172.21.15.110:0/650575718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570113800 msgr2=0x7fd57010d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd56f7fe700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570113800 0x7fd57010d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd56f7fe700 1 -- 172.21.15.110:0/650575718 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd560005040 con 0x7fd5700fda50 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd56f7fe700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 0x7fd5701132c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd56000b5c0 tx=0x7fd560009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:42:13.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd56cff9700 1 -- 172.21.15.110:0/650575718 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56000e040 con 0x7fd5700fda50 2024-04-25T15:42:13.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd56cff9700 1 -- 172.21.15.110:0/650575718 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd560009950 con 0x7fd5700fda50 2024-04-25T15:42:13.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd57010d9e0 con 0x7fd5700fda50 2024-04-25T15:42:13.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd56cff9700 1 -- 172.21.15.110:0/650575718 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd56001c3f0 con 0x7fd5700fda50 2024-04-25T15:42:13.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.091+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd57010df20 con 0x7fd5700fda50 2024-04-25T15:42:13.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.092+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd570102660 con 0x7fd5700fda50 2024-04-25T15:42:13.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.092+0000 7fd56cff9700 1 -- 172.21.15.110:0/650575718 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7fd5600137d0 con 0x7fd5700fda50 2024-04-25T15:42:13.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.093+0000 7fd56cff9700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd55c069360 0x7fd55c06b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:13.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.093+0000 7fd56cff9700 1 -- 172.21.15.110:0/650575718 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7fd560087340 con 0x7fd5700fda50 2024-04-25T15:42:13.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.093+0000 7fd567fff700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd55c069360 0x7fd55c06b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:13.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.093+0000 7fd567fff700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd55c069360 0x7fd55c06b810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd558006fd0 tx=0x7fd558006dd0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:42:13.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.097+0000 7fd56cff9700 1 -- 172.21.15.110:0/650575718 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd560058280 con 0x7fd5700fda50 2024-04-25T15:42:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:12 smithi193 ceph-mon[33747]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:42:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:12 smithi193 ceph-mon[33747]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:42:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:12 smithi193 ceph-mon[33747]: Reconfiguring mon.smithi110 (unknown last config time)... 2024-04-25T15:42:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:12 smithi193 ceph-mon[33747]: Reconfiguring daemon mon.smithi110 on smithi110 2024-04-25T15:42:13.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.289+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fd57010e910 con 0x7fd5700fda50 2024-04-25T15:42:13.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.289+0000 7fd56cff9700 1 -- 172.21.15.110:0/650575718 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7fd560057e10 con 0x7fd5700fda50 2024-04-25T15:42:13.290 INFO:teuthology.orchestra.run.smithi110.stdout:# minimal ceph.conf for e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:42:13.290 INFO:teuthology.orchestra.run.smithi110.stdout:[global] 2024-04-25T15:42:13.290 INFO:teuthology.orchestra.run.smithi110.stdout: fsid = e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:42:13.291 INFO:teuthology.orchestra.run.smithi110.stdout: mon_host = [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] 2024-04-25T15:42:13.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.291+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd55c069360 msgr2=0x7fd55c06b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:13.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.291+0000 7fd575c6a700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd55c069360 0x7fd55c06b810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd558006fd0 tx=0x7fd558006dd0 comp rx=0 tx=0).stop 2024-04-25T15:42:13.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.291+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 msgr2=0x7fd5701132c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:13.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.291+0000 7fd575c6a700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 0x7fd5701132c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd56000b5c0 tx=0x7fd560009e30 comp rx=0 tx=0).stop 2024-04-25T15:42:13.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.292+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 shutdown_connections 2024-04-25T15:42:13.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.292+0000 7fd575c6a700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5700fda50 0x7fd5701132c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:13.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.292+0000 7fd575c6a700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd55c069360 0x7fd55c06b810 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:13.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.292+0000 7fd575c6a700 1 --2- 172.21.15.110:0/650575718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd570113800 0x7fd57010d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:13.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.292+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 >> 172.21.15.110:0/650575718 conn(0x7fd5700f74e0 msgr2=0x7fd5700f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:42:13.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.292+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 shutdown_connections 2024-04-25T15:42:13.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:13.292+0000 7fd575c6a700 1 -- 172.21.15.110:0/650575718 wait complete. 2024-04-25T15:42:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:13 smithi110 ceph-mon[26344]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:13 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/650575718' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:13 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:13 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:13 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.lvgbkf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:42:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:13 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:42:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:13 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:14.163 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T15:42:14.164 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:42:14.164 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T15:42:14.189 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:42:14.189 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:42:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:13 smithi193 ceph-mon[33747]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:13 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/650575718' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:13 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:13 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:14.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:13 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.lvgbkf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:42:14.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:13 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:42:14.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:13 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:14.255 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:42:14.255 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T15:42:14.286 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:42:14.286 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:42:14.353 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T15:42:14.353 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:42:14.354 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T15:42:14.369 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T15:42:14.369 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-04-25T15:42:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T15:42:14.424 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T15:42:14.424 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T15:42:14.425 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:42:14.425 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T15:42:14.425 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-04-25 15:41:32.133007594 +0000 2024-04-25T15:42:14.425 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-25 15:36:23.122536975 +0000 2024-04-25T15:42:14.425 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-25 15:36:23.122536975 +0000 2024-04-25T15:42:14.425 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-04-25 15:36:23.122536975 +0000 2024-04-25T15:42:14.425 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T15:42:14.488 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-04-25T15:42:14.489 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-04-25T15:42:14.489 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 9.5091e-05 s, 5.4 MB/s 2024-04-25T15:42:14.490 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T15:42:14.546 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-04-25 15:41:32.134007578 +0000 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-25 15:36:23.461530884 +0000 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-25 15:36:23.461530884 +0000 2024-04-25T15:42:14.601 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-04-25 15:36:23.461530884 +0000 2024-04-25T15:42:14.602 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T15:42:14.664 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-04-25T15:42:14.664 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-04-25T15:42:14.664 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 9.2724e-05 s, 5.5 MB/s 2024-04-25T15:42:14.665 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T15:42:14.721 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-04-25T15:42:14.776 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T15:42:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T15:42:14.777 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T15:42:14.777 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:42:14.777 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T15:42:14.777 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-04-25 15:41:32.134007578 +0000 2024-04-25T15:42:14.777 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-25 15:36:23.790524972 +0000 2024-04-25T15:42:14.777 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-25 15:36:23.790524972 +0000 2024-04-25T15:42:14.777 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-04-25 15:36:23.790524972 +0000 2024-04-25T15:42:14.777 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T15:42:14.838 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-04-25T15:42:14.838 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-04-25T15:42:14.838 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000106548 s, 4.8 MB/s 2024-04-25T15:42:14.839 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T15:42:14.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:14 smithi110 ceph-mon[26344]: Reconfiguring mgr.smithi110.lvgbkf (unknown last config time)... 2024-04-25T15:42:14.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:14 smithi110 ceph-mon[26344]: Reconfiguring daemon mgr.smithi110.lvgbkf on smithi110 2024-04-25T15:42:14.877 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-04-25T15:42:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T15:42:14.932 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T15:42:14.932 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T15:42:14.932 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:42:14.932 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T15:42:14.933 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-04-25 15:41:32.134007578 +0000 2024-04-25T15:42:14.933 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-25 15:36:24.129518881 +0000 2024-04-25T15:42:14.933 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-25 15:36:24.129518881 +0000 2024-04-25T15:42:14.933 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-04-25 15:36:24.129518881 +0000 2024-04-25T15:42:14.933 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T15:42:14.994 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-04-25T15:42:14.995 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-04-25T15:42:14.995 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 8.396e-05 s, 6.1 MB/s 2024-04-25T15:42:14.996 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T15:42:15.052 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:42:15.052 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T15:42:15.071 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T15:42:15.071 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 15:42:02.022450918 +0000 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 15:36:23.131295160 +0000 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 15:36:23.131295160 +0000 2024-04-25T15:42:15.129 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 15:36:23.131295160 +0000 2024-04-25T15:42:15.129 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T15:42:15.192 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:14 smithi193 ceph-mon[33747]: Reconfiguring mgr.smithi110.lvgbkf (unknown last config time)... 2024-04-25T15:42:15.192 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:14 smithi193 ceph-mon[33747]: Reconfiguring daemon mgr.smithi110.lvgbkf on smithi110 2024-04-25T15:42:15.195 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T15:42:15.195 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T15:42:15.195 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000376024 s, 1.4 MB/s 2024-04-25T15:42:15.196 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T15:42:15.255 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-04-25T15:42:15.312 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T15:42:15.312 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T15:42:15.312 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T15:42:15.312 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:42:15.313 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T15:42:15.313 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 15:42:02.022450918 +0000 2024-04-25T15:42:15.313 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 15:36:23.471289391 +0000 2024-04-25T15:42:15.313 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 15:36:23.471289391 +0000 2024-04-25T15:42:15.313 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 15:36:23.471289391 +0000 2024-04-25T15:42:15.313 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T15:42:15.377 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T15:42:15.377 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T15:42:15.377 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000237575 s, 2.2 MB/s 2024-04-25T15:42:15.378 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T15:42:15.435 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-04-25T15:42:15.490 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T15:42:15.490 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T15:42:15.490 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T15:42:15.491 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:42:15.491 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T15:42:15.491 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 15:42:02.022450918 +0000 2024-04-25T15:42:15.491 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 15:36:23.802283775 +0000 2024-04-25T15:42:15.491 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 15:36:23.802283775 +0000 2024-04-25T15:42:15.491 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 15:36:23.802283775 +0000 2024-04-25T15:42:15.491 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T15:42:15.553 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T15:42:15.553 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T15:42:15.553 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00018505 s, 2.8 MB/s 2024-04-25T15:42:15.555 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T15:42:15.612 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-04-25T15:42:15.667 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T15:42:15.668 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T15:42:15.668 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T15:42:15.668 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:42:15.668 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T15:42:15.668 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 15:42:02.023450903 +0000 2024-04-25T15:42:15.668 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 15:36:24.317275037 +0000 2024-04-25T15:42:15.668 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 15:36:24.317275037 +0000 2024-04-25T15:42:15.668 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 15:36:24.317275037 +0000 2024-04-25T15:42:15.668 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T15:42:15.729 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T15:42:15.729 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T15:42:15.729 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00023332 s, 2.2 MB/s 2024-04-25T15:42:15.731 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T15:42:15.787 INFO:tasks.cephadm:Deploying osd.0 on smithi110 with /dev/vg_nvme/lv_4... 2024-04-25T15:42:15.788 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T15:42:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:15 smithi193 ceph-mon[33747]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:15 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:15 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:15 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:42:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:15 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:16.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:15 smithi110 ceph-mon[26344]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:16.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:15 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:16.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:15 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:16.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:15 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:42:16.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:15 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:16.738 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:42:17.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:16 smithi110 ceph-mon[26344]: Reconfiguring crash.smithi110 (monmap changed)... 2024-04-25T15:42:17.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:16 smithi110 ceph-mon[26344]: Reconfiguring daemon crash.smithi110 on smithi110 2024-04-25T15:42:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:16 smithi193 ceph-mon[33747]: Reconfiguring crash.smithi110 (monmap changed)... 2024-04-25T15:42:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:16 smithi193 ceph-mon[33747]: Reconfiguring daemon crash.smithi110 on smithi110 2024-04-25T15:42:18.013 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:17 smithi110 ceph-mon[26344]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:18.013 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:18.013 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:18.013 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:17 smithi110 ceph-mon[26344]: Reconfiguring alertmanager.smithi110 (dependencies changed)... 2024-04-25T15:42:18.013 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:17 smithi110 ceph-mon[26344]: Reconfiguring daemon alertmanager.smithi110 on smithi110 2024-04-25T15:42:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:17 smithi193 ceph-mon[33747]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:17 smithi193 ceph-mon[33747]: Reconfiguring alertmanager.smithi110 (dependencies changed)... 2024-04-25T15:42:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:17 smithi193 ceph-mon[33747]: Reconfiguring daemon alertmanager.smithi110 on smithi110 2024-04-25T15:42:19.778 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T15:42:19.778 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T15:42:19.778 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-04-25T15:42:19.778 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-04-25T15:42:19.778 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0102152 s, 1.0 GB/s 2024-04-25T15:42:19.778 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-04-25T15:42:20.088 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:19 smithi110 ceph-mon[26344]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:19 smithi193 ceph-mon[33747]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:20.897 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-04-25T15:42:21.491 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:42:21.881 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:21 smithi110 ceph-mon[26344]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:22.041 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:21 smithi193 ceph-mon[33747]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:23.799 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.796+0000 7f683a260700 1 -- 172.21.15.110:0/4152140773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fdff0 msgr2=0x7f68340fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.796+0000 7f683a260700 1 --2- 172.21.15.110:0/4152140773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fdff0 0x7f68340fe460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6828004940 tx=0x7f6828009e30 comp rx=0 tx=0).stop 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.797+0000 7f683a260700 1 -- 172.21.15.110:0/4152140773 shutdown_connections 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.797+0000 7f683a260700 1 --2- 172.21.15.110:0/4152140773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fdff0 0x7f68340fe460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.797+0000 7f683a260700 1 --2- 172.21.15.110:0/4152140773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68340fce80 0x7f68340fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.797+0000 7f683a260700 1 -- 172.21.15.110:0/4152140773 >> 172.21.15.110:0/4152140773 conn(0x7f68340f8520 msgr2=0x7f68340fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.797+0000 7f683a260700 1 -- 172.21.15.110:0/4152140773 shutdown_connections 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.797+0000 7f683a260700 1 -- 172.21.15.110:0/4152140773 wait complete. 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.797+0000 7f683a260700 1 Processor -- start 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f683a260700 1 -- start start 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f683a260700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fce80 0x7f683410d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f683a260700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68340fdff0 0x7f683410def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f683a260700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683410e4f0 con 0x7f68340fce80 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f683a260700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683410ed40 con 0x7f68340fdff0 2024-04-25T15:42:23.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f68337fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fce80 0x7f683410d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:23.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f68337fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fce80 0x7f683410d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43912/0 (socket says 172.21.15.110:43912) 2024-04-25T15:42:23.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f68337fe700 1 -- 172.21.15.110:0/3252440135 learned_addr learned my addr 172.21.15.110:0/3252440135 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:42:23.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f6832ffd700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68340fdff0 0x7f683410def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:23.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f68337fe700 1 -- 172.21.15.110:0/3252440135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68340fdff0 msgr2=0x7f683410def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:23.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f68337fe700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68340fdff0 0x7f683410def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:23.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.798+0000 7f68337fe700 1 -- 172.21.15.110:0/3252440135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6828005040 con 0x7f68340fce80 2024-04-25T15:42:23.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.799+0000 7f68337fe700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fce80 0x7f683410d9b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f682400eb40 tx=0x7f682400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:42:23.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.799+0000 7f681bfff700 1 -- 172.21.15.110:0/3252440135 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6824007cc0 con 0x7f68340fce80 2024-04-25T15:42:23.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.799+0000 7f681bfff700 1 -- 172.21.15.110:0/3252440135 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f682400f040 con 0x7f68340fce80 2024-04-25T15:42:23.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.799+0000 7f681bfff700 1 -- 172.21.15.110:0/3252440135 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f682400ca10 con 0x7f68340fce80 2024-04-25T15:42:23.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.799+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683410efc0 con 0x7f68340fce80 2024-04-25T15:42:23.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.799+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f683410f4b0 con 0x7f68340fce80 2024-04-25T15:42:23.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.800+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683405f5c0 con 0x7f68340fce80 2024-04-25T15:42:23.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.801+0000 7f681bfff700 1 -- 172.21.15.110:0/3252440135 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f682400cc80 con 0x7f68340fce80 2024-04-25T15:42:23.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.801+0000 7f681bfff700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f681c069210 0x7f681c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:42:23.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.801+0000 7f681bfff700 1 -- 172.21.15.110:0/3252440135 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1366+0+0 (secure 0 0 0) 0x7f6824085c70 con 0x7f68340fce80 2024-04-25T15:42:23.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.801+0000 7f6832ffd700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f681c069210 0x7f681c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:42:23.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.802+0000 7f6832ffd700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f681c069210 0x7f681c06b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6828009b30 tx=0x7f6828009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:42:23.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:23.806+0000 7f681bfff700 1 -- 172.21.15.110:0/3252440135 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f682404e250 con 0x7f68340fce80 2024-04-25T15:42:24.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:24.023+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f683410f790 con 0x7f681c069210 2024-04-25T15:42:24.024 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:23 smithi110 ceph-mon[26344]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:23 smithi193 ceph-mon[33747]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:25.108 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:24 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:42:25.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:24 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:42:25.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:24 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:25.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:24 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:25.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:24 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:24 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:42:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:24 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:42:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:24 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:24 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:24 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:25 smithi110 ceph-mon[26344]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:42:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:25 smithi110 ceph-mon[26344]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:25 smithi110 ceph-mon[26344]: Reconfiguring grafana.smithi110 (dependencies changed)... 2024-04-25T15:42:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:25 smithi110 ceph-mon[26344]: Reconfiguring daemon grafana.smithi110 on smithi110 2024-04-25T15:42:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:42:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:25 smithi193 ceph-mon[33747]: from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:42:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:25 smithi193 ceph-mon[33747]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:25 smithi193 ceph-mon[33747]: Reconfiguring grafana.smithi110 (dependencies changed)... 2024-04-25T15:42:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:25 smithi193 ceph-mon[33747]: Reconfiguring daemon grafana.smithi110 on smithi110 2024-04-25T15:42:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:42:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:27 smithi110 ceph-mon[26344]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:28.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:27 smithi193 ceph-mon[33747]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:30.060 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:29 smithi110 ceph-mon[26344]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:30.060 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:30.060 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:29 smithi193 ceph-mon[33747]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:31.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:30 smithi110 ceph-mon[26344]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-04-25T15:42:31.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:30 smithi110 ceph-mon[26344]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-04-25T15:42:31.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:30 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2494242623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a0a2ad-2c02-4108-85a8-3ef716509956"}]: dispatch 2024-04-25T15:42:31.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:30 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2494242623' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a0a2ad-2c02-4108-85a8-3ef716509956"}]': finished 2024-04-25T15:42:31.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:30 smithi110 ceph-mon[26344]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T15:42:31.109 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:30 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:30 smithi193 ceph-mon[33747]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-04-25T15:42:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:30 smithi193 ceph-mon[33747]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-04-25T15:42:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:30 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2494242623' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "50a0a2ad-2c02-4108-85a8-3ef716509956"}]: dispatch 2024-04-25T15:42:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:30 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2494242623' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "50a0a2ad-2c02-4108-85a8-3ef716509956"}]': finished 2024-04-25T15:42:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:30 smithi193 ceph-mon[33747]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T15:42:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:30 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:31 smithi110 ceph-mon[26344]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:31 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3610291230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:42:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:31 smithi193 ceph-mon[33747]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:31 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3610291230' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:42:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:33 smithi110 ceph-mon[26344]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:33 smithi193 ceph-mon[33747]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:35 smithi110 ceph-mon[26344]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:35 smithi193 ceph-mon[33747]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:37.980 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:37 smithi110 ceph-mon[26344]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:37 smithi193 ceph-mon[33747]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:40.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:39 smithi110 ceph-mon[26344]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:39 smithi193 ceph-mon[33747]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:41.065 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:42:41.065 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:41.065 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:41.065 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:42:41.065 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:41.092 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:42:41.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:41.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:41.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:42:41.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:41 smithi110 ceph-mon[26344]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:41 smithi110 ceph-mon[26344]: Reconfiguring crash.smithi193 (monmap changed)... 2024-04-25T15:42:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:41 smithi110 ceph-mon[26344]: Reconfiguring daemon crash.smithi193 on smithi193 2024-04-25T15:42:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:41 smithi193 ceph-mon[33747]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:41 smithi193 ceph-mon[33747]: Reconfiguring crash.smithi193 (monmap changed)... 2024-04-25T15:42:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:41 smithi193 ceph-mon[33747]: Reconfiguring daemon crash.smithi193 on smithi193 2024-04-25T15:42:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:43.001 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:42 smithi110 ceph-mon[26344]: Reconfiguring mgr.smithi193.mgoteh (monmap changed)... 2024-04-25T15:42:43.001 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:42 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:42:43.001 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:42 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:42:43.002 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:42 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:43.002 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:42 smithi110 ceph-mon[26344]: Reconfiguring daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:42:43.002 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:42 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T15:42:43.002 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:42 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:42 smithi193 ceph-mon[33747]: Reconfiguring mgr.smithi193.mgoteh (monmap changed)... 2024-04-25T15:42:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:42 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:42:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:42 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:42:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:42 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:42 smithi193 ceph-mon[33747]: Reconfiguring daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:42:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:42 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T15:42:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:42 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:43.940 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:43 smithi110 ceph-mon[26344]: Deploying daemon osd.0 on smithi110 2024-04-25T15:42:43.940 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:43 smithi110 ceph-mon[26344]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:43.941 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:43 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:43.941 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:43 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:43.941 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:43 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:42:43.941 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:43 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:42:43.941 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:43 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:43 smithi193 ceph-mon[33747]: Deploying daemon osd.0 on smithi110 2024-04-25T15:42:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:43 smithi193 ceph-mon[33747]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:43 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:43 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:43 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:42:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:43 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:42:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:43 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:45.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: Reconfiguring mon.smithi193 (monmap changed)... 2024-04-25T15:42:45.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: Reconfiguring daemon mon.smithi193 on smithi193 2024-04-25T15:42:45.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:42:45.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110:9093"}]: dispatch 2024-04-25T15:42:45.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:42:45.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110:3000"}]: dispatch 2024-04-25T15:42:45.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:42:45.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110:9095"}]: dispatch 2024-04-25T15:42:45.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:42:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: Reconfiguring mon.smithi193 (monmap changed)... 2024-04-25T15:42:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: Reconfiguring daemon mon.smithi193 on smithi193 2024-04-25T15:42:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110:9093"}]: dispatch 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110:3000"}]: dispatch 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110:9095"}]: dispatch 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:42:45.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:45 smithi110 ceph-mon[26344]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:42:45.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:45 smithi110 ceph-mon[26344]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110:9093"}]: dispatch 2024-04-25T15:42:45.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:45 smithi110 ceph-mon[26344]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:42:45.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:45 smithi110 ceph-mon[26344]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110:3000"}]: dispatch 2024-04-25T15:42:45.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:45 smithi110 ceph-mon[26344]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:42:45.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:45 smithi110 ceph-mon[26344]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:45.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:45 smithi110 ceph-mon[26344]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110:9095"}]: dispatch 2024-04-25T15:42:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:45 smithi193 ceph-mon[33747]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:42:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:45 smithi193 ceph-mon[33747]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110:9093"}]: dispatch 2024-04-25T15:42:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:45 smithi193 ceph-mon[33747]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:42:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:45 smithi193 ceph-mon[33747]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110:3000"}]: dispatch 2024-04-25T15:42:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:45 smithi193 ceph-mon[33747]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:42:46.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:45 smithi193 ceph-mon[33747]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:46.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:45 smithi193 ceph-mon[33747]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110:9095"}]: dispatch 2024-04-25T15:42:47.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:47.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:46 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:47.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:46 smithi110 ceph-mon[26344]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:47.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:46 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:47.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:46 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:47.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:46 smithi193 ceph-mon[33747]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:49.294 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:49 smithi110 ceph-mon[26344]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:49.294 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:49 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:49.295 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:49 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:49.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:49 smithi193 ceph-mon[33747]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:49.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:49 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:49.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:49 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:50.876 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:50.876 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:50.876 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:50.876 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:42:50.876 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:50.876 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:42:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:42:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:42:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:42:52.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:51 smithi110 ceph-mon[26344]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:51 smithi193 ceph-mon[33747]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:52.698 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 0 on host 'smithi110' 2024-04-25T15:42:52.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.697+0000 7f681bfff700 1 -- 172.21.15.110:0/3252440135 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f683410f790 con 0x7f681c069210 2024-04-25T15:42:52.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f681c069210 msgr2=0x7f681c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f681c069210 0x7f681c06b6c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6828009b30 tx=0x7f6828009e30 comp rx=0 tx=0).stop 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fce80 msgr2=0x7f683410d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fce80 0x7f683410d9b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f682400eb40 tx=0x7f682400b040 comp rx=0 tx=0).stop 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 shutdown_connections 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f68340fce80 0x7f683410d9b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f681c069210 0x7f681c06b6c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 --2- 172.21.15.110:0/3252440135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68340fdff0 0x7f683410def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 >> 172.21.15.110:0/3252440135 conn(0x7f68340f8520 msgr2=0x7f68340f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:42:52.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 shutdown_connections 2024-04-25T15:42:52.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:42:52.700+0000 7f683a260700 1 -- 172.21.15.110:0/3252440135 wait complete. 2024-04-25T15:42:53.315 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:53 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:53.315 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:53 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:53.315 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:53 smithi110 ceph-mon[26344]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:53.315 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:53 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:53.315 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:53 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:53.315 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:53 smithi110 ceph-mon[26344]: from='osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T15:42:53.337 DEBUG:teuthology.orchestra.run.smithi110:osd.0> sudo journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T15:42:53.340 INFO:tasks.cephadm:Deploying osd.1 on smithi110 with /dev/vg_nvme/lv_3... 2024-04-25T15:42:53.341 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T15:42:53.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:53 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:53.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:53 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:53.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:53 smithi193 ceph-mon[33747]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:53.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:53 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:53.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:53 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:53.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:53 smithi193 ceph-mon[33747]: from='osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T15:42:54.698 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:42:54.930 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:54 smithi110 ceph-mon[26344]: from='osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T15:42:54.931 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:54 smithi110 ceph-mon[26344]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T15:42:54.931 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:54 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:54.931 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:54 smithi110 ceph-mon[26344]: from='osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:42:54.931 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:42:54 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[45308]: 2024-04-25T15:42:54.659+0000 7f48c952e700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T15:42:54.932 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:42:54 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[45308]: 2024-04-25T15:42:54.662+0000 7f48c4329700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:42:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:54 smithi193 ceph-mon[33747]: from='osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T15:42:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:54 smithi193 ceph-mon[33747]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T15:42:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:54 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:54 smithi193 ceph-mon[33747]: from='osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:42:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:55 smithi110 ceph-mon[26344]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:55 smithi110 ceph-mon[26344]: from='osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-25T15:42:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:55 smithi110 ceph-mon[26344]: osdmap e8: 1 total, 0 up, 1 in 2024-04-25T15:42:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:42:55.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:55 smithi193 ceph-mon[33747]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T15:42:55.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:55 smithi193 ceph-mon[33747]: from='osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-25T15:42:55.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:55 smithi193 ceph-mon[33747]: osdmap e8: 1 total, 0 up, 1 in 2024-04-25T15:42:55.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:55.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:55.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:42:56.712 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:56 smithi110 ceph-mon[26344]: purged_snaps scrub starts 2024-04-25T15:42:56.712 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:56 smithi110 ceph-mon[26344]: purged_snaps scrub ok 2024-04-25T15:42:56.712 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:56 smithi110 ceph-mon[26344]: osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] boot 2024-04-25T15:42:56.712 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:56 smithi110 ceph-mon[26344]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T15:42:56.712 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:56 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:56 smithi193 ceph-mon[33747]: purged_snaps scrub starts 2024-04-25T15:42:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:56 smithi193 ceph-mon[33747]: purged_snaps scrub ok 2024-04-25T15:42:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:56 smithi193 ceph-mon[33747]: osd.0 [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] boot 2024-04-25T15:42:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:56 smithi193 ceph-mon[33747]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T15:42:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:56 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:42:57.262 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T15:42:57.262 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T15:42:57.262 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-04-25T15:42:57.262 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-04-25T15:42:57.262 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00956496 s, 1.1 GB/s 2024-04-25T15:42:57.262 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-04-25T15:42:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:57 smithi110 ceph-mon[26344]: pgmap v35: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:42:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:57 smithi110 ceph-mon[26344]: osdmap e10: 1 total, 1 up, 1 in 2024-04-25T15:42:57.899 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-04-25T15:42:57.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:57 smithi193 ceph-mon[33747]: pgmap v35: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:42:57.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:57 smithi193 ceph-mon[33747]: osdmap e10: 1 total, 1 up, 1 in 2024-04-25T15:42:58.344 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:42:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:59 smithi110 ceph-mon[26344]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:42:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:59 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:42:59 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:59.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:59 smithi193 ceph-mon[33747]: pgmap v37: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:42:59.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:59 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:42:59.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:42:59 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:00.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.632+0000 7fdd7dceb700 1 -- 172.21.15.110:0/1506224674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 msgr2=0x7fdd780fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:00.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.632+0000 7fdd7dceb700 1 --2- 172.21.15.110:0/1506224674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 0x7fdd780fd3e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c004940 tx=0x7fdd6c009e30 comp rx=0 tx=0).stop 2024-04-25T15:43:00.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.632+0000 7fdd7dceb700 1 -- 172.21.15.110:0/1506224674 shutdown_connections 2024-04-25T15:43:00.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 --2- 172.21.15.110:0/1506224674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 0x7fdd780fd3e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:00.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 --2- 172.21.15.110:0/1506224674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd780fbcc0 0x7fdd780fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:00.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 -- 172.21.15.110:0/1506224674 >> 172.21.15.110:0/1506224674 conn(0x7fdd780f7440 msgr2=0x7fdd780f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:43:00.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 -- 172.21.15.110:0/1506224674 shutdown_connections 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 -- 172.21.15.110:0/1506224674 wait complete. 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 Processor -- start 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 -- start start 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd780fbcc0 0x7fdd7810c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 0x7fdd7810cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7810d380 con 0x7fdd780fcf70 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.633+0000 7fdd7dceb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7810d4f0 con 0x7fdd780fbcc0 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 0x7fdd7810cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd76ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 0x7fdd7810cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46720/0 (socket says 172.21.15.110:46720) 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd76ffd700 1 -- 172.21.15.110:0/3215892481 learned_addr learned my addr 172.21.15.110:0/3215892481 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd777fe700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd780fbcc0 0x7fdd7810c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd76ffd700 1 -- 172.21.15.110:0/3215892481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd780fbcc0 msgr2=0x7fdd7810c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd76ffd700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd780fbcc0 0x7fdd7810c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:00.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd76ffd700 1 -- 172.21.15.110:0/3215892481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c005040 con 0x7fdd780fcf70 2024-04-25T15:43:00.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd76ffd700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 0x7fdd7810cd80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009fd0 tx=0x7fdd6c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:43:00.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd5ffff700 1 -- 172.21.15.110:0/3215892481 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6c00e040 con 0x7fdd780fcf70 2024-04-25T15:43:00.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd5ffff700 1 -- 172.21.15.110:0/3215892481 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd6c00be00 con 0x7fdd780fcf70 2024-04-25T15:43:00.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd78066250 con 0x7fdd780fcf70 2024-04-25T15:43:00.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd5ffff700 1 -- 172.21.15.110:0/3215892481 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd6c0137f0 con 0x7fdd780fcf70 2024-04-25T15:43:00.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.634+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd78066810 con 0x7fdd780fcf70 2024-04-25T15:43:00.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.635+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7804c710 con 0x7fdd780fcf70 2024-04-25T15:43:00.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.636+0000 7fdd5ffff700 1 -- 172.21.15.110:0/3215892481 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7fdd6c013a60 con 0x7fdd780fcf70 2024-04-25T15:43:00.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.636+0000 7fdd5ffff700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fdd60069210 0x7fdd6006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:00.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.636+0000 7fdd777fe700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fdd60069210 0x7fdd6006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:00.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.636+0000 7fdd5ffff700 1 -- 172.21.15.110:0/3215892481 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1978+0+0 (secure 0 0 0) 0x7fdd6c089c20 con 0x7fdd780fcf70 2024-04-25T15:43:00.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.637+0000 7fdd777fe700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fdd60069210 0x7fdd6006b6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdd680041d0 tx=0x7fdd680043c0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:43:00.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.641+0000 7fdd5ffff700 1 -- 172.21.15.110:0/3215892481 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fdd6c05a890 con 0x7fdd780fcf70 2024-04-25T15:43:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:00.837+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd78066cf0 con 0x7fdd60069210 2024-04-25T15:43:01.498 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:01 smithi110 ceph-mon[26344]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:01.498 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:01 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:43:01.498 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:01 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:43:01.498 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:01 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:01.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:01 smithi193 ceph-mon[33747]: pgmap v38: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:01.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:01 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:43:01.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:01 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:43:01.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:01 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:02.582 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:02 smithi110 ceph-mon[26344]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:43:02.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:02 smithi193 ceph-mon[33747]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:43:03.593 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:03 smithi110 ceph-mon[26344]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:03.593 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:03.594 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:03.594 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:03.594 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:03.594 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:03.594 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:03.594 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:03 smithi193 ceph-mon[33747]: pgmap v39: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:03.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:04.454 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:04 smithi110 ceph-mon[26344]: Detected new or changed devices on smithi110 2024-04-25T15:43:04.455 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:04 smithi110 ceph-mon[26344]: Adjusting osd_memory_target on smithi110 to 12864M 2024-04-25T15:43:04.455 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:04 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:04.717 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:04 smithi193 ceph-mon[33747]: Detected new or changed devices on smithi110 2024-04-25T15:43:04.717 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:04 smithi193 ceph-mon[33747]: Adjusting osd_memory_target on smithi110 to 12864M 2024-04-25T15:43:04.717 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:04 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:05.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:05 smithi110 ceph-mon[26344]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:05.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:05 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3167572870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90511b8f-4444-44ff-b30c-789349b354f9"}]: dispatch 2024-04-25T15:43:05.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:05 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3167572870' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90511b8f-4444-44ff-b30c-789349b354f9"}]': finished 2024-04-25T15:43:05.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:05 smithi110 ceph-mon[26344]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T15:43:05.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:05 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:05.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:05 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/4046515915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:43:05.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:05 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:05.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:05 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:05 smithi193 ceph-mon[33747]: pgmap v40: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:05 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3167572870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90511b8f-4444-44ff-b30c-789349b354f9"}]: dispatch 2024-04-25T15:43:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:05 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3167572870' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90511b8f-4444-44ff-b30c-789349b354f9"}]': finished 2024-04-25T15:43:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:05 smithi193 ceph-mon[33747]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T15:43:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:05 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:05 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/4046515915' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:43:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:05 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:05.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:05 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:07.395 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:07 smithi110 ceph-mon[26344]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:07 smithi193 ceph-mon[33747]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:08 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:08 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:08 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:08 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:08 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:08 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:09.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:09 smithi193 ceph-mon[33747]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:10.099 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:09 smithi110 ceph-mon[26344]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:10.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:10 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:43:11.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:10 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:43:11.988 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:11 smithi110 ceph-mon[26344]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:11.988 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T15:43:11.988 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:11.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:11 smithi193 ceph-mon[33747]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:11.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T15:43:11.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:12.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:12 smithi193 ceph-mon[33747]: Deploying daemon osd.1 on smithi110 2024-04-25T15:43:13.022 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:12 smithi110 ceph-mon[26344]: Deploying daemon osd.1 on smithi110 2024-04-25T15:43:13.945 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:13 smithi110 ceph-mon[26344]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:13 smithi193 ceph-mon[33747]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:15.829 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:15 smithi110 ceph-mon[26344]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:15 smithi193 ceph-mon[33747]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:17.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:17 smithi193 ceph-mon[33747]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:17.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:17.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:17.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:17.780 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:17 smithi110 ceph-mon[26344]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:17.780 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:17.780 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:17.780 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:19.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.292+0000 7fdd5ffff700 1 -- 172.21.15.110:0/3215892481 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdd78066cf0 con 0x7fdd60069210 2024-04-25T15:43:19.293 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 1 on host 'smithi110' 2024-04-25T15:43:19.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fdd60069210 msgr2=0x7fdd6006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fdd60069210 0x7fdd6006b6c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdd680041d0 tx=0x7fdd680043c0 comp rx=0 tx=0).stop 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 msgr2=0x7fdd7810cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 0x7fdd7810cd80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdd6c009fd0 tx=0x7fdd6c009e30 comp rx=0 tx=0).stop 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 shutdown_connections 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fdd60069210 0x7fdd6006b6c0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd780fbcc0 0x7fdd7810c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 --2- 172.21.15.110:0/3215892481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd780fcf70 0x7fdd7810cd80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 >> 172.21.15.110:0/3215892481 conn(0x7fdd780f7440 msgr2=0x7fdd780f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 shutdown_connections 2024-04-25T15:43:19.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:19.295+0000 7fdd7dceb700 1 -- 172.21.15.110:0/3215892481 wait complete. 2024-04-25T15:43:19.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:19 smithi110 ceph-mon[26344]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:19.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:19.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:19.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:19 smithi193 ceph-mon[33747]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:19.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:19.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:20.200 DEBUG:teuthology.orchestra.run.smithi110:osd.1> sudo journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T15:43:20.202 INFO:tasks.cephadm:Deploying osd.2 on smithi110 with /dev/vg_nvme/lv_2... 2024-04-25T15:43:20.202 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T15:43:20.493 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:20 smithi110 ceph-mon[26344]: from='osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T15:43:20.493 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[53015]: 2024-04-25T15:43:20.274+0000 7fdf70cbf540 -1 osd.1 0 log_to_monitors true 2024-04-25T15:43:20.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:20 smithi193 ceph-mon[33747]: from='osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T15:43:20.898 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:43:21.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:21 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[53015]: 2024-04-25T15:43:21.389+0000 7fdf67491700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T15:43:21.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:21 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[53015]: 2024-04-25T15:43:21.393+0000 7fdf62a8d700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:43:21.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:21 smithi110 ceph-mon[26344]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:21.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:21 smithi110 ceph-mon[26344]: from='osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T15:43:21.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:21 smithi110 ceph-mon[26344]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T15:43:21.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:21 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:21.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:21 smithi110 ceph-mon[26344]: from='osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:43:21.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:21 smithi193 ceph-mon[33747]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:21.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:21 smithi193 ceph-mon[33747]: from='osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T15:43:21.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:21 smithi193 ceph-mon[33747]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T15:43:21.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:21 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:21.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:21 smithi193 ceph-mon[33747]: from='osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:43:22.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:22 smithi110 ceph-mon[26344]: from='osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-25T15:43:22.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:22 smithi110 ceph-mon[26344]: osdmap e13: 2 total, 1 up, 2 in 2024-04-25T15:43:22.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:22 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:22.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:22 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:22.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:22 smithi193 ceph-mon[33747]: from='osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-25T15:43:22.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:22 smithi193 ceph-mon[33747]: osdmap e13: 2 total, 1 up, 2 in 2024-04-25T15:43:22.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:22 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:22.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:22 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:23.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:23 smithi110 ceph-mon[26344]: purged_snaps scrub starts 2024-04-25T15:43:23.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:23 smithi110 ceph-mon[26344]: purged_snaps scrub ok 2024-04-25T15:43:23.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:23 smithi110 ceph-mon[26344]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:23.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:23 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:23.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:23 smithi110 ceph-mon[26344]: osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] boot 2024-04-25T15:43:23.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:23 smithi110 ceph-mon[26344]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T15:43:23.604 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:23 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:23.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:23 smithi193 ceph-mon[33747]: purged_snaps scrub starts 2024-04-25T15:43:23.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:23 smithi193 ceph-mon[33747]: purged_snaps scrub ok 2024-04-25T15:43:23.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:23 smithi193 ceph-mon[33747]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-04-25T15:43:23.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:23 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:23.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:23 smithi193 ceph-mon[33747]: osd.1 [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] boot 2024-04-25T15:43:23.655 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:23 smithi193 ceph-mon[33747]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T15:43:23.656 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:23 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:43:23.895 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T15:43:23.895 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T15:43:23.896 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-04-25T15:43:23.896 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-04-25T15:43:23.896 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00927661 s, 1.1 GB/s 2024-04-25T15:43:23.896 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-04-25T15:43:24.488 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:24 smithi110 ceph-mon[26344]: osdmap e15: 2 total, 2 up, 2 in 2024-04-25T15:43:24.519 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-04-25T15:43:24.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:24 smithi193 ceph-mon[33747]: osdmap e15: 2 total, 2 up, 2 in 2024-04-25T15:43:25.227 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:43:25.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:25 smithi110 ceph-mon[26344]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:25.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:43:25.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:25.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:25.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:25 smithi193 ceph-mon[33747]: pgmap v55: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:25.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:43:25.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:25.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:27.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 -- 172.21.15.110:0/1890996168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b4066240 msgr2=0x7f90b40ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:27.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 --2- 172.21.15.110:0/1890996168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b4066240 0x7f90b40ffa20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f90a4004940 tx=0x7f90a4009e30 comp rx=0 tx=0).stop 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 -- 172.21.15.110:0/1890996168 shutdown_connections 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 --2- 172.21.15.110:0/1890996168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b40fff60 0x7f90b4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 --2- 172.21.15.110:0/1890996168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b4066240 0x7f90b40ffa20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 -- 172.21.15.110:0/1890996168 >> 172.21.15.110:0/1890996168 conn(0x7f90b40f7430 msgr2=0x7f90b40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 -- 172.21.15.110:0/1890996168 shutdown_connections 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 -- 172.21.15.110:0/1890996168 wait complete. 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.020+0000 7f90ba37d700 1 Processor -- start 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90ba37d700 1 -- start start 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90ba37d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4066240 0x7f90b40fd450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90ba37d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40fff60 0x7f90b40fb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90ba37d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40fbf60 con 0x7f90b40fff60 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90ba37d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40fc0d0 con 0x7f90b4066240 2024-04-25T15:43:27.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40fff60 0x7f90b40fb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40fff60 0x7f90b40fb920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54506/0 (socket says 172.21.15.110:54506) 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b37fe700 1 -- 172.21.15.110:0/3129170045 learned_addr learned my addr 172.21.15.110:0/3129170045 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b3fff700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4066240 0x7f90b40fd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b37fe700 1 -- 172.21.15.110:0/3129170045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4066240 msgr2=0x7f90b40fd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b37fe700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4066240 0x7f90b40fd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b37fe700 1 -- 172.21.15.110:0/3129170045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a4005040 con 0x7f90b40fff60 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b37fe700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40fff60 0x7f90b40fb920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90a800b790 tx=0x7f90a8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.021+0000 7f90b0ff9700 1 -- 172.21.15.110:0/3129170045 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a800f6f0 con 0x7f90b40fff60 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.022+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b40fc3b0 con 0x7f90b40fff60 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.022+0000 7f90b0ff9700 1 -- 172.21.15.110:0/3129170045 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f90a800fd20 con 0x7f90b40fff60 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.022+0000 7f90b0ff9700 1 -- 172.21.15.110:0/3129170045 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90a80103a0 con 0x7f90b40fff60 2024-04-25T15:43:27.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.022+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b4115b10 con 0x7f90b40fff60 2024-04-25T15:43:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.023+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b405f5c0 con 0x7f90b40fff60 2024-04-25T15:43:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.023+0000 7f90b0ff9700 1 -- 172.21.15.110:0/3129170045 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f90a8011400 con 0x7f90b40fff60 2024-04-25T15:43:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.023+0000 7f90b0ff9700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f909c069210 0x7f909c06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.023+0000 7f90b0ff9700 1 -- 172.21.15.110:0/3129170045 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2410+0+0 (secure 0 0 0) 0x7f90a8086260 con 0x7f90b40fff60 2024-04-25T15:43:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.023+0000 7f90b3fff700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f909c069210 0x7f909c06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:27.025 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.024+0000 7f90b3fff700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f909c069210 0x7f909c06b6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f90b40fd1e0 tx=0x7f90a4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:43:27.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.028+0000 7f90b0ff9700 1 -- 172.21.15.110:0/3129170045 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f90a8053260 con 0x7f90b40fff60 2024-04-25T15:43:27.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:27.226+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f90b40f8f20 con 0x7f909c069210 2024-04-25T15:43:27.584 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:27 smithi110 ceph-mon[26344]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:27.584 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:43:27.584 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:43:27.584 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:27.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:27 smithi193 ceph-mon[33747]: pgmap v56: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:27.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:43:27.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:43:27.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:28.691 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:28 smithi110 ceph-mon[26344]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:43:28.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:28 smithi193 ceph-mon[33747]: from='client.14280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:43:29.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:29 smithi193 ceph-mon[33747]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:29 smithi110 ceph-mon[26344]: pgmap v57: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: Detected new or changed devices on smithi110 2024-04-25T15:43:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: Adjusting osd_memory_target on smithi110 to 6432M 2024-04-25T15:43:31.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:31.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:31.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:31.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:31.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:31.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:31.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: Detected new or changed devices on smithi110 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: Adjusting osd_memory_target on smithi110 to 6432M 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:31.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:32.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:32 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2129675302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c74ff664-988d-4d4b-a248-176d11f44010"}]: dispatch 2024-04-25T15:43:32.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:32 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2129675302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c74ff664-988d-4d4b-a248-176d11f44010"}]': finished 2024-04-25T15:43:32.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:32 smithi110 ceph-mon[26344]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T15:43:32.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:32 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:32.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:32 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1353030490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:43:32.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:32 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2129675302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c74ff664-988d-4d4b-a248-176d11f44010"}]: dispatch 2024-04-25T15:43:32.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:32 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2129675302' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c74ff664-988d-4d4b-a248-176d11f44010"}]': finished 2024-04-25T15:43:32.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:32 smithi193 ceph-mon[33747]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T15:43:32.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:32 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:32.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:32 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1353030490' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:43:33.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:33 smithi110 ceph-mon[26344]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:33.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:33 smithi193 ceph-mon[33747]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:35.721 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:35 smithi110 ceph-mon[26344]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:35.721 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:35.721 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:35.721 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:35.721 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:35.721 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:35 smithi193 ceph-mon[33747]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:35.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:37.840 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:37 smithi110 ceph-mon[26344]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:37.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:37 smithi193 ceph-mon[33747]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:39 smithi110 ceph-mon[26344]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:39 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T15:43:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:39 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:39 smithi110 ceph-mon[26344]: Deploying daemon osd.2 on smithi110 2024-04-25T15:43:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:39 smithi193 ceph-mon[33747]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:39 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T15:43:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:39 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:39 smithi193 ceph-mon[33747]: Deploying daemon osd.2 on smithi110 2024-04-25T15:43:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:43:40.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:43:41.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:41 smithi110 ceph-mon[26344]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:41 smithi193 ceph-mon[33747]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:43 smithi110 ceph-mon[26344]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:43 smithi193 ceph-mon[33747]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:45.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:45.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:45.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:44 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:44 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:45.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:45 smithi193 ceph-mon[33747]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:45 smithi110 ceph-mon[26344]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:46.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.929+0000 7f90b0ff9700 1 -- 172.21.15.110:0/3129170045 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f90b40f8f20 con 0x7f909c069210 2024-04-25T15:43:46.931 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 2 on host 'smithi110' 2024-04-25T15:43:46.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f909c069210 msgr2=0x7f909c06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:46.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f909c069210 0x7f909c06b6c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f90b40fd1e0 tx=0x7f90a4009e30 comp rx=0 tx=0).stop 2024-04-25T15:43:46.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40fff60 msgr2=0x7f90b40fb920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:46.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40fff60 0x7f90b40fb920 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f90a800b790 tx=0x7f90a8009250 comp rx=0 tx=0).stop 2024-04-25T15:43:46.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 shutdown_connections 2024-04-25T15:43:46.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f909c069210 0x7f909c06b6c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:46.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90b4066240 0x7f90b40fd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:46.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 --2- 172.21.15.110:0/3129170045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90b40fff60 0x7f90b40fb920 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:46.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 >> 172.21.15.110:0/3129170045 conn(0x7f90b40f7430 msgr2=0x7f90b40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:43:46.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.932+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 shutdown_connections 2024-04-25T15:43:46.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:43:46.933+0000 7f90ba37d700 1 -- 172.21.15.110:0/3129170045 wait complete. 2024-04-25T15:43:47.570 DEBUG:teuthology.orchestra.run.smithi110:osd.2> sudo journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T15:43:47.572 INFO:tasks.cephadm:Deploying osd.3 on smithi193 with /dev/vg_nvme/lv_4... 2024-04-25T15:43:47.572 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T15:43:47.829 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi193/config 2024-04-25T15:43:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:47 smithi110 ceph-mon[26344]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:47 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:47 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:47 smithi110 ceph-mon[26344]: from='osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T15:43:47.863 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:47 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[60352]: 2024-04-25T15:43:47.631+0000 7fcdcc316540 -1 osd.2 0 log_to_monitors true 2024-04-25T15:43:47.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:47 smithi193 ceph-mon[33747]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:47.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:47 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:47.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:47 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:47.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:47 smithi193 ceph-mon[33747]: from='osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T15:43:49.225 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:48 smithi193 ceph-mon[33747]: from='osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T15:43:49.225 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:48 smithi193 ceph-mon[33747]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T15:43:49.225 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:48 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:49.225 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:48 smithi193 ceph-mon[33747]: from='osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:43:49.225 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:48 smithi193 ceph-mon[33747]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:49.337 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:48 smithi110 ceph-mon[26344]: from='osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T15:43:49.338 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:48 smithi110 ceph-mon[26344]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T15:43:49.338 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:48 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:49.338 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:48 smithi110 ceph-mon[26344]: from='osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:43:49.338 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:48 smithi110 ceph-mon[26344]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:49.338 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:48 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[60352]: 2024-04-25T15:43:48.890+0000 7fcdc2ae8700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T15:43:49.338 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:48 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[60352]: 2024-04-25T15:43:48.896+0000 7fcdbe8e5700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:43:49.781 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T15:43:49.781 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T15:43:49.781 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-04-25T15:43:49.781 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-04-25T15:43:49.781 INFO:teuthology.orchestra.run.smithi193.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114488 s, 916 MB/s 2024-04-25T15:43:49.781 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 2024-04-25T15:43:49.781 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-04-25T15:43:50.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:49 smithi193 ceph-mon[33747]: from='osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-25T15:43:50.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:49 smithi193 ceph-mon[33747]: osdmap e18: 3 total, 2 up, 3 in 2024-04-25T15:43:50.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:49 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:50.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:49 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:50.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:49 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:50.191 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:49 smithi110 ceph-mon[26344]: from='osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-04-25T15:43:50.191 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:49 smithi110 ceph-mon[26344]: osdmap e18: 3 total, 2 up, 3 in 2024-04-25T15:43:50.191 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:49 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:50.191 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:49 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:50.191 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:49 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:50.380 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_4 2024-04-25T15:43:50.636 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi193/config 2024-04-25T15:43:50.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: purged_snaps scrub starts 2024-04-25T15:43:50.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: purged_snaps scrub ok 2024-04-25T15:43:50.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] boot 2024-04-25T15:43:50.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T15:43:50.930 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:50.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:50.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-25T15:43:50.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:50.931 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: purged_snaps scrub starts 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: purged_snaps scrub ok 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: osd.2 [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] boot 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.008+0000 7f803a1bf700 1 -- 172.21.15.193:0/133343089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fdfd0 msgr2=0x7f80340fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.008+0000 7f803a1bf700 1 --2- 172.21.15.193:0/133343089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fdfd0 0x7f80340fe440 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f801c004940 tx=0x7f801c009e30 comp rx=0 tx=0).stop 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.008+0000 7f803a1bf700 1 -- 172.21.15.193:0/133343089 shutdown_connections 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.008+0000 7f803a1bf700 1 --2- 172.21.15.193:0/133343089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fdfd0 0x7f80340fe440 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.008+0000 7f803a1bf700 1 --2- 172.21.15.193:0/133343089 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80340fce60 0x7f80340fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.008+0000 7f803a1bf700 1 -- 172.21.15.193:0/133343089 >> 172.21.15.193:0/133343089 conn(0x7f80340f8540 msgr2=0x7f80340fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.008+0000 7f803a1bf700 1 -- 172.21.15.193:0/133343089 shutdown_connections 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.008+0000 7f803a1bf700 1 -- 172.21.15.193:0/133343089 wait complete. 2024-04-25T15:43:52.012 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f803a1bf700 1 Processor -- start 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f803a1bf700 1 -- start start 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f803a1bf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fce60 0x7f803410d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f803a1bf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80340fdfd0 0x7f803410ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f803a1bf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803410e3f0 con 0x7f80340fdfd0 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f803a1bf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803410e560 con 0x7f80340fce60 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f80337fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fce60 0x7f803410d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f80337fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fce60 0x7f803410d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:34642/0 (socket says 172.21.15.193:34642) 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f80337fe700 1 -- 172.21.15.193:0/1988308062 learned_addr learned my addr 172.21.15.193:0/1988308062 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f802affd700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80340fdfd0 0x7f803410ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f80337fe700 1 -- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80340fdfd0 msgr2=0x7f803410ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f80337fe700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80340fdfd0 0x7f803410ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f80337fe700 1 -- 172.21.15.193:0/1988308062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c005040 con 0x7f80340fce60 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.009+0000 7f80337fe700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fce60 0x7f803410d8b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f802400eb40 tx=0x7f802400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:43:52.013 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.010+0000 7f8030ff9700 1 -- 172.21.15.193:0/1988308062 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8024007cc0 con 0x7f80340fce60 2024-04-25T15:43:52.014 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.010+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8034112f60 con 0x7f80340fce60 2024-04-25T15:43:52.014 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.010+0000 7f8030ff9700 1 -- 172.21.15.193:0/1988308062 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f802400f040 con 0x7f80340fce60 2024-04-25T15:43:52.014 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.010+0000 7f8030ff9700 1 -- 172.21.15.193:0/1988308062 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f802400ca10 con 0x7f80340fce60 2024-04-25T15:43:52.014 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.010+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8034113410 con 0x7f80340fce60 2024-04-25T15:43:52.014 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.011+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803404c710 con 0x7f80340fce60 2024-04-25T15:43:52.014 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.011+0000 7f8030ff9700 1 -- 172.21.15.193:0/1988308062 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 18) v1 ==== 86575+0+0 (secure 0 0 0) 0x7f802400cc80 con 0x7f80340fce60 2024-04-25T15:43:52.015 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.011+0000 7f8030ff9700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8020069210 0x7f802006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:43:52.015 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.012+0000 7f8030ff9700 1 -- 172.21.15.193:0/1988308062 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3228+0+0 (secure 0 0 0) 0x7f8024086370 con 0x7f80340fce60 2024-04-25T15:43:52.015 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.012+0000 7f802affd700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8020069210 0x7f802006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:43:52.015 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.012+0000 7f802affd700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8020069210 0x7f802006b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f801c009b30 tx=0x7f801c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:43:52.016 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.016+0000 7f8030ff9700 1 -- 172.21.15.193:0/1988308062 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f802404e250 con 0x7f80340fce60 2024-04-25T15:43:52.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:51 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-25T15:43:52.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:51 smithi110 ceph-mon[26344]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T15:43:52.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:51 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:43:52.212 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.211+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f8034113600 con 0x7f8020069210 2024-04-25T15:43:52.212 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:51 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-04-25T15:43:52.212 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:51 smithi193 ceph-mon[33747]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T15:43:52.212 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:51 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T15:43:52.520 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62282]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T15:43:52.520 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62282]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:52.520 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62282]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:52.520 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62282]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:52.772 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62307]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T15:43:52.772 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62307]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:52.772 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62307]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:52.772 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62307]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:52.916 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:43:52.916+0000 7f8030ff9700 1 -- 172.21.15.193:0/1988308062 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8024053440 con 0x7f80340fce60 2024-04-25T15:43:53.062 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62316]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T15:43:53.062 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62316]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:53.062 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62316]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:53.062 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:52 smithi110 sudo[62316]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:53.062 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:52 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T15:43:53.062 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:52 smithi110 ceph-mon[26344]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T15:43:53.062 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:52 smithi110 ceph-mon[26344]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:43:53.062 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:52 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:43:53.062 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:52 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:43:53.062 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:52 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:53.063 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:52 smithi110 ceph-mon[26344]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:43:53.063 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:52 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:52 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T15:43:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:52 smithi193 ceph-mon[33747]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T15:43:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:52 smithi193 ceph-mon[33747]: from='client.24131 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:43:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:52 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:43:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:52 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:43:53.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:52 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:53.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:52 smithi193 ceph-mon[33747]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:43:53.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:52 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:53.314 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[62335]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T15:43:53.314 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[62335]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:53.315 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[62335]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:53.315 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[62335]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:53.578 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[63946]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T15:43:53.578 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[63946]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:53.578 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[63946]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:53.578 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[63946]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:53.839 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[64597]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T15:43:53.839 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[64597]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:53.839 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[64597]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:53.839 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[64597]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:53.839 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[64602]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T15:43:53.839 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[64602]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:53.840 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[64602]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:53.840 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:53 smithi110 sudo[64602]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:54 smithi110 sudo[64605]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T15:43:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:54 smithi110 sudo[64605]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:54 smithi110 sudo[64605]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:54 smithi110 sudo[64605]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:54 smithi110 ceph-mon[26344]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T15:43:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:54 smithi110 ceph-mon[26344]: mgrmap e19: smithi110.lvgbkf(active, since 2m), standbys: smithi193.mgoteh 2024-04-25T15:43:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:54 smithi110 ceph-mon[26344]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T15:43:54.341 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 ceph-mon[33747]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T15:43:54.341 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 ceph-mon[33747]: mgrmap e19: smithi110.lvgbkf(active, since 2m), standbys: smithi193.mgoteh 2024-04-25T15:43:54.341 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 ceph-mon[33747]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T15:43:54.598 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 sudo[37140]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T15:43:54.598 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 sudo[37140]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:54.599 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 sudo[37140]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:54.599 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 sudo[37140]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 sudo[37161]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T15:43:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 sudo[37161]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T15:43:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 sudo[37161]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T15:43:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:54 smithi193 sudo[37161]: pam_unix(sudo:session): session closed for user root 2024-04-25T15:43:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: Detected new or changed devices on smithi110 2024-04-25T15:43:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T15:43:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:43:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:43:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:43:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T15:43:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:43:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: Adjusting osd_memory_target on smithi110 to 4288M 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/2380240126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd48aeea-c692-4258-a8df-848bf440b6ab"}]: dispatch 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd48aeea-c692-4258-a8df-848bf440b6ab"}]: dispatch 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd48aeea-c692-4258-a8df-848bf440b6ab"}]': finished 2024-04-25T15:43:55.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T15:43:55.365 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:43:55.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: Detected new or changed devices on smithi110 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 872 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: Adjusting osd_memory_target on smithi110 to 4288M 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='client.? 172.21.15.193:0/2380240126' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd48aeea-c692-4258-a8df-848bf440b6ab"}]: dispatch 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "dd48aeea-c692-4258-a8df-848bf440b6ab"}]: dispatch 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "dd48aeea-c692-4258-a8df-848bf440b6ab"}]': finished 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T15:43:55.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:43:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:56 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:56 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:56 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:56 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:43:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:56 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/4051580997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:43:56.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:56 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:43:56.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:56 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:43:56.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:56 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:43:56.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:56 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:43:56.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:56 smithi193 ceph-mon[33747]: from='client.? 172.21.15.193:0/4051580997' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:43:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:57 smithi110 ceph-mon[26344]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:43:57.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:57 smithi193 ceph-mon[33747]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:43:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:43:59 smithi110 ceph-mon[26344]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:43:59.682 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:43:59 smithi193 ceph-mon[33747]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:44:01.479 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:01 smithi193 ceph-mon[33747]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-25T15:44:01.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:01 smithi110 ceph-mon[26344]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-04-25T15:44:02.399 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:02 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T15:44:02.399 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:02 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:02.399 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:02 smithi193 ceph-mon[33747]: Deploying daemon osd.3 on smithi193 2024-04-25T15:44:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:02 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T15:44:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:02 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:02.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:02 smithi110 ceph-mon[26344]: Deploying daemon osd.3 on smithi193 2024-04-25T15:44:03.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:03 smithi110 ceph-mon[26344]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-25T15:44:03.614 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:03 smithi193 ceph-mon[33747]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-04-25T15:44:05.469 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:05 smithi193 ceph-mon[33747]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-25T15:44:05.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:05 smithi110 ceph-mon[26344]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-04-25T15:44:07.900 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:07 smithi193 ceph-mon[33747]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-04-25T15:44:07.901 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:07 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:07.901 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:07 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:07.901 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:07 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:07 smithi110 ceph-mon[26344]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 69 KiB/s wr, 4 op/s 2024-04-25T15:44:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:07 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:07 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:07 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:09.626 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.625+0000 7f8030ff9700 1 -- 172.21.15.193:0/1988308062 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8034113600 con 0x7f8020069210 2024-04-25T15:44:09.627 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 3 on host 'smithi193' 2024-04-25T15:44:09.628 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8020069210 msgr2=0x7f802006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8020069210 0x7f802006b6c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f801c009b30 tx=0x7f801c009e30 comp rx=0 tx=0).stop 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fce60 msgr2=0x7f803410d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fce60 0x7f803410d8b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f802400eb40 tx=0x7f802400b040 comp rx=0 tx=0).stop 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 shutdown_connections 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8020069210 0x7f802006b6c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80340fce60 0x7f803410d8b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 --2- 172.21.15.193:0/1988308062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80340fdfd0 0x7f803410ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 >> 172.21.15.193:0/1988308062 conn(0x7f80340f8540 msgr2=0x7f80340f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:44:09.629 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 shutdown_connections 2024-04-25T15:44:09.630 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:09.628+0000 7f803a1bf700 1 -- 172.21.15.193:0/1988308062 wait complete. 2024-04-25T15:44:09.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:09 smithi193 ceph-mon[33747]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-25T15:44:09.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:09 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:09.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:09 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:09 smithi110 ceph-mon[26344]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-25T15:44:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:09 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:09 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:10.212 DEBUG:teuthology.orchestra.run.smithi193:osd.3> sudo journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T15:44:10.215 INFO:tasks.cephadm:Deploying osd.4 on smithi193 with /dev/vg_nvme/lv_3... 2024-04-25T15:44:10.215 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T15:44:10.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:10 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:44:10.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:10 smithi193 ceph-mon[33747]: from='osd.3 [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T15:44:10.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:10 smithi193 ceph-mon[33747]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T15:44:10.740 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:44:10 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[39361]: 2024-04-25T15:44:10.501+0000 7f2d72b81540 -1 osd.3 0 log_to_monitors true 2024-04-25T15:44:10.878 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi193/config 2024-04-25T15:44:11.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:10 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:44:11.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:10 smithi110 ceph-mon[26344]: from='osd.3 [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T15:44:11.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:10 smithi110 ceph-mon[26344]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T15:44:11.931 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[39361]: 2024-04-25T15:44:11.638+0000 7f2d69353700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T15:44:11.932 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[39361]: 2024-04-25T15:44:11.643+0000 7f2d6414e700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:44:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-mon[33747]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-25T15:44:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-mon[33747]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T15:44:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-mon[33747]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T15:44:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-mon[33747]: from='osd.3 [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:44:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-mon[33747]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:44:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:11.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:11 smithi110 ceph-mon[26344]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-04-25T15:44:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:11 smithi110 ceph-mon[26344]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T15:44:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:11 smithi110 ceph-mon[26344]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T15:44:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:11 smithi110 ceph-mon[26344]: from='osd.3 [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:44:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:11 smithi110 ceph-mon[26344]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:44:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:12.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:12.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:12 smithi193 ceph-mon[33747]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T15:44:12.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:12 smithi193 ceph-mon[33747]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T15:44:12.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:12 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:12.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:12 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:12.746 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T15:44:12.746 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T15:44:12.746 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-04-25T15:44:12.746 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-04-25T15:44:12.746 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125162 s, 838 MB/s 2024-04-25T15:44:12.746 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-04-25T15:44:13.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:12 smithi110 ceph-mon[26344]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T15:44:13.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:12 smithi110 ceph-mon[26344]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T15:44:13.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:12 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:13.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:12 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:13.471 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_3 2024-04-25T15:44:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:13 smithi193 ceph-mon[33747]: purged_snaps scrub starts 2024-04-25T15:44:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:13 smithi193 ceph-mon[33747]: purged_snaps scrub ok 2024-04-25T15:44:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:13 smithi193 ceph-mon[33747]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:44:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:13 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:13 smithi193 ceph-mon[33747]: osd.3 [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] boot 2024-04-25T15:44:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:13 smithi193 ceph-mon[33747]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T15:44:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:13 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:13.911 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi193/config 2024-04-25T15:44:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:13 smithi110 ceph-mon[26344]: purged_snaps scrub starts 2024-04-25T15:44:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:13 smithi110 ceph-mon[26344]: purged_snaps scrub ok 2024-04-25T15:44:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:13 smithi110 ceph-mon[26344]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-04-25T15:44:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:13 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:13 smithi110 ceph-mon[26344]: osd.3 [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] boot 2024-04-25T15:44:14.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:13 smithi110 ceph-mon[26344]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T15:44:14.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:13 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:44:14.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:14 smithi193 ceph-mon[33747]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T15:44:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:14 smithi110 ceph-mon[26344]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T15:44:15.650 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.649+0000 7f2554437700 1 -- 172.21.15.193:0/2460264150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 msgr2=0x7f254c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:15.650 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.649+0000 7f2554437700 1 --2- 172.21.15.193:0/2460264150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 0x7f254c0fc150 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2548004940 tx=0x7f2548009e30 comp rx=0 tx=0).stop 2024-04-25T15:44:15.650 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.649+0000 7f2554437700 1 -- 172.21.15.193:0/2460264150 shutdown_connections 2024-04-25T15:44:15.650 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.649+0000 7f2554437700 1 --2- 172.21.15.193:0/2460264150 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f254c0fcff0 0x7f254c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:15.650 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.649+0000 7f2554437700 1 --2- 172.21.15.193:0/2460264150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 0x7f254c0fc150 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:15.651 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.649+0000 7f2554437700 1 -- 172.21.15.193:0/2460264150 >> 172.21.15.193:0/2460264150 conn(0x7f254c0f74c0 msgr2=0x7f254c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:44:15.651 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.649+0000 7f2554437700 1 -- 172.21.15.193:0/2460264150 shutdown_connections 2024-04-25T15:44:15.651 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.649+0000 7f2554437700 1 -- 172.21.15.193:0/2460264150 wait complete. 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f2554437700 1 Processor -- start 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f2554437700 1 -- start start 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f2554437700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 0x7f254c10fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f2554437700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f254c0fcff0 0x7f254c110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f2554437700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c110700 con 0x7f254c0fcff0 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f2554437700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c110f50 con 0x7f254c0fbd40 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f25521d3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 0x7f254c10fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f25521d3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 0x7f254c10fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:45380/0 (socket says 172.21.15.193:45380) 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f25521d3700 1 -- 172.21.15.193:0/3604284352 learned_addr learned my addr 172.21.15.193:0/3604284352 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.650+0000 7f25519d2700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f254c0fcff0 0x7f254c110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:44:15.652 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.651+0000 7f25521d3700 1 -- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f254c0fcff0 msgr2=0x7f254c110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:15.653 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.651+0000 7f25521d3700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f254c0fcff0 0x7f254c110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:15.653 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.651+0000 7f25521d3700 1 -- 172.21.15.193:0/3604284352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548005040 con 0x7f254c0fbd40 2024-04-25T15:44:15.653 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.651+0000 7f25521d3700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 0x7f254c10fbc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2548009150 tx=0x7f2548009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:44:15.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.653+0000 7f2542ffd700 1 -- 172.21.15.193:0/3604284352 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254800e040 con 0x7f254c0fbd40 2024-04-25T15:44:15.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.653+0000 7f2542ffd700 1 -- 172.21.15.193:0/3604284352 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f254800bbf0 con 0x7f254c0fbd40 2024-04-25T15:44:15.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.653+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c1111d0 con 0x7f254c0fbd40 2024-04-25T15:44:15.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.653+0000 7f2542ffd700 1 -- 172.21.15.193:0/3604284352 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f254801c630 con 0x7f254c0fbd40 2024-04-25T15:44:15.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.653+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c111710 con 0x7f254c0fbd40 2024-04-25T15:44:15.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.654+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c101980 con 0x7f254c0fbd40 2024-04-25T15:44:15.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.654+0000 7f2542ffd700 1 -- 172.21.15.193:0/3604284352 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f254801c790 con 0x7f254c0fbd40 2024-04-25T15:44:15.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.655+0000 7f2542ffd700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f25380692b0 0x7f253806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:44:15.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.655+0000 7f2542ffd700 1 -- 172.21.15.193:0/3604284352 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 3770+0+0 (secure 0 0 0) 0x7f2548087db0 con 0x7f254c0fbd40 2024-04-25T15:44:15.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.655+0000 7f25519d2700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f25380692b0 0x7f253806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:44:15.660 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.659+0000 7f2542ffd700 1 -- 172.21.15.193:0/3604284352 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2548058150 con 0x7f254c0fbd40 2024-04-25T15:44:15.661 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.661+0000 7f25519d2700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f25380692b0 0x7f253806b760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f253c009fd0 tx=0x7f253c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:44:15.862 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:15.861+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f254c111980 con 0x7f25380692b0 2024-04-25T15:44:15.863 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:15 smithi193 ceph-mon[33747]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:15.863 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:15 smithi193 ceph-mon[33747]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T15:44:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:15 smithi110 ceph-mon[26344]: pgmap v92: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:15 smithi110 ceph-mon[26344]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T15:44:16.694 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:16 smithi193 ceph-mon[33747]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T15:44:16.694 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:16 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:44:16.694 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:16 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:44:16.694 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:16 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:16 smithi110 ceph-mon[26344]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T15:44:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:16 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:44:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:16 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:44:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:16 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: Detected new or changed devices on smithi193 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: Adjusting osd_memory_target on smithi193 to 15936M 2024-04-25T15:44:17.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:17.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:17.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:44:17.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:17.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:17.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:17.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:44:17.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:17 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:44:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: pgmap v95: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T15:44:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: Detected new or changed devices on smithi193 2024-04-25T15:44:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: Adjusting osd_memory_target on smithi193 to 15936M 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:44:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:17 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:19 smithi193 ceph-mon[33747]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:19 smithi193 ceph-mon[33747]: from='client.? 172.21.15.193:0/1284842930' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b0a0e9a-5308-421d-8091-fac8f7649509"}]: dispatch 2024-04-25T15:44:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:19 smithi193 ceph-mon[33747]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b0a0e9a-5308-421d-8091-fac8f7649509"}]: dispatch 2024-04-25T15:44:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:19 smithi193 ceph-mon[33747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b0a0e9a-5308-421d-8091-fac8f7649509"}]': finished 2024-04-25T15:44:19.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:19 smithi193 ceph-mon[33747]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T15:44:19.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:19.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:19 smithi193 ceph-mon[33747]: from='client.? 172.21.15.193:0/2801513643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:44:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:19 smithi110 ceph-mon[26344]: pgmap v97: 1 pgs: 1 remapped+peering; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:19 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/1284842930' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b0a0e9a-5308-421d-8091-fac8f7649509"}]: dispatch 2024-04-25T15:44:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:19 smithi110 ceph-mon[26344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3b0a0e9a-5308-421d-8091-fac8f7649509"}]: dispatch 2024-04-25T15:44:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:19 smithi110 ceph-mon[26344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3b0a0e9a-5308-421d-8091-fac8f7649509"}]': finished 2024-04-25T15:44:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:19 smithi110 ceph-mon[26344]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T15:44:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:19 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/2801513643' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:44:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:21 smithi193 ceph-mon[33747]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T15:44:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:21 smithi110 ceph-mon[26344]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-25T15:44:23.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:23 smithi193 ceph-mon[33747]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T15:44:24.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:23 smithi110 ceph-mon[26344]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T15:44:25.972 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:25 smithi193 ceph-mon[33747]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T15:44:25.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:44:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:25 smithi110 ceph-mon[26344]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T15:44:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:44:26.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:26 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T15:44:26.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:26 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:27.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:26 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T15:44:27.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:26 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:27.883 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:27 smithi193 ceph-mon[33747]: Deploying daemon osd.4 on smithi193 2024-04-25T15:44:27.884 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:27 smithi193 ceph-mon[33747]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T15:44:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:27 smithi110 ceph-mon[26344]: Deploying daemon osd.4 on smithi193 2024-04-25T15:44:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:27 smithi110 ceph-mon[26344]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T15:44:29.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:29 smithi193 ceph-mon[33747]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T15:44:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:29 smithi110 ceph-mon[26344]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T15:44:31.961 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:31 smithi193 ceph-mon[33747]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:31.961 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:31.961 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:31.961 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:31 smithi110 ceph-mon[26344]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:33.932 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:33 smithi193 ceph-mon[33747]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:33 smithi110 ceph-mon[26344]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:34.277 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.276+0000 7f2542ffd700 1 -- 172.21.15.193:0/3604284352 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f254c111980 con 0x7f25380692b0 2024-04-25T15:44:34.278 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 4 on host 'smithi193' 2024-04-25T15:44:34.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f25380692b0 msgr2=0x7f253806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:34.279 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f25380692b0 0x7f253806b760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f253c009fd0 tx=0x7f253c00b040 comp rx=0 tx=0).stop 2024-04-25T15:44:34.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 msgr2=0x7f254c10fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:34.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 0x7f254c10fbc0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2548009150 tx=0x7f2548009e30 comp rx=0 tx=0).stop 2024-04-25T15:44:34.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 shutdown_connections 2024-04-25T15:44:34.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f25380692b0 0x7f253806b760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:34.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254c0fbd40 0x7f254c10fbc0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:34.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 --2- 172.21.15.193:0/3604284352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f254c0fcff0 0x7f254c110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:34.280 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 >> 172.21.15.193:0/3604284352 conn(0x7f254c0f74c0 msgr2=0x7f254c0f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:44:34.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 shutdown_connections 2024-04-25T15:44:34.281 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:34.278+0000 7f2554437700 1 -- 172.21.15.193:0/3604284352 wait complete. 2024-04-25T15:44:34.893 DEBUG:teuthology.orchestra.run.smithi193:osd.4> sudo journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T15:44:34.895 INFO:tasks.cephadm:Deploying osd.5 on smithi193 with /dev/vg_nvme/lv_2... 2024-04-25T15:44:34.896 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T15:44:35.210 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi193/config 2024-04-25T15:44:35.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:35.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:35.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:35 smithi193 ceph-mon[33747]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:35.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:35 smithi193 ceph-mon[33747]: from='osd.4 [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T15:44:35.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:35 smithi193 ceph-mon[33747]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T15:44:35.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:35.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:35 smithi110 ceph-mon[26344]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:35 smithi110 ceph-mon[26344]: from='osd.4 [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T15:44:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:35 smithi110 ceph-mon[26344]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T15:44:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:36.573 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:36 smithi193 ceph-mon[33747]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T15:44:36.573 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:36 smithi193 ceph-mon[33747]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T15:44:36.573 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:36 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:36.573 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:36 smithi193 ceph-mon[33747]: from='osd.4 [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:44:36.573 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:36 smithi193 ceph-mon[33747]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:44:36.573 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:44:36 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[46086]: 2024-04-25T15:44:36.257+0000 7f78b10de700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T15:44:36.573 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:44:36 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[46086]: 2024-04-25T15:44:36.265+0000 7f78adedd700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:44:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:36 smithi110 ceph-mon[26344]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T15:44:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:36 smithi110 ceph-mon[26344]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T15:44:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:36 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:36 smithi110 ceph-mon[26344]: from='osd.4 [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:44:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:36 smithi110 ceph-mon[26344]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:44:37.012 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T15:44:37.012 INFO:teuthology.orchestra.run.smithi193.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T15:44:37.012 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10+0 records in 2024-04-25T15:44:37.012 INFO:teuthology.orchestra.run.smithi193.stderr:10+0 records out 2024-04-25T15:44:37.012 INFO:teuthology.orchestra.run.smithi193.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123907 s, 846 MB/s 2024-04-25T15:44:37.012 INFO:teuthology.orchestra.run.smithi193.stderr:--> Zapping successful for: 2024-04-25T15:44:37.586 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:37 smithi193 ceph-mon[33747]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T15:44:37.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:37 smithi193 ceph-mon[33747]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T15:44:37.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:37 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:37.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:37 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:37.587 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:37 smithi193 ceph-mon[33747]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:37.611 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi193:vg_nvme/lv_2 2024-04-25T15:44:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:37 smithi110 ceph-mon[26344]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T15:44:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:37 smithi110 ceph-mon[26344]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T15:44:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:37 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:37 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:37 smithi110 ceph-mon[26344]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-04-25T15:44:38.290 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:38 smithi193 ceph-mon[33747]: purged_snaps scrub starts 2024-04-25T15:44:38.290 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:38 smithi193 ceph-mon[33747]: purged_snaps scrub ok 2024-04-25T15:44:38.290 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:38 smithi193 ceph-mon[33747]: osd.4 [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] boot 2024-04-25T15:44:38.290 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:38 smithi193 ceph-mon[33747]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T15:44:38.290 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:38 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:38.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:38 smithi110 ceph-mon[26344]: purged_snaps scrub starts 2024-04-25T15:44:38.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:38 smithi110 ceph-mon[26344]: purged_snaps scrub ok 2024-04-25T15:44:38.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:38 smithi110 ceph-mon[26344]: osd.4 [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] boot 2024-04-25T15:44:38.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:38 smithi110 ceph-mon[26344]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T15:44:38.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:38 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:44:38.774 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi193/config 2024-04-25T15:44:39.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:39 smithi193 ceph-mon[33747]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:39.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:39 smithi193 ceph-mon[33747]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T15:44:39.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:39 smithi110 ceph-mon[26344]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:39.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:39 smithi110 ceph-mon[26344]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T15:44:40.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.418+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/3845791213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e4066f30 msgr2=0x7fe5e40ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:40.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.418+0000 7fe5e9ca7700 1 --2- 172.21.15.193:0/3845791213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e4066f30 0x7fe5e40ff9b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4004940 tx=0x7fe5d4009e30 comp rx=0 tx=0).stop 2024-04-25T15:44:40.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.418+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/3845791213 shutdown_connections 2024-04-25T15:44:40.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.418+0000 7fe5e9ca7700 1 --2- 172.21.15.193:0/3845791213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5e40ffef0 0x7fe5e4102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:40.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.418+0000 7fe5e9ca7700 1 --2- 172.21.15.193:0/3845791213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e4066f30 0x7fe5e40ff9b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:40.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.418+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/3845791213 >> 172.21.15.193:0/3845791213 conn(0x7fe5e40f7440 msgr2=0x7fe5e40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:44:40.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.418+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/3845791213 shutdown_connections 2024-04-25T15:44:40.422 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.418+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/3845791213 wait complete. 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e9ca7700 1 Processor -- start 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e9ca7700 1 -- start start 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e9ca7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5e4066f30 0x7fe5e410d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e9ca7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e40ffef0 0x7fe5e410dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e9ca7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e410e4a0 con 0x7fe5e4066f30 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e9ca7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e410e610 con 0x7fe5e40ffef0 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e2ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e40ffef0 0x7fe5e410dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e2ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e40ffef0 0x7fe5e410dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:44978/0 (socket says 172.21.15.193:44978) 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.419+0000 7fe5e2ffd700 1 -- 172.21.15.193:0/2345251169 learned_addr learned my addr 172.21.15.193:0/2345251169 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5e2ffd700 1 -- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5e4066f30 msgr2=0x7fe5e410d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5e37fe700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5e4066f30 0x7fe5e410d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5e2ffd700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5e4066f30 0x7fe5e410d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5e2ffd700 1 -- 172.21.15.193:0/2345251169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4005040 con 0x7fe5e40ffef0 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5e2ffd700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e40ffef0 0x7fe5e410dea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c830 tx=0x7fe5d8004470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:44:40.423 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5cbfff700 1 -- 172.21.15.193:0/2345251169 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5d800a760 con 0x7fe5e40ffef0 2024-04-25T15:44:40.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5cbfff700 1 -- 172.21.15.193:0/2345251169 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5d800ad90 con 0x7fe5e40ffef0 2024-04-25T15:44:40.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5e4113070 con 0x7fe5e40ffef0 2024-04-25T15:44:40.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5cbfff700 1 -- 172.21.15.193:0/2345251169 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5d80113a0 con 0x7fe5e40ffef0 2024-04-25T15:44:40.424 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.420+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5e4113630 con 0x7fe5e40ffef0 2024-04-25T15:44:40.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.421+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5e404c710 con 0x7fe5e40ffef0 2024-04-25T15:44:40.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.421+0000 7fe5cbfff700 1 -- 172.21.15.193:0/2345251169 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe5d8012400 con 0x7fe5e40ffef0 2024-04-25T15:44:40.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.422+0000 7fe5cbfff700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe5cc0692a0 0x7fe5cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:44:40.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.422+0000 7fe5cbfff700 1 -- 172.21.15.193:0/2345251169 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7fe5d8086c60 con 0x7fe5e40ffef0 2024-04-25T15:44:40.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.422+0000 7fe5e37fe700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe5cc0692a0 0x7fe5cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:44:40.425 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.423+0000 7fe5e37fe700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe5cc0692a0 0x7fe5cc06b750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400b970 tx=0x7fe5d4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:44:40.428 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.426+0000 7fe5cbfff700 1 -- 172.21.15.193:0/2345251169 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe5d8056fb0 con 0x7fe5e40ffef0 2024-04-25T15:44:40.626 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:40.624+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5e40f8f30 con 0x7fe5cc0692a0 2024-04-25T15:44:41.384 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:41.384 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:44:41.384 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:44:41.384 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:44:41.385 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:41.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:44:41.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi193:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:44:41.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T15:44:41.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T15:44:41.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:41.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:44:41.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:42.390 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:42 smithi193 ceph-mon[33747]: Detected new or changed devices on smithi193 2024-04-25T15:44:42.391 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:42 smithi193 ceph-mon[33747]: Adjusting osd_memory_target on smithi193 to 7968M 2024-04-25T15:44:42.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:42 smithi110 ceph-mon[26344]: Detected new or changed devices on smithi193 2024-04-25T15:44:42.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:42 smithi110 ceph-mon[26344]: Adjusting osd_memory_target on smithi193 to 7968M 2024-04-25T15:44:43.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:43 smithi110 ceph-mon[26344]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:43.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:43 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/1841979114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6cf2f8b-8105-475c-8a79-ce9bbcdf8644"}]: dispatch 2024-04-25T15:44:43.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:43 smithi110 ceph-mon[26344]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6cf2f8b-8105-475c-8a79-ce9bbcdf8644"}]: dispatch 2024-04-25T15:44:43.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:43 smithi110 ceph-mon[26344]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6cf2f8b-8105-475c-8a79-ce9bbcdf8644"}]': finished 2024-04-25T15:44:43.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:43 smithi110 ceph-mon[26344]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T15:44:43.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:43 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:44:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:43 smithi193 ceph-mon[33747]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:43 smithi193 ceph-mon[33747]: from='client.? 172.21.15.193:0/1841979114' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6cf2f8b-8105-475c-8a79-ce9bbcdf8644"}]: dispatch 2024-04-25T15:44:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:43 smithi193 ceph-mon[33747]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6cf2f8b-8105-475c-8a79-ce9bbcdf8644"}]: dispatch 2024-04-25T15:44:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:43 smithi193 ceph-mon[33747]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6cf2f8b-8105-475c-8a79-ce9bbcdf8644"}]': finished 2024-04-25T15:44:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:43 smithi193 ceph-mon[33747]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T15:44:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:43 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:44:44.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:44 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/2300765519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:44:44.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:44 smithi193 ceph-mon[33747]: from='client.? 172.21.15.193:0/2300765519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T15:44:45.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:45 smithi110 ceph-mon[26344]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:45.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:45 smithi193 ceph-mon[33747]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:47.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:47 smithi193 ceph-mon[33747]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:47.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:47 smithi110 ceph-mon[26344]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:49.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:49 smithi110 ceph-mon[26344]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:49.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:49 smithi193 ceph-mon[33747]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:50.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T15:44:50.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:50 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:50.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T15:44:50.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:50 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:44:51.580 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:51 smithi193 ceph-mon[33747]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:51.581 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:51 smithi193 ceph-mon[33747]: Deploying daemon osd.5 on smithi193 2024-04-25T15:44:51.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:51 smithi110 ceph-mon[26344]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:51.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:51 smithi110 ceph-mon[26344]: Deploying daemon osd.5 on smithi193 2024-04-25T15:44:53.483 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:53 smithi193 ceph-mon[33747]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:53.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:53 smithi110 ceph-mon[26344]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:55.391 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:55 smithi193 ceph-mon[33747]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:55.391 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:55.391 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:55.391 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:55.391 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:44:55.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:55 smithi110 ceph-mon[26344]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:55.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:55.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:55.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:44:55.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:44:57.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:57 smithi193 ceph-mon[33747]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:57.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:57 smithi110 ceph-mon[26344]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:57.693 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.692+0000 7fe5cbfff700 1 -- 172.21.15.193:0/2345251169 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe5e40f8f30 con 0x7fe5cc0692a0 2024-04-25T15:44:57.693 INFO:teuthology.orchestra.run.smithi193.stdout:Created osd(s) 5 on host 'smithi193' 2024-04-25T15:44:57.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.694+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe5cc0692a0 msgr2=0x7fe5cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:57.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.694+0000 7fe5e9ca7700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe5cc0692a0 0x7fe5cc06b750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe5d400b970 tx=0x7fe5d4009e30 comp rx=0 tx=0).stop 2024-04-25T15:44:57.695 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.694+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e40ffef0 msgr2=0x7fe5e410dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:44:57.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.694+0000 7fe5e9ca7700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e40ffef0 0x7fe5e410dea0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe5d800c830 tx=0x7fe5d8004470 comp rx=0 tx=0).stop 2024-04-25T15:44:57.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.695+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 shutdown_connections 2024-04-25T15:44:57.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.695+0000 7fe5e9ca7700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5e4066f30 0x7fe5e410d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:57.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.695+0000 7fe5e9ca7700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe5cc0692a0 0x7fe5cc06b750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:57.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.695+0000 7fe5e9ca7700 1 --2- 172.21.15.193:0/2345251169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5e40ffef0 0x7fe5e410dea0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:44:57.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.695+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 >> 172.21.15.193:0/2345251169 conn(0x7fe5e40f7440 msgr2=0x7fe5e40f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:44:57.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.695+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 shutdown_connections 2024-04-25T15:44:57.696 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:44:57.695+0000 7fe5e9ca7700 1 -- 172.21.15.193:0/2345251169 wait complete. 2024-04-25T15:44:58.490 DEBUG:teuthology.orchestra.run.smithi193:osd.5> sudo journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T15:44:58.493 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-25T15:44:58.493 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T15:44:58.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:58 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:58.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:58 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:58.758 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:44:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:58 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:58 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:59.239 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:44:59 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[52847]: 2024-04-25T15:44:59.064+0000 7fc9ee46a540 -1 osd.5 0 log_to_monitors true 2024-04-25T15:44:59.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:59 smithi193 ceph-mon[33747]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:44:59.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:59 smithi193 ceph-mon[33747]: from='osd.5 [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T15:44:59.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:59 smithi193 ceph-mon[33747]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T15:44:59.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:59 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:44:59.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:44:59 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:00.040 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:59 smithi110 ceph-mon[26344]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:45:00.040 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:59 smithi110 ceph-mon[26344]: from='osd.5 [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T15:45:00.040 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:59 smithi110 ceph-mon[26344]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T15:45:00.040 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:59 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:00.040 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:44:59 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:00.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.392+0000 7f958fc42700 1 -- 172.21.15.110:0/4001545921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fce60 msgr2=0x7f95880fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.392+0000 7f958fc42700 1 --2- 172.21.15.110:0/4001545921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fce60 0x7f95880fd270 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9578004940 tx=0x7f9578009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.392+0000 7f958fc42700 1 -- 172.21.15.110:0/4001545921 shutdown_connections 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.392+0000 7f958fc42700 1 --2- 172.21.15.110:0/4001545921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95880fdfd0 0x7f95880fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.392+0000 7f958fc42700 1 --2- 172.21.15.110:0/4001545921 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fce60 0x7f95880fd270 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.392+0000 7f958fc42700 1 -- 172.21.15.110:0/4001545921 >> 172.21.15.110:0/4001545921 conn(0x7f95880f8540 msgr2=0x7f95880fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.392+0000 7f958fc42700 1 -- 172.21.15.110:0/4001545921 shutdown_connections 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.392+0000 7f958fc42700 1 -- 172.21.15.110:0/4001545921 wait complete. 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958fc42700 1 Processor -- start 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958fc42700 1 -- start start 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958fc42700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95880fce60 0x7f958810d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958fc42700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fdfd0 0x7f958810de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958fc42700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958810e500 con 0x7f95880fdfd0 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958fc42700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9588112ea0 con 0x7f95880fce60 2024-04-25T15:45:00.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958d1dd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fdfd0 0x7f958810de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958d1dd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fdfd0 0x7f958810de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43852/0 (socket says 172.21.15.110:43852) 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958d1dd700 1 -- 172.21.15.110:0/3444064350 learned_addr learned my addr 172.21.15.110:0/3444064350 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958d9de700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95880fce60 0x7f958810d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958d1dd700 1 -- 172.21.15.110:0/3444064350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95880fce60 msgr2=0x7f958810d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958d1dd700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95880fce60 0x7f958810d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.393+0000 7f958d1dd700 1 -- 172.21.15.110:0/3444064350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578005040 con 0x7f95880fdfd0 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.394+0000 7f958d1dd700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fdfd0 0x7f958810de70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f958400b790 tx=0x7f9584009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.394+0000 7f957e7fc700 1 -- 172.21.15.110:0/3444064350 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f958400f6f0 con 0x7f95880fdfd0 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.394+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9588113100 con 0x7f95880fdfd0 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.394+0000 7f957e7fc700 1 -- 172.21.15.110:0/3444064350 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f958400fd20 con 0x7f95880fdfd0 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.394+0000 7f957e7fc700 1 -- 172.21.15.110:0/3444064350 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95840103a0 con 0x7f95880fdfd0 2024-04-25T15:45:00.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.394+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9588113670 con 0x7f95880fdfd0 2024-04-25T15:45:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.395+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958804c710 con 0x7f95880fdfd0 2024-04-25T15:45:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.395+0000 7f957e7fc700 1 -- 172.21.15.110:0/3444064350 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9584011400 con 0x7f95880fdfd0 2024-04-25T15:45:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.396+0000 7f957e7fc700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f95740692a0 0x7f957406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.396+0000 7f958d9de700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f95740692a0 0x7f957406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:00.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.396+0000 7f957e7fc700 1 -- 172.21.15.110:0/3444064350 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4338+0+0 (secure 0 0 0) 0x7f9584086b20 con 0x7f95880fdfd0 2024-04-25T15:45:00.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.396+0000 7f958d9de700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f95740692a0 0x7f957406b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9578009fd0 tx=0x7f9578009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:00.401 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.400+0000 7f957e7fc700 1 -- 172.21.15.110:0/3444064350 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9584056d40 con 0x7f95880fdfd0 2024-04-25T15:45:00.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.586+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f9588105940 con 0x7f95880fdfd0 2024-04-25T15:45:00.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.587+0000 7f957e7fc700 1 -- 172.21.15.110:0/3444064350 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v37) v1 ==== 74+0+130 (secure 0 0 0) 0x7f95840568d0 con 0x7f95880fdfd0 2024-04-25T15:45:00.588 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:00.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f95740692a0 msgr2=0x7f957406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:00.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f95740692a0 0x7f957406b750 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9578009fd0 tx=0x7f9578009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:00.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fdfd0 msgr2=0x7f958810de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:00.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fdfd0 0x7f958810de70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f958400b790 tx=0x7f9584009250 comp rx=0 tx=0).stop 2024-04-25T15:45:00.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 shutdown_connections 2024-04-25T15:45:00.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f95740692a0 0x7f957406b750 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:00.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95880fce60 0x7f958810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:00.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 --2- 172.21.15.110:0/3444064350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95880fdfd0 0x7f958810de70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:00.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 >> 172.21.15.110:0/3444064350 conn(0x7f95880f8540 msgr2=0x7f95880f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:00.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 shutdown_connections 2024-04-25T15:45:00.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:00.589+0000 7f958fc42700 1 -- 172.21.15.110:0/3444064350 wait complete. 2024-04-25T15:45:00.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:00 smithi110 ceph-mon[26344]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T15:45:00.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:00 smithi110 ceph-mon[26344]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T15:45:00.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:00 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:45:00.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:00 smithi110 ceph-mon[26344]: from='osd.5 [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:45:00.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:00 smithi110 ceph-mon[26344]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:45:00.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:00 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3444064350' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T15:45:00.950 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:45:00 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[52847]: 2024-04-25T15:45:00.670+0000 7fc9e643f700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T15:45:00.950 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:45:00 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[52847]: 2024-04-25T15:45:00.677+0000 7fc9df236700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:45:00.950 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:00 smithi193 ceph-mon[33747]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T15:45:00.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:00 smithi193 ceph-mon[33747]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T15:45:00.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:00 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:45:00.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:00 smithi193 ceph-mon[33747]: from='osd.5 [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:45:00.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:00 smithi193 ceph-mon[33747]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:45:00.951 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:00 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3444064350' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T15:45:01.241 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":37,"num_osds":6,"num_up_osds":5,"osd_up_since":1714059877,"num_in_osds":6,"osd_in_since":1714059882,"num_remapped_pgs":0} 2024-04-25T15:45:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:01 smithi110 ceph-mon[26344]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:45:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:01 smithi110 ceph-mon[26344]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T15:45:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:01 smithi110 ceph-mon[26344]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T15:45:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:01 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:45:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:01 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:45:01.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:01 smithi193 ceph-mon[33747]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-04-25T15:45:01.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:01 smithi193 ceph-mon[33747]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]': finished 2024-04-25T15:45:01.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:01 smithi193 ceph-mon[33747]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T15:45:01.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:01 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:45:01.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:01 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:45:02.243 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T15:45:02.502 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:02.954 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:02 smithi110 ceph-mon[26344]: purged_snaps scrub starts 2024-04-25T15:45:02.954 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:02 smithi110 ceph-mon[26344]: purged_snaps scrub ok 2024-04-25T15:45:02.954 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:02 smithi110 ceph-mon[26344]: osd.5 [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] boot 2024-04-25T15:45:02.954 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:02 smithi110 ceph-mon[26344]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T15:45:02.954 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:02 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:45:02.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:02 smithi193 ceph-mon[33747]: purged_snaps scrub starts 2024-04-25T15:45:02.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:02 smithi193 ceph-mon[33747]: purged_snaps scrub ok 2024-04-25T15:45:02.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:02 smithi193 ceph-mon[33747]: osd.5 [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] boot 2024-04-25T15:45:02.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:02 smithi193 ceph-mon[33747]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T15:45:02.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:02 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:45:03.935 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:03.935 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T15:45:03.935 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: Detected new or changed devices on smithi193 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: Adjusting osd_memory_target on smithi193 to 5312M 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:45:03.936 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: Detected new or changed devices on smithi193 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: Adjusting osd_memory_target on smithi193 to 5312M 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:45:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:45:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:45:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:45:04.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 -- 172.21.15.110:0/1979258520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80662c0 msgr2=0x7fe2b80ff9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:04.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 --2- 172.21.15.110:0/1979258520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80662c0 0x7fe2b80ff9a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009ce0 tx=0x7fe2b0004d10 comp rx=0 tx=0).stop 2024-04-25T15:45:04.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 -- 172.21.15.110:0/1979258520 shutdown_connections 2024-04-25T15:45:04.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 --2- 172.21.15.110:0/1979258520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b80ffee0 0x7fe2b8102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:04.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 --2- 172.21.15.110:0/1979258520 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80662c0 0x7fe2b80ff9a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:04.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 -- 172.21.15.110:0/1979258520 >> 172.21.15.110:0/1979258520 conn(0x7fe2b80f7430 msgr2=0x7fe2b80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:04.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 -- 172.21.15.110:0/1979258520 shutdown_connections 2024-04-25T15:45:04.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 -- 172.21.15.110:0/1979258520 wait complete. 2024-04-25T15:45:04.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.149+0000 7fe2bf033700 1 Processor -- start 2024-04-25T15:45:04.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2bf033700 1 -- start start 2024-04-25T15:45:04.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2bf033700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b80662c0 0x7fe2b80fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:04.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2bf033700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80ffee0 0x7fe2b80fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:04.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2bf033700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80fbe70 con 0x7fe2b80ffee0 2024-04-25T15:45:04.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2bf033700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80fbfb0 con 0x7fe2b80662c0 2024-04-25T15:45:04.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2b7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80ffee0 0x7fe2b80fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:04.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2b7fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80ffee0 0x7fe2b80fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43876/0 (socket says 172.21.15.110:43876) 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2b7fff700 1 -- 172.21.15.110:0/2433936720 learned_addr learned my addr 172.21.15.110:0/2433936720 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2bcdcf700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b80662c0 0x7fe2b80fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2b7fff700 1 -- 172.21.15.110:0/2433936720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b80662c0 msgr2=0x7fe2b80fd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2b7fff700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b80662c0 0x7fe2b80fd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2b7fff700 1 -- 172.21.15.110:0/2433936720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b0009940 con 0x7fe2b80ffee0 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.150+0000 7fe2bcdcf700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b80662c0 0x7fe2b80fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.151+0000 7fe2b7fff700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80ffee0 0x7fe2b80fb8a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00b790 tx=0x7fe2ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.151+0000 7fe2b57fa700 1 -- 172.21.15.110:0/2433936720 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2ac00f770 con 0x7fe2b80ffee0 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.151+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b80fc290 con 0x7fe2b80ffee0 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.151+0000 7fe2b57fa700 1 -- 172.21.15.110:0/2433936720 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe2ac00fda0 con 0x7fe2b80ffee0 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.151+0000 7fe2b57fa700 1 -- 172.21.15.110:0/2433936720 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2ac00d310 con 0x7fe2b80ffee0 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.151+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b8115c40 con 0x7fe2b80ffee0 2024-04-25T15:45:04.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.152+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b805f5c0 con 0x7fe2b80ffee0 2024-04-25T15:45:04.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.152+0000 7fe2b57fa700 1 -- 172.21.15.110:0/2433936720 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe2ac010420 con 0x7fe2b80ffee0 2024-04-25T15:45:04.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.152+0000 7fe2b57fa700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe2a00692a0 0x7fe2a006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:04.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.153+0000 7fe2b57fa700 1 -- 172.21.15.110:0/2433936720 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe2ac086e80 con 0x7fe2b80ffee0 2024-04-25T15:45:04.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.153+0000 7fe2bcdcf700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe2a00692a0 0x7fe2a006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:04.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.153+0000 7fe2bcdcf700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe2a00692a0 0x7fe2a006b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0015040 tx=0x7fe2b0004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:04.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.157+0000 7fe2b57fa700 1 -- 172.21.15.110:0/2433936720 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe2ac0534f0 con 0x7fe2b80ffee0 2024-04-25T15:45:04.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.341+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe2b804bed0 con 0x7fe2b80ffee0 2024-04-25T15:45:04.342 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.342+0000 7fe2b57fa700 1 -- 172.21.15.110:0/2433936720 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7fe2ac056b10 con 0x7fe2b80ffee0 2024-04-25T15:45:04.343 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:04.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe2a00692a0 msgr2=0x7fe2a006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe2a00692a0 0x7fe2a006b750 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0015040 tx=0x7fe2b0004d10 comp rx=0 tx=0).stop 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80ffee0 msgr2=0x7fe2b80fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80ffee0 0x7fe2b80fb8a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac00b790 tx=0x7fe2ac009250 comp rx=0 tx=0).stop 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 shutdown_connections 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe2a00692a0 0x7fe2a006b750 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2b80662c0 0x7fe2b80fd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 --2- 172.21.15.110:0/2433936720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b80ffee0 0x7fe2b80fb8a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 >> 172.21.15.110:0/2433936720 conn(0x7fe2b80f7430 msgr2=0x7fe2b80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 shutdown_connections 2024-04-25T15:45:04.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:04.344+0000 7fe2bf033700 1 -- 172.21.15.110:0/2433936720 wait complete. 2024-04-25T15:45:04.983 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:04 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2433936720' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T15:45:04.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:04 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2433936720' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T15:45:05.006 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1714059901,"num_in_osds":6,"osd_in_since":1714059882,"num_remapped_pgs":0} 2024-04-25T15:45:05.007 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T15:45:05.257 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:05.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:05 smithi193 ceph-mon[33747]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:06.048 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:05 smithi110 ceph-mon[26344]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:06.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 -- 172.21.15.110:0/4198103122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c101d90 msgr2=0x7f3a3c102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:06.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 --2- 172.21.15.110:0/4198103122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c101d90 0x7f3a3c102160 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a34009ce0 tx=0x7f3a34004d10 comp rx=0 tx=0).stop 2024-04-25T15:45:06.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 -- 172.21.15.110:0/4198103122 shutdown_connections 2024-04-25T15:45:06.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 --2- 172.21.15.110:0/4198103122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a3c0fbd40 0x7f3a3c0fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:06.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 --2- 172.21.15.110:0/4198103122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c101d90 0x7f3a3c102160 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:06.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 -- 172.21.15.110:0/4198103122 >> 172.21.15.110:0/4198103122 conn(0x7f3a3c071d00 msgr2=0x7f3a3c072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:06.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 -- 172.21.15.110:0/4198103122 shutdown_connections 2024-04-25T15:45:06.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 -- 172.21.15.110:0/4198103122 wait complete. 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 Processor -- start 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.933+0000 7f3a42df2700 1 -- start start 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a42df2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c0fbd40 0x7f3a3c10ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a42df2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a3c101d90 0x7f3a3c10f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a42df2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c10f8a0 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a42df2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c10f9e0 con 0x7f3a3c101d90 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a40b8e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c0fbd40 0x7f3a3c10ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a40b8e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c0fbd40 0x7f3a3c10ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:43902/0 (socket says 172.21.15.110:43902) 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a40b8e700 1 -- 172.21.15.110:0/1803819517 learned_addr learned my addr 172.21.15.110:0/1803819517 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a40b8e700 1 -- 172.21.15.110:0/1803819517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a3c101d90 msgr2=0x7f3a3c10f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a40b8e700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a3c101d90 0x7f3a3c10f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a40b8e700 1 -- 172.21.15.110:0/1803819517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a34009940 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a40b8e700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c0fbd40 0x7f3a3c10ee20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400be90 tx=0x7f3a34004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:06.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a397fa700 1 -- 172.21.15.110:0/1803819517 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a3400e040 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a397fa700 1 -- 172.21.15.110:0/1803819517 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a34008ea0 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.934+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c117cd0 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.935+0000 7f3a397fa700 1 -- 172.21.15.110:0/1803819517 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a3401c940 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.935+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c118210 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.935+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c0fd430 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.936+0000 7f3a397fa700 1 -- 172.21.15.110:0/1803819517 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3a34012d00 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.936+0000 7f3a397fa700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3a280692a0 0x7f3a2806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.936+0000 7f3a397fa700 1 -- 172.21.15.110:0/1803819517 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3a34088a50 con 0x7f3a3c0fbd40 2024-04-25T15:45:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.936+0000 7f3a3bfff700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3a280692a0 0x7f3a2806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:06.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.937+0000 7f3a3bfff700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3a280692a0 0x7f3a2806b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c066000 tx=0x7f3a24007d40 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:06.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:06.941+0000 7f3a397fa700 1 -- 172.21.15.110:0/1803819517 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3a34055040 con 0x7f3a3c0fbd40 2024-04-25T15:45:07.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.128+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a3c05f5c0 con 0x7f3a3c0fbd40 2024-04-25T15:45:07.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.128+0000 7f3a397fa700 1 -- 172.21.15.110:0/1803819517 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11133 (secure 0 0 0) 0x7f3a34004e10 con 0x7f3a3c0fbd40 2024-04-25T15:45:07.129 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:07.130 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":40,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","created":"2024-04-25T15:39:31.460681+0000","modified":"2024-04-25T15:45:02.670519+0000","last_up_change":"2024-04-25T15:45:01.665598+0000","last_in_change":"2024-04-25T15:44:42.991633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T15:43:50.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"50a0a2ad-2c02-4108-85a8-3ef716509956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6803","nonce":3841288978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6805","nonce":3841288978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6809","nonce":3841288978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6807","nonce":3841288978}]},"public_addr":"172.21.15.110:6803/3841288978","cluster_addr":"172.21.15.110:6805/3841288978","heartbeat_back_addr":"172.21.15.110:6809/3841288978","heartbeat_front_addr":"172.21.15.110:6807/3841288978","state":["exists","up"]},{"osd":1,"uuid":"90511b8f-4444-44ff-b30c-789349b354f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6811","nonce":1447602330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6813","nonce":1447602330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6817","nonce":1447602330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6815","nonce":1447602330}]},"public_addr":"172.21.15.110:6811/1447602330","cluster_addr":"172.21.15.110:6813/1447602330","heartbeat_back_addr":"172.21.15.110:6817/1447602330","heartbeat_front_addr":"172.21.15.110:6815/1447602330","state":["exists","up"]},{"osd":2,"uuid":"c74ff664-988d-4d4b-a248-176d11f44010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6819","nonce":3448336555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6821","nonce":3448336555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6825","nonce":3448336555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6823","nonce":3448336555}]},"public_addr":"172.21.15.110:6819/3448336555","cluster_addr":"172.21.15.110:6821/3448336555","heartbeat_back_addr":"172.21.15.110:6825/3448336555","heartbeat_front_addr":"172.21.15.110:6823/3448336555","state":["exists","up"]},{"osd":3,"uuid":"dd48aeea-c692-4258-a8df-848bf440b6ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6801","nonce":1286847191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6803","nonce":1286847191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6807","nonce":1286847191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6805","nonce":1286847191}]},"public_addr":"172.21.15.193:6801/1286847191","cluster_addr":"172.21.15.193:6803/1286847191","heartbeat_back_addr":"172.21.15.193:6807/1286847191","heartbeat_front_addr":"172.21.15.193:6805/1286847191","state":["exists","up"]},{"osd":4,"uuid":"3b0a0e9a-5308-421d-8091-fac8f7649509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6809","nonce":190042456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6811","nonce":190042456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6815","nonce":190042456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6813","nonce":190042456}]},"public_addr":"172.21.15.193:6809/190042456","cluster_addr":"172.21.15.193:6811/190042456","heartbeat_back_addr":"172.21.15.193:6815/190042456","heartbeat_front_addr":"172.21.15.193:6813/190042456","state":["exists","up"]},{"osd":5,"uuid":"c6cf2f8b-8105-475c-8a79-ce9bbcdf8644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6817","nonce":1742452500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6819","nonce":1742452500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6823","nonce":1742452500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6821","nonce":1742452500}]},"public_addr":"172.21.15.193:6817/1742452500","cluster_addr":"172.21.15.193:6819/1742452500","heartbeat_back_addr":"172.21.15.193:6823/1742452500","heartbeat_front_addr":"172.21.15.193:6821/1742452500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:42:54.075221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:21.305107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:48.660473+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:11.495485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:35.758085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:45:00.087588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6801/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2245327398":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/920532485":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2468496169":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/1601321214":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/3933829355":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/1729346603":"2024-04-26T15:40:32.961057+0000","172.21.15.110:6800/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:6801/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/322959174":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/668880779":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6801/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/2068930777":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1116386591":"2024-04-26T15:40:32.961057+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3a280692a0 msgr2=0x7f3a2806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3a280692a0 0x7f3a2806b750 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c066000 tx=0x7f3a24007d40 comp rx=0 tx=0).stop 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c0fbd40 msgr2=0x7f3a3c10ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c0fbd40 0x7f3a3c10ee20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3a3400be90 tx=0x7f3a34004d10 comp rx=0 tx=0).stop 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 shutdown_connections 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3a3c0fbd40 0x7f3a3c10ee20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3a280692a0 0x7f3a2806b750 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 --2- 172.21.15.110:0/1803819517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a3c101d90 0x7f3a3c10f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 >> 172.21.15.110:0/1803819517 conn(0x7f3a3c071d00 msgr2=0x7f3a3c0fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 shutdown_connections 2024-04-25T15:45:07.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:07.131+0000 7f3a42df2700 1 -- 172.21.15.110:0/1803819517 wait complete. 2024-04-25T15:45:07.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:07 smithi110 ceph-mon[26344]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:07.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:07 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1803819517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:45:07.891 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T15:43:50.541493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-04-25T15:45:07.891 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T15:45:07.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:07 smithi193 ceph-mon[33747]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:07.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:07 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1803819517' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:45:08.143 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/4213246277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 msgr2=0x7f4e240fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 --2- 172.21.15.110:0/4213246277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 0x7f4e240fc1b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c004940 tx=0x7f4e0c009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/4213246277 shutdown_connections 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 --2- 172.21.15.110:0/4213246277 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 0x7f4e240fc1b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 --2- 172.21.15.110:0/4213246277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e24101d90 0x7f4e24102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/4213246277 >> 172.21.15.110:0/4213246277 conn(0x7f4e24071d00 msgr2=0x7f4e24072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/4213246277 shutdown_connections 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/4213246277 wait complete. 2024-04-25T15:45:09.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.767+0000 7f4e2a1d0700 1 Processor -- start 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e2a1d0700 1 -- start start 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e2a1d0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 0x7f4e24113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e2a1d0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e24101d90 0x7f4e241140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e2a1d0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e24114710 con 0x7f4e240fbd40 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e2a1d0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2410db00 con 0x7f4e24101d90 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e237fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 0x7f4e24113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e237fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 0x7f4e24113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57560/0 (socket says 172.21.15.110:57560) 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e237fe700 1 -- 172.21.15.110:0/2808802685 learned_addr learned my addr 172.21.15.110:0/2808802685 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e1affd700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e24101d90 0x7f4e241140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e237fe700 1 -- 172.21.15.110:0/2808802685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e24101d90 msgr2=0x7f4e241140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e237fe700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e24101d90 0x7f4e241140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e237fe700 1 -- 172.21.15.110:0/2808802685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c005040 con 0x7f4e240fbd40 2024-04-25T15:45:09.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e237fe700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 0x7f4e24113b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4e14002a50 tx=0x7f4e140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:09.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e20ff9700 1 -- 172.21.15.110:0/2808802685 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e14007cc0 con 0x7f4e240fbd40 2024-04-25T15:45:09.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e2410dde0 con 0x7f4e240fbd40 2024-04-25T15:45:09.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.768+0000 7f4e20ff9700 1 -- 172.21.15.110:0/2808802685 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e1400f040 con 0x7f4e240fbd40 2024-04-25T15:45:09.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.769+0000 7f4e20ff9700 1 -- 172.21.15.110:0/2808802685 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e14009640 con 0x7f4e240fbd40 2024-04-25T15:45:09.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.769+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e2410e3a0 con 0x7f4e240fbd40 2024-04-25T15:45:09.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.770+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e240fd430 con 0x7f4e240fbd40 2024-04-25T15:45:09.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.770+0000 7f4e20ff9700 1 -- 172.21.15.110:0/2808802685 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4e140097a0 con 0x7f4e240fbd40 2024-04-25T15:45:09.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.770+0000 7f4e20ff9700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4e100692a0 0x7f4e1006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:09.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.770+0000 7f4e20ff9700 1 -- 172.21.15.110:0/2808802685 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4e14086c90 con 0x7f4e240fbd40 2024-04-25T15:45:09.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.770+0000 7f4e1affd700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4e100692a0 0x7f4e1006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:09.773 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.771+0000 7f4e1affd700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4e100692a0 0x7f4e1006b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00b3b0 tx=0x7f4e0c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:09.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.774+0000 7f4e20ff9700 1 -- 172.21.15.110:0/2808802685 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4e14056d10 con 0x7f4e240fbd40 2024-04-25T15:45:09.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:09 smithi110 ceph-mon[26344]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:09.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.959+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e24108df0 con 0x7f4e240fbd40 2024-04-25T15:45:09.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.960+0000 7f4e20ff9700 1 -- 172.21.15.110:0/2808802685 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e140568a0 con 0x7f4e240fbd40 2024-04-25T15:45:09.961 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4e100692a0 msgr2=0x7f4e1006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4e100692a0 0x7f4e1006b750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c00b3b0 tx=0x7f4e0c009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 msgr2=0x7f4e24113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 0x7f4e24113b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4e14002a50 tx=0x7f4e140046b0 comp rx=0 tx=0).stop 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 shutdown_connections 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e240fbd40 0x7f4e24113b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4e100692a0 0x7f4e1006b750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 --2- 172.21.15.110:0/2808802685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e24101d90 0x7f4e241140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 >> 172.21.15.110:0/2808802685 conn(0x7f4e24071d00 msgr2=0x7f4e240fa820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 shutdown_connections 2024-04-25T15:45:09.963 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:09.962+0000 7f4e2a1d0700 1 -- 172.21.15.110:0/2808802685 wait complete. 2024-04-25T15:45:09.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:09 smithi193 ceph-mon[33747]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:10.605 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T15:45:10.606 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T15:45:10.859 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:10.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:10 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2808802685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:45:10.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:10 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:45:10.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:10 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2808802685' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:45:10.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:10 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:45:11.925 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:11 smithi110 ceph-mon[26344]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:11.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:11 smithi193 ceph-mon[33747]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:12.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 -- 172.21.15.110:0/118591507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 msgr2=0x7fab74102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:12.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 --2- 172.21.15.110:0/118591507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 0x7fab74102380 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab64004940 tx=0x7fab64009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:12.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 -- 172.21.15.110:0/118591507 shutdown_connections 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 --2- 172.21.15.110:0/118591507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 0x7fab74102380 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 --2- 172.21.15.110:0/118591507 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab74066a70 0x7fab740ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 -- 172.21.15.110:0/118591507 >> 172.21.15.110:0/118591507 conn(0x7fab740f7490 msgr2=0x7fab740f98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 -- 172.21.15.110:0/118591507 shutdown_connections 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 -- 172.21.15.110:0/118591507 wait complete. 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 Processor -- start 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 -- start start 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab74066a70 0x7fab7410fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 0x7fab74110040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab74110640 con 0x7fab740fffa0 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.502+0000 7fab7b6c4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab741107b0 con 0x7fab74066a70 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab78c5f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 0x7fab74110040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab78c5f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 0x7fab74110040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57574/0 (socket says 172.21.15.110:57574) 2024-04-25T15:45:12.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab78c5f700 1 -- 172.21.15.110:0/69081750 learned_addr learned my addr 172.21.15.110:0/69081750 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab79460700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab74066a70 0x7fab7410fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab78c5f700 1 -- 172.21.15.110:0/69081750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab74066a70 msgr2=0x7fab7410fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab78c5f700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab74066a70 0x7fab7410fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab78c5f700 1 -- 172.21.15.110:0/69081750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab64005040 con 0x7fab740fffa0 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab78c5f700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 0x7fab74110040 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab6400b3b0 tx=0x7fab64009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab69ffb700 1 -- 172.21.15.110:0/69081750 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6400e040 con 0x7fab740fffa0 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab69ffb700 1 -- 172.21.15.110:0/69081750 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab6400bbf0 con 0x7fab740fffa0 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab741151b0 con 0x7fab740fffa0 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab69ffb700 1 -- 172.21.15.110:0/69081750 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6401c630 con 0x7fab740fffa0 2024-04-25T15:45:12.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.503+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab74115770 con 0x7fab740fffa0 2024-04-25T15:45:12.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.504+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab7404c710 con 0x7fab740fffa0 2024-04-25T15:45:12.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.505+0000 7fab69ffb700 1 -- 172.21.15.110:0/69081750 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fab64026440 con 0x7fab740fffa0 2024-04-25T15:45:12.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.505+0000 7fab69ffb700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fab600692a0 0x7fab6006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:12.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.505+0000 7fab69ffb700 1 -- 172.21.15.110:0/69081750 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fab64087a90 con 0x7fab740fffa0 2024-04-25T15:45:12.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.505+0000 7fab79460700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fab600692a0 0x7fab6006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:12.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.506+0000 7fab79460700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fab600692a0 0x7fab6006b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab740667b0 tx=0x7fab7000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:12.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.510+0000 7fab69ffb700 1 -- 172.21.15.110:0/69081750 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fab64057ca0 con 0x7fab740fffa0 2024-04-25T15:45:12.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.774+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fab741058d0 con 0x7fab740fffa0 2024-04-25T15:45:12.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.778+0000 7fab69ffb700 1 -- 172.21.15.110:0/69081750 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v14) v1 ==== 170+0+59 (secure 0 0 0) 0x7fab64057830 con 0x7fab740fffa0 2024-04-25T15:45:12.779 INFO:teuthology.orchestra.run.smithi110.stdout:[client.0] 2024-04-25T15:45:12.779 INFO:teuthology.orchestra.run.smithi110.stdout: key = AQCIeipmRtA/LhAAj7kblRDmaMz+Lt4a3fwZnQ== 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fab600692a0 msgr2=0x7fab6006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fab600692a0 0x7fab6006b750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fab740667b0 tx=0x7fab7000b040 comp rx=0 tx=0).stop 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 msgr2=0x7fab74110040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 0x7fab74110040 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab6400b3b0 tx=0x7fab64009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 shutdown_connections 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fab600692a0 0x7fab6006b750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab74066a70 0x7fab7410fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 --2- 172.21.15.110:0/69081750 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab740fffa0 0x7fab74110040 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 >> 172.21.15.110:0/69081750 conn(0x7fab740f7490 msgr2=0x7fab74100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:12.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 shutdown_connections 2024-04-25T15:45:12.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:12.780+0000 7fab7b6c4700 1 -- 172.21.15.110:0/69081750 wait complete. 2024-04-25T15:45:13.499 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:45:13.499 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T15:45:13.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T15:45:13.536 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T15:45:13.801 INFO:teuthology.orchestra.run.smithi193.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi193/config 2024-04-25T15:45:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:13 smithi110 ceph-mon[26344]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:13 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/69081750' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:45:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:13 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/69081750' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T15:45:13.916 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:13 smithi193 ceph-mon[33747]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:13.916 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:13 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/69081750' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:45:13.916 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:13 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/69081750' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 -- 172.21.15.193:0/617281955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 msgr2=0x7fc81c0fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 --2- 172.21.15.193:0/617281955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 0x7fc81c0fd410 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc804004940 tx=0x7fc804009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 -- 172.21.15.193:0/617281955 shutdown_connections 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 --2- 172.21.15.193:0/617281955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 0x7fc81c0fd410 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 --2- 172.21.15.193:0/617281955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc81c0fbcf0 0x7fc81c0fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 -- 172.21.15.193:0/617281955 >> 172.21.15.193:0/617281955 conn(0x7fc81c0f7430 msgr2=0x7fc81c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 -- 172.21.15.193:0/617281955 shutdown_connections 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 -- 172.21.15.193:0/617281955 wait complete. 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 Processor -- start 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.168+0000 7fc820ecd700 1 -- start start 2024-04-25T15:45:15.171 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc820ecd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc81c0fbcf0 0x7fc81c10c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc820ecd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 0x7fc81c10cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc820ecd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c10d340 con 0x7fc81c0fbcf0 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc820ecd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c10d4b0 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc811d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 0x7fc81c10cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc811d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 0x7fc81c10cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55236/0 (socket says 172.21.15.193:55236) 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc811d9b700 1 -- 172.21.15.193:0/4255763615 learned_addr learned my addr 172.21.15.193:0/4255763615 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc81a59c700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc81c0fbcf0 0x7fc81c10c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc811d9b700 1 -- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc81c0fbcf0 msgr2=0x7fc81c10c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc811d9b700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc81c0fbcf0 0x7fc81c10c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc811d9b700 1 -- 172.21.15.193:0/4255763615 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc804005040 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc81a59c700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc81c0fbcf0 0x7fc81c10c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc811d9b700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 0x7fc81c10cd40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc80400b3b0 tx=0x7fc804009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc8137fe700 1 -- 172.21.15.193:0/4255763615 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80400e040 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.172 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c111eb0 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc8137fe700 1 -- 172.21.15.193:0/4255763615 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc80400bcc0 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.169+0000 7fc8137fe700 1 -- 172.21.15.193:0/4255763615 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80401c630 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.173 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.170+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c112440 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.174 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.171+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c04c710 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.174 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.171+0000 7fc8137fe700 1 -- 172.21.15.193:0/4255763615 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fc804013420 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.174 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.171+0000 7fc8137fe700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fc8080692a0 0x7fc80806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:15.174 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.171+0000 7fc8137fe700 1 -- 172.21.15.193:0/4255763615 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fc804087b90 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.174 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.171+0000 7fc81a59c700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fc8080692a0 0x7fc80806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:15.174 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.172+0000 7fc81a59c700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fc8080692a0 0x7fc80806b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009fd0 tx=0x7fc80c009ee0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:15.177 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.176+0000 7fc8137fe700 1 -- 172.21.15.193:0/4255763615 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fc804057c90 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.436 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.435+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fc81c1047e0 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.440 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.440+0000 7fc8137fe700 1 -- 172.21.15.193:0/4255763615 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fc804057820 con 0x7fc81c0fcfa0 2024-04-25T15:45:15.440 INFO:teuthology.orchestra.run.smithi193.stdout:[client.1] 2024-04-25T15:45:15.441 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQCLeipmo5IOGhAAFEa0QDCFHCFQGY6UBEBdww== 2024-04-25T15:45:15.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fc8080692a0 msgr2=0x7fc80806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:15.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fc8080692a0 0x7fc80806b750 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc80c009fd0 tx=0x7fc80c009ee0 comp rx=0 tx=0).stop 2024-04-25T15:45:15.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 msgr2=0x7fc81c10cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:15.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 0x7fc81c10cd40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc80400b3b0 tx=0x7fc804009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:15.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 shutdown_connections 2024-04-25T15:45:15.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc81c0fbcf0 0x7fc81c10c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:15.443 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fc8080692a0 0x7fc80806b750 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:15.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 --2- 172.21.15.193:0/4255763615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c0fcfa0 0x7fc81c10cd40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:15.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 >> 172.21.15.193:0/4255763615 conn(0x7fc81c0f7430 msgr2=0x7fc81c0f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:15.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 shutdown_connections 2024-04-25T15:45:15.444 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:45:15.442+0000 7fc820ecd700 1 -- 172.21.15.193:0/4255763615 wait complete. 2024-04-25T15:45:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:15 smithi193 ceph-mon[33747]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:15 smithi193 ceph-mon[33747]: from='client.? 172.21.15.193:0/4255763615' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:45:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:15 smithi193 ceph-mon[33747]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:45:15.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:15 smithi193 ceph-mon[33747]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T15:45:16.016 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:45:16.016 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-25T15:45:16.016 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T15:45:16.051 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T15:45:16.051 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T15:45:16.051 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T15:45:16.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:15 smithi110 ceph-mon[26344]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:16.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:15 smithi110 ceph-mon[26344]: from='client.? 172.21.15.193:0/4255763615' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:45:16.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:15 smithi110 ceph-mon[26344]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T15:45:16.077 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:15 smithi110 ceph-mon[26344]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T15:45:16.313 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:17.759 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:17 smithi110 ceph-mon[26344]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:17.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.970+0000 7f84409ab700 1 -- 172.21.15.110:0/3349116976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c105970 msgr2=0x7f843c0fefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:17.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.970+0000 7f84409ab700 1 --2- 172.21.15.110:0/3349116976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c105970 0x7f843c0fefe0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8430004940 tx=0x7f8430009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:17.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 -- 172.21.15.110:0/3349116976 shutdown_connections 2024-04-25T15:45:17.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 --2- 172.21.15.110:0/3349116976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c105970 0x7f843c0fefe0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:17.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 --2- 172.21.15.110:0/3349116976 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c104fd0 0x7f843c1053a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:17.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 -- 172.21.15.110:0/3349116976 >> 172.21.15.110:0/3349116976 conn(0x7f843c0fab80 msgr2=0x7f843c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 -- 172.21.15.110:0/3349116976 shutdown_connections 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 -- 172.21.15.110:0/3349116976 wait complete. 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 Processor -- start 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 -- start start 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c104fd0 0x7f843c115b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c105970 0x7f843c1160d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c1167b0 con 0x7f843c104fd0 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.971+0000 7f84409ab700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c10fd00 con 0x7f843c105970 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f843a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c104fd0 0x7f843c115b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f843a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c104fd0 0x7f843c115b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57600/0 (socket says 172.21.15.110:57600) 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f843a59c700 1 -- 172.21.15.110:0/3119261655 learned_addr learned my addr 172.21.15.110:0/3119261655 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f8439d9b700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c105970 0x7f843c1160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f843a59c700 1 -- 172.21.15.110:0/3119261655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c105970 msgr2=0x7f843c1160d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:17.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f843a59c700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c105970 0x7f843c1160d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:17.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f843a59c700 1 -- 172.21.15.110:0/3119261655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8430005040 con 0x7f843c104fd0 2024-04-25T15:45:17.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f843a59c700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c104fd0 0x7f843c115b90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b790 tx=0x7f842c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:17.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f842affd700 1 -- 172.21.15.110:0/3119261655 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842c00f770 con 0x7f843c104fd0 2024-04-25T15:45:17.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c10ff80 con 0x7f843c104fd0 2024-04-25T15:45:17.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f842affd700 1 -- 172.21.15.110:0/3119261655 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f842c00fda0 con 0x7f843c104fd0 2024-04-25T15:45:17.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f842affd700 1 -- 172.21.15.110:0/3119261655 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842c00d310 con 0x7f843c104fd0 2024-04-25T15:45:17.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.972+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c110540 con 0x7f843c104fd0 2024-04-25T15:45:17.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.973+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c100720 con 0x7f843c104fd0 2024-04-25T15:45:17.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.974+0000 7f842affd700 1 -- 172.21.15.110:0/3119261655 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f842c010420 con 0x7f843c104fd0 2024-04-25T15:45:17.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.974+0000 7f842affd700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f84240692a0 0x7f842406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:17.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.974+0000 7f842affd700 1 -- 172.21.15.110:0/3119261655 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f842c086e70 con 0x7f843c104fd0 2024-04-25T15:45:17.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.974+0000 7f8439d9b700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f84240692a0 0x7f842406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:17.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.974+0000 7f8439d9b700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f84240692a0 0x7f842406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f843000b3b0 tx=0x7f8430009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:17.979 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:17.978+0000 7f842affd700 1 -- 172.21.15.110:0/3119261655 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f842c056f70 con 0x7f843c104fd0 2024-04-25T15:45:17.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:17 smithi193 ceph-mon[33747]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:18.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.218+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f843c04c710 con 0x7f843c104fd0 2024-04-25T15:45:18.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.221+0000 7f842affd700 1 -- 172.21.15.110:0/3119261655 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+167188 (secure 0 0 0) 0x7f843c04c710 con 0x7f843c104fd0 2024-04-25T15:45:18.224 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:18.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f84240692a0 msgr2=0x7f842406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:18.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f84240692a0 0x7f842406b750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f843000b3b0 tx=0x7f8430009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c104fd0 msgr2=0x7f843c115b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c104fd0 0x7f843c115b90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f842c00b790 tx=0x7f842c009250 comp rx=0 tx=0).stop 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 shutdown_connections 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f843c104fd0 0x7f843c115b90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f84240692a0 0x7f842406b750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 --2- 172.21.15.110:0/3119261655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c105970 0x7f843c1160d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 >> 172.21.15.110:0/3119261655 conn(0x7f843c0fab80 msgr2=0x7f843c0fba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 shutdown_connections 2024-04-25T15:45:18.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:18.223+0000 7f84409ab700 1 -- 172.21.15.110:0/3119261655 wait complete. 2024-04-25T15:45:18.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:18 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3119261655' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T15:45:18.885 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":19,"flags":0,"active_gid":14210,"active_name":"smithi110.lvgbkf","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":1619647096},{"type":"v1","addr":"172.21.15.110:6801","nonce":1619647096}]},"active_addr":"172.21.15.110:6801/1619647096","active_change":"2024-04-25T15:41:40.227762+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":14230,"name":"smithi193.mgoteh","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.110:8443/","prometheus":"http://172.21.15.110:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3611960738}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":4219932763}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":3260352407}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.110:0","nonce":1043733842}]}]} 2024-04-25T15:45:18.889 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T15:45:18.889 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T15:45:18.889 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T15:45:18.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:18 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3119261655' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T15:45:19.176 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:19 smithi110 ceph-mon[26344]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:19 smithi193 ceph-mon[33747]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:20.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.820+0000 7fbe63a82700 1 -- 172.21.15.110:0/2679483512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 msgr2=0x7fbe5c0fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:20.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.820+0000 7fbe63a82700 1 --2- 172.21.15.110:0/2679483512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 0x7fbe5c0fcdf0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c004940 tx=0x7fbe4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:20.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 -- 172.21.15.110:0/2679483512 shutdown_connections 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 --2- 172.21.15.110:0/2679483512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 0x7fbe5c0fcdf0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 --2- 172.21.15.110:0/2679483512 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe5c102de0 0x7fbe5c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 -- 172.21.15.110:0/2679483512 >> 172.21.15.110:0/2679483512 conn(0x7fbe5c0f8990 msgr2=0x7fbe5c0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 -- 172.21.15.110:0/2679483512 shutdown_connections 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 -- 172.21.15.110:0/2679483512 wait complete. 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 Processor -- start 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 -- start start 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe5c102de0 0x7fbe5c115b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 0x7fbe5c1160c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c1167a0 con 0x7fbe5c103780 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe63a82700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c10fcf0 con 0x7fbe5c102de0 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe6101d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 0x7fbe5c1160c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe6101d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 0x7fbe5c1160c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46748/0 (socket says 172.21.15.110:46748) 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.821+0000 7fbe6101d700 1 -- 172.21.15.110:0/3262580350 learned_addr learned my addr 172.21.15.110:0/3262580350 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe6101d700 1 -- 172.21.15.110:0/3262580350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe5c102de0 msgr2=0x7fbe5c115b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:20.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe6181e700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe5c102de0 0x7fbe5c115b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe6101d700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe5c102de0 0x7fbe5c115b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe6101d700 1 -- 172.21.15.110:0/3262580350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4c005040 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe6101d700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 0x7fbe5c1160c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009100 tx=0x7fbe4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe527fc700 1 -- 172.21.15.110:0/3262580350 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4c00e040 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe527fc700 1 -- 172.21.15.110:0/3262580350 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe4c013420 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe5c10ff10 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe527fc700 1 -- 172.21.15.110:0/3262580350 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe4c01c610 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.822+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe5c1104d0 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.823+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe5c1072e0 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.823+0000 7fbe527fc700 1 -- 172.21.15.110:0/3262580350 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fbe4c013590 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.824+0000 7fbe527fc700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbe480692a0 0x7fbe4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.824+0000 7fbe6181e700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbe480692a0 0x7fbe4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.824+0000 7fbe527fc700 1 -- 172.21.15.110:0/3262580350 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fbe4c0880e0 con 0x7fbe5c103780 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.824+0000 7fbe6181e700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbe480692a0 0x7fbe4806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800a900 tx=0x7fbe58008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:20.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:20.828+0000 7fbe527fc700 1 -- 172.21.15.110:0/3262580350 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fbe4c0581e0 con 0x7fbe5c103780 2024-04-25T15:45:21.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.015+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe5c111230 con 0x7fbe5c103780 2024-04-25T15:45:21.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.016+0000 7fbe527fc700 1 -- 172.21.15.110:0/3262580350 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11133 (secure 0 0 0) 0x7fbe4c057d70 con 0x7fbe5c103780 2024-04-25T15:45:21.017 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:21.017 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":40,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","created":"2024-04-25T15:39:31.460681+0000","modified":"2024-04-25T15:45:02.670519+0000","last_up_change":"2024-04-25T15:45:01.665598+0000","last_in_change":"2024-04-25T15:44:42.991633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T15:43:50.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"50a0a2ad-2c02-4108-85a8-3ef716509956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6803","nonce":3841288978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6805","nonce":3841288978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6809","nonce":3841288978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6807","nonce":3841288978}]},"public_addr":"172.21.15.110:6803/3841288978","cluster_addr":"172.21.15.110:6805/3841288978","heartbeat_back_addr":"172.21.15.110:6809/3841288978","heartbeat_front_addr":"172.21.15.110:6807/3841288978","state":["exists","up"]},{"osd":1,"uuid":"90511b8f-4444-44ff-b30c-789349b354f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6811","nonce":1447602330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6813","nonce":1447602330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6817","nonce":1447602330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6815","nonce":1447602330}]},"public_addr":"172.21.15.110:6811/1447602330","cluster_addr":"172.21.15.110:6813/1447602330","heartbeat_back_addr":"172.21.15.110:6817/1447602330","heartbeat_front_addr":"172.21.15.110:6815/1447602330","state":["exists","up"]},{"osd":2,"uuid":"c74ff664-988d-4d4b-a248-176d11f44010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6819","nonce":3448336555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6821","nonce":3448336555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6825","nonce":3448336555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6823","nonce":3448336555}]},"public_addr":"172.21.15.110:6819/3448336555","cluster_addr":"172.21.15.110:6821/3448336555","heartbeat_back_addr":"172.21.15.110:6825/3448336555","heartbeat_front_addr":"172.21.15.110:6823/3448336555","state":["exists","up"]},{"osd":3,"uuid":"dd48aeea-c692-4258-a8df-848bf440b6ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6801","nonce":1286847191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6803","nonce":1286847191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6807","nonce":1286847191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6805","nonce":1286847191}]},"public_addr":"172.21.15.193:6801/1286847191","cluster_addr":"172.21.15.193:6803/1286847191","heartbeat_back_addr":"172.21.15.193:6807/1286847191","heartbeat_front_addr":"172.21.15.193:6805/1286847191","state":["exists","up"]},{"osd":4,"uuid":"3b0a0e9a-5308-421d-8091-fac8f7649509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6809","nonce":190042456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6811","nonce":190042456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6815","nonce":190042456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6813","nonce":190042456}]},"public_addr":"172.21.15.193:6809/190042456","cluster_addr":"172.21.15.193:6811/190042456","heartbeat_back_addr":"172.21.15.193:6815/190042456","heartbeat_front_addr":"172.21.15.193:6813/190042456","state":["exists","up"]},{"osd":5,"uuid":"c6cf2f8b-8105-475c-8a79-ce9bbcdf8644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6817","nonce":1742452500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6819","nonce":1742452500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6823","nonce":1742452500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6821","nonce":1742452500}]},"public_addr":"172.21.15.193:6817/1742452500","cluster_addr":"172.21.15.193:6819/1742452500","heartbeat_back_addr":"172.21.15.193:6823/1742452500","heartbeat_front_addr":"172.21.15.193:6821/1742452500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:42:54.075221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:21.305107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:48.660473+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:11.495485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:35.758085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:45:00.087588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6801/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2245327398":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/920532485":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2468496169":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/1601321214":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/3933829355":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/1729346603":"2024-04-26T15:40:32.961057+0000","172.21.15.110:6800/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:6801/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/322959174":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/668880779":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6801/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/2068930777":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1116386591":"2024-04-26T15:40:32.961057+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbe480692a0 msgr2=0x7fbe4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbe480692a0 0x7fbe4806b750 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800a900 tx=0x7fbe58008040 comp rx=0 tx=0).stop 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 msgr2=0x7fbe5c1160c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 0x7fbe5c1160c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009100 tx=0x7fbe4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 shutdown_connections 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fbe480692a0 0x7fbe4806b750 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe5c102de0 0x7fbe5c115b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 --2- 172.21.15.110:0/3262580350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe5c103780 0x7fbe5c1160c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 >> 172.21.15.110:0/3262580350 conn(0x7fbe5c0f8990 msgr2=0x7fbe5c0fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 shutdown_connections 2024-04-25T15:45:21.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:21.018+0000 7fbe63a82700 1 -- 172.21.15.110:0/3262580350 wait complete. 2024-04-25T15:45:21.687 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T15:45:21.687 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T15:45:21.940 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:21 smithi193 ceph-mon[33747]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:21 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3262580350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:45:22.061 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:21 smithi110 ceph-mon[26344]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:22.061 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:21 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3262580350' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:45:23.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.516+0000 7f4b35388700 1 -- 172.21.15.110:0/253455022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 msgr2=0x7f4b300fbd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.516+0000 7f4b35388700 1 --2- 172.21.15.110:0/253455022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 0x7f4b300fbd20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4b24004940 tx=0x7f4b24009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.516+0000 7f4b35388700 1 -- 172.21.15.110:0/253455022 shutdown_connections 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.516+0000 7f4b35388700 1 --2- 172.21.15.110:0/253455022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 0x7f4b300fbd20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.516+0000 7f4b35388700 1 --2- 172.21.15.110:0/253455022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b30101d10 0x7f4b301020e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.516+0000 7f4b35388700 1 -- 172.21.15.110:0/253455022 >> 172.21.15.110:0/253455022 conn(0x7f4b300f78e0 msgr2=0x7f4b300f9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.516+0000 7f4b35388700 1 -- 172.21.15.110:0/253455022 shutdown_connections 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.516+0000 7f4b35388700 1 -- 172.21.15.110:0/253455022 wait complete. 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b35388700 1 Processor -- start 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b35388700 1 -- start start 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b35388700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b30101d10 0x7f4b300fd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b35388700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 0x7f4b300fded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b35388700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300ff8e0 con 0x7f4b301026b0 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b35388700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b300fe410 con 0x7f4b30101d10 2024-04-25T15:45:23.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b2e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 0x7f4b300fded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b2e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 0x7f4b300fded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46764/0 (socket says 172.21.15.110:46764) 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b2e7fc700 1 -- 172.21.15.110:0/3731016721 learned_addr learned my addr 172.21.15.110:0/3731016721 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b2effd700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b30101d10 0x7f4b300fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b2e7fc700 1 -- 172.21.15.110:0/3731016721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b30101d10 msgr2=0x7f4b300fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b2e7fc700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b30101d10 0x7f4b300fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b2e7fc700 1 -- 172.21.15.110:0/3731016721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b24005040 con 0x7f4b301026b0 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b2e7fc700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 0x7f4b300fded0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009100 tx=0x7f4b24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.517+0000 7f4b177fe700 1 -- 172.21.15.110:0/3731016721 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2400e040 con 0x7f4b301026b0 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.518+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b300fe690 con 0x7f4b301026b0 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.518+0000 7f4b177fe700 1 -- 172.21.15.110:0/3731016721 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b24009af0 con 0x7f4b301026b0 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.518+0000 7f4b177fe700 1 -- 172.21.15.110:0/3731016721 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b2401c3f0 con 0x7f4b301026b0 2024-04-25T15:45:23.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.518+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b3011a0b0 con 0x7f4b301026b0 2024-04-25T15:45:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.519+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b300fd460 con 0x7f4b301026b0 2024-04-25T15:45:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.519+0000 7f4b177fe700 1 -- 172.21.15.110:0/3731016721 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4b2400baf0 con 0x7f4b301026b0 2024-04-25T15:45:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.519+0000 7f4b177fe700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4b180692a0 0x7f4b1806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.519+0000 7f4b177fe700 1 -- 172.21.15.110:0/3731016721 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4b24088100 con 0x7f4b301026b0 2024-04-25T15:45:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.519+0000 7f4b2effd700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4b180692a0 0x7f4b1806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:23.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.520+0000 7f4b2effd700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4b180692a0 0x7f4b1806b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4b28007970 tx=0x7f4b28009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:23.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.524+0000 7f4b177fe700 1 -- 172.21.15.110:0/3731016721 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4b24054770 con 0x7f4b301026b0 2024-04-25T15:45:23.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.710+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b3011a2f0 con 0x7f4b301026b0 2024-04-25T15:45:23.711 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.711+0000 7f4b177fe700 1 -- 172.21.15.110:0/3731016721 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11133 (secure 0 0 0) 0x7f4b24057d90 con 0x7f4b301026b0 2024-04-25T15:45:23.712 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:23.712 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":40,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","created":"2024-04-25T15:39:31.460681+0000","modified":"2024-04-25T15:45:02.670519+0000","last_up_change":"2024-04-25T15:45:01.665598+0000","last_in_change":"2024-04-25T15:44:42.991633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T15:43:50.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"50a0a2ad-2c02-4108-85a8-3ef716509956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6803","nonce":3841288978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6805","nonce":3841288978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6809","nonce":3841288978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6807","nonce":3841288978}]},"public_addr":"172.21.15.110:6803/3841288978","cluster_addr":"172.21.15.110:6805/3841288978","heartbeat_back_addr":"172.21.15.110:6809/3841288978","heartbeat_front_addr":"172.21.15.110:6807/3841288978","state":["exists","up"]},{"osd":1,"uuid":"90511b8f-4444-44ff-b30c-789349b354f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6811","nonce":1447602330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6813","nonce":1447602330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6817","nonce":1447602330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6815","nonce":1447602330}]},"public_addr":"172.21.15.110:6811/1447602330","cluster_addr":"172.21.15.110:6813/1447602330","heartbeat_back_addr":"172.21.15.110:6817/1447602330","heartbeat_front_addr":"172.21.15.110:6815/1447602330","state":["exists","up"]},{"osd":2,"uuid":"c74ff664-988d-4d4b-a248-176d11f44010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6819","nonce":3448336555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6821","nonce":3448336555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6825","nonce":3448336555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6823","nonce":3448336555}]},"public_addr":"172.21.15.110:6819/3448336555","cluster_addr":"172.21.15.110:6821/3448336555","heartbeat_back_addr":"172.21.15.110:6825/3448336555","heartbeat_front_addr":"172.21.15.110:6823/3448336555","state":["exists","up"]},{"osd":3,"uuid":"dd48aeea-c692-4258-a8df-848bf440b6ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6801","nonce":1286847191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6803","nonce":1286847191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6807","nonce":1286847191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6805","nonce":1286847191}]},"public_addr":"172.21.15.193:6801/1286847191","cluster_addr":"172.21.15.193:6803/1286847191","heartbeat_back_addr":"172.21.15.193:6807/1286847191","heartbeat_front_addr":"172.21.15.193:6805/1286847191","state":["exists","up"]},{"osd":4,"uuid":"3b0a0e9a-5308-421d-8091-fac8f7649509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6809","nonce":190042456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6811","nonce":190042456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6815","nonce":190042456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6813","nonce":190042456}]},"public_addr":"172.21.15.193:6809/190042456","cluster_addr":"172.21.15.193:6811/190042456","heartbeat_back_addr":"172.21.15.193:6815/190042456","heartbeat_front_addr":"172.21.15.193:6813/190042456","state":["exists","up"]},{"osd":5,"uuid":"c6cf2f8b-8105-475c-8a79-ce9bbcdf8644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6817","nonce":1742452500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6819","nonce":1742452500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6823","nonce":1742452500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6821","nonce":1742452500}]},"public_addr":"172.21.15.193:6817/1742452500","cluster_addr":"172.21.15.193:6819/1742452500","heartbeat_back_addr":"172.21.15.193:6823/1742452500","heartbeat_front_addr":"172.21.15.193:6821/1742452500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:42:54.075221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:21.305107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:48.660473+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:11.495485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:35.758085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:45:00.087588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6801/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2245327398":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/920532485":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2468496169":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/1601321214":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/3933829355":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/1729346603":"2024-04-26T15:40:32.961057+0000","172.21.15.110:6800/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:6801/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/322959174":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/668880779":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6801/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/2068930777":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1116386591":"2024-04-26T15:40:32.961057+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:45:23.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4b180692a0 msgr2=0x7f4b1806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:23.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4b180692a0 0x7f4b1806b750 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4b28007970 tx=0x7f4b28009da0 comp rx=0 tx=0).stop 2024-04-25T15:45:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 msgr2=0x7f4b300fded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 0x7f4b300fded0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009100 tx=0x7f4b24009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 shutdown_connections 2024-04-25T15:45:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4b180692a0 0x7f4b1806b750 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b30101d10 0x7f4b300fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 --2- 172.21.15.110:0/3731016721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b301026b0 0x7f4b300fded0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 >> 172.21.15.110:0/3731016721 conn(0x7f4b300f78e0 msgr2=0x7f4b300f8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:23.715 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 shutdown_connections 2024-04-25T15:45:23.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:23.713+0000 7f4b35388700 1 -- 172.21.15.110:0/3731016721 wait complete. 2024-04-25T15:45:23.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:23 smithi193 ceph-mon[33747]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:23 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3731016721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:45:24.064 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:23 smithi110 ceph-mon[26344]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:24.064 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:23 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3731016721' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:45:24.367 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T15:45:24.368 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T15:45:24.369 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T15:45:24.369 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T15:45:24.370 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T15:45:24.370 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T15:45:24.735 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:25.815 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:25 smithi110 ceph-mon[26344]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:25.815 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:45:25.873 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:25.897 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:25.976 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:25 smithi193 ceph-mon[33747]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:25.976 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:45:26.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.558+0000 7f7fff94c700 1 -- 172.21.15.110:0/4224964270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 msgr2=0x7f7ff8102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:26.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.558+0000 7f7fff94c700 1 --2- 172.21.15.110:0/4224964270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 0x7f7ff8102160 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4004940 tx=0x7f7ff4009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.558+0000 7f7fff94c700 1 -- 172.21.15.110:0/4224964270 shutdown_connections 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.558+0000 7f7fff94c700 1 --2- 172.21.15.110:0/4224964270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff80fbd40 0x7f7ff80fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.558+0000 7f7fff94c700 1 --2- 172.21.15.110:0/4224964270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 0x7f7ff8102160 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.558+0000 7f7fff94c700 1 -- 172.21.15.110:0/4224964270 >> 172.21.15.110:0/4224964270 conn(0x7f7ff8071d00 msgr2=0x7f7ff8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7fff94c700 1 -- 172.21.15.110:0/4224964270 shutdown_connections 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7fff94c700 1 -- 172.21.15.110:0/4224964270 wait complete. 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7fff94c700 1 Processor -- start 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7fff94c700 1 -- start start 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7fff94c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff80fbd40 0x7f7ff8113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7fff94c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 0x7f7ff81140b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7fff94c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8114700 con 0x7f7ff8101d90 2024-04-25T15:45:26.562 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7fff94c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff810daf0 con 0x7f7ff80fbd40 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7ffcee7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 0x7f7ff81140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7ffcee7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 0x7f7ff81140b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46778/0 (socket says 172.21.15.110:46778) 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7ffcee7700 1 -- 172.21.15.110:0/3067872571 learned_addr learned my addr 172.21.15.110:0/3067872571 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.559+0000 7f7ffd6e8700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff80fbd40 0x7f7ff8113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7ffcee7700 1 -- 172.21.15.110:0/3067872571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff80fbd40 msgr2=0x7f7ff8113b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7ffcee7700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff80fbd40 0x7f7ff8113b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7ffcee7700 1 -- 172.21.15.110:0/3067872571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4005040 con 0x7f7ff8101d90 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7ffcee7700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 0x7f7ff81140b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8002a50 tx=0x7f7fe80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fe8007cc0 con 0x7f7ff8101d90 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fe800f040 con 0x7f7ff8101d90 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff810ddd0 con 0x7f7ff8101d90 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fe800da30 con 0x7f7ff8101d90 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.560+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff810e390 con 0x7f7ff8101d90 2024-04-25T15:45:26.563 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.561+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7ff810e8b0 con 0x7f7ff8101d90 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.561+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7fe800db90 con 0x7f7ff8101d90 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7fedffb700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7fe40692a0 0x7f7fe406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f7fe8086c10 con 0x7f7ff8101d90 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7ffd6e8700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7fe40692a0 0x7f7fe406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7fedffb700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] conn(0x7f7fe406ef70 0x7f7fe4071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 --> [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7fe4071a30 con 0x7f7fe406ef70 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fe808c070 con 0x7f7ff8101d90 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7ffdee9700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] conn(0x7f7fe406ef70 0x7f7fe4071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7ffd6e8700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7fe40692a0 0x7f7fe406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4004890 tx=0x7f7ff4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.562+0000 7f7ffdee9700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] conn(0x7f7fe406ef70 0x7f7fe4071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:26.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.563+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 <== osd.0 v2:172.21.15.110:6802/3841288978 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f7fe4071a30 con 0x7f7fe406ef70 2024-04-25T15:45:26.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.578+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 --> [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7ff8065fb0 con 0x7f7fe406ef70 2024-04-25T15:45:26.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.578+0000 7f7fedffb700 1 -- 172.21.15.110:0/3067872571 <== osd.0 v2:172.21.15.110:6802/3841288978 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7ff8065fb0 con 0x7f7fe406ef70 2024-04-25T15:45:26.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.578+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] conn(0x7f7fe406ef70 msgr2=0x7f7fe4071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:26.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.578+0000 7f7fff94c700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] conn(0x7f7fe406ef70 0x7f7fe4071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.578+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7fe40692a0 msgr2=0x7f7fe406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7fe40692a0 0x7f7fe406b750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4004890 tx=0x7f7ff4009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 msgr2=0x7f7ff81140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 0x7f7ff81140b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7fe8002a50 tx=0x7f7fe80046b0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 shutdown_connections 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7fe40692a0 0x7f7fe406b750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ff80fbd40 0x7f7ff8113b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:6802/3841288978,v1:172.21.15.110:6803/3841288978] conn(0x7f7fe406ef70 0x7f7fe4071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 --2- 172.21.15.110:0/3067872571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7ff8101d90 0x7f7ff81140b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 >> 172.21.15.110:0/3067872571 conn(0x7f7ff8071d00 msgr2=0x7f7ff80655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 shutdown_connections 2024-04-25T15:45:26.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:26.579+0000 7f7fff94c700 1 -- 172.21.15.110:0/3067872571 wait complete. 2024-04-25T15:45:27.214 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:27.214 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:27.215 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:27.970 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:27 smithi110 ceph-mon[26344]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:27.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:27 smithi193 ceph-mon[33747]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:29.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:29 smithi193 ceph-mon[33747]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:29 smithi110 ceph-mon[26344]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:31.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 -- 172.21.15.110:0/2418598122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8101d90 msgr2=0x7f52d8102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:31.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 --2- 172.21.15.110:0/2418598122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8101d90 0x7f52d8102160 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f52d0009ce0 tx=0x7f52d0004d10 comp rx=0 tx=0).stop 2024-04-25T15:45:31.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 -- 172.21.15.110:0/2418598122 shutdown_connections 2024-04-25T15:45:31.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 --2- 172.21.15.110:0/2418598122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52d80fbd40 0x7f52d80fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:31.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 --2- 172.21.15.110:0/2418598122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d8101d90 0x7f52d8102160 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:31.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 -- 172.21.15.110:0/2418598122 >> 172.21.15.110:0/2418598122 conn(0x7f52d8071d00 msgr2=0x7f52d8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:31.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 -- 172.21.15.110:0/2418598122 shutdown_connections 2024-04-25T15:45:31.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 -- 172.21.15.110:0/2418598122 wait complete. 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.476+0000 7f52dd71e700 1 Processor -- start 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52dd71e700 1 -- start start 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52dd71e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d80fbd40 0x7f52d810ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52dd71e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52d8101d90 0x7f52d810f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52dd71e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d810f920 con 0x7f52d80fbd40 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52dd71e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d810fa60 con 0x7f52d8101d90 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d80fbd40 0x7f52d810ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d80fbd40 0x7f52d810ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53760/0 (socket says 172.21.15.110:53760) 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d6ffd700 1 -- 172.21.15.110:0/4080288622 learned_addr learned my addr 172.21.15.110:0/4080288622 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:31.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d67fc700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52d8101d90 0x7f52d810f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:31.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d6ffd700 1 -- 172.21.15.110:0/4080288622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52d8101d90 msgr2=0x7f52d810f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:31.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d6ffd700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52d8101d90 0x7f52d810f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:31.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d6ffd700 1 -- 172.21.15.110:0/4080288622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d0009940 con 0x7f52d80fbd40 2024-04-25T15:45:31.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d67fc700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52d8101d90 0x7f52d810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:45:31.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.477+0000 7f52d6ffd700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d80fbd40 0x7f52d810ee10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f52d000be90 tx=0x7f52d0004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:31.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.478+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d000e040 con 0x7f52d80fbd40 2024-04-25T15:45:31.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.478+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52d0008ea0 con 0x7f52d80fbd40 2024-04-25T15:45:31.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.478+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d8117d20 con 0x7f52d80fbd40 2024-04-25T15:45:31.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.478+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d001c940 con 0x7f52d80fbd40 2024-04-25T15:45:31.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.478+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d81182b0 con 0x7f52d80fbd40 2024-04-25T15:45:31.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.479+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f52d80fd430 con 0x7f52d80fbd40 2024-04-25T15:45:31.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.479+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f52d001cbb0 con 0x7f52d80fbd40 2024-04-25T15:45:31.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.479+0000 7f52bf7fe700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52c00692a0 0x7f52c006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:31.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.479+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f52d0088a80 con 0x7f52d80fbd40 2024-04-25T15:45:31.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.479+0000 7f52d67fc700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52c00692a0 0x7f52c006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:31.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.480+0000 7f52bf7fe700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] conn(0x7f52c006ef70 0x7f52c0071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:31.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.480+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 --> [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f52c0071a30 con 0x7f52c006ef70 2024-04-25T15:45:31.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.480+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f52d008d220 con 0x7f52d80fbd40 2024-04-25T15:45:31.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.480+0000 7f52d77fe700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] conn(0x7f52c006ef70 0x7f52c0071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:31.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.480+0000 7f52d67fc700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52c00692a0 0x7f52c006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52d8066040 tx=0x7f52cc008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:31.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.480+0000 7f52d77fe700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] conn(0x7f52c006ef70 0x7f52c0071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:31.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.481+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 <== osd.1 v2:172.21.15.110:6810/1447602330 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f52c0071a30 con 0x7f52c006ef70 2024-04-25T15:45:31.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.496+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 --> [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f52d80faa10 con 0x7f52c006ef70 2024-04-25T15:45:31.497 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.496+0000 7f52bf7fe700 1 -- 172.21.15.110:0/4080288622 <== osd.1 v2:172.21.15.110:6810/1447602330 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f52d80faa10 con 0x7f52c006ef70 2024-04-25T15:45:31.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] conn(0x7f52c006ef70 msgr2=0x7f52c0071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:31.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] conn(0x7f52c006ef70 0x7f52c0071380 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:31.498 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52c00692a0 msgr2=0x7f52c006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52c00692a0 0x7f52c006b750 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52d8066040 tx=0x7f52cc008040 comp rx=0 tx=0).stop 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d80fbd40 msgr2=0x7f52d810ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d80fbd40 0x7f52d810ee10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f52d000be90 tx=0x7f52d0004d10 comp rx=0 tx=0).stop 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 shutdown_connections 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6810/1447602330,v1:172.21.15.110:6811/1447602330] conn(0x7f52c006ef70 0x7f52c0071380 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52d80fbd40 0x7f52d810ee10 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52c00692a0 0x7f52c006b750 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 --2- 172.21.15.110:0/4080288622 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52d8101d90 0x7f52d810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 >> 172.21.15.110:0/4080288622 conn(0x7f52d8071d00 msgr2=0x7f52d80fa0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 shutdown_connections 2024-04-25T15:45:31.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:31.497+0000 7f52dd71e700 1 -- 172.21.15.110:0/4080288622 wait complete. 2024-04-25T15:45:31.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:31 smithi110 ceph-mon[26344]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:31.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:31 smithi193 ceph-mon[33747]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:32.727 INFO:teuthology.orchestra.run.smithi110.stdout:38654705697 2024-04-25T15:45:32.727 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T15:45:33.499 INFO:teuthology.orchestra.run.smithi110.stdout:60129542173 2024-04-25T15:45:33.499 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T15:45:33.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 -- 172.21.15.110:0/3833498332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 msgr2=0x7feb400fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:33.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 --2- 172.21.15.110:0/3833498332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 0x7feb400fc1b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb28004940 tx=0x7feb28009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:33.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 -- 172.21.15.110:0/3833498332 shutdown_connections 2024-04-25T15:45:33.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 --2- 172.21.15.110:0/3833498332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 0x7feb400fc1b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 --2- 172.21.15.110:0/3833498332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40101d90 0x7feb40102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 -- 172.21.15.110:0/3833498332 >> 172.21.15.110:0/3833498332 conn(0x7feb40071d00 msgr2=0x7feb40072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 -- 172.21.15.110:0/3833498332 shutdown_connections 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 -- 172.21.15.110:0/3833498332 wait complete. 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.695+0000 7feb45675700 1 Processor -- start 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb45675700 1 -- start start 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb45675700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 0x7feb40112a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb45675700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40101d90 0x7feb40112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb45675700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb40113610 con 0x7feb400fbd40 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb45675700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4010ca00 con 0x7feb40101d90 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb3effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 0x7feb40112a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb3effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 0x7feb40112a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53782/0 (socket says 172.21.15.110:53782) 2024-04-25T15:45:33.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb3effd700 1 -- 172.21.15.110:0/2090197189 learned_addr learned my addr 172.21.15.110:0/2090197189 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb3effd700 1 -- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40101d90 msgr2=0x7feb40112fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb367fc700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40101d90 0x7feb40112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb3effd700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40101d90 0x7feb40112fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb3effd700 1 -- 172.21.15.110:0/2090197189 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb28005040 con 0x7feb400fbd40 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.696+0000 7feb3effd700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 0x7feb40112a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb3000eb40 tx=0x7feb3000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.697+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb30007cc0 con 0x7feb400fbd40 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.697+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb3000f040 con 0x7feb400fbd40 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.697+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4010cc80 con 0x7feb400fbd40 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.697+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb30009620 con 0x7feb400fbd40 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.697+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4010d1c0 con 0x7feb400fbd40 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.697+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feb4010d6e0 con 0x7feb400fbd40 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.698+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7feb30009780 con 0x7feb400fbd40 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.699+0000 7feb37fff700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7feb2c0692b0 0x7feb2c06b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:33.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.699+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7feb30086d40 con 0x7feb400fbd40 2024-04-25T15:45:33.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.699+0000 7feb367fc700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7feb2c0692b0 0x7feb2c06b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:33.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.699+0000 7feb37fff700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] conn(0x7feb2c06ef80 0x7feb2c071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:33.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.699+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 --> [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb2c071a40 con 0x7feb2c06ef80 2024-04-25T15:45:33.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.699+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb30087120 con 0x7feb400fbd40 2024-04-25T15:45:33.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.699+0000 7feb367fc700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7feb2c0692b0 0x7feb2c06b760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb2800b0d0 tx=0x7feb28009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:33.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.699+0000 7feb3f7fe700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] conn(0x7feb2c06ef80 0x7feb2c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:33.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.700+0000 7feb3f7fe700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] conn(0x7feb2c06ef80 0x7feb2c071390 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:33.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.700+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 <== osd.4 v2:172.21.15.193:6808/190042456 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7feb2c071a40 con 0x7feb2c06ef80 2024-04-25T15:45:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.716+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 --> [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7feb4010d8a0 con 0x7feb2c06ef80 2024-04-25T15:45:33.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb37fff700 1 -- 172.21.15.110:0/2090197189 <== osd.4 v2:172.21.15.193:6808/190042456 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7feb4010d8a0 con 0x7feb2c06ef80 2024-04-25T15:45:33.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] conn(0x7feb2c06ef80 msgr2=0x7feb2c071390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] conn(0x7feb2c06ef80 0x7feb2c071390 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7feb2c0692b0 msgr2=0x7feb2c06b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7feb2c0692b0 0x7feb2c06b760 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb2800b0d0 tx=0x7feb28009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 msgr2=0x7feb40112a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 0x7feb40112a80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7feb3000eb40 tx=0x7feb3000b040 comp rx=0 tx=0).stop 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 shutdown_connections 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:6808/190042456,v1:172.21.15.193:6809/190042456] conn(0x7feb2c06ef80 0x7feb2c071390 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb400fbd40 0x7feb40112a80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7feb2c0692b0 0x7feb2c06b760 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 --2- 172.21.15.110:0/2090197189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb40101d90 0x7feb40112fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.717+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 >> 172.21.15.110:0/2090197189 conn(0x7feb40071d00 msgr2=0x7feb400f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:33.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.718+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 shutdown_connections 2024-04-25T15:45:33.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:33.718+0000 7feb45675700 1 -- 172.21.15.110:0/2090197189 wait complete. 2024-04-25T15:45:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:33 smithi193 ceph-mon[33747]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:33.994 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:33 smithi110 ceph-mon[26344]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:34.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.663+0000 7fe3eee64700 1 -- 172.21.15.110:0/2383147188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8103760 msgr2=0x7fe3e80fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.663+0000 7fe3eee64700 1 --2- 172.21.15.110:0/2383147188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8103760 0x7fe3e80fcdd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4004940 tx=0x7fe3d4009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:34.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.663+0000 7fe3eee64700 1 -- 172.21.15.110:0/2383147188 shutdown_connections 2024-04-25T15:45:34.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.663+0000 7fe3eee64700 1 --2- 172.21.15.110:0/2383147188 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8103760 0x7fe3e80fcdd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.663+0000 7fe3eee64700 1 --2- 172.21.15.110:0/2383147188 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3e8102dc0 0x7fe3e8103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.663+0000 7fe3eee64700 1 -- 172.21.15.110:0/2383147188 >> 172.21.15.110:0/2383147188 conn(0x7fe3e80f8930 msgr2=0x7fe3e80fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.663+0000 7fe3eee64700 1 -- 172.21.15.110:0/2383147188 shutdown_connections 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.663+0000 7fe3eee64700 1 -- 172.21.15.110:0/2383147188 wait complete. 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3eee64700 1 Processor -- start 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3eee64700 1 -- start start 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3eee64700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8102dc0 0x7fe3e8070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3eee64700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3e8103760 0x7fe3e80707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3eee64700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8072240 con 0x7fe3e8102dc0 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3eee64700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e8070ce0 con 0x7fe3e8103760 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3ecc00700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8102dc0 0x7fe3e8070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3ecc00700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8102dc0 0x7fe3e8070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53796/0 (socket says 172.21.15.110:53796) 2024-04-25T15:45:34.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.664+0000 7fe3ecc00700 1 -- 172.21.15.110:0/3205865794 learned_addr learned my addr 172.21.15.110:0/3205865794 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3e7fff700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3e8103760 0x7fe3e80707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3ecc00700 1 -- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3e8103760 msgr2=0x7fe3e80707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3ecc00700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3e8103760 0x7fe3e80707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3ecc00700 1 -- 172.21.15.110:0/3205865794 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3d4005040 con 0x7fe3e8102dc0 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3e7fff700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3e8103760 0x7fe3e80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3ecc00700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8102dc0 0x7fe3e8070260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00b790 tx=0x7fe3dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3dc00f770 con 0x7fe3e8102dc0 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e8070fc0 con 0x7fe3e8102dc0 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3dc00fda0 con 0x7fe3e8102dc0 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3dc00d310 con 0x7fe3e8102dc0 2024-04-25T15:45:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.665+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e811a1f0 con 0x7fe3e8102dc0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.666+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe3e80fe510 con 0x7fe3e8102dc0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.666+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe3dc010420 con 0x7fe3e8102dc0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.667+0000 7fe3e57fa700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3d80692a0 0x7fe3d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.667+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fe3dc086e80 con 0x7fe3e8102dc0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.667+0000 7fe3e7fff700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3d80692a0 0x7fe3d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.667+0000 7fe3e57fa700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] conn(0x7fe3d806ef70 0x7fe3d8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.667+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 --> [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe3d8071a30 con 0x7fe3d806ef70 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.667+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3dc08c480 con 0x7fe3e8102dc0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.667+0000 7fe3e7fff700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3d80692a0 0x7fe3d806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009fd0 tx=0x7fe3d4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.667+0000 7fe3ed401700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] conn(0x7fe3d806ef70 0x7fe3d8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.668+0000 7fe3ed401700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] conn(0x7fe3d806ef70 0x7fe3d8071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:34.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.669+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 <== osd.3 v2:172.21.15.193:6800/1286847191 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fe3d8071a30 con 0x7fe3d806ef70 2024-04-25T15:45:34.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 --> [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe3e805e5e0 con 0x7fe3d806ef70 2024-04-25T15:45:34.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3e57fa700 1 -- 172.21.15.110:0/3205865794 <== osd.3 v2:172.21.15.193:6800/1286847191 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fe3e805e5e0 con 0x7fe3d806ef70 2024-04-25T15:45:34.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] conn(0x7fe3d806ef70 msgr2=0x7fe3d8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] conn(0x7fe3d806ef70 0x7fe3d8071380 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3d80692a0 msgr2=0x7fe3d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3d80692a0 0x7fe3d806b750 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4009fd0 tx=0x7fe3d4009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8102dc0 msgr2=0x7fe3e8070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8102dc0 0x7fe3e8070260 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00b790 tx=0x7fe3dc009250 comp rx=0 tx=0).stop 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 shutdown_connections 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:6800/1286847191,v1:172.21.15.193:6801/1286847191] conn(0x7fe3d806ef70 0x7fe3d8071380 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3e8102dc0 0x7fe3e8070260 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3d80692a0 0x7fe3d806b750 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 --2- 172.21.15.110:0/3205865794 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3e8103760 0x7fe3e80707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 >> 172.21.15.110:0/3205865794 conn(0x7fe3e80f8930 msgr2=0x7fe3e80fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:34.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 shutdown_connections 2024-04-25T15:45:34.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.684+0000 7fe3eee64700 1 -- 172.21.15.110:0/3205865794 wait complete. 2024-04-25T15:45:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.866+0000 7f12cbcae700 1 -- 172.21.15.110:0/2218266517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 msgr2=0x7f12c40fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.866+0000 7f12cbcae700 1 --2- 172.21.15.110:0/2218266517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 0x7f12c40fbcf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f12b4004940 tx=0x7f12b4009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.866+0000 7f12cbcae700 1 -- 172.21.15.110:0/2218266517 shutdown_connections 2024-04-25T15:45:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.866+0000 7f12cbcae700 1 --2- 172.21.15.110:0/2218266517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 0x7f12c40fbcf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.866+0000 7f12cbcae700 1 --2- 172.21.15.110:0/2218266517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c4101ce0 0x7f12c41020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.866+0000 7f12cbcae700 1 -- 172.21.15.110:0/2218266517 >> 172.21.15.110:0/2218266517 conn(0x7f12c40f7850 msgr2=0x7f12c40f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.866+0000 7f12cbcae700 1 -- 172.21.15.110:0/2218266517 shutdown_connections 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.866+0000 7f12cbcae700 1 -- 172.21.15.110:0/2218266517 wait complete. 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12cbcae700 1 Processor -- start 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12cbcae700 1 -- start start 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12cbcae700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c4101ce0 0x7f12c4113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12cbcae700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 0x7f12c41140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12cbcae700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c41146f0 con 0x7f12c4102680 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12cbcae700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c410dae0 con 0x7f12c4101ce0 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12c9249700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 0x7f12c41140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12c9249700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 0x7f12c41140a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53820/0 (socket says 172.21.15.110:53820) 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12c9249700 1 -- 172.21.15.110:0/4253264478 learned_addr learned my addr 172.21.15.110:0/4253264478 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:34.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12c9a4a700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c4101ce0 0x7f12c4113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:34.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12c9249700 1 -- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c4101ce0 msgr2=0x7f12c4113b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.867+0000 7f12c9249700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c4101ce0 0x7f12c4113b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.868+0000 7f12c9249700 1 -- 172.21.15.110:0/4253264478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b4005040 con 0x7f12c4102680 2024-04-25T15:45:34.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.868+0000 7f12c9249700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 0x7f12c41140a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009100 tx=0x7f12b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.868+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12b400e040 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.868+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c410dd60 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.868+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12b4009af0 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.868+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12b401c3f0 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.868+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c410e320 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.869+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f12c410e800 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.870+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f12b400baf0 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.870+0000 7f12ba7fc700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f12b00692f0 0x7f12b006b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.870+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f12b4088180 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.870+0000 7f12c9a4a700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f12b00692f0 0x7f12b006b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.870+0000 7f12ba7fc700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] conn(0x7f12b006efc0 0x7f12b00713d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.870+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 --> [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f12b0071a80 con 0x7f12b006efc0 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.870+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b408a720 con 0x7f12c4102680 2024-04-25T15:45:34.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.871+0000 7f12ca24b700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] conn(0x7f12b006efc0 0x7f12b00713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:34.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.871+0000 7f12c9a4a700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f12b00692f0 0x7f12b006b7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12c0004eb0 tx=0x7f12c000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:34.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.871+0000 7f12ca24b700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] conn(0x7f12b006efc0 0x7f12b00713d0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:34.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.872+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 <== osd.5 v2:172.21.15.193:6816/1742452500 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f12b0071a80 con 0x7f12b006efc0 2024-04-25T15:45:34.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.886+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 --> [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f12c40f9900 con 0x7f12b006efc0 2024-04-25T15:45:34.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12ba7fc700 1 -- 172.21.15.110:0/4253264478 <== osd.5 v2:172.21.15.193:6816/1742452500 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f12c40f9900 con 0x7f12b006efc0 2024-04-25T15:45:34.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] conn(0x7f12b006efc0 msgr2=0x7f12b00713d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] conn(0x7f12b006efc0 0x7f12b00713d0 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f12b00692f0 msgr2=0x7f12b006b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f12b00692f0 0x7f12b006b7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f12c0004eb0 tx=0x7f12c000b040 comp rx=0 tx=0).stop 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 msgr2=0x7f12c41140a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 0x7f12c41140a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009100 tx=0x7f12b4009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 shutdown_connections 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:6816/1742452500,v1:172.21.15.193:6817/1742452500] conn(0x7f12b006efc0 0x7f12b00713d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f12b00692f0 0x7f12b006b7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12c4101ce0 0x7f12c4113b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 --2- 172.21.15.110:0/4253264478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12c4102680 0x7f12c41140a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 >> 172.21.15.110:0/4253264478 conn(0x7f12c40f7850 msgr2=0x7f12c40f8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 shutdown_connections 2024-04-25T15:45:34.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:34.887+0000 7f12cbcae700 1 -- 172.21.15.110:0/4253264478 wait complete. 2024-04-25T15:45:35.606 INFO:teuthology.orchestra.run.smithi110.stdout:146028888078 2024-04-25T15:45:35.607 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T15:45:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:35 smithi193 ceph-mon[33747]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:35 smithi110 ceph-mon[26344]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:36.250 INFO:teuthology.orchestra.run.smithi110.stdout:111669149716 2024-04-25T15:45:36.250 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T15:45:36.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.427+0000 7f4f0493f700 1 -- 172.21.15.110:0/4053914903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 msgr2=0x7f4f000fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.427+0000 7f4f0493f700 1 --2- 172.21.15.110:0/4053914903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 0x7f4f000fe1e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0004970 tx=0x7f4ef0009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.427+0000 7f4f0493f700 1 -- 172.21.15.110:0/4053914903 shutdown_connections 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.427+0000 7f4f0493f700 1 --2- 172.21.15.110:0/4053914903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f000fe7b0 0x7f4f00106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.427+0000 7f4f0493f700 1 --2- 172.21.15.110:0/4053914903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 0x7f4f000fe1e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.427+0000 7f4f0493f700 1 -- 172.21.15.110:0/4053914903 >> 172.21.15.110:0/4053914903 conn(0x7f4f000f78d0 msgr2=0x7f4f000f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.427+0000 7f4f0493f700 1 -- 172.21.15.110:0/4053914903 shutdown_connections 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4f0493f700 1 -- 172.21.15.110:0/4053914903 wait complete. 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4f0493f700 1 Processor -- start 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4f0493f700 1 -- start start 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4f0493f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 0x7f4f000fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4f0493f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f000fe7b0 0x7f4f000fc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4f0493f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000fc7a0 con 0x7f4f000fde10 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4f0493f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000fc8e0 con 0x7f4f000fe7b0 2024-04-25T15:45:36.677 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4efe59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 0x7f4f000fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:36.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4efe59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 0x7f4f000fbcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53840/0 (socket says 172.21.15.110:53840) 2024-04-25T15:45:36.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4efe59c700 1 -- 172.21.15.110:0/355936636 learned_addr learned my addr 172.21.15.110:0/355936636 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:36.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.428+0000 7f4efdd9b700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f000fe7b0 0x7f4f000fc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:36.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4efe59c700 1 -- 172.21.15.110:0/355936636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f000fe7b0 msgr2=0x7f4f000fc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:36.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4efe59c700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f000fe7b0 0x7f4f000fc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:36.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4efe59c700 1 -- 172.21.15.110:0/355936636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef0005040 con 0x7f4f000fde10 2024-04-25T15:45:36.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4efe59c700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 0x7f4f000fbcd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0000c00 tx=0x7f4ef0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:36.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ef000e040 con 0x7f4f000fde10 2024-04-25T15:45:36.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ef000bc10 con 0x7f4f000fde10 2024-04-25T15:45:36.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f000fcb30 con 0x7f4f000fde10 2024-04-25T15:45:36.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ef001c630 con 0x7f4f000fde10 2024-04-25T15:45:36.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.429+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f0011a200 con 0x7f4f000fde10 2024-04-25T15:45:36.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.430+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4f000666b0 con 0x7f4f000fde10 2024-04-25T15:45:36.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.431+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4ef000bd80 con 0x7f4f000fde10 2024-04-25T15:45:36.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.431+0000 7f4eeeffd700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4ee80692a0 0x7f4ee806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:36.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.431+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4ef0088100 con 0x7f4f000fde10 2024-04-25T15:45:36.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.431+0000 7f4efdd9b700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4ee80692a0 0x7f4ee806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:36.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.431+0000 7f4eeeffd700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] conn(0x7f4ee806ef70 0x7f4ee8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:36.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.431+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 --> [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ee8071a30 con 0x7f4ee806ef70 2024-04-25T15:45:36.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.431+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ef008a6f0 con 0x7f4f000fde10 2024-04-25T15:45:36.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.431+0000 7f4efed9d700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] conn(0x7f4ee806ef70 0x7f4ee8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:36.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.432+0000 7f4efdd9b700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4ee80692a0 0x7f4ee806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4007fd0 tx=0x7f4ef4009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:36.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.432+0000 7f4efed9d700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] conn(0x7f4ee806ef70 0x7f4ee8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:36.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.432+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 <== osd.2 v2:172.21.15.110:6818/3448336555 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f4ee8071a30 con 0x7f4ee806ef70 2024-04-25T15:45:36.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 --> [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4f0005e5e0 con 0x7f4ee806ef70 2024-04-25T15:45:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4eeeffd700 1 -- 172.21.15.110:0/355936636 <== osd.2 v2:172.21.15.110:6818/3448336555 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4f0005e5e0 con 0x7f4ee806ef70 2024-04-25T15:45:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] conn(0x7f4ee806ef70 msgr2=0x7f4ee8071380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] conn(0x7f4ee806ef70 0x7f4ee8071380 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4ee80692a0 msgr2=0x7f4ee806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4ee80692a0 0x7f4ee806b750 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4ef4007fd0 tx=0x7f4ef4009040 comp rx=0 tx=0).stop 2024-04-25T15:45:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 msgr2=0x7f4f000fbcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 0x7f4f000fbcd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0000c00 tx=0x7f4ef0009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:36.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 shutdown_connections 2024-04-25T15:45:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6818/3448336555,v1:172.21.15.110:6819/3448336555] conn(0x7f4ee806ef70 0x7f4ee8071380 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f000fde10 0x7f4f000fbcd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4ee80692a0 0x7f4ee806b750 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 --2- 172.21.15.110:0/355936636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f000fe7b0 0x7f4f000fc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 >> 172.21.15.110:0/355936636 conn(0x7f4f000f78d0 msgr2=0x7f4f000ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 shutdown_connections 2024-04-25T15:45:36.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:36.449+0000 7f4f0493f700 1 -- 172.21.15.110:0/355936636 wait complete. 2024-04-25T15:45:37.404 INFO:teuthology.orchestra.run.smithi110.stdout:81604378649 2024-04-25T15:45:37.404 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T15:45:37.762 INFO:teuthology.orchestra.run.smithi110.stdout:167503724553 2024-04-25T15:45:37.762 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T15:45:37.794 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:37.794 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:37.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:37 smithi193 ceph-mon[33747]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:38.034 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:37 smithi110 ceph-mon[26344]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:39.086 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:39.131 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:39.131 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:39.832 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:39 smithi110 ceph-mon[26344]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:39 smithi193 ceph-mon[33747]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:40.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:45:41.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:45:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:41 smithi110 ceph-mon[26344]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:41 smithi193 ceph-mon[33747]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:42.505 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.183+0000 7f349fefd700 1 -- 172.21.15.110:0/2091123727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 msgr2=0x7f34981031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.183+0000 7f349fefd700 1 --2- 172.21.15.110:0/2091123727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 0x7f34981031b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3494004940 tx=0x7f3494009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.183+0000 7f349fefd700 1 -- 172.21.15.110:0/2091123727 shutdown_connections 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.183+0000 7f349fefd700 1 --2- 172.21.15.110:0/2091123727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3498103780 0x7f34980fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.183+0000 7f349fefd700 1 --2- 172.21.15.110:0/2091123727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 0x7f34981031b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.183+0000 7f349fefd700 1 -- 172.21.15.110:0/2091123727 >> 172.21.15.110:0/2091123727 conn(0x7f34980f89b0 msgr2=0x7f34980fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.183+0000 7f349fefd700 1 -- 172.21.15.110:0/2091123727 shutdown_connections 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.183+0000 7f349fefd700 1 -- 172.21.15.110:0/2091123727 wait complete. 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349fefd700 1 Processor -- start 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349fefd700 1 -- start start 2024-04-25T15:45:43.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349fefd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 0x7f3498113b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349fefd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3498103780 0x7f34981140a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349fefd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498114780 con 0x7f3498102de0 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349fefd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f349810dae0 con 0x7f3498103780 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349dc99700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 0x7f3498113b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349dc99700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 0x7f3498113b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54628/0 (socket says 172.21.15.110:54628) 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349dc99700 1 -- 172.21.15.110:0/4152074768 learned_addr learned my addr 172.21.15.110:0/4152074768 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349dc99700 1 -- 172.21.15.110:0/4152074768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3498103780 msgr2=0x7f34981140a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349d498700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3498103780 0x7f34981140a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349dc99700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3498103780 0x7f34981140a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349dc99700 1 -- 172.21.15.110:0/4152074768 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494005040 con 0x7f3498102de0 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349d498700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3498103780 0x7f34981140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f349dc99700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 0x7f3498113b60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3494009150 tx=0x7f3494009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.184+0000 7f348e7fc700 1 -- 172.21.15.110:0/4152074768 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f349400e040 con 0x7f3498102de0 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.185+0000 7f348e7fc700 1 -- 172.21.15.110:0/4152074768 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3494013420 con 0x7f3498102de0 2024-04-25T15:45:43.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.185+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349810dd60 con 0x7f3498102de0 2024-04-25T15:45:43.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.185+0000 7f348e7fc700 1 -- 172.21.15.110:0/4152074768 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f349401c630 con 0x7f3498102de0 2024-04-25T15:45:43.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.185+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f349810e320 con 0x7f3498102de0 2024-04-25T15:45:43.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.186+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34981000f0 con 0x7f3498102de0 2024-04-25T15:45:43.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.186+0000 7f348e7fc700 1 -- 172.21.15.110:0/4152074768 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3494013590 con 0x7f3498102de0 2024-04-25T15:45:43.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.186+0000 7f348e7fc700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f34840692b0 0x7f348406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:43.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.186+0000 7f348e7fc700 1 -- 172.21.15.110:0/4152074768 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f34940880d0 con 0x7f3498102de0 2024-04-25T15:45:43.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.186+0000 7f349d498700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f34840692b0 0x7f348406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:43.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.187+0000 7f349d498700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f34840692b0 0x7f348406b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3488006fd0 tx=0x7f3488008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:43.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.191+0000 7f348e7fc700 1 -- 172.21.15.110:0/4152074768 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f34940581d0 con 0x7f3498102de0 2024-04-25T15:45:43.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.381+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3498109f50 con 0x7f3498102de0 2024-04-25T15:45:43.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.382+0000 7f348e7fc700 1 -- 172.21.15.110:0/4152074768 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3494057d60 con 0x7f3498102de0 2024-04-25T15:45:43.383 INFO:teuthology.orchestra.run.smithi110.stdout:111669149717 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.383+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f34840692b0 msgr2=0x7f348406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f34840692b0 0x7f348406b760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3488006fd0 tx=0x7f3488008040 comp rx=0 tx=0).stop 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 msgr2=0x7f3498113b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 0x7f3498113b60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3494009150 tx=0x7f3494009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 shutdown_connections 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3498102de0 0x7f3498113b60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f34840692b0 0x7f348406b760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 --2- 172.21.15.110:0/4152074768 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3498103780 0x7f34981140a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 >> 172.21.15.110:0/4152074768 conn(0x7f34980f89b0 msgr2=0x7f34980fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 shutdown_connections 2024-04-25T15:45:43.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:43.384+0000 7f349fefd700 1 -- 172.21.15.110:0/4152074768 wait complete. 2024-04-25T15:45:43.794 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:43 smithi110 ceph-mon[26344]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:43.794 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:43 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/4152074768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.189+0000 7f732c04d700 1 -- 172.21.15.110:0/4040364337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73241026a0 msgr2=0x7f73240fbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.189+0000 7f732c04d700 1 --2- 172.21.15.110:0/4040364337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73241026a0 0x7f73240fbd10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7314004940 tx=0x7f7314009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.189+0000 7f732c04d700 1 -- 172.21.15.110:0/4040364337 shutdown_connections 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.189+0000 7f732c04d700 1 --2- 172.21.15.110:0/4040364337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73241026a0 0x7f73240fbd10 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.189+0000 7f732c04d700 1 --2- 172.21.15.110:0/4040364337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7324101d00 0x7f73241020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.189+0000 7f732c04d700 1 -- 172.21.15.110:0/4040364337 >> 172.21.15.110:0/4040364337 conn(0x7f73240f7870 msgr2=0x7f73240f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.189+0000 7f732c04d700 1 -- 172.21.15.110:0/4040364337 shutdown_connections 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.189+0000 7f732c04d700 1 -- 172.21.15.110:0/4040364337 wait complete. 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f732c04d700 1 Processor -- start 2024-04-25T15:45:44.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f732c04d700 1 -- start start 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f732c04d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7324101d00 0x7f732410eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f732c04d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73241026a0 0x7f732410f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f732c04d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732410f5b0 con 0x7f7324101d00 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f732c04d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f732410f6f0 con 0x7f73241026a0 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f7329de9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7324101d00 0x7f732410eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f7329de9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7324101d00 0x7f732410eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54648/0 (socket says 172.21.15.110:54648) 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f7329de9700 1 -- 172.21.15.110:0/3434608549 learned_addr learned my addr 172.21.15.110:0/3434608549 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.190+0000 7f73295e8700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73241026a0 0x7f732410f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f7329de9700 1 -- 172.21.15.110:0/3434608549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73241026a0 msgr2=0x7f732410f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f7329de9700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73241026a0 0x7f732410f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f7329de9700 1 -- 172.21.15.110:0/3434608549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7314005040 con 0x7f7324101d00 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f7329de9700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7324101d00 0x7f732410eb30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f732000cac0 tx=0x7f7320007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f731a7fc700 1 -- 172.21.15.110:0/3434608549 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f732000a980 con 0x7f7324101d00 2024-04-25T15:45:44.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7324117ba0 con 0x7f7324101d00 2024-04-25T15:45:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f731a7fc700 1 -- 172.21.15.110:0/3434608549 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7320012440 con 0x7f7324101d00 2024-04-25T15:45:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f731a7fc700 1 -- 172.21.15.110:0/3434608549 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73200115e0 con 0x7f7324101d00 2024-04-25T15:45:44.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.191+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7324117f70 con 0x7f7324101d00 2024-04-25T15:45:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.192+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73240fd450 con 0x7f7324101d00 2024-04-25T15:45:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.192+0000 7f731a7fc700 1 -- 172.21.15.110:0/3434608549 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f73200125b0 con 0x7f7324101d00 2024-04-25T15:45:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.193+0000 7f731a7fc700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f73100692a0 0x7f731006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.193+0000 7f731a7fc700 1 -- 172.21.15.110:0/3434608549 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f73200875b0 con 0x7f7324101d00 2024-04-25T15:45:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.193+0000 7f73295e8700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f73100692a0 0x7f731006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:44.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.193+0000 7f73295e8700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f73100692a0 0x7f731006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7324066a40 tx=0x7f7314009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:44.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.197+0000 7f731a7fc700 1 -- 172.21.15.110:0/3434608549 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7320053ba0 con 0x7f7324101d00 2024-04-25T15:45:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:43 smithi193 ceph-mon[33747]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:43 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/4152074768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T15:45:44.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 -- 172.21.15.110:0/1387188763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52800fc4d0 msgr2=0x7f52800fc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:44.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 --2- 172.21.15.110:0/1387188763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52800fc4d0 0x7f52800fc940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5270004940 tx=0x7f5270009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:44.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 -- 172.21.15.110:0/1387188763 shutdown_connections 2024-04-25T15:45:44.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 --2- 172.21.15.110:0/1387188763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52800fc4d0 0x7f52800fc940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 --2- 172.21.15.110:0/1387188763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5280101d70 0x7f5280102140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 -- 172.21.15.110:0/1387188763 >> 172.21.15.110:0/1387188763 conn(0x7f5280072320 msgr2=0x7f5280072720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:44.377 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 -- 172.21.15.110:0/1387188763 shutdown_connections 2024-04-25T15:45:44.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 -- 172.21.15.110:0/1387188763 wait complete. 2024-04-25T15:45:44.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.376+0000 7f5285e8d700 1 Processor -- start 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f5285e8d700 1 -- start start 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f5285e8d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52800fc4d0 0x7f5280115b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f5285e8d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5280101d70 0x7f52801160b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f5285e8d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5280116700 con 0x7f5280101d70 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f5285e8d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528010fce0 con 0x7f52800fc4d0 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f527effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5280101d70 0x7f52801160b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f527effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5280101d70 0x7f52801160b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54670/0 (socket says 172.21.15.110:54670) 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f527effd700 1 -- 172.21.15.110:0/3127464759 learned_addr learned my addr 172.21.15.110:0/3127464759 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f527f7fe700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52800fc4d0 0x7f5280115b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f527effd700 1 -- 172.21.15.110:0/3127464759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52800fc4d0 msgr2=0x7f5280115b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f527effd700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52800fc4d0 0x7f5280115b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f527effd700 1 -- 172.21.15.110:0/3127464759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5270005040 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.377+0000 7f527effd700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5280101d70 0x7f52801160b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f527000b8b0 tx=0x7f5270009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.378+0000 7f5267fff700 1 -- 172.21.15.110:0/3127464759 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527000e040 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.378+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528010ff60 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.378+0000 7f5267fff700 1 -- 172.21.15.110:0/3127464759 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5270004290 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.378+0000 7f5267fff700 1 -- 172.21.15.110:0/3127464759 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527001c3f0 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.378+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5280110520 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.379+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52800fd4e0 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.379+0000 7f5267fff700 1 -- 172.21.15.110:0/3127464759 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5270013420 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.380+0000 7f5267fff700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52680692a0 0x7f526806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.380+0000 7f527f7fe700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52680692a0 0x7f526806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.380+0000 7f5267fff700 1 -- 172.21.15.110:0/3127464759 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f52700880b0 con 0x7f5280101d70 2024-04-25T15:45:44.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.380+0000 7f527f7fe700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52680692a0 0x7f526806b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5280065c80 tx=0x7f527400b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:44.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.384+0000 7f5267fff700 1 -- 172.21.15.110:0/3127464759 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f52700581b0 con 0x7f5280101d70 2024-04-25T15:45:44.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.395+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f732404c710 con 0x7f7324101d00 2024-04-25T15:45:44.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.396+0000 7f731a7fc700 1 -- 172.21.15.110:0/3434608549 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7320012860 con 0x7f7324101d00 2024-04-25T15:45:44.398 INFO:teuthology.orchestra.run.smithi110.stdout:38654705700 2024-04-25T15:45:44.398 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f73100692a0 msgr2=0x7f731006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:44.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f73100692a0 0x7f731006b750 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7324066a40 tx=0x7f7314009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:44.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7324101d00 msgr2=0x7f732410eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:44.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7324101d00 0x7f732410eb30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f732000cac0 tx=0x7f7320007ee0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 shutdown_connections 2024-04-25T15:45:44.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7324101d00 0x7f732410eb30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f73100692a0 0x7f731006b750 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 --2- 172.21.15.110:0/3434608549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73241026a0 0x7f732410f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 >> 172.21.15.110:0/3434608549 conn(0x7f73240f7870 msgr2=0x7f73240f8810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:44.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 shutdown_connections 2024-04-25T15:45:44.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.398+0000 7f732c04d700 1 -- 172.21.15.110:0/3434608549 wait complete. 2024-04-25T15:45:44.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.579+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5280108df0 con 0x7f5280101d70 2024-04-25T15:45:44.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.579+0000 7f5267fff700 1 -- 172.21.15.110:0/3127464759 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5270057d40 con 0x7f5280101d70 2024-04-25T15:45:44.580 INFO:teuthology.orchestra.run.smithi110.stdout:81604378650 2024-04-25T15:45:44.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52680692a0 msgr2=0x7f526806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:44.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52680692a0 0x7f526806b750 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5280065c80 tx=0x7f527400b040 comp rx=0 tx=0).stop 2024-04-25T15:45:44.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5280101d70 msgr2=0x7f52801160b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:44.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5280101d70 0x7f52801160b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f527000b8b0 tx=0x7f5270009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:44.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 shutdown_connections 2024-04-25T15:45:44.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52680692a0 0x7f526806b750 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52800fc4d0 0x7f5280115b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 --2- 172.21.15.110:0/3127464759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5280101d70 0x7f52801160b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:44.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.581+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 >> 172.21.15.110:0/3127464759 conn(0x7f5280072320 msgr2=0x7f52800f9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:44.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.582+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 shutdown_connections 2024-04-25T15:45:44.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:44.582+0000 7f5285e8d700 1 -- 172.21.15.110:0/3127464759 wait complete. 2024-04-25T15:45:45.007 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:44 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3434608549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T15:45:45.007 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:44 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3127464759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T15:45:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:44 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3434608549' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T15:45:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:44 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3127464759' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T15:45:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:45 smithi110 ceph-mon[26344]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:46.168 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705697 got 38654705700 for osd.0 2024-04-25T15:45:46.169 DEBUG:teuthology.parallel:result is None 2024-04-25T15:45:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:45 smithi193 ceph-mon[33747]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 -- 172.21.15.110:0/451612066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 msgr2=0x7fad3c1021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 --2- 172.21.15.110:0/451612066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 0x7fad3c1021e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fad30004940 tx=0x7fad30009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 -- 172.21.15.110:0/451612066 shutdown_connections 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 --2- 172.21.15.110:0/451612066 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad3c0fc630 0x7fad3c0fcaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 --2- 172.21.15.110:0/451612066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 0x7fad3c1021e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 -- 172.21.15.110:0/451612066 >> 172.21.15.110:0/451612066 conn(0x7fad3c0714d0 msgr2=0x7fad3c0718d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 -- 172.21.15.110:0/451612066 shutdown_connections 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 -- 172.21.15.110:0/451612066 wait complete. 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 Processor -- start 2024-04-25T15:45:46.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.370+0000 7fad42b0e700 1 -- start start 2024-04-25T15:45:46.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad42b0e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad3c0fc630 0x7fad3c113920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad42b0e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 0x7fad3c113e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad42b0e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c1144b0 con 0x7fad3c101e10 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad42b0e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c10dab0 con 0x7fad3c0fc630 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad3bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 0x7fad3c113e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad3bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 0x7fad3c113e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54690/0 (socket says 172.21.15.110:54690) 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad3bfff700 1 -- 172.21.15.110:0/2321518711 learned_addr learned my addr 172.21.15.110:0/2321518711 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad408aa700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad3c0fc630 0x7fad3c113920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad3bfff700 1 -- 172.21.15.110:0/2321518711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad3c0fc630 msgr2=0x7fad3c113920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad3bfff700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad3c0fc630 0x7fad3c113920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad3bfff700 1 -- 172.21.15.110:0/2321518711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad30005040 con 0x7fad3c101e10 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad3bfff700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 0x7fad3c113e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c830 tx=0x7fad28004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad397fa700 1 -- 172.21.15.110:0/2321518711 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad280106f0 con 0x7fad3c101e10 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c10dd90 con 0x7fad3c101e10 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad397fa700 1 -- 172.21.15.110:0/2321518711 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad28010d20 con 0x7fad3c101e10 2024-04-25T15:45:46.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad397fa700 1 -- 172.21.15.110:0/2321518711 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad280113a0 con 0x7fad3c101e10 2024-04-25T15:45:46.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.371+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c10e2d0 con 0x7fad3c101e10 2024-04-25T15:45:46.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.372+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c0fd640 con 0x7fad3c101e10 2024-04-25T15:45:46.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.373+0000 7fad397fa700 1 -- 172.21.15.110:0/2321518711 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fad28012400 con 0x7fad3c101e10 2024-04-25T15:45:46.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.373+0000 7fad397fa700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fad2c0692a0 0x7fad2c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:46.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.373+0000 7fad408aa700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fad2c0692a0 0x7fad2c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:46.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.373+0000 7fad397fa700 1 -- 172.21.15.110:0/2321518711 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fad28086e20 con 0x7fad3c101e10 2024-04-25T15:45:46.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.374+0000 7fad408aa700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fad2c0692a0 0x7fad2c06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad30004f90 tx=0x7fad30009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:46.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.377+0000 7fad397fa700 1 -- 172.21.15.110:0/2321518711 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fad28056f20 con 0x7fad3c101e10 2024-04-25T15:45:46.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.569+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fad3c108f00 con 0x7fad3c101e10 2024-04-25T15:45:46.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.569+0000 7fad397fa700 1 -- 172.21.15.110:0/2321518711 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fad28056ab0 con 0x7fad3c101e10 2024-04-25T15:45:46.570 INFO:teuthology.orchestra.run.smithi110.stdout:167503724555 2024-04-25T15:45:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fad2c0692a0 msgr2=0x7fad2c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fad2c0692a0 0x7fad2c06b750 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad30004f90 tx=0x7fad30009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 msgr2=0x7fad3c113e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 0x7fad3c113e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fad2800c830 tx=0x7fad28004470 comp rx=0 tx=0).stop 2024-04-25T15:45:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 shutdown_connections 2024-04-25T15:45:46.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fad2c0692a0 0x7fad2c06b750 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:46.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad3c0fc630 0x7fad3c113920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:46.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 --2- 172.21.15.110:0/2321518711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad3c101e10 0x7fad3c113e60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:46.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.571+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 >> 172.21.15.110:0/2321518711 conn(0x7fad3c0714d0 msgr2=0x7fad3c105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:46.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.572+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 shutdown_connections 2024-04-25T15:45:46.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:46.572+0000 7fad42b0e700 1 -- 172.21.15.110:0/2321518711 wait complete. 2024-04-25T15:45:47.087 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:46 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2321518711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T15:45:47.109 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378649 got 81604378650 for osd.2 2024-04-25T15:45:47.110 DEBUG:teuthology.parallel:result is None 2024-04-25T15:45:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:46 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2321518711' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T15:45:47.803 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724555 for osd.5 2024-04-25T15:45:47.803 DEBUG:teuthology.parallel:result is None 2024-04-25T15:45:48.111 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:47 smithi110 ceph-mon[26344]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:47 smithi193 ceph-mon[33747]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:48.524 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149716 got 111669149717 for osd.3 2024-04-25T15:45:48.524 DEBUG:teuthology.parallel:result is None 2024-04-25T15:45:49.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.474+0000 7f0fab4e0700 1 -- 172.21.15.110:0/3168401546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa40fbd40 msgr2=0x7f0fa40fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.474+0000 7f0fab4e0700 1 --2- 172.21.15.110:0/3168401546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa40fbd40 0x7f0fa40fc1b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f94004940 tx=0x7f0f94009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.474+0000 7f0fab4e0700 1 -- 172.21.15.110:0/3168401546 shutdown_connections 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.474+0000 7f0fab4e0700 1 --2- 172.21.15.110:0/3168401546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa40fbd40 0x7f0fa40fc1b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.474+0000 7f0fab4e0700 1 --2- 172.21.15.110:0/3168401546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa4101d90 0x7f0fa4102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.474+0000 7f0fab4e0700 1 -- 172.21.15.110:0/3168401546 >> 172.21.15.110:0/3168401546 conn(0x7f0fa4071d00 msgr2=0x7f0fa4072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.474+0000 7f0fab4e0700 1 -- 172.21.15.110:0/3168401546 shutdown_connections 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.474+0000 7f0fab4e0700 1 -- 172.21.15.110:0/3168401546 wait complete. 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fab4e0700 1 Processor -- start 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fab4e0700 1 -- start start 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fab4e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa40fbd40 0x7f0fa4113c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fab4e0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa4101d90 0x7f0fa4114190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fab4e0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa4114870 con 0x7f0fa4101d90 2024-04-25T15:45:49.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fab4e0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa410dbb0 con 0x7f0fa40fbd40 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fa8a7b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa4101d90 0x7f0fa4114190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fa8a7b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa4101d90 0x7f0fa4114190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54700/0 (socket says 172.21.15.110:54700) 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fa8a7b700 1 -- 172.21.15.110:0/2117741224 learned_addr learned my addr 172.21.15.110:0/2117741224 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fa8a7b700 1 -- 172.21.15.110:0/2117741224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa40fbd40 msgr2=0x7f0fa4113c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fa8a7b700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa40fbd40 0x7f0fa4113c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fa8a7b700 1 -- 172.21.15.110:0/2117741224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f94005040 con 0x7f0fa4101d90 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0fa8a7b700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa4101d90 0x7f0fa4114190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400b3b0 tx=0x7f0f94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.475+0000 7f0f99ffb700 1 -- 172.21.15.110:0/2117741224 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f9400e030 con 0x7f0fa4101d90 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.476+0000 7f0f99ffb700 1 -- 172.21.15.110:0/2117741224 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f94009430 con 0x7f0fa4101d90 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.476+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa410ddd0 con 0x7f0fa4101d90 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.476+0000 7f0f99ffb700 1 -- 172.21.15.110:0/2117741224 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f94013e50 con 0x7f0fa4101d90 2024-04-25T15:45:49.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.476+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa410e390 con 0x7f0fa4101d90 2024-04-25T15:45:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.477+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa40fd430 con 0x7f0fa4101d90 2024-04-25T15:45:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.477+0000 7f0f99ffb700 1 -- 172.21.15.110:0/2117741224 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0f940136b0 con 0x7f0fa4101d90 2024-04-25T15:45:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.478+0000 7f0f99ffb700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0f90069230 0x7f0f9006b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.478+0000 7f0f99ffb700 1 -- 172.21.15.110:0/2117741224 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0f94088a50 con 0x7f0fa4101d90 2024-04-25T15:45:49.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.478+0000 7f0fa927c700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0f90069230 0x7f0f9006b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:49.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.478+0000 7f0fa927c700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0f90069230 0x7f0f9006b6e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0097c0 tx=0x7f0f9c006ce0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:49.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.482+0000 7f0f99ffb700 1 -- 172.21.15.110:0/2117741224 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0f94058b50 con 0x7f0fa4101d90 2024-04-25T15:45:49.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.671+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f0fa410df60 con 0x7f0fa4101d90 2024-04-25T15:45:49.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.671+0000 7f0f99ffb700 1 -- 172.21.15.110:0/2117741224 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0fa410df60 con 0x7f0fa4101d90 2024-04-25T15:45:49.672 INFO:teuthology.orchestra.run.smithi110.stdout:60129542176 2024-04-25T15:45:49.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0f90069230 msgr2=0x7f0f9006b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0f90069230 0x7f0f9006b6e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0f9c0097c0 tx=0x7f0f9c006ce0 comp rx=0 tx=0).stop 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa4101d90 msgr2=0x7f0fa4114190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa4101d90 0x7f0fa4114190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400b3b0 tx=0x7f0f94009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 shutdown_connections 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0f90069230 0x7f0f9006b6e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa40fbd40 0x7f0fa4113c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 --2- 172.21.15.110:0/2117741224 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0fa4101d90 0x7f0fa4114190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 >> 172.21.15.110:0/2117741224 conn(0x7f0fa4071d00 msgr2=0x7f0fa40fa0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 shutdown_connections 2024-04-25T15:45:49.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:49.674+0000 7f0fab4e0700 1 -- 172.21.15.110:0/2117741224 wait complete. 2024-04-25T15:45:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:49 smithi110 ceph-mon[26344]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:49 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2117741224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T15:45:50.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.120+0000 7fd23a4a3700 1 -- 172.21.15.110:0/557091565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd234101d90 msgr2=0x7fd234102160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:50.122 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.120+0000 7fd23a4a3700 1 --2- 172.21.15.110:0/557091565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd234101d90 0x7fd234102160 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd224004940 tx=0x7fd224009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:50.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.121+0000 7fd23a4a3700 1 -- 172.21.15.110:0/557091565 shutdown_connections 2024-04-25T15:45:50.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.121+0000 7fd23a4a3700 1 --2- 172.21.15.110:0/557091565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd2340fbd40 0x7fd2340fc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:50.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.121+0000 7fd23a4a3700 1 --2- 172.21.15.110:0/557091565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd234101d90 0x7fd234102160 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:50.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.121+0000 7fd23a4a3700 1 -- 172.21.15.110:0/557091565 >> 172.21.15.110:0/557091565 conn(0x7fd234071d00 msgr2=0x7fd234072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:50.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.121+0000 7fd23a4a3700 1 -- 172.21.15.110:0/557091565 shutdown_connections 2024-04-25T15:45:50.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.121+0000 7fd23a4a3700 1 -- 172.21.15.110:0/557091565 wait complete. 2024-04-25T15:45:50.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd23a4a3700 1 Processor -- start 2024-04-25T15:45:50.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd23a4a3700 1 -- start start 2024-04-25T15:45:50.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd23a4a3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2340fbd40 0x7fd23410ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:50.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd23a4a3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd23410f220 0x7fd23410f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:50.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd23a4a3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd234117de0 con 0x7fd2340fbd40 2024-04-25T15:45:50.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd23a4a3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd234117f20 con 0x7fd23410f220 2024-04-25T15:45:50.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd233fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2340fbd40 0x7fd23410ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:50.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd233fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2340fbd40 0x7fd23410ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41206/0 (socket says 172.21.15.110:41206) 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.122+0000 7fd233fff700 1 -- 172.21.15.110:0/1949715596 learned_addr learned my addr 172.21.15.110:0/1949715596 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd2337fe700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd23410f220 0x7fd23410f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd233fff700 1 -- 172.21.15.110:0/1949715596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd23410f220 msgr2=0x7fd23410f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd233fff700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd23410f220 0x7fd23410f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd233fff700 1 -- 172.21.15.110:0/1949715596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd224005040 con 0x7fd2340fbd40 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd2337fe700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd23410f220 0x7fd23410f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd233fff700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2340fbd40 0x7fd23410ece0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd224000c00 tx=0x7fd224009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd230ff9700 1 -- 172.21.15.110:0/1949715596 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd22400e040 con 0x7fd2340fbd40 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd230ff9700 1 -- 172.21.15.110:0/1949715596 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd224013420 con 0x7fd2340fbd40 2024-04-25T15:45:50.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd230ff9700 1 -- 172.21.15.110:0/1949715596 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd22401c630 con 0x7fd2340fbd40 2024-04-25T15:45:50.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd234118120 con 0x7fd2340fbd40 2024-04-25T15:45:50.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.123+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd234118690 con 0x7fd2340fbd40 2024-04-25T15:45:50.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.124+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2340fd430 con 0x7fd2340fbd40 2024-04-25T15:45:50.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.125+0000 7fd230ff9700 1 -- 172.21.15.110:0/1949715596 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd224013a90 con 0x7fd2340fbd40 2024-04-25T15:45:50.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.125+0000 7fd230ff9700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd21c0692a0 0x7fd21c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:50.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.125+0000 7fd230ff9700 1 -- 172.21.15.110:0/1949715596 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7fd224087fe0 con 0x7fd2340fbd40 2024-04-25T15:45:50.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.125+0000 7fd2337fe700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd21c0692a0 0x7fd21c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:50.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.126+0000 7fd2337fe700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd21c0692a0 0x7fd21c06b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd228007b10 tx=0x7fd228008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:50.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.130+0000 7fd230ff9700 1 -- 172.21.15.110:0/1949715596 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd224054650 con 0x7fd2340fbd40 2024-04-25T15:45:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:49 smithi193 ceph-mon[33747]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:49 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2117741224' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T15:45:50.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.318+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fd23405f5c0 con 0x7fd2340fbd40 2024-04-25T15:45:50.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.319+0000 7fd230ff9700 1 -- 172.21.15.110:0/1949715596 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd224057c70 con 0x7fd2340fbd40 2024-04-25T15:45:50.320 INFO:teuthology.orchestra.run.smithi110.stdout:146028888081 2024-04-25T15:45:50.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd21c0692a0 msgr2=0x7fd21c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:50.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd21c0692a0 0x7fd21c06b750 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd228007b10 tx=0x7fd228008040 comp rx=0 tx=0).stop 2024-04-25T15:45:50.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2340fbd40 msgr2=0x7fd23410ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:50.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2340fbd40 0x7fd23410ece0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd224000c00 tx=0x7fd224009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:50.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 shutdown_connections 2024-04-25T15:45:50.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd2340fbd40 0x7fd23410ece0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:50.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd21c0692a0 0x7fd21c06b750 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:50.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 --2- 172.21.15.110:0/1949715596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd23410f220 0x7fd23410f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:50.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 >> 172.21.15.110:0/1949715596 conn(0x7fd234071d00 msgr2=0x7fd234104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:50.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 shutdown_connections 2024-04-25T15:45:50.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:50.321+0000 7fd23a4a3700 1 -- 172.21.15.110:0/1949715596 wait complete. 2024-04-25T15:45:50.634 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542173 got 60129542176 for osd.1 2024-04-25T15:45:50.634 DEBUG:teuthology.parallel:result is None 2024-04-25T15:45:50.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:50 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1949715596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T15:45:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:50 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1949715596' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T15:45:51.281 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888078 got 146028888081 for osd.4 2024-04-25T15:45:51.281 DEBUG:teuthology.parallel:result is None 2024-04-25T15:45:51.281 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T15:45:51.282 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:45:51.537 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:51.931 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:51 smithi110 ceph-mon[26344]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:51 smithi193 ceph-mon[33747]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:53.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.058+0000 7f65d4980700 1 -- 172.21.15.110:0/3521883581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 msgr2=0x7f65d0106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:53.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.058+0000 7f65d4980700 1 --2- 172.21.15.110:0/3521883581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 0x7f65d0106e50 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f65c4004940 tx=0x7f65c4009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:53.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 -- 172.21.15.110:0/3521883581 shutdown_connections 2024-04-25T15:45:53.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 --2- 172.21.15.110:0/3521883581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 0x7f65d0106e50 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:53.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 --2- 172.21.15.110:0/3521883581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d00fde70 0x7f65d00fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:53.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 -- 172.21.15.110:0/3521883581 >> 172.21.15.110:0/3521883581 conn(0x7f65d00f7860 msgr2=0x7f65d00f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:53.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 -- 172.21.15.110:0/3521883581 shutdown_connections 2024-04-25T15:45:53.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 -- 172.21.15.110:0/3521883581 wait complete. 2024-04-25T15:45:53.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 Processor -- start 2024-04-25T15:45:53.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 -- start start 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d00fde70 0x7f65d00fd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 0x7f65d00fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00fc1e0 con 0x7f65d00fe810 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65d4980700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65d00fc350 con 0x7f65d00fde70 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.059+0000 7f65cdd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 0x7f65d00fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65cdd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 0x7f65d00fbca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41224/0 (socket says 172.21.15.110:41224) 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65cdd9b700 1 -- 172.21.15.110:0/2002487957 learned_addr learned my addr 172.21.15.110:0/2002487957 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65ce59c700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d00fde70 0x7f65d00fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65cdd9b700 1 -- 172.21.15.110:0/2002487957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d00fde70 msgr2=0x7f65d00fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65cdd9b700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d00fde70 0x7f65d00fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65cdd9b700 1 -- 172.21.15.110:0/2002487957 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c4005040 con 0x7f65d00fe810 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65cdd9b700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 0x7f65d00fbca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009380 tx=0x7f65c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65beffd700 1 -- 172.21.15.110:0/2002487957 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c400e040 con 0x7f65d00fe810 2024-04-25T15:45:53.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65d00fc630 con 0x7f65d00fe810 2024-04-25T15:45:53.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65beffd700 1 -- 172.21.15.110:0/2002487957 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f65c4009af0 con 0x7f65d00fe810 2024-04-25T15:45:53.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65beffd700 1 -- 172.21.15.110:0/2002487957 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65c401c3f0 con 0x7f65d00fe810 2024-04-25T15:45:53.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.060+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65d011e630 con 0x7f65d00fe810 2024-04-25T15:45:53.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.061+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65d0066af0 con 0x7f65d00fe810 2024-04-25T15:45:53.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.062+0000 7f65beffd700 1 -- 172.21.15.110:0/2002487957 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f65c4013420 con 0x7f65d00fe810 2024-04-25T15:45:53.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.062+0000 7f65beffd700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f65b80692a0 0x7f65b806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:53.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.062+0000 7f65beffd700 1 -- 172.21.15.110:0/2002487957 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f65c4088100 con 0x7f65d00fe810 2024-04-25T15:45:53.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.062+0000 7f65ce59c700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f65b80692a0 0x7f65b806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:53.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.063+0000 7f65ce59c700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f65b80692a0 0x7f65b806b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65c0004f40 tx=0x7f65c000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:53.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.067+0000 7f65beffd700 1 -- 172.21.15.110:0/2002487957 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f65c40582b0 con 0x7f65d00fe810 2024-04-25T15:45:53.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.255+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65d0109e30 con 0x7f65b80692a0 2024-04-25T15:45:53.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.257+0000 7f65beffd700 1 -- 172.21.15.110:0/2002487957 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27877 (secure 0 0 0) 0x7f65d0109e30 con 0x7f65b80692a0 2024-04-25T15:45:53.258 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:53.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f65b80692a0 msgr2=0x7f65b806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:53.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f65b80692a0 0x7f65b806b750 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65c0004f40 tx=0x7f65c000b040 comp rx=0 tx=0).stop 2024-04-25T15:45:53.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 msgr2=0x7f65d00fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:53.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 0x7f65d00fbca0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65c4009380 tx=0x7f65c4009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 shutdown_connections 2024-04-25T15:45:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f65b80692a0 0x7f65b806b750 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65d00fde70 0x7f65d00fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 --2- 172.21.15.110:0/2002487957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65d00fe810 0x7f65d00fbca0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 >> 172.21.15.110:0/2002487957 conn(0x7f65d00f7860 msgr2=0x7f65d01016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.259+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 shutdown_connections 2024-04-25T15:45:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:53.260+0000 7f65d4980700 1 -- 172.21.15.110:0/2002487957 wait complete. 2024-04-25T15:45:53.261 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-04-25T15:45:53.917 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":154,"stamp":"2024-04-25T15:45:52.273321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001954"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T15:45:01.668709+0000","last_change":"2024-04-25T15:44:37.257783+0000","last_active":"2024-04-25T15:45:01.668709+0000","last_peered":"2024-04-25T15:45:01.668709+0000","last_clean":"2024-04-25T15:45:01.668709+0000","last_became_active":"2024-04-25T15:44:37.257426+0000","last_became_peered":"2024-04-25T15:44:37.257426+0000","last_unstale":"2024-04-25T15:45:01.668709+0000","last_undegraded":"2024-04-25T15:45:01.668709+0000","last_fullsized":"2024-04-25T15:45:01.668709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:43:50.897010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:43:50.897010+0000","last_clean_scrub_stamp":"2024-04-25T15:43:50.897010+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:38.211423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.441}]},{"osd":2,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.64700000000000002}]},{"osd":1,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.72499999999999998}]},{"osd":2,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.65700000000000003}]},{"osd":4,"last update":"Thu Apr 25 15:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.308},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.42599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 15:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.248}]},{"osd":1,"last update":"Thu Apr 25 15:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.28100000000000003}]},{"osd":3,"last update":"Thu Apr 25 15:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 15:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.21199999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Apr 25 15:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Apr 25 15:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 15:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.318,"15min":0.315},"min":{"1min":0.22900000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.307},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.33000000000000002}]},{"osd":2,"last update":"Thu Apr 25 15:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.49299999999999999}]},{"osd":4,"last update":"Thu Apr 25 15:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.503}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:45:53.918 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T15:45:54.041 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:53 smithi193 ceph-mon[33747]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:53 smithi110 ceph-mon[26344]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:54.181 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:54.998 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:54 smithi110 ceph-mon[26344]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:45:55.220 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:54 smithi193 ceph-mon[33747]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:45:55.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 -- 172.21.15.110:0/3733032807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 msgr2=0x7f37800fbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 --2- 172.21.15.110:0/3733032807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 0x7f37800fbd50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3770004940 tx=0x7f3770009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 -- 172.21.15.110:0/3733032807 shutdown_connections 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 --2- 172.21.15.110:0/3733032807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 0x7f37800fbd50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 --2- 172.21.15.110:0/3733032807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3780101d40 0x7f3780102110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 -- 172.21.15.110:0/3733032807 >> 172.21.15.110:0/3733032807 conn(0x7f37800f78d0 msgr2=0x7f37800f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 -- 172.21.15.110:0/3733032807 shutdown_connections 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 -- 172.21.15.110:0/3733032807 wait complete. 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 Processor -- start 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 -- start start 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3780101d40 0x7f3780112910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 0x7f3780112e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:55.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3780113530 con 0x7f37801026e0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.916+0000 7f378780e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378010caa0 con 0x7f3780101d40 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3784da9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 0x7f3780112e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3784da9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 0x7f3780112e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41238/0 (socket says 172.21.15.110:41238) 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3784da9700 1 -- 172.21.15.110:0/4027483179 learned_addr learned my addr 172.21.15.110:0/4027483179 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f37855aa700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3780101d40 0x7f3780112910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3784da9700 1 -- 172.21.15.110:0/4027483179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3780101d40 msgr2=0x7f3780112910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3784da9700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3780101d40 0x7f3780112910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3784da9700 1 -- 172.21.15.110:0/4027483179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3770005040 con 0x7f37801026e0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3784da9700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 0x7f3780112e50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3770009100 tx=0x7f3770009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3775ffb700 1 -- 172.21.15.110:0/4027483179 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f377000e040 con 0x7f37801026e0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3775ffb700 1 -- 172.21.15.110:0/4027483179 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3770013420 con 0x7f37801026e0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f378010cd20 con 0x7f37801026e0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f3775ffb700 1 -- 172.21.15.110:0/4027483179 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f377001c610 con 0x7f37801026e0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.917+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378010d260 con 0x7f37801026e0 2024-04-25T15:45:55.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.918+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37800ff050 con 0x7f37801026e0 2024-04-25T15:45:55.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.919+0000 7f3775ffb700 1 -- 172.21.15.110:0/4027483179 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3770013590 con 0x7f37801026e0 2024-04-25T15:45:55.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.920+0000 7f3775ffb700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f376c0692f0 0x7f376c06b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:55.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.920+0000 7f3775ffb700 1 -- 172.21.15.110:0/4027483179 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f3770088180 con 0x7f37801026e0 2024-04-25T15:45:55.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.920+0000 7f37855aa700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f376c0692f0 0x7f376c06b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:55.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.920+0000 7f37855aa700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f376c0692f0 0x7f376c06b7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f377c006fd0 tx=0x7f377c008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:55.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:55.924+0000 7f3775ffb700 1 -- 172.21.15.110:0/4027483179 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f37700581e0 con 0x7f37801026e0 2024-04-25T15:45:56.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.103+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f378010df60 con 0x7f376c0692f0 2024-04-25T15:45:56.104 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:55 smithi110 ceph-mon[26344]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:56.104 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:45:56.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.105+0000 7f3775ffb700 1 -- 172.21.15.110:0/4027483179 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27889 (secure 0 0 0) 0x7f378010df60 con 0x7f376c0692f0 2024-04-25T15:45:56.106 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f376c0692f0 msgr2=0x7f376c06b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f376c0692f0 0x7f376c06b7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f377c006fd0 tx=0x7f377c008040 comp rx=0 tx=0).stop 2024-04-25T15:45:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 msgr2=0x7f3780112e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 0x7f3780112e50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f3770009100 tx=0x7f3770009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 shutdown_connections 2024-04-25T15:45:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f376c0692f0 0x7f376c06b7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3780101d40 0x7f3780112910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:56.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 --2- 172.21.15.110:0/4027483179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37801026e0 0x7f3780112e50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:56.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 >> 172.21.15.110:0/4027483179 conn(0x7f37800f78d0 msgr2=0x7f37801062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:56.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 shutdown_connections 2024-04-25T15:45:56.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:56.107+0000 7f378780e700 1 -- 172.21.15.110:0/4027483179 wait complete. 2024-04-25T15:45:56.109 INFO:teuthology.orchestra.run.smithi110.stderr:dumped all 2024-04-25T15:45:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:55 smithi193 ceph-mon[33747]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:45:56.892 INFO:teuthology.orchestra.run.smithi110.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-04-25T15:45:54.273719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":1788024,"kb_used_data":3876,"kb_used_omap":9,"kb_used_meta":1784054,"kb_avail":560682888,"statfs":{"total":575970213888,"available":574139277312,"internally_reserved":0,"allocated":3969024,"data_stored":2606262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":10224,"internal_metadata":1826871312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002005"},"pg_stats":[{"pgid":"1.0","version":"22'117","reported_seq":189,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T15:45:01.668709+0000","last_change":"2024-04-25T15:44:37.257783+0000","last_active":"2024-04-25T15:45:01.668709+0000","last_peered":"2024-04-25T15:45:01.668709+0000","last_clean":"2024-04-25T15:45:01.668709+0000","last_became_active":"2024-04-25T15:44:37.257426+0000","last_became_peered":"2024-04-25T15:44:37.257426+0000","last_unstale":"2024-04-25T15:45:01.668709+0000","last_undegraded":"2024-04-25T15:45:01.668709+0000","last_fullsized":"2024-04-25T15:45:01.668709+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T15:43:50.897010+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T15:43:50.897010+0000","last_clean_scrub_stamp":"2024-04-25T15:43:50.897010+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:38.211423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297712,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447440,"statfs":{"total":95995035648,"available":95690178560,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.441}]},{"osd":2,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.41399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]},{"osd":3,"up_from":26,"seq":111669149720,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298296,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446856,"statfs":{"total":95995035648,"available":95689580544,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.504}]},{"osd":1,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.51800000000000002}]},{"osd":2,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.53500000000000003}]},{"osd":4,"last update":"Thu Apr 25 15:45:38 2024","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.40200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":297720,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447432,"statfs":{"total":95995035648,"available":95690170368,"internally_reserved":0,"allocated":364544,"data_stored":139193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 15:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"last":0.248}]},{"osd":1,"last update":"Thu Apr 25 15:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.502,"5min":0.502,"15min":0.502},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.28100000000000003}]},{"osd":3,"last update":"Thu Apr 25 15:45:15 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Thu Apr 25 15:45:41 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Apr 25 15:45:27 2024","interfaces":[{"interface":"back","average":{"1min":0.33100000000000002,"5min":0.32100000000000001,"15min":0.31900000000000001},"min":{"1min":0.21199999999999999,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.496,"5min":0.496,"15min":0.496},"last":0.24199999999999999},{"interface":"front","average":{"1min":0.34899999999999998,"5min":0.32700000000000001,"15min":0.32300000000000001},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Apr 25 15:44:51 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"last":0.32200000000000001}]},{"osd":3,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.52000000000000002}]},{"osd":4,"last update":"Thu Apr 25 15:45:37 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]}]},{"osd":1,"up_from":14,"seq":60129542178,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298292,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446860,"statfs":{"total":95995035648,"available":95689584640,"internally_reserved":0,"allocated":958464,"data_stored":729561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Apr 25 15:45:28 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.318,"15min":0.315},"min":{"1min":0.22900000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.34200000000000003,"15min":0.34000000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.24299999999999999}]},{"osd":2,"last update":"Thu Apr 25 15:44:50 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.307}]},{"osd":3,"last update":"Thu Apr 25 15:45:14 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81200000000000006,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.5}]},{"osd":4,"last update":"Thu Apr 25 15:45:39 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.46600000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T15:45:56.893 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T15:45:56.893 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T15:45:56.893 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T15:45:56.893 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T15:45:57.146 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:45:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:57 smithi110 ceph-mon[26344]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:45:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:57 smithi110 ceph-mon[26344]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:58.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:57 smithi193 ceph-mon[33747]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T15:45:58.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:57 smithi193 ceph-mon[33747]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:45:58.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 -- 172.21.15.110:0/1710680793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540103780 msgr2=0x7f85400fcdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 --2- 172.21.15.110:0/1710680793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540103780 0x7f85400fcdf0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f8530004940 tx=0x7f8530009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 -- 172.21.15.110:0/1710680793 shutdown_connections 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 --2- 172.21.15.110:0/1710680793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540103780 0x7f85400fcdf0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 --2- 172.21.15.110:0/1710680793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8540102de0 0x7f85401031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 -- 172.21.15.110:0/1710680793 >> 172.21.15.110:0/1710680793 conn(0x7f85400f8990 msgr2=0x7f85400fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 -- 172.21.15.110:0/1710680793 shutdown_connections 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 -- 172.21.15.110:0/1710680793 wait complete. 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 Processor -- start 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.804+0000 7f854728d700 1 -- start start 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f854728d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540102de0 0x7f8540070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f854728d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8540103780 0x7f85400707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f854728d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85400721b0 con 0x7f8540102de0 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f854728d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8540070ce0 con 0x7f8540103780 2024-04-25T15:45:58.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8545029700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540102de0 0x7f8540070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8545029700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540102de0 0x7f8540070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41248/0 (socket says 172.21.15.110:41248) 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8545029700 1 -- 172.21.15.110:0/1550327700 learned_addr learned my addr 172.21.15.110:0/1550327700 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8544828700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8540103780 0x7f85400707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8545029700 1 -- 172.21.15.110:0/1550327700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8540103780 msgr2=0x7f85400707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8545029700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8540103780 0x7f85400707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8545029700 1 -- 172.21.15.110:0/1550327700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530005040 con 0x7f8540102de0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8544828700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8540103780 0x7f85400707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8545029700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540102de0 0x7f8540070260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f853c002a50 tx=0x7f853c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8535ffb700 1 -- 172.21.15.110:0/1550327700 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f853c007cc0 con 0x7f8540102de0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8535ffb700 1 -- 172.21.15.110:0/1550327700 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f853c00f040 con 0x7f8540102de0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8540070fc0 con 0x7f8540102de0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.805+0000 7f8535ffb700 1 -- 172.21.15.110:0/1550327700 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f853c009640 con 0x7f8540102de0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.806+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f854011a140 con 0x7f8540102de0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.806+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85400fe530 con 0x7f8540102de0 2024-04-25T15:45:58.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.807+0000 7f8535ffb700 1 -- 172.21.15.110:0/1550327700 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f853c0097a0 con 0x7f8540102de0 2024-04-25T15:45:58.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.807+0000 7f8535ffb700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f852c0692a0 0x7f852c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:45:58.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.807+0000 7f8535ffb700 1 -- 172.21.15.110:0/1550327700 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f853c086c70 con 0x7f8540102de0 2024-04-25T15:45:58.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.807+0000 7f8544828700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f852c0692a0 0x7f852c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:45:58.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.808+0000 7f8544828700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f852c0692a0 0x7f852c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f853000b3b0 tx=0x7f8530009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:45:58.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:58.811+0000 7f8535ffb700 1 -- 172.21.15.110:0/1550327700 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f853c053260 con 0x7f8540102de0 2024-04-25T15:45:59.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.039+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f854004c710 con 0x7f8540102de0 2024-04-25T15:45:59.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.039+0000 7f8535ffb700 1 -- 172.21.15.110:0/1550327700 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f853c056880 con 0x7f8540102de0 2024-04-25T15:45:59.040 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:45:59.040 INFO:teuthology.orchestra.run.smithi110.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f852c0692a0 msgr2=0x7f852c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f852c0692a0 0x7f852c06b750 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f853000b3b0 tx=0x7f8530009e30 comp rx=0 tx=0).stop 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540102de0 msgr2=0x7f8540070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540102de0 0x7f8540070260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f853c002a50 tx=0x7f853c0046b0 comp rx=0 tx=0).stop 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 shutdown_connections 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8540102de0 0x7f8540070260 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f852c0692a0 0x7f852c06b750 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 --2- 172.21.15.110:0/1550327700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8540103780 0x7f85400707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 >> 172.21.15.110:0/1550327700 conn(0x7f85400f8990 msgr2=0x7f85400f9970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 shutdown_connections 2024-04-25T15:45:59.042 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:45:59.041+0000 7f854728d700 1 -- 172.21.15.110:0/1550327700 wait complete. 2024-04-25T15:45:59.838 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T15:45:59.839 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T15:45:59.839 INFO:teuthology.run_tasks:Running task print... 2024-04-25T15:45:59.850 INFO:teuthology.task.print:**** done end installing quincy cephadm ... 2024-04-25T15:45:59.851 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:45:59.865 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:45:59.866 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-25T15:46:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:59 smithi110 ceph-mon[26344]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:45:59 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1550327700' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T15:46:00.119 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:59 smithi193 ceph-mon[33747]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:45:59 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1550327700' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T15:46:01.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.875+0000 7f1f5f057700 1 -- 172.21.15.110:0/3075301408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fcf70 msgr2=0x7f1f580fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:01.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.875+0000 7f1f5f057700 1 --2- 172.21.15.110:0/3075301408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fcf70 0x7f1f580fd3e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1f44004940 tx=0x7f1f44009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:01.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.875+0000 7f1f5f057700 1 -- 172.21.15.110:0/3075301408 shutdown_connections 2024-04-25T15:46:01.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.875+0000 7f1f5f057700 1 --2- 172.21.15.110:0/3075301408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fcf70 0x7f1f580fd3e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:01.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.875+0000 7f1f5f057700 1 --2- 172.21.15.110:0/3075301408 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f580fbcc0 0x7f1f580fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:01.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.875+0000 7f1f5f057700 1 -- 172.21.15.110:0/3075301408 >> 172.21.15.110:0/3075301408 conn(0x7f1f580f7440 msgr2=0x7f1f580f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.875+0000 7f1f5f057700 1 -- 172.21.15.110:0/3075301408 shutdown_connections 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.875+0000 7f1f5f057700 1 -- 172.21.15.110:0/3075301408 wait complete. 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5f057700 1 Processor -- start 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5f057700 1 -- start start 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5f057700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fbcc0 0x7f1f5810c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5f057700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f580fcf70 0x7f1f5810cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5f057700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5810d3e0 con 0x7f1f580fbcc0 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5f057700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f58111d80 con 0x7f1f580fcf70 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5cdf3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fbcc0 0x7f1f5810c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5cdf3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fbcc0 0x7f1f5810c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46012/0 (socket says 172.21.15.110:46012) 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.876+0000 7f1f5cdf3700 1 -- 172.21.15.110:0/3709710769 learned_addr learned my addr 172.21.15.110:0/3709710769 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f57fff700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f580fcf70 0x7f1f5810cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f5cdf3700 1 -- 172.21.15.110:0/3709710769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f580fcf70 msgr2=0x7f1f5810cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:01.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f5cdf3700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f580fcf70 0x7f1f5810cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f5cdf3700 1 -- 172.21.15.110:0/3709710769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f44005040 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f57fff700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f580fcf70 0x7f1f5810cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f5cdf3700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fbcc0 0x7f1f5810c810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b790 tx=0x7f1f4c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f557fa700 1 -- 172.21.15.110:0/3709710769 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f4c00f770 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f557fa700 1 -- 172.21.15.110:0/3709710769 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1f4c00fda0 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f58111fe0 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f557fa700 1 -- 172.21.15.110:0/3709710769 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f4c00d310 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.877+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f581124c0 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.878+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5804c710 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.878+0000 7f1f557fa700 1 -- 172.21.15.110:0/3709710769 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1f4c010420 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.879+0000 7f1f557fa700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1f480692a0 0x7f1f4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.879+0000 7f1f57fff700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1f480692a0 0x7f1f4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.879+0000 7f1f557fa700 1 -- 172.21.15.110:0/3709710769 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1f4c086e50 con 0x7f1f580fbcc0 2024-04-25T15:46:01.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.879+0000 7f1f57fff700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1f480692a0 0x7f1f4806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400b970 tx=0x7f1f44009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:01.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:01.883+0000 7f1f557fa700 1 -- 172.21.15.110:0/3709710769 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1f4c056f50 con 0x7f1f580fbcc0 2024-04-25T15:46:02.070 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.069+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f1f58104830 con 0x7f1f580fbcc0 2024-04-25T15:46:02.070 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:01 smithi110 ceph-mon[26344]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:02.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.073+0000 7f1f557fa700 1 -- 172.21.15.110:0/3709710769 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1f4c056ae0 con 0x7f1f580fbcc0 2024-04-25T15:46:02.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.075+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1f480692a0 msgr2=0x7f1f4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:02.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.075+0000 7f1f5f057700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1f480692a0 0x7f1f4806b750 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400b970 tx=0x7f1f44009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.075+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fbcc0 msgr2=0x7f1f5810c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.075+0000 7f1f5f057700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fbcc0 0x7f1f5810c810 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b790 tx=0x7f1f4c009250 comp rx=0 tx=0).stop 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.076+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 shutdown_connections 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.076+0000 7f1f5f057700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1f580fbcc0 0x7f1f5810c810 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.076+0000 7f1f5f057700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1f480692a0 0x7f1f4806b750 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.076+0000 7f1f5f057700 1 --2- 172.21.15.110:0/3709710769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f580fcf70 0x7f1f5810cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.076+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 >> 172.21.15.110:0/3709710769 conn(0x7f1f580f7440 msgr2=0x7f1f580f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.076+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 shutdown_connections 2024-04-25T15:46:02.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:02.076+0000 7f1f5f057700 1 -- 172.21.15.110:0/3709710769 wait complete. 2024-04-25T15:46:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:01 smithi193 ceph-mon[33747]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:02.871 INFO:teuthology.run_tasks:Running task print... 2024-04-25T15:46:02.880 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-25T15:46:02.880 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:46:02.892 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:46:02.893 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T15:46:03.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:03 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3709710769' entity='client.admin' 2024-04-25T15:46:03.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:46:03.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:03 smithi110 ceph-mon[26344]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:03.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:03.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:46:03.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:03 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:03.142 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:03 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3709710769' entity='client.admin' 2024-04-25T15:46:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:46:03.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:03 smithi193 ceph-mon[33747]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:03.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:03.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:46:03.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:03 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:04.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.836+0000 7f5a53793700 1 -- 172.21.15.110:0/2207208399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 msgr2=0x7f5a4c0fd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.836+0000 7f5a53793700 1 --2- 172.21.15.110:0/2207208399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 0x7f5a4c0fd410 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c004940 tx=0x7f5a3c009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.836+0000 7f5a53793700 1 -- 172.21.15.110:0/2207208399 shutdown_connections 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.836+0000 7f5a53793700 1 --2- 172.21.15.110:0/2207208399 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 0x7f5a4c0fd410 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.836+0000 7f5a53793700 1 --2- 172.21.15.110:0/2207208399 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a4c0fbcf0 0x7f5a4c0fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.836+0000 7f5a53793700 1 -- 172.21.15.110:0/2207208399 >> 172.21.15.110:0/2207208399 conn(0x7f5a4c0f7450 msgr2=0x7f5a4c0f9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.836+0000 7f5a53793700 1 -- 172.21.15.110:0/2207208399 shutdown_connections 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.836+0000 7f5a53793700 1 -- 172.21.15.110:0/2207208399 wait complete. 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a53793700 1 Processor -- start 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a53793700 1 -- start start 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a53793700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a4c0fbcf0 0x7f5a4c0ffc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a53793700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 0x7f5a4c0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:04.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a53793700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0fe820 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a53793700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4c0fe990 con 0x7f5a4c0fbcf0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a50d2e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 0x7f5a4c0fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a50d2e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 0x7f5a4c0fe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46028/0 (socket says 172.21.15.110:46028) 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a50d2e700 1 -- 172.21.15.110:0/405318653 learned_addr learned my addr 172.21.15.110:0/405318653 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a5152f700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a4c0fbcf0 0x7f5a4c0ffc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a50d2e700 1 -- 172.21.15.110:0/405318653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a4c0fbcf0 msgr2=0x7f5a4c0ffc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a50d2e700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a4c0fbcf0 0x7f5a4c0ffc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a50d2e700 1 -- 172.21.15.110:0/405318653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c005040 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a50d2e700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 0x7f5a4c0fe2e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009150 tx=0x7f5a3c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a41ffb700 1 -- 172.21.15.110:0/405318653 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3c00e040 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4c0fec10 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a41ffb700 1 -- 172.21.15.110:0/405318653 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a3c00bcc0 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.837+0000 7f5a41ffb700 1 -- 172.21.15.110:0/405318653 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3c01c630 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.838+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a4c115a90 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.838+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a4c04c710 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.839+0000 7f5a41ffb700 1 -- 172.21.15.110:0/405318653 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5a3c013980 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.839+0000 7f5a41ffb700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5a380692a0 0x7f5a3806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:04.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.839+0000 7f5a41ffb700 1 -- 172.21.15.110:0/405318653 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f5a3c088200 con 0x7f5a4c0fcfa0 2024-04-25T15:46:04.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.839+0000 7f5a5152f700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5a380692a0 0x7f5a3806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:04.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.840+0000 7f5a5152f700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5a380692a0 0x7f5a3806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a48006fd0 tx=0x7f5a48008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:04.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:04.844+0000 7f5a41ffb700 1 -- 172.21.15.110:0/405318653 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5a3c058300 con 0x7f5a4c0fcfa0 2024-04-25T15:46:05.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.042+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a4c05e3a0 con 0x7f5a380692a0 2024-04-25T15:46:05.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.043+0000 7f5a41ffb700 1 -- 172.21.15.110:0/405318653 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f5a4c05e3a0 con 0x7f5a380692a0 2024-04-25T15:46:05.044 INFO:teuthology.orchestra.run.smithi110.stdout:Backend: cephadm 2024-04-25T15:46:05.044 INFO:teuthology.orchestra.run.smithi110.stdout:Available: Yes 2024-04-25T15:46:05.044 INFO:teuthology.orchestra.run.smithi110.stdout:Paused: No 2024-04-25T15:46:05.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5a380692a0 msgr2=0x7f5a3806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:05.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5a380692a0 0x7f5a3806b750 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5a48006fd0 tx=0x7f5a48008040 comp rx=0 tx=0).stop 2024-04-25T15:46:05.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 msgr2=0x7f5a4c0fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:05.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 0x7f5a4c0fe2e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c009150 tx=0x7f5a3c009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:05.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 shutdown_connections 2024-04-25T15:46:05.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5a380692a0 0x7f5a3806b750 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:05.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a4c0fbcf0 0x7f5a4c0ffc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:05.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 --2- 172.21.15.110:0/405318653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a4c0fcfa0 0x7f5a4c0fe2e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:05.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.045+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 >> 172.21.15.110:0/405318653 conn(0x7f5a4c0f7450 msgr2=0x7f5a4c066800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:05.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.046+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 shutdown_connections 2024-04-25T15:46:05.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:05.046+0000 7f5a53793700 1 -- 172.21.15.110:0/405318653 wait complete. 2024-04-25T15:46:05.601 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:05 smithi110 ceph-mon[26344]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:05.625 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T15:46:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:05 smithi193 ceph-mon[33747]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:05.876 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:06.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:06 smithi110 ceph-mon[26344]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:06.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:06 smithi193 ceph-mon[33747]: from='client.14440 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:07.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.475+0000 7f0b4de44700 1 -- 172.21.15.110:0/1553232877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fce60 msgr2=0x7f0b480fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:07.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.475+0000 7f0b4de44700 1 --2- 172.21.15.110:0/1553232877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fce60 0x7f0b480fd270 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b38004940 tx=0x7f0b38009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:07.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.475+0000 7f0b4de44700 1 -- 172.21.15.110:0/1553232877 shutdown_connections 2024-04-25T15:46:07.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.475+0000 7f0b4de44700 1 --2- 172.21.15.110:0/1553232877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b480fdfd0 0x7f0b480fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:07.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.475+0000 7f0b4de44700 1 --2- 172.21.15.110:0/1553232877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fce60 0x7f0b480fd270 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:07.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.475+0000 7f0b4de44700 1 -- 172.21.15.110:0/1553232877 >> 172.21.15.110:0/1553232877 conn(0x7f0b480f8540 msgr2=0x7f0b480fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:07.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.476+0000 7f0b4de44700 1 -- 172.21.15.110:0/1553232877 shutdown_connections 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.476+0000 7f0b4de44700 1 -- 172.21.15.110:0/1553232877 wait complete. 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.476+0000 7f0b4de44700 1 Processor -- start 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.476+0000 7f0b4de44700 1 -- start start 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.476+0000 7f0b4de44700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b480fce60 0x7f0b4810d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.476+0000 7f0b4de44700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fdfd0 0x7f0b4810de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.476+0000 7f0b4de44700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4810e450 con 0x7f0b480fdfd0 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.476+0000 7f0b4de44700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4810e5c0 con 0x7f0b480fce60 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b3effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fdfd0 0x7f0b4810de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b3effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fdfd0 0x7f0b4810de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:46046/0 (socket says 172.21.15.110:46046) 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b3effd700 1 -- 172.21.15.110:0/4109103005 learned_addr learned my addr 172.21.15.110:0/4109103005 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b477fe700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b480fce60 0x7f0b4810d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b3effd700 1 -- 172.21.15.110:0/4109103005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b480fce60 msgr2=0x7f0b4810d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b3effd700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b480fce60 0x7f0b4810d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:07.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b3effd700 1 -- 172.21.15.110:0/4109103005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b38005040 con 0x7f0b480fdfd0 2024-04-25T15:46:07.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b3effd700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fdfd0 0x7f0b4810de50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000c830 tx=0x7f0b30004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:07.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b44ff9700 1 -- 172.21.15.110:0/4109103005 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b300106f0 con 0x7f0b480fdfd0 2024-04-25T15:46:07.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b44ff9700 1 -- 172.21.15.110:0/4109103005 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b30010d20 con 0x7f0b480fdfd0 2024-04-25T15:46:07.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b48113020 con 0x7f0b480fdfd0 2024-04-25T15:46:07.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b44ff9700 1 -- 172.21.15.110:0/4109103005 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b300113a0 con 0x7f0b480fdfd0 2024-04-25T15:46:07.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.477+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b481135b0 con 0x7f0b480fdfd0 2024-04-25T15:46:07.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.478+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4804c710 con 0x7f0b480fdfd0 2024-04-25T15:46:07.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.479+0000 7f0b44ff9700 1 -- 172.21.15.110:0/4109103005 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0b30012400 con 0x7f0b480fdfd0 2024-04-25T15:46:07.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.479+0000 7f0b44ff9700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0b340692a0 0x7f0b3406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:07.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.479+0000 7f0b44ff9700 1 -- 172.21.15.110:0/4109103005 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f0b30086e20 con 0x7f0b480fdfd0 2024-04-25T15:46:07.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.479+0000 7f0b477fe700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0b340692a0 0x7f0b3406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:07.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.480+0000 7f0b477fe700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0b340692a0 0x7f0b3406b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b3800b970 tx=0x7f0b38009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:07.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.483+0000 7f0b44ff9700 1 -- 172.21.15.110:0/4109103005 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0b30056f20 con 0x7f0b480fdfd0 2024-04-25T15:46:07.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:07 smithi110 ceph-mon[26344]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:07.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.680+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b48102360 con 0x7f0b340692a0 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.688+0000 7f0b44ff9700 1 -- 172.21.15.110:0/4109103005 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2682 (secure 0 0 0) 0x7f0b48102360 con 0x7f0b340692a0 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (3m) 2m ago 5m 25.9M - 0.25.0 c8568f914cd2 b995d416435c 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (5m) 2m ago 5m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (4m) 68s ago 4m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (3m) 2m ago 4m 82.5M - 9.4.7 954c08fa6188 6cf7f080ae79 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:9283 running (6m) 2m ago 6m 469M - 17.2.7-902-gc61472c8 0de407c1b3af c623b709ef43 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (4m) 68s ago 4m 430M - 17.2.7-902-gc61472c8 0de407c1b3af 30a2f136ed24 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (6m) 2m ago 6m 41.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (4m) 68s ago 4m 40.5M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:46:07.689 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (5m) 2m ago 5m 16.0M - 1.5.0 0da6a335fe13 5f8f351cfd11 2024-04-25T15:46:07.690 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (4m) 68s ago 4m 15.4M - 1.5.0 0da6a335fe13 13afa4d53352 2024-04-25T15:46:07.690 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (3m) 2m ago 3m 35.9M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:46:07.690 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (2m) 2m ago 2m 38.9M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:46:07.690 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (2m) 2m ago 2m 10.7M 4288M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:46:07.690 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (2m) 68s ago 2m 41.3M 5312M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:46:07.690 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (97s) 68s ago 96s 38.6M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:46:07.690 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (72s) 68s ago 72s 13.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:46:07.690 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (3m) 2m ago 4m 34.7M - 2.43.0 a07b618ecd1d ce7fd55a0368 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0b340692a0 msgr2=0x7f0b3406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0b340692a0 0x7f0b3406b750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0b3800b970 tx=0x7f0b38009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fdfd0 msgr2=0x7f0b4810de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fdfd0 0x7f0b4810de50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b3000c830 tx=0x7f0b30004470 comp rx=0 tx=0).stop 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 shutdown_connections 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0b340692a0 0x7f0b3406b750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b480fce60 0x7f0b4810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 --2- 172.21.15.110:0/4109103005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b480fdfd0 0x7f0b4810de50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 >> 172.21.15.110:0/4109103005 conn(0x7f0b480f8540 msgr2=0x7f0b480f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 shutdown_connections 2024-04-25T15:46:07.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:07.690+0000 7f0b4de44700 1 -- 172.21.15.110:0/4109103005 wait complete. 2024-04-25T15:46:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:07 smithi193 ceph-mon[33747]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:08.333 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T15:46:08.582 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:08.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:08 smithi193 ceph-mon[33747]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:08.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:08 smithi110 ceph-mon[26344]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:09.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:09 smithi110 ceph-mon[26344]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:09.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:09 smithi193 ceph-mon[33747]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:10.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.158+0000 7f1e8e01a700 1 -- 172.21.15.110:0/3553476882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88070830 msgr2=0x7f1e88070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.158+0000 7f1e8e01a700 1 --2- 172.21.15.110:0/3553476882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88070830 0x7f1e88070ca0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c004940 tx=0x7f1e7c009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.158+0000 7f1e8e01a700 1 -- 172.21.15.110:0/3553476882 shutdown_connections 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.158+0000 7f1e8e01a700 1 --2- 172.21.15.110:0/3553476882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88070830 0x7f1e88070ca0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.158+0000 7f1e8e01a700 1 --2- 172.21.15.110:0/3553476882 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88071de0 0x7f1e880702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.158+0000 7f1e8e01a700 1 -- 172.21.15.110:0/3553476882 >> 172.21.15.110:0/3553476882 conn(0x7f1e880f8520 msgr2=0x7f1e880fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.158+0000 7f1e8e01a700 1 -- 172.21.15.110:0/3553476882 shutdown_connections 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.158+0000 7f1e8e01a700 1 -- 172.21.15.110:0/3553476882 wait complete. 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e8e01a700 1 Processor -- start 2024-04-25T15:46:10.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e8e01a700 1 -- start start 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e8e01a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88070830 0x7f1e8810fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e8e01a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88071de0 0x7f1e88110090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e8e01a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e88110720 con 0x7f1e88071de0 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e8e01a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e881150c0 con 0x7f1e88070830 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e86ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88071de0 0x7f1e88110090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e86ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88071de0 0x7f1e88110090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49340/0 (socket says 172.21.15.110:49340) 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e86ffd700 1 -- 172.21.15.110:0/100877643 learned_addr learned my addr 172.21.15.110:0/100877643 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.159+0000 7f1e877fe700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88070830 0x7f1e8810fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e86ffd700 1 -- 172.21.15.110:0/100877643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88070830 msgr2=0x7f1e8810fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e86ffd700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88070830 0x7f1e8810fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:10.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e86ffd700 1 -- 172.21.15.110:0/100877643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e7c005040 con 0x7f1e88071de0 2024-04-25T15:46:10.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e86ffd700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88071de0 0x7f1e88110090 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009150 tx=0x7f1e7c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:10.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e6ffff700 1 -- 172.21.15.110:0/100877643 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e7c00e040 con 0x7f1e88071de0 2024-04-25T15:46:10.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e881152c0 con 0x7f1e88071de0 2024-04-25T15:46:10.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e6ffff700 1 -- 172.21.15.110:0/100877643 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e7c004290 con 0x7f1e88071de0 2024-04-25T15:46:10.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e6ffff700 1 -- 172.21.15.110:0/100877643 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e7c01c630 con 0x7f1e88071de0 2024-04-25T15:46:10.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.160+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e88115830 con 0x7f1e88071de0 2024-04-25T15:46:10.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.161+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8804c710 con 0x7f1e88071de0 2024-04-25T15:46:10.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.161+0000 7f1e6ffff700 1 -- 172.21.15.110:0/100877643 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f1e7c00bbf0 con 0x7f1e88071de0 2024-04-25T15:46:10.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.162+0000 7f1e6ffff700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1e700692a0 0x7f1e7006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:10.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.162+0000 7f1e6ffff700 1 -- 172.21.15.110:0/100877643 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f1e7c087b20 con 0x7f1e88071de0 2024-04-25T15:46:10.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.162+0000 7f1e877fe700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1e700692a0 0x7f1e7006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:10.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.162+0000 7f1e877fe700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1e700692a0 0x7f1e7006b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e88071b20 tx=0x7f1e780091a0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:10.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.166+0000 7f1e6ffff700 1 -- 172.21.15.110:0/100877643 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f1e7c057c20 con 0x7f1e88071de0 2024-04-25T15:46:10.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.363+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e88103dc0 con 0x7f1e700692a0 2024-04-25T15:46:10.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.367+0000 7f1e6ffff700 1 -- 172.21.15.110:0/100877643 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1179 (secure 0 0 0) 0x7f1e88103dc0 con 0x7f1e700692a0 2024-04-25T15:46:10.368 INFO:teuthology.orchestra.run.smithi110.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T15:46:10.368 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-25T15:46:10.368 INFO:teuthology.orchestra.run.smithi110.stdout:crash 2/2 2m ago 5m * 2024-04-25T15:46:10.368 INFO:teuthology.orchestra.run.smithi110.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-25T15:46:10.369 INFO:teuthology.orchestra.run.smithi110.stdout:mgr 2/2 2m ago 5m count:2 2024-04-25T15:46:10.369 INFO:teuthology.orchestra.run.smithi110.stdout:mon 2/2 2m ago 5m smithi110:172.21.15.110=smithi110;smithi193:172.21.15.193=smithi193;count:2 2024-04-25T15:46:10.369 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-25T15:46:10.369 INFO:teuthology.orchestra.run.smithi110.stdout:osd 6 2m ago - 2024-04-25T15:46:10.369 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-04-25T15:46:10.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1e700692a0 msgr2=0x7f1e7006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:10.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1e700692a0 0x7f1e7006b750 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1e88071b20 tx=0x7f1e780091a0 comp rx=0 tx=0).stop 2024-04-25T15:46:10.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88071de0 msgr2=0x7f1e88110090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:10.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88071de0 0x7f1e88110090 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1e7c009150 tx=0x7f1e7c009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:10.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 shutdown_connections 2024-04-25T15:46:10.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f1e700692a0 0x7f1e7006b750 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:10.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e88070830 0x7f1e8810fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:10.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 --2- 172.21.15.110:0/100877643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1e88071de0 0x7f1e88110090 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:10.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.369+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 >> 172.21.15.110:0/100877643 conn(0x7f1e880f8520 msgr2=0x7f1e880f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:10.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.370+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 shutdown_connections 2024-04-25T15:46:10.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:10.370+0000 7f1e8e01a700 1 -- 172.21.15.110:0/100877643 wait complete. 2024-04-25T15:46:10.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:10 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:46:10.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:10 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:46:10.974 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T15:46:11.228 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:11.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:11 smithi110 ceph-mon[26344]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:11.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:11 smithi110 ceph-mon[26344]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:11.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:11 smithi193 ceph-mon[33747]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:11.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:11 smithi193 ceph-mon[33747]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:12.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.813+0000 7f4db2bd0700 1 -- 172.21.15.110:0/3958501044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 msgr2=0x7f4dac102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:12.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.813+0000 7f4db2bd0700 1 --2- 172.21.15.110:0/3958501044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 0x7f4dac102350 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4da0004940 tx=0x7f4da0009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:12.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.813+0000 7f4db2bd0700 1 -- 172.21.15.110:0/3958501044 shutdown_connections 2024-04-25T15:46:12.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.813+0000 7f4db2bd0700 1 --2- 172.21.15.110:0/3958501044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 0x7f4dac102350 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:12.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.813+0000 7f4db2bd0700 1 --2- 172.21.15.110:0/3958501044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dac066eb0 0x7f4dac0ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:12.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.813+0000 7f4db2bd0700 1 -- 172.21.15.110:0/3958501044 >> 172.21.15.110:0/3958501044 conn(0x7f4dac0f7440 msgr2=0x7f4dac0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.813+0000 7f4db2bd0700 1 -- 172.21.15.110:0/3958501044 shutdown_connections 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.813+0000 7f4db2bd0700 1 -- 172.21.15.110:0/3958501044 wait complete. 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.814+0000 7f4db2bd0700 1 Processor -- start 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.814+0000 7f4db2bd0700 1 -- start start 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.814+0000 7f4db2bd0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dac066eb0 0x7f4dac10d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.814+0000 7f4db2bd0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 0x7f4dac10ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.814+0000 7f4db2bd0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac10e560 con 0x7f4dac0fff70 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.814+0000 7f4db2bd0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dac112eb0 con 0x7f4dac066eb0 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.814+0000 7f4dabfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 0x7f4dac10ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4dabfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 0x7f4dac10ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49368/0 (socket says 172.21.15.110:49368) 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4dabfff700 1 -- 172.21.15.110:0/1061242260 learned_addr learned my addr 172.21.15.110:0/1061242260 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4db096c700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dac066eb0 0x7f4dac10d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:12.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4dabfff700 1 -- 172.21.15.110:0/1061242260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dac066eb0 msgr2=0x7f4dac10d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:12.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4dabfff700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dac066eb0 0x7f4dac10d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:12.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4dabfff700 1 -- 172.21.15.110:0/1061242260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0005040 con 0x7f4dac0fff70 2024-04-25T15:46:12.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4dabfff700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 0x7f4dac10ded0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4da000b3b0 tx=0x7f4da0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:12.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4da97fa700 1 -- 172.21.15.110:0/1061242260 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4da000e040 con 0x7f4dac0fff70 2024-04-25T15:46:12.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dac1130b0 con 0x7f4dac0fff70 2024-04-25T15:46:12.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4da97fa700 1 -- 172.21.15.110:0/1061242260 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4da000bcc0 con 0x7f4dac0fff70 2024-04-25T15:46:12.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4da97fa700 1 -- 172.21.15.110:0/1061242260 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4da001c630 con 0x7f4dac0fff70 2024-04-25T15:46:12.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.815+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dac113620 con 0x7f4dac0fff70 2024-04-25T15:46:12.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.816+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dac04c710 con 0x7f4dac0fff70 2024-04-25T15:46:12.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.816+0000 7f4da97fa700 1 -- 172.21.15.110:0/1061242260 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f4da0013980 con 0x7f4dac0fff70 2024-04-25T15:46:12.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.817+0000 7f4da97fa700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4d940692a0 0x7f4d9406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:12.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.817+0000 7f4da97fa700 1 -- 172.21.15.110:0/1061242260 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4da0087ce0 con 0x7f4dac0fff70 2024-04-25T15:46:12.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.817+0000 7f4db096c700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4d940692a0 0x7f4d9406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:12.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.817+0000 7f4db096c700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4d940692a0 0x7f4d9406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c006fd0 tx=0x7f4d9c008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:12.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:12.821+0000 7f4da97fa700 1 -- 172.21.15.110:0/1061242260 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4da0057d00 con 0x7f4dac0fff70 2024-04-25T15:46:13.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.014+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f4dac0f8f30 con 0x7f4d940692a0 2024-04-25T15:46:13.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.015+0000 7f4da97fa700 1 -- 172.21.15.110:0/1061242260 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f4dac0f8f30 con 0x7f4d940692a0 2024-04-25T15:46:13.016 INFO:teuthology.orchestra.run.smithi110.stdout:HOST ADDR LABELS STATUS 2024-04-25T15:46:13.016 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 172.21.15.110 2024-04-25T15:46:13.016 INFO:teuthology.orchestra.run.smithi110.stdout:smithi193 172.21.15.193 2024-04-25T15:46:13.016 INFO:teuthology.orchestra.run.smithi110.stdout:2 hosts in cluster 2024-04-25T15:46:13.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4d940692a0 msgr2=0x7f4d9406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:13.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4d940692a0 0x7f4d9406b750 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c006fd0 tx=0x7f4d9c008040 comp rx=0 tx=0).stop 2024-04-25T15:46:13.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 msgr2=0x7f4dac10ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:13.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 0x7f4dac10ded0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f4da000b3b0 tx=0x7f4da0009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:13.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 shutdown_connections 2024-04-25T15:46:13.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f4d940692a0 0x7f4d9406b750 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:13.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4dac066eb0 0x7f4dac10d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:13.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 --2- 172.21.15.110:0/1061242260 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4dac0fff70 0x7f4dac10ded0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:13.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 >> 172.21.15.110:0/1061242260 conn(0x7f4dac0f7440 msgr2=0x7f4dac0f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:13.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 shutdown_connections 2024-04-25T15:46:13.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:13.017+0000 7f4db2bd0700 1 -- 172.21.15.110:0/1061242260 wait complete. 2024-04-25T15:46:13.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:13 smithi110 ceph-mon[26344]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:13.695 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T15:46:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:13 smithi193 ceph-mon[33747]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:13.945 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:14.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:14 smithi110 ceph-mon[26344]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:14.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:14 smithi193 ceph-mon[33747]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 -- 172.21.15.110:0/295265418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 msgr2=0x7f47140ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 --2- 172.21.15.110:0/295265418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 0x7f47140ffa70 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4710004940 tx=0x7f4710009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 -- 172.21.15.110:0/295265418 shutdown_connections 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 --2- 172.21.15.110:0/295265418 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47140fffb0 0x7f4714102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 --2- 172.21.15.110:0/295265418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 0x7f47140ffa70 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 -- 172.21.15.110:0/295265418 >> 172.21.15.110:0/295265418 conn(0x7f47140f7420 msgr2=0x7f47140f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 -- 172.21.15.110:0/295265418 shutdown_connections 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 -- 172.21.15.110:0/295265418 wait complete. 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 Processor -- start 2024-04-25T15:46:15.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.496+0000 7f471c111700 1 -- start start 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f471c111700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 0x7f471410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f471c111700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47140fffb0 0x7f471410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f471c111700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471410e5e0 con 0x7f4714066ef0 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f471c111700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714112ee0 con 0x7f47140fffb0 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f4719ead700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 0x7f471410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f4719ead700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 0x7f471410da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49390/0 (socket says 172.21.15.110:49390) 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f4719ead700 1 -- 172.21.15.110:0/3318583455 learned_addr learned my addr 172.21.15.110:0/3318583455 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f47196ac700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47140fffb0 0x7f471410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f4719ead700 1 -- 172.21.15.110:0/3318583455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47140fffb0 msgr2=0x7f471410df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f4719ead700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47140fffb0 0x7f471410df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f4719ead700 1 -- 172.21.15.110:0/3318583455 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710005040 con 0x7f4714066ef0 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f47196ac700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47140fffb0 0x7f471410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:46:15.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f4719ead700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 0x7f471410da10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f471000b3b0 tx=0x7f4710009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:15.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f470a7fc700 1 -- 172.21.15.110:0/3318583455 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471000e040 con 0x7f4714066ef0 2024-04-25T15:46:15.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f470a7fc700 1 -- 172.21.15.110:0/3318583455 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f471000bcc0 con 0x7f4714066ef0 2024-04-25T15:46:15.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47141130e0 con 0x7f4714066ef0 2024-04-25T15:46:15.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f470a7fc700 1 -- 172.21.15.110:0/3318583455 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f471001c630 con 0x7f4714066ef0 2024-04-25T15:46:15.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.497+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4714113650 con 0x7f4714066ef0 2024-04-25T15:46:15.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.498+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471404c710 con 0x7f4714066ef0 2024-04-25T15:46:15.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.499+0000 7f470a7fc700 1 -- 172.21.15.110:0/3318583455 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f47100139f0 con 0x7f4714066ef0 2024-04-25T15:46:15.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.499+0000 7f470a7fc700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f47000692a0 0x7f470006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:15.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.499+0000 7f470a7fc700 1 -- 172.21.15.110:0/3318583455 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f4710087c60 con 0x7f4714066ef0 2024-04-25T15:46:15.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.499+0000 7f47196ac700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f47000692a0 0x7f470006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:15.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.500+0000 7f47196ac700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f47000692a0 0x7f470006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47140673b0 tx=0x7f4704008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:15.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.503+0000 7f470a7fc700 1 -- 172.21.15.110:0/3318583455 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f4710057d60 con 0x7f4714066ef0 2024-04-25T15:46:15.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:15 smithi110 ceph-mon[26344]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:15.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.698+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f47140f8f60 con 0x7f47000692a0 2024-04-25T15:46:15.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.699+0000 7f470a7fc700 1 -- 172.21.15.110:0/3318583455 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+399 (secure 0 0 0) 0x7f47140f8f60 con 0x7f47000692a0 2024-04-25T15:46:15.700 INFO:teuthology.orchestra.run.smithi110.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T15:46:15.700 INFO:teuthology.orchestra.run.smithi110.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-04-25T15:46:15.700 INFO:teuthology.orchestra.run.smithi110.stdout:smithi193 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500HB400BGN 372G No 72s ago Has a FileSystem, LVM detected 2024-04-25T15:46:15.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f47000692a0 msgr2=0x7f470006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:15.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f47000692a0 0x7f470006b750 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f47140673b0 tx=0x7f4704008040 comp rx=0 tx=0).stop 2024-04-25T15:46:15.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 msgr2=0x7f471410da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:15.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 0x7f471410da10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f471000b3b0 tx=0x7f4710009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:15.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 shutdown_connections 2024-04-25T15:46:15.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4714066ef0 0x7f471410da10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:15.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f47000692a0 0x7f470006b750 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:15.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 --2- 172.21.15.110:0/3318583455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47140fffb0 0x7f471410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:15.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 >> 172.21.15.110:0/3318583455 conn(0x7f47140f7420 msgr2=0x7f47140f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:15.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 shutdown_connections 2024-04-25T15:46:15.703 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:15.701+0000 7f471c111700 1 -- 172.21.15.110:0/3318583455 wait complete. 2024-04-25T15:46:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:15 smithi193 ceph-mon[33747]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:16.316 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:46:16.326 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:46:16.326 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-25T15:46:16.578 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:16.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:16 smithi110 ceph-mon[26344]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:16.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:16 smithi193 ceph-mon[33747]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:17.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:17 smithi110 ceph-mon[26344]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:17.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:17 smithi193 ceph-mon[33747]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:18.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 -- 172.21.15.110:0/1094755898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fcf60 msgr2=0x7f20c80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:18.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 --2- 172.21.15.110:0/1094755898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fcf60 0x7f20c80fd3d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f20b0004940 tx=0x7f20b0009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:18.063 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 -- 172.21.15.110:0/1094755898 shutdown_connections 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 --2- 172.21.15.110:0/1094755898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fcf60 0x7f20c80fd3d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 --2- 172.21.15.110:0/1094755898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c80fbcb0 0x7f20c80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 -- 172.21.15.110:0/1094755898 >> 172.21.15.110:0/1094755898 conn(0x7f20c80f7430 msgr2=0x7f20c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 -- 172.21.15.110:0/1094755898 shutdown_connections 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 -- 172.21.15.110:0/1094755898 wait complete. 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.061+0000 7f20ce13e700 1 Processor -- start 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20ce13e700 1 -- start start 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20ce13e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fbcb0 0x7f20c8110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20ce13e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c80fcf60 0x7f20c8112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20ce13e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80663c0 con 0x7f20c80fbcb0 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20ce13e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8066500 con 0x7f20c80fcf60 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20c77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fbcb0 0x7f20c8110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:18.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20c77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fbcb0 0x7f20c8110710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49412/0 (socket says 172.21.15.110:49412) 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20c77fe700 1 -- 172.21.15.110:0/1024671208 learned_addr learned my addr 172.21.15.110:0/1024671208 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20beffd700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c80fcf60 0x7f20c8112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20c77fe700 1 -- 172.21.15.110:0/1024671208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c80fcf60 msgr2=0x7f20c8112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20c77fe700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c80fcf60 0x7f20c8112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20c77fe700 1 -- 172.21.15.110:0/1024671208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b0005040 con 0x7f20c80fbcb0 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.062+0000 7f20c77fe700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fbcb0 0x7f20c8110710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20b800b790 tx=0x7f20b8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.063+0000 7f20c4ff9700 1 -- 172.21.15.110:0/1024671208 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20b800f6f0 con 0x7f20c80fbcb0 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.063+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c81132c0 con 0x7f20c80fbcb0 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.063+0000 7f20c4ff9700 1 -- 172.21.15.110:0/1024671208 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20b800fd20 con 0x7f20c80fbcb0 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.063+0000 7f20c4ff9700 1 -- 172.21.15.110:0/1024671208 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20b80103a0 con 0x7f20c80fbcb0 2024-04-25T15:46:18.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.063+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c8113850 con 0x7f20c80fbcb0 2024-04-25T15:46:18.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.064+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c805f5c0 con 0x7f20c80fbcb0 2024-04-25T15:46:18.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.064+0000 7f20c4ff9700 1 -- 172.21.15.110:0/1024671208 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f20b8011400 con 0x7f20c80fbcb0 2024-04-25T15:46:18.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.064+0000 7f20c4ff9700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f20b40692a0 0x7f20b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:18.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.064+0000 7f20c4ff9700 1 -- 172.21.15.110:0/1024671208 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4634+0+0 (secure 0 0 0) 0x7f20b8086c40 con 0x7f20c80fbcb0 2024-04-25T15:46:18.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.065+0000 7f20beffd700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f20b40692a0 0x7f20b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:18.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.065+0000 7f20beffd700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f20b40692a0 0x7f20b406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20b000b970 tx=0x7f20b0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:18.069 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.069+0000 7f20c4ff9700 1 -- 172.21.15.110:0/1024671208 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f20b80532b0 con 0x7f20c80fbcb0 2024-04-25T15:46:18.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:18.280+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f20c8101200 con 0x7f20b40692a0 2024-04-25T15:46:18.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:18 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-25T15:46:19.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:18 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-25T15:46:19.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.592+0000 7f20c4ff9700 1 -- 172.21.15.110:0/1024671208 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f20c8101200 con 0x7f20b40692a0 2024-04-25T15:46:19.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f20b40692a0 msgr2=0x7f20b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:19.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f20b40692a0 0x7f20b406b750 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20b000b970 tx=0x7f20b0009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:19.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fbcb0 msgr2=0x7f20c8110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:19.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fbcb0 0x7f20c8110710 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f20b800b790 tx=0x7f20b8009250 comp rx=0 tx=0).stop 2024-04-25T15:46:19.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 shutdown_connections 2024-04-25T15:46:19.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20c80fbcb0 0x7f20c8110710 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:19.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f20b40692a0 0x7f20b406b750 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:19.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 --2- 172.21.15.110:0/1024671208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c80fcf60 0x7f20c8112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:19.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 >> 172.21.15.110:0/1024671208 conn(0x7f20c80f7430 msgr2=0x7f20c80f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:19.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 shutdown_connections 2024-04-25T15:46:19.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:19.595+0000 7f20ce13e700 1 -- 172.21.15.110:0/1024671208 wait complete. 2024-04-25T15:46:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[26340]: 2024-04-25T15:46:19.572+0000 7f8935502700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T15:46:19.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: fsmap cephfs:0 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.mvvbtv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.mvvbtv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:46:19.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:19 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: fsmap cephfs:0 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:46:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:20.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:46:20.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:20.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.mvvbtv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:46:20.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.mvvbtv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:46:20.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:19 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:20.273 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T15:46:21.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:20 smithi110 ceph-mon[26344]: Saving service mds.cephfs spec with placement count:4 2024-04-25T15:46:21.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:20 smithi110 ceph-mon[26344]: Deploying daemon mds.cephfs.smithi110.mvvbtv on smithi110 2024-04-25T15:46:21.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:20 smithi110 ceph-mon[26344]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T15:46:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:20 smithi193 ceph-mon[33747]: Saving service mds.cephfs spec with placement count:4 2024-04-25T15:46:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:20 smithi193 ceph-mon[33747]: Deploying daemon mds.cephfs.smithi110.mvvbtv on smithi110 2024-04-25T15:46:21.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:20 smithi193 ceph-mon[33747]: osdmap e44: 6 total, 6 up, 6 in 2024-04-25T15:46:21.577 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:22.017 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:21 smithi110 ceph-mon[26344]: pgmap v171: 65 pgs: 9 creating+peering, 55 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:22.017 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:21 smithi110 ceph-mon[26344]: osdmap e45: 6 total, 6 up, 6 in 2024-04-25T15:46:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:21 smithi193 ceph-mon[33747]: pgmap v171: 65 pgs: 9 creating+peering, 55 unknown, 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:21 smithi193 ceph-mon[33747]: osdmap e45: 6 total, 6 up, 6 in 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.423+0000 7fd40c4d3700 1 -- 172.21.15.110:0/3965770078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fce60 msgr2=0x7fd4040fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.423+0000 7fd40c4d3700 1 --2- 172.21.15.110:0/3965770078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fce60 0x7fd4040fd270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4004940 tx=0x7fd3f4009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.423+0000 7fd40c4d3700 1 -- 172.21.15.110:0/3965770078 shutdown_connections 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.423+0000 7fd40c4d3700 1 --2- 172.21.15.110:0/3965770078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4040fdfd0 0x7fd4040fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.423+0000 7fd40c4d3700 1 --2- 172.21.15.110:0/3965770078 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fce60 0x7fd4040fd270 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.423+0000 7fd40c4d3700 1 -- 172.21.15.110:0/3965770078 >> 172.21.15.110:0/3965770078 conn(0x7fd4040f8540 msgr2=0x7fd4040fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.423+0000 7fd40c4d3700 1 -- 172.21.15.110:0/3965770078 shutdown_connections 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.423+0000 7fd40c4d3700 1 -- 172.21.15.110:0/3965770078 wait complete. 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd40c4d3700 1 Processor -- start 2024-04-25T15:46:23.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd40c4d3700 1 -- start start 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd40c4d3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4040fce60 0x7fd404071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd40c4d3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fdfd0 0x7fd4040702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd40c4d3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4040707f0 con 0x7fd4040fdfd0 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd40c4d3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd404070960 con 0x7fd4040fce60 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd409a6e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fdfd0 0x7fd4040702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd409a6e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fdfd0 0x7fd4040702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55132/0 (socket says 172.21.15.110:55132) 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd409a6e700 1 -- 172.21.15.110:0/1060361883 learned_addr learned my addr 172.21.15.110:0/1060361883 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd409a6e700 1 -- 172.21.15.110:0/1060361883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4040fce60 msgr2=0x7fd404071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd40a26f700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4040fce60 0x7fd404071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd409a6e700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4040fce60 0x7fd404071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd409a6e700 1 -- 172.21.15.110:0/1060361883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f4005040 con 0x7fd4040fdfd0 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd409a6e700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fdfd0 0x7fd4040702b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b790 tx=0x7fd400009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd3faffd700 1 -- 172.21.15.110:0/1060361883 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40000f770 con 0x7fd4040fdfd0 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.424+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd404070c40 con 0x7fd4040fdfd0 2024-04-25T15:46:23.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.425+0000 7fd3faffd700 1 -- 172.21.15.110:0/1060361883 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd40000fda0 con 0x7fd4040fdfd0 2024-04-25T15:46:23.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.425+0000 7fd3faffd700 1 -- 172.21.15.110:0/1060361883 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd40000d310 con 0x7fd4040fdfd0 2024-04-25T15:46:23.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.425+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd404115a40 con 0x7fd4040fdfd0 2024-04-25T15:46:23.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.426+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40404c710 con 0x7fd4040fdfd0 2024-04-25T15:46:23.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.426+0000 7fd3faffd700 1 -- 172.21.15.110:0/1060361883 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd400010420 con 0x7fd4040fdfd0 2024-04-25T15:46:23.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.426+0000 7fd3faffd700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3f00692a0 0x7fd3f006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:23.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.427+0000 7fd3faffd700 1 -- 172.21.15.110:0/1060361883 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fd4000871d0 con 0x7fd4040fdfd0 2024-04-25T15:46:23.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.427+0000 7fd40a26f700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3f00692a0 0x7fd3f006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:23.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.427+0000 7fd40a26f700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3f00692a0 0x7fd3f006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4000c00 tx=0x7fd3f4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:23.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.431+0000 7fd3faffd700 1 -- 172.21.15.110:0/1060361883 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd400056f90 con 0x7fd4040fdfd0 2024-04-25T15:46:23.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.667+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd404105940 con 0x7fd4040fdfd0 2024-04-25T15:46:23.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.668+0000 7fd3faffd700 1 -- 172.21.15.110:0/1060361883 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1076 (secure 0 0 0) 0x7fd400056b20 con 0x7fd4040fdfd0 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:e2 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 2 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:19.573140+0000 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:46:23.669 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:in 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:up {} 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:46:23.670 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:46:23.671 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:46:23.671 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:46:23.671 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data disabled 2024-04-25T15:46:23.671 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:46:23.671 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 0 2024-04-25T15:46:23.671 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:23.671 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3f00692a0 msgr2=0x7fd3f006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3f00692a0 0x7fd3f006b750 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd3f4000c00 tx=0x7fd3f4009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fdfd0 msgr2=0x7fd4040702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fdfd0 0x7fd4040702b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd40000b790 tx=0x7fd400009250 comp rx=0 tx=0).stop 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 shutdown_connections 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3f00692a0 0x7fd3f006b750 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4040fce60 0x7fd404071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 --2- 172.21.15.110:0/1060361883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd4040fdfd0 0x7fd4040702b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 >> 172.21.15.110:0/1060361883 conn(0x7fd4040f8540 msgr2=0x7fd4040f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 shutdown_connections 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:23.670+0000 7fd40c4d3700 1 -- 172.21.15.110:0/1060361883 wait complete. 2024-04-25T15:46:23.672 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 2 2024-04-25T15:46:23.965 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:23 smithi110 ceph-mon[26344]: pgmap v174: 65 pgs: 9 creating+peering, 18 unknown, 38 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:23.965 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:23 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1060361883' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:46:23.965 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:23 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:23.965 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:23 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:23.965 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:23 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:23.965 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:23 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ewxwfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:46:23.965 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:23 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ewxwfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:46:23.965 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:23 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:24.139 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:23 smithi193 ceph-mon[33747]: pgmap v174: 65 pgs: 9 creating+peering, 18 unknown, 38 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:24.139 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:23 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1060361883' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:46:24.139 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:23 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:24.139 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:23 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:24.139 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:23 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:24.139 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:23 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ewxwfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:46:24.139 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:23 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ewxwfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:46:24.139 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:23 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:24.273 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:46:24.288 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:46:24.288 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 1' 2024-04-25T15:46:24.555 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:25.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: Deploying daemon mds.cephfs.smithi193.ewxwfw on smithi193 2024-04-25T15:46:25.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] up:boot 2024-04-25T15:46:25.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: daemon mds.cephfs.smithi110.mvvbtv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-25T15:46:25.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:46:25.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T15:46:25.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: Cluster is now healthy 2024-04-25T15:46:25.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: fsmap cephfs:0 1 up:standby 2024-04-25T15:46:25.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:46:25.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:creating} 2024-04-25T15:46:25.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:24 smithi110 ceph-mon[26344]: daemon mds.cephfs.smithi110.mvvbtv is now active in filesystem cephfs as rank 0 2024-04-25T15:46:25.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: Deploying daemon mds.cephfs.smithi193.ewxwfw on smithi193 2024-04-25T15:46:25.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] up:boot 2024-04-25T15:46:25.142 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: daemon mds.cephfs.smithi110.mvvbtv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-25T15:46:25.142 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:46:25.142 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T15:46:25.142 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: Cluster is now healthy 2024-04-25T15:46:25.142 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: fsmap cephfs:0 1 up:standby 2024-04-25T15:46:25.142 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:46:25.142 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:creating} 2024-04-25T15:46:25.142 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:24 smithi193 ceph-mon[33747]: daemon mds.cephfs.smithi110.mvvbtv is now active in filesystem cephfs as rank 0 2024-04-25T15:46:25.970 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:25 smithi110 ceph-mon[26344]: pgmap v175: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:25 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] up:active 2024-04-25T15:46:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:25 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 2024-04-25T15:46:25.971 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:46:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:25 smithi193 ceph-mon[33747]: pgmap v175: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:25 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] up:active 2024-04-25T15:46:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:25 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 2024-04-25T15:46:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:46:26.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 -- 172.21.15.110:0/1779939451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 msgr2=0x7f35940ff9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:26.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 --2- 172.21.15.110:0/1779939451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 0x7f35940ff9d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3584004940 tx=0x7f3584009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:26.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 -- 172.21.15.110:0/1779939451 shutdown_connections 2024-04-25T15:46:26.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 --2- 172.21.15.110:0/1779939451 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35940fff10 0x7f3594102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:26.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 --2- 172.21.15.110:0/1779939451 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 0x7f35940ff9d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:26.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 -- 172.21.15.110:0/1779939451 >> 172.21.15.110:0/1779939451 conn(0x7f35940f7420 msgr2=0x7f35940f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:26.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 -- 172.21.15.110:0/1779939451 shutdown_connections 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 -- 172.21.15.110:0/1779939451 wait complete. 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 Processor -- start 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.214+0000 7f359b28c700 1 -- start start 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f359b28c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 0x7f359410d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f359b28c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35940fff10 0x7f359410de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f359b28c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359410e440 con 0x7f3594066ed0 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f359b28c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359410e5b0 con 0x7f35940fff10 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3599028700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 0x7f359410d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3599028700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 0x7f359410d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55162/0 (socket says 172.21.15.110:55162) 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3599028700 1 -- 172.21.15.110:0/1500432890 learned_addr learned my addr 172.21.15.110:0/1500432890 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3598827700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35940fff10 0x7f359410de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3599028700 1 -- 172.21.15.110:0/1500432890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35940fff10 msgr2=0x7f359410de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3599028700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35940fff10 0x7f359410de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3599028700 1 -- 172.21.15.110:0/1500432890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3584005040 con 0x7f3594066ed0 2024-04-25T15:46:26.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3599028700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 0x7f359410d900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3584000c00 tx=0x7f3584009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3589ffb700 1 -- 172.21.15.110:0/1500432890 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358400e040 con 0x7f3594066ed0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3594112f60 con 0x7f3594066ed0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3589ffb700 1 -- 172.21.15.110:0/1500432890 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f358400bcc0 con 0x7f3594066ed0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.215+0000 7f3589ffb700 1 -- 172.21.15.110:0/1500432890 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358401c630 con 0x7f3594066ed0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.216+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3594113490 con 0x7f3594066ed0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.216+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359404c710 con 0x7f3594066ed0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.217+0000 7f3589ffb700 1 -- 172.21.15.110:0/1500432890 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3584013980 con 0x7f3594066ed0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.217+0000 7f3589ffb700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35800692a0 0x7f358006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.218+0000 7f3589ffb700 1 -- 172.21.15.110:0/1500432890 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3584088530 con 0x7f3594066ed0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.218+0000 7f3598827700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35800692a0 0x7f358006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.218+0000 7f3598827700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35800692a0 0x7f358006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3590009fd0 tx=0x7f3590009f60 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:26.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.222+0000 7f3589ffb700 1 -- 172.21.15.110:0/1500432890 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f35840583a0 con 0x7f3594066ed0 2024-04-25T15:46:26.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:26.475+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f35941058d0 con 0x7f3594066ed0 2024-04-25T15:46:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.kdhjqx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:46:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.kdhjqx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:46:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:27.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: Deploying daemon mds.cephfs.smithi110.kdhjqx on smithi110 2024-04-25T15:46:27.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: pgmap v176: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s wr, 3 op/s 2024-04-25T15:46:27.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:boot 2024-04-25T15:46:27.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 1 up:standby 2024-04-25T15:46:27.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:46:27.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:27 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1500432890' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-25T15:46:27.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.366+0000 7f3589ffb700 1 -- 172.21.15.110:0/1500432890 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v7) v1 ==== 105+0+0 (secure 0 0 0) 0x7f3584057f30 con 0x7f3594066ed0 2024-04-25T15:46:27.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35800692a0 msgr2=0x7f358006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:27.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35800692a0 0x7f358006b750 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3590009fd0 tx=0x7f3590009f60 comp rx=0 tx=0).stop 2024-04-25T15:46:27.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 msgr2=0x7f359410d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:27.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 0x7f359410d900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3584000c00 tx=0x7f3584009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:27.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 shutdown_connections 2024-04-25T15:46:27.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3594066ed0 0x7f359410d900 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:27.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35800692a0 0x7f358006b750 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:27.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 --2- 172.21.15.110:0/1500432890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35940fff10 0x7f359410de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:27.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.368+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 >> 172.21.15.110:0/1500432890 conn(0x7f35940f7420 msgr2=0x7f35940f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:27.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.369+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 shutdown_connections 2024-04-25T15:46:27.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:27.369+0000 7f359b28c700 1 -- 172.21.15.110:0/1500432890 wait complete. 2024-04-25T15:46:27.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:27.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:27.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:27.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.kdhjqx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:46:27.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.kdhjqx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:46:27.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:27.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: Deploying daemon mds.cephfs.smithi110.kdhjqx on smithi110 2024-04-25T15:46:27.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: pgmap v176: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.0 KiB/s wr, 3 op/s 2024-04-25T15:46:27.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:boot 2024-04-25T15:46:27.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 1 up:standby 2024-04-25T15:46:27.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:46:27.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:27 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1500432890' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2024-04-25T15:46:28.139 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:46:28.148 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:46:28.149 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-25T15:46:28.623 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:28 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1500432890' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-25T15:46:28.623 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:28 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 1 up:standby 2024-04-25T15:46:28.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:28 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1500432890' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2024-04-25T15:46:28.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:28 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 1 up:standby 2024-04-25T15:46:28.786 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 3 op/s 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.jtyzhl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.jtyzhl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: Deploying daemon mds.cephfs.smithi193.jtyzhl on smithi193 2024-04-25T15:46:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:boot 2024-04-25T15:46:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 2 up:standby 2024-04-25T15:46:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:29 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:46:30.235 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 3 op/s 2024-04-25T15:46:30.235 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:30.235 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:30.235 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:30.235 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.jtyzhl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:46:30.236 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.jtyzhl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T15:46:30.236 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:30.236 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: Deploying daemon mds.cephfs.smithi193.jtyzhl on smithi193 2024-04-25T15:46:30.236 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:boot 2024-04-25T15:46:30.236 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 2 up:standby 2024-04-25T15:46:30.236 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:29 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:46:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.478+0000 7f6088205700 1 -- 172.21.15.110:0/3957722404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 msgr2=0x7f60800fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.478+0000 7f6088205700 1 --2- 172.21.15.110:0/3957722404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 0x7f60800fe450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f607c004940 tx=0x7f607c009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.478+0000 7f6088205700 1 -- 172.21.15.110:0/3957722404 shutdown_connections 2024-04-25T15:46:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.478+0000 7f6088205700 1 --2- 172.21.15.110:0/3957722404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 0x7f60800fe450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.478+0000 7f6088205700 1 --2- 172.21.15.110:0/3957722404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60800fce70 0x7f60800fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.478+0000 7f6088205700 1 -- 172.21.15.110:0/3957722404 >> 172.21.15.110:0/3957722404 conn(0x7f60800f8550 msgr2=0x7f60800fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:30.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.478+0000 7f6088205700 1 -- 172.21.15.110:0/3957722404 shutdown_connections 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.478+0000 7f6088205700 1 -- 172.21.15.110:0/3957722404 wait complete. 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f6088205700 1 Processor -- start 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f6088205700 1 -- start start 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f6088205700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60800fce70 0x7f6080071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f6088205700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 0x7f60800702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f6088205700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6080072200 con 0x7f60800fdfe0 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f6088205700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800707f0 con 0x7f60800fce70 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f60857a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 0x7f60800702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f60857a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 0x7f60800702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39964/0 (socket says 172.21.15.110:39964) 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f60857a0700 1 -- 172.21.15.110:0/1405414133 learned_addr learned my addr 172.21.15.110:0/1405414133 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f6085fa1700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60800fce70 0x7f6080071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f60857a0700 1 -- 172.21.15.110:0/1405414133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60800fce70 msgr2=0x7f6080071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:30.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f60857a0700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60800fce70 0x7f6080071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:30.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f60857a0700 1 -- 172.21.15.110:0/1405414133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607c005040 con 0x7f60800fdfe0 2024-04-25T15:46:30.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.479+0000 7f6085fa1700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60800fce70 0x7f6080071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:46:30.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.480+0000 7f60857a0700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 0x7f60800702b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f607c00b3b0 tx=0x7f607c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:30.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.480+0000 7f60767fc700 1 -- 172.21.15.110:0/1405414133 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f607c00e040 con 0x7f60800fdfe0 2024-04-25T15:46:30.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.480+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6080070a40 con 0x7f60800fdfe0 2024-04-25T15:46:30.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.480+0000 7f60767fc700 1 -- 172.21.15.110:0/1405414133 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f607c00bcc0 con 0x7f60800fdfe0 2024-04-25T15:46:30.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.480+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6080071000 con 0x7f60800fdfe0 2024-04-25T15:46:30.483 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.480+0000 7f60767fc700 1 -- 172.21.15.110:0/1405414133 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f607c01c630 con 0x7f60800fdfe0 2024-04-25T15:46:30.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.481+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608004c710 con 0x7f60800fdfe0 2024-04-25T15:46:30.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.481+0000 7f60767fc700 1 -- 172.21.15.110:0/1405414133 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f607c013980 con 0x7f60800fdfe0 2024-04-25T15:46:30.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.482+0000 7f60767fc700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f606c0692a0 0x7f606c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:30.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.482+0000 7f6085fa1700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f606c0692a0 0x7f606c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:30.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.482+0000 7f60767fc700 1 -- 172.21.15.110:0/1405414133 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f607c087fd0 con 0x7f60800fdfe0 2024-04-25T15:46:30.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.482+0000 7f6085fa1700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f606c0692a0 0x7f606c06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f60700041d0 tx=0x7f6070009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:30.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.486+0000 7f60767fc700 1 -- 172.21.15.110:0/1405414133 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f607c057d10 con 0x7f60800fdfe0 2024-04-25T15:46:30.721 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:30.720+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f60801058d0 con 0x7f60800fdfe0 2024-04-25T15:46:31.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.313+0000 7f60767fc700 1 -- 172.21.15.110:0/1405414133 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f60801058d0 con 0x7f60800fdfe0 2024-04-25T15:46:31.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f606c0692a0 msgr2=0x7f606c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:31.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f606c0692a0 0x7f606c06b750 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f60700041d0 tx=0x7f6070009040 comp rx=0 tx=0).stop 2024-04-25T15:46:31.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 msgr2=0x7f60800702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:31.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 0x7f60800702b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f607c00b3b0 tx=0x7f607c009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:31.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 shutdown_connections 2024-04-25T15:46:31.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f606c0692a0 0x7f606c06b750 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:31.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f60800fce70 0x7f6080071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:31.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 --2- 172.21.15.110:0/1405414133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60800fdfe0 0x7f60800702b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:31.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.315+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 >> 172.21.15.110:0/1405414133 conn(0x7f60800f8550 msgr2=0x7f60800f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:31.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.316+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 shutdown_connections 2024-04-25T15:46:31.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:31.316+0000 7f6088205700 1 -- 172.21.15.110:0/1405414133 wait complete. 2024-04-25T15:46:31.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:31 smithi110 ceph-mon[26344]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T15:46:31.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:31.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:31 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1405414133' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-25T15:46:31.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:31.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:31.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:31 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:31.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:31 smithi193 ceph-mon[33747]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s wr, 4 op/s 2024-04-25T15:46:31.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:31.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:31 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1405414133' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-25T15:46:31.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:31.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:31.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:31 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:31.995 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:46:32.005 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:46:32.005 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-04-25T15:46:32.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:32 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1405414133' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-25T15:46:32.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:32 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:boot 2024-04-25T15:46:32.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:32 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:32.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:32 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:32.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:32 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:46:32.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:32 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:32.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:32 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:46:32.588 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:32 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1405414133' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-25T15:46:32.588 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:32 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:boot 2024-04-25T15:46:32.588 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:32 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:32.588 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:32 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:32.588 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:32 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:46:32.588 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:32 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:32.588 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:32 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:46:32.911 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:33.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:33 smithi110 ceph-mon[26344]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-04-25T15:46:33.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:33 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] up:active 2024-04-25T15:46:33.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:33 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:33.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:33 smithi193 ceph-mon[33747]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s wr, 4 op/s 2024-04-25T15:46:33.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:33 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] up:active 2024-04-25T15:46:33.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:33 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:34.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.520+0000 7fec1cba5700 1 -- 172.21.15.110:0/1951913367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 msgr2=0x7fec180fd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:34.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.520+0000 7fec1cba5700 1 --2- 172.21.15.110:0/1951913367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 0x7fec180fd2a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fec08004940 tx=0x7fec08009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:34.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.520+0000 7fec1cba5700 1 -- 172.21.15.110:0/1951913367 shutdown_connections 2024-04-25T15:46:34.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.520+0000 7fec1cba5700 1 --2- 172.21.15.110:0/1951913367 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec180fe090 0x7fec180fe500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:34.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.520+0000 7fec1cba5700 1 --2- 172.21.15.110:0/1951913367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 0x7fec180fd2a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:34.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.520+0000 7fec1cba5700 1 -- 172.21.15.110:0/1951913367 >> 172.21.15.110:0/1951913367 conn(0x7fec180f85c0 msgr2=0x7fec180fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:34.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.520+0000 7fec1cba5700 1 -- 172.21.15.110:0/1951913367 shutdown_connections 2024-04-25T15:46:34.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.520+0000 7fec1cba5700 1 -- 172.21.15.110:0/1951913367 wait complete. 2024-04-25T15:46:34.523 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:34 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:standby 2024-04-25T15:46:34.524 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:34 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:34.524 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:34 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:34.524 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:34 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:34.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.521+0000 7fec1cba5700 1 Processor -- start 2024-04-25T15:46:34.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.521+0000 7fec1cba5700 1 -- start start 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.521+0000 7fec1cba5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 0x7fec1810da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.521+0000 7fec1cba5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec180fe090 0x7fec1810df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.521+0000 7fec1cba5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1810e590 con 0x7fec180fce90 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.521+0000 7fec1cba5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1810ede0 con 0x7fec180fe090 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec1659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 0x7fec1810da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec1659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 0x7fec1810da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39978/0 (socket says 172.21.15.110:39978) 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec1659c700 1 -- 172.21.15.110:0/2796047705 learned_addr learned my addr 172.21.15.110:0/2796047705 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec15d9b700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec180fe090 0x7fec1810df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec1659c700 1 -- 172.21.15.110:0/2796047705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec180fe090 msgr2=0x7fec1810df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec1659c700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec180fe090 0x7fec1810df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec1659c700 1 -- 172.21.15.110:0/2796047705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec08005040 con 0x7fec180fce90 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec1659c700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 0x7fec1810da50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec0800b970 tx=0x7fec08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:34.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec06ffd700 1 -- 172.21.15.110:0/2796047705 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0800e040 con 0x7fec180fce90 2024-04-25T15:46:34.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec06ffd700 1 -- 172.21.15.110:0/2796047705 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec08004290 con 0x7fec180fce90 2024-04-25T15:46:34.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.522+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1810f060 con 0x7fec180fce90 2024-04-25T15:46:34.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.523+0000 7fec06ffd700 1 -- 172.21.15.110:0/2796047705 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec0801c3f0 con 0x7fec180fce90 2024-04-25T15:46:34.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.523+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1810f5a0 con 0x7fec180fce90 2024-04-25T15:46:34.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.524+0000 7fec06ffd700 1 -- 172.21.15.110:0/2796047705 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fec08013420 con 0x7fec180fce90 2024-04-25T15:46:34.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.524+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1805f5c0 con 0x7fec180fce90 2024-04-25T15:46:34.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.525+0000 7fec06ffd700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fec000690a0 0x7fec0006b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:34.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.525+0000 7fec06ffd700 1 -- 172.21.15.110:0/2796047705 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fec08089e00 con 0x7fec180fce90 2024-04-25T15:46:34.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.525+0000 7fec15d9b700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fec000690a0 0x7fec0006b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:34.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.525+0000 7fec15d9b700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fec000690a0 0x7fec0006b550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0041d0 tx=0x7fec0c0043c0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:34.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.533+0000 7fec06ffd700 1 -- 172.21.15.110:0/2796047705 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fec0808d060 con 0x7fec180fce90 2024-04-25T15:46:34.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:34 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:standby 2024-04-25T15:46:34.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:34 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:34.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:34 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:34.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:34 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:34.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:34.774+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fec1810f820 con 0x7fec180fce90 2024-04-25T15:46:35.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.398+0000 7fec06ffd700 1 -- 172.21.15.110:0/2796047705 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v13) v1 ==== 168+0+0 (secure 0 0 0) 0x7fec08057b20 con 0x7fec180fce90 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fec000690a0 msgr2=0x7fec0006b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fec000690a0 0x7fec0006b550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fec0c0041d0 tx=0x7fec0c0043c0 comp rx=0 tx=0).stop 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 msgr2=0x7fec1810da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 0x7fec1810da50 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fec0800b970 tx=0x7fec08009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 shutdown_connections 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec180fce90 0x7fec1810da50 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fec000690a0 0x7fec0006b550 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 --2- 172.21.15.110:0/2796047705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec180fe090 0x7fec1810df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:35.402 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 >> 172.21.15.110:0/2796047705 conn(0x7fec180f85c0 msgr2=0x7fec180f9d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:35.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 shutdown_connections 2024-04-25T15:46:35.403 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:35.400+0000 7fec1cba5700 1 -- 172.21.15.110:0/2796047705 wait complete. 2024-04-25T15:46:35.403 INFO:teuthology.orchestra.run.smithi110.stderr:inline data enabled 2024-04-25T15:46:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:35 smithi193 ceph-mon[33747]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-25T15:46:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:35 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:standby 2024-04-25T15:46:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:35 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:35 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2796047705' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:46:35.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:35 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:35 smithi110 ceph-mon[26344]: pgmap v180: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-25T15:46:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:35 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:standby 2024-04-25T15:46:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:35 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:35 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2796047705' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T15:46:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:35 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:36.170 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T15:46:36.180 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:46:36.180 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T15:46:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:36 smithi110 ceph-mon[26344]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-25T15:46:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:36 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2796047705' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T15:46:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:36 smithi110 ceph-mon[26344]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:standby 2024-04-25T15:46:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:36 smithi110 ceph-mon[26344]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:36.626 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:36.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:36 smithi193 ceph-mon[33747]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-04-25T15:46:36.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:36 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2796047705' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T15:46:36.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:36 smithi193 ceph-mon[33747]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:standby 2024-04-25T15:46:36.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:36 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:46:37.637 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:37 smithi110 ceph-mon[26344]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-25T15:46:37.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:37 smithi193 ceph-mon[33747]: pgmap v181: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-04-25T15:46:39.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.441+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3464628895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fcf90 msgr2=0x7f2f180fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:39.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.441+0000 7f2f1f8c8700 1 --2- 172.21.15.110:0/3464628895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fcf90 0x7f2f180fd400 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f08004940 tx=0x7f2f08009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:39.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.441+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3464628895 shutdown_connections 2024-04-25T15:46:39.444 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.441+0000 7f2f1f8c8700 1 --2- 172.21.15.110:0/3464628895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fcf90 0x7f2f180fd400 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.442+0000 7f2f1f8c8700 1 --2- 172.21.15.110:0/3464628895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f180fbce0 0x7f2f180fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.442+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3464628895 >> 172.21.15.110:0/3464628895 conn(0x7f2f180f7420 msgr2=0x7f2f180f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.442+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3464628895 shutdown_connections 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.442+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3464628895 wait complete. 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.442+0000 7f2f1f8c8700 1 Processor -- start 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.442+0000 7f2f1f8c8700 1 -- start start 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1f8c8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fbce0 0x7f2f18066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1f8c8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f180fcf90 0x7f2f18112650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1f8c8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f18112b90 con 0x7f2f180fbce0 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1f8c8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f18112cd0 con 0x7f2f180fcf90 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1d664700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fbce0 0x7f2f18066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1d664700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fbce0 0x7f2f18066cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40002/0 (socket says 172.21.15.110:40002) 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1d664700 1 -- 172.21.15.110:0/3109243579 learned_addr learned my addr 172.21.15.110:0/3109243579 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1ce63700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f180fcf90 0x7f2f18112650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:39.445 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1d664700 1 -- 172.21.15.110:0/3109243579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f180fcf90 msgr2=0x7f2f18112650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1d664700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f180fcf90 0x7f2f18112650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1d664700 1 -- 172.21.15.110:0/3109243579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08005040 con 0x7f2f180fbce0 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1ce63700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f180fcf90 0x7f2f18112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1d664700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fbce0 0x7f2f18066cb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2f14002a80 tx=0x7f2f140046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f0dffb700 1 -- 172.21.15.110:0/3109243579 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f14007cc0 con 0x7f2f180fbce0 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f0dffb700 1 -- 172.21.15.110:0/3109243579 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2f1400f040 con 0x7f2f180fbce0 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f18112fb0 con 0x7f2f180fbce0 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.443+0000 7f2f0dffb700 1 -- 172.21.15.110:0/3109243579 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f140096f0 con 0x7f2f180fbce0 2024-04-25T15:46:39.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.444+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f181134e0 con 0x7f2f180fbce0 2024-04-25T15:46:39.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.445+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1804c710 con 0x7f2f180fbce0 2024-04-25T15:46:39.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.445+0000 7f2f0dffb700 1 -- 172.21.15.110:0/3109243579 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2f1400d690 con 0x7f2f180fbce0 2024-04-25T15:46:39.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.445+0000 7f2f0dffb700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2f040692a0 0x7f2f0406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:39.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.445+0000 7f2f0dffb700 1 -- 172.21.15.110:0/3109243579 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2f14086980 con 0x7f2f180fbce0 2024-04-25T15:46:39.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.445+0000 7f2f1ce63700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2f040692a0 0x7f2f0406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:39.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.446+0000 7f2f1ce63700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2f040692a0 0x7f2f0406b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800b3b0 tx=0x7f2f08009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:39.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.450+0000 7f2f0dffb700 1 -- 172.21.15.110:0/3109243579 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2f1404d250 con 0x7f2f180fbce0 2024-04-25T15:46:39.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:39 smithi110 ceph-mon[26344]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 1 op/s 2024-04-25T15:46:39.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.717+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2f18113700 con 0x7f2f180fbce0 2024-04-25T15:46:39.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.718+0000 7f2f0dffb700 1 -- 172.21.15.110:0/3109243579 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f2f140566c0 con 0x7f2f180fbce0 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:46:39.719 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:46:39.720 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:46:39.721 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2f040692a0 msgr2=0x7f2f0406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2f040692a0 0x7f2f0406b750 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800b3b0 tx=0x7f2f08009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fbce0 msgr2=0x7f2f18066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fbce0 0x7f2f18066cb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2f14002a80 tx=0x7f2f140046b0 comp rx=0 tx=0).stop 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 shutdown_connections 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f180fbce0 0x7f2f18066cb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2f040692a0 0x7f2f0406b750 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 --2- 172.21.15.110:0/3109243579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f180fcf90 0x7f2f18112650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 >> 172.21.15.110:0/3109243579 conn(0x7f2f180f7420 msgr2=0x7f2f180f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 shutdown_connections 2024-04-25T15:46:39.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:39.720+0000 7f2f1f8c8700 1 -- 172.21.15.110:0/3109243579 wait complete. 2024-04-25T15:46:39.723 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:46:39.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:39 smithi193 ceph-mon[33747]: pgmap v182: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 1 op/s 2024-04-25T15:46:40.418 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3109243579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:46:40.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:40.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:40.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:46:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:46:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3109243579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:46:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:46:41.288 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:41.667 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:41 smithi110 ceph-mon[26344]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 1 op/s 2024-04-25T15:46:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:41 smithi193 ceph-mon[33747]: pgmap v183: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 1 op/s 2024-04-25T15:46:42.807 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:42 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:42.807 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:42 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:42.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 -- 172.21.15.110:0/4021075719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 msgr2=0x7f52240fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:42.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 --2- 172.21.15.110:0/4021075719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 0x7f52240fc0c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f5214004940 tx=0x7f5214009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:42.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 -- 172.21.15.110:0/4021075719 shutdown_connections 2024-04-25T15:46:42.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 --2- 172.21.15.110:0/4021075719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52240fcf60 0x7f52240fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:42.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 --2- 172.21.15.110:0/4021075719 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 0x7f52240fc0c0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:42.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 -- 172.21.15.110:0/4021075719 >> 172.21.15.110:0/4021075719 conn(0x7f52240f7430 msgr2=0x7f52240f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 -- 172.21.15.110:0/4021075719 shutdown_connections 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 -- 172.21.15.110:0/4021075719 wait complete. 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 Processor -- start 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.823+0000 7f52297d5700 1 -- start start 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f52297d5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 0x7f522410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f52297d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52240fcf60 0x7f522410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f52297d5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522410e590 con 0x7f52240fbcb0 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f52297d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522410ed90 con 0x7f52240fcf60 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 0x7f522410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f5222ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 0x7f522410da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37568/0 (socket says 172.21.15.110:37568) 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f5222ffd700 1 -- 172.21.15.110:0/2583849336 learned_addr learned my addr 172.21.15.110:0/2583849336 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f52227fc700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52240fcf60 0x7f522410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f5222ffd700 1 -- 172.21.15.110:0/2583849336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52240fcf60 msgr2=0x7f522410df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:42.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f5222ffd700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52240fcf60 0x7f522410df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:42.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f5222ffd700 1 -- 172.21.15.110:0/2583849336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214005040 con 0x7f52240fbcb0 2024-04-25T15:46:42.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f5222ffd700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 0x7f522410da50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5214009150 tx=0x7f5214009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:42.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f520b7fe700 1 -- 172.21.15.110:0/2583849336 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521400e040 con 0x7f52240fbcb0 2024-04-25T15:46:42.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522410f010 con 0x7f52240fbcb0 2024-04-25T15:46:42.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f520b7fe700 1 -- 172.21.15.110:0/2583849336 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5214004290 con 0x7f52240fbcb0 2024-04-25T15:46:42.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522410f550 con 0x7f52240fbcb0 2024-04-25T15:46:42.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.824+0000 7f520b7fe700 1 -- 172.21.15.110:0/2583849336 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521401c630 con 0x7f52240fbcb0 2024-04-25T15:46:42.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.825+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5224101af0 con 0x7f52240fbcb0 2024-04-25T15:46:42.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.826+0000 7f520b7fe700 1 -- 172.21.15.110:0/2583849336 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f521400bbf0 con 0x7f52240fbcb0 2024-04-25T15:46:42.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.826+0000 7f520b7fe700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f520c0692a0 0x7f520c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:42.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.826+0000 7f520b7fe700 1 -- 172.21.15.110:0/2583849336 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5214087e70 con 0x7f52240fbcb0 2024-04-25T15:46:42.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.826+0000 7f52227fc700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f520c0692a0 0x7f520c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:42.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.827+0000 7f52227fc700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f520c0692a0 0x7f520c06b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5218007fd0 tx=0x7f5218009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:42.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:42.831+0000 7f520b7fe700 1 -- 172.21.15.110:0/2583849336 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5214057c30 con 0x7f52240fbcb0 2024-04-25T15:46:43.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.072+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f522405f5c0 con 0x7f52240fbcb0 2024-04-25T15:46:43.073 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.072+0000 7f520b7fe700 1 -- 172.21.15.110:0/2583849336 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4689 (secure 0 0 0) 0x7f52140577c0 con 0x7f52240fbcb0 2024-04-25T15:46:43.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f520c0692a0 msgr2=0x7f520c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:43.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f520c0692a0 0x7f520c06b750 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5218007fd0 tx=0x7f5218009040 comp rx=0 tx=0).stop 2024-04-25T15:46:43.076 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 msgr2=0x7f522410da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 0x7f522410da50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5214009150 tx=0x7f5214009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 shutdown_connections 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52240fbcb0 0x7f522410da50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f520c0692a0 0x7f520c06b750 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 --2- 172.21.15.110:0/2583849336 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52240fcf60 0x7f522410df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.074+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 >> 172.21.15.110:0/2583849336 conn(0x7f52240f7430 msgr2=0x7f52240f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.075+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 shutdown_connections 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:43.075+0000 7f52297d5700 1 -- 172.21.15.110:0/2583849336 wait complete. 2024-04-25T15:46:43.077 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:46:43.085 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:46:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:42 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:42 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:44.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:43 smithi110 ceph-mon[26344]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-25T15:46:44.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:43 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2583849336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:46:44.192 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-25T15:46:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:43 smithi193 ceph-mon[33747]: pgmap v184: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-25T15:46:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:43 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2583849336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:46:44.866 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:45.900 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:45 smithi110 ceph-mon[26344]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-25T15:46:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:45 smithi193 ceph-mon[33747]: pgmap v185: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2024-04-25T15:46:46.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.744+0000 7f92e5fa1700 1 -- 172.21.15.110:0/1160186920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fcda0 msgr2=0x7f92e00fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:46.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.744+0000 7f92e5fa1700 1 --2- 172.21.15.110:0/1160186920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fcda0 0x7f92e00fd210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f92c8004940 tx=0x7f92c8009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:46.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.744+0000 7f92e5fa1700 1 -- 172.21.15.110:0/1160186920 shutdown_connections 2024-04-25T15:46:46.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.744+0000 7f92e5fa1700 1 --2- 172.21.15.110:0/1160186920 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fcda0 0x7f92e00fd210 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:46.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.744+0000 7f92e5fa1700 1 --2- 172.21.15.110:0/1160186920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e00fbaf0 0x7f92e00fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.744+0000 7f92e5fa1700 1 -- 172.21.15.110:0/1160186920 >> 172.21.15.110:0/1160186920 conn(0x7f92e00f7270 msgr2=0x7f92e00f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.744+0000 7f92e5fa1700 1 -- 172.21.15.110:0/1160186920 shutdown_connections 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.744+0000 7f92e5fa1700 1 -- 172.21.15.110:0/1160186920 wait complete. 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e5fa1700 1 Processor -- start 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e5fa1700 1 -- start start 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e5fa1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fbaf0 0x7f92e010c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e5fa1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e00fcda0 0x7f92e010cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e5fa1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e010d2d0 con 0x7f92e00fbaf0 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e5fa1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e010dac0 con 0x7f92e00fcda0 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e4f9f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fbaf0 0x7f92e010c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e4f9f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fbaf0 0x7f92e010c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37588/0 (socket says 172.21.15.110:37588) 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e4f9f700 1 -- 172.21.15.110:0/4159697351 learned_addr learned my addr 172.21.15.110:0/4159697351 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92dffff700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e00fcda0 0x7f92e010cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e4f9f700 1 -- 172.21.15.110:0/4159697351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e00fcda0 msgr2=0x7f92e010cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:46.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e4f9f700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e00fcda0 0x7f92e010cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:46.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.745+0000 7f92e4f9f700 1 -- 172.21.15.110:0/4159697351 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c8005040 con 0x7f92e00fbaf0 2024-04-25T15:46:46.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.746+0000 7f92e4f9f700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fbaf0 0x7f92e010c700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f92d800bd20 tx=0x7f92d800ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:46.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.746+0000 7f92dd7fa700 1 -- 172.21.15.110:0/4159697351 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d800b2e0 con 0x7f92e00fbaf0 2024-04-25T15:46:46.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.746+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e010dda0 con 0x7f92e00fbaf0 2024-04-25T15:46:46.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.746+0000 7f92dd7fa700 1 -- 172.21.15.110:0/4159697351 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92d800ede0 con 0x7f92e00fbaf0 2024-04-25T15:46:46.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.746+0000 7f92dd7fa700 1 -- 172.21.15.110:0/4159697351 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92d8017470 con 0x7f92e00fbaf0 2024-04-25T15:46:46.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.746+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e010e360 con 0x7f92e00fbaf0 2024-04-25T15:46:46.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.747+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e00faef0 con 0x7f92e00fbaf0 2024-04-25T15:46:46.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.747+0000 7f92dd7fa700 1 -- 172.21.15.110:0/4159697351 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f92d8014070 con 0x7f92e00fbaf0 2024-04-25T15:46:46.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.748+0000 7f92dd7fa700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f92cc0692a0 0x7f92cc06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:46.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.748+0000 7f92dd7fa700 1 -- 172.21.15.110:0/4159697351 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f92d8089080 con 0x7f92e00fbaf0 2024-04-25T15:46:46.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.748+0000 7f92dffff700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f92cc0692a0 0x7f92cc06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:46.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.748+0000 7f92dffff700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f92cc0692a0 0x7f92cc06b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92c800b3b0 tx=0x7f92c8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:46.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:46.752+0000 7f92dd7fa700 1 -- 172.21.15.110:0/4159697351 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f92d8058dc0 con 0x7f92e00fbaf0 2024-04-25T15:46:47.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.018+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7f92e010e640 con 0x7f92e00fbaf0 2024-04-25T15:46:47.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.019+0000 7f92dd7fa700 1 -- 172.21.15.110:0/4159697351 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v13) v1 ==== 78+0+99 (secure 0 0 0) 0x7f92d8058950 con 0x7f92e00fbaf0 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f92cc0692a0 msgr2=0x7f92cc06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f92cc0692a0 0x7f92cc06b750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f92c800b3b0 tx=0x7f92c8009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fbaf0 msgr2=0x7f92e010c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fbaf0 0x7f92e010c700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f92d800bd20 tx=0x7f92d800ba50 comp rx=0 tx=0).stop 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 shutdown_connections 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92e00fbaf0 0x7f92e010c700 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f92cc0692a0 0x7f92cc06b750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 --2- 172.21.15.110:0/4159697351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92e00fcda0 0x7f92e010cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 >> 172.21.15.110:0/4159697351 conn(0x7f92e00f7270 msgr2=0x7f92e00f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 shutdown_connections 2024-04-25T15:46:47.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.021+0000 7f92e5fa1700 1 -- 172.21.15.110:0/4159697351 wait complete. 2024-04-25T15:46:47.034 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:46:47.601 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-25T15:46:47.613 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T15:46:47.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.726+0000 7fe6f651d640 1 -- 172.21.15.110:0/1794195017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fb060 msgr2=0x7fe6f00fb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:47.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.726+0000 7fe6f651d640 1 --2- 172.21.15.110:0/1794195017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fb060 0x7fe6f00fb470 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00048d0 tx=0x7fe6e0009da0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.726+0000 7fe6f651d640 1 -- 172.21.15.110:0/1794195017 shutdown_connections 2024-04-25T15:46:47.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.726+0000 7fe6f651d640 1 --2- 172.21.15.110:0/1794195017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f00fc330 0x7fe6f00fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.726+0000 7fe6f651d640 1 --2- 172.21.15.110:0/1794195017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fb060 0x7fe6f00fb470 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.726+0000 7fe6f651d640 1 -- 172.21.15.110:0/1794195017 >> 172.21.15.110:0/1794195017 conn(0x7fe6f00f6840 msgr2=0x7fe6f00f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:47.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.726+0000 7fe6f651d640 1 -- 172.21.15.110:0/1794195017 shutdown_connections 2024-04-25T15:46:47.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.726+0000 7fe6f651d640 1 -- 172.21.15.110:0/1794195017 wait complete. 2024-04-25T15:46:47.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6f651d640 1 Processor -- start 2024-04-25T15:46:47.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6f651d640 1 -- start start 2024-04-25T15:46:47.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6f651d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f00fb060 0x7fe6f0065ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:47.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6f651d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fc330 0x7fe6f0111ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:47.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6f651d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f0111ff0 con 0x7fe6f00fc330 2024-04-25T15:46:47.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6f651d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f0112130 con 0x7fe6f00fb060 2024-04-25T15:46:47.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6ef7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fc330 0x7fe6f0111ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:47.730 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6ef7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fc330 0x7fe6f0111ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37618/0 (socket says 172.21.15.110:37618) 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.727+0000 7fe6ef7fe640 1 -- 172.21.15.110:0/1361917285 learned_addr learned my addr 172.21.15.110:0/1361917285 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6effff640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f00fb060 0x7fe6f0065ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6ef7fe640 1 -- 172.21.15.110:0/1361917285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f00fb060 msgr2=0x7fe6f0065ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6ef7fe640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f00fb060 0x7fe6f0065ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6ef7fe640 1 -- 172.21.15.110:0/1361917285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e0005040 con 0x7fe6f00fc330 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6effff640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f00fb060 0x7fe6f0065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6ef7fe640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fc330 0x7fe6f0111ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400c830 tx=0x7fe6e4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6ecff9640 1 -- 172.21.15.110:0/1361917285 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6e400a850 con 0x7fe6f00fc330 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6ecff9640 1 -- 172.21.15.110:0/1361917285 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6e400ae80 con 0x7fe6f00fc330 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f0112390 con 0x7fe6f00fc330 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6ecff9640 1 -- 172.21.15.110:0/1361917285 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6e4012ea0 con 0x7fe6f00fc330 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.728+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f0112880 con 0x7fe6f00fc330 2024-04-25T15:46:47.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.729+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f004bcc0 con 0x7fe6f00fc330 2024-04-25T15:46:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.730+0000 7fe6ecff9640 1 -- 172.21.15.110:0/1361917285 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe6e4012400 con 0x7fe6f00fc330 2024-04-25T15:46:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.730+0000 7fe6ecff9640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe6d8069210 0x7fe6d806b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.730+0000 7fe6ecff9640 1 -- 172.21.15.110:0/1361917285 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe6e4087400 con 0x7fe6f00fc330 2024-04-25T15:46:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.730+0000 7fe6effff640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe6d8069210 0x7fe6d806b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:47.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.731+0000 7fe6effff640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe6d8069210 0x7fe6d806b6c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00048a0 tx=0x7fe6e0009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:47.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.735+0000 7fe6ecff9640 1 -- 172.21.15.110:0/1361917285 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe6e4053730 con 0x7fe6f00fc330 2024-04-25T15:46:47.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:47 smithi110 ceph-mon[26344]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:47 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/4159697351' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-25T15:46:47.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.903+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6f0103bb0 con 0x7fe6f00fc330 2024-04-25T15:46:47.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.903+0000 7fe6ecff9640 1 -- 172.21.15.110:0/1361917285 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+14318 (secure 0 0 0) 0x7fe6e40126b0 con 0x7fe6f00fc330 2024-04-25T15:46:47.904 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:47.904 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":45,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","created":"2024-04-25T15:39:31.460681+0000","modified":"2024-04-25T15:46:21.578896+0000","last_up_change":"2024-04-25T15:45:01.665598+0000","last_in_change":"2024-04-25T15:44:42.991633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T15:43:50.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-25T15:46:18.514536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-25T15:46:18.805544+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"50a0a2ad-2c02-4108-85a8-3ef716509956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6803","nonce":3841288978}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6805","nonce":3841288978}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6809","nonce":3841288978}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":3841288978},{"type":"v1","addr":"172.21.15.110:6807","nonce":3841288978}]},"public_addr":"172.21.15.110:6803/3841288978","cluster_addr":"172.21.15.110:6805/3841288978","heartbeat_back_addr":"172.21.15.110:6809/3841288978","heartbeat_front_addr":"172.21.15.110:6807/3841288978","state":["exists","up"]},{"osd":1,"uuid":"90511b8f-4444-44ff-b30c-789349b354f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6811","nonce":1447602330}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6813","nonce":1447602330}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6817","nonce":1447602330}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1447602330},{"type":"v1","addr":"172.21.15.110:6815","nonce":1447602330}]},"public_addr":"172.21.15.110:6811/1447602330","cluster_addr":"172.21.15.110:6813/1447602330","heartbeat_back_addr":"172.21.15.110:6817/1447602330","heartbeat_front_addr":"172.21.15.110:6815/1447602330","state":["exists","up"]},{"osd":2,"uuid":"c74ff664-988d-4d4b-a248-176d11f44010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6819","nonce":3448336555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6821","nonce":3448336555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6825","nonce":3448336555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3448336555},{"type":"v1","addr":"172.21.15.110:6823","nonce":3448336555}]},"public_addr":"172.21.15.110:6819/3448336555","cluster_addr":"172.21.15.110:6821/3448336555","heartbeat_back_addr":"172.21.15.110:6825/3448336555","heartbeat_front_addr":"172.21.15.110:6823/3448336555","state":["exists","up"]},{"osd":3,"uuid":"dd48aeea-c692-4258-a8df-848bf440b6ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6801","nonce":1286847191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6803","nonce":1286847191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6807","nonce":1286847191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1286847191},{"type":"v1","addr":"172.21.15.193:6805","nonce":1286847191}]},"public_addr":"172.21.15.193:6801/1286847191","cluster_addr":"172.21.15.193:6803/1286847191","heartbeat_back_addr":"172.21.15.193:6807/1286847191","heartbeat_front_addr":"172.21.15.193:6805/1286847191","state":["exists","up"]},{"osd":4,"uuid":"3b0a0e9a-5308-421d-8091-fac8f7649509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6809","nonce":190042456}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6811","nonce":190042456}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6815","nonce":190042456}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":190042456},{"type":"v1","addr":"172.21.15.193:6813","nonce":190042456}]},"public_addr":"172.21.15.193:6809/190042456","cluster_addr":"172.21.15.193:6811/190042456","heartbeat_back_addr":"172.21.15.193:6815/190042456","heartbeat_front_addr":"172.21.15.193:6813/190042456","state":["exists","up"]},{"osd":5,"uuid":"c6cf2f8b-8105-475c-8a79-ce9bbcdf8644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6817","nonce":1742452500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6819","nonce":1742452500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6823","nonce":1742452500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":1742452500},{"type":"v1","addr":"172.21.15.193:6821","nonce":1742452500}]},"public_addr":"172.21.15.193:6817/1742452500","cluster_addr":"172.21.15.193:6819/1742452500","heartbeat_back_addr":"172.21.15.193:6823/1742452500","heartbeat_front_addr":"172.21.15.193:6821/1742452500","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:42:54.075221+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:21.305107+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:48.660473+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:11.495485+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:35.758085+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:45:00.087588+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.110:6801/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2245327398":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/920532485":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2468496169":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/1601321214":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/3933829355":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/1729346603":"2024-04-26T15:40:32.961057+0000","172.21.15.110:6800/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:6801/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/322959174":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/668880779":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6801/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/2068930777":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1116386591":"2024-04-26T15:40:32.961057+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T15:46:47.906 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe6d8069210 msgr2=0x7fe6d806b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe6d8069210 0x7fe6d806b6c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00048a0 tx=0x7fe6e0009da0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fc330 msgr2=0x7fe6f0111ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fc330 0x7fe6f0111ab0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400c830 tx=0x7fe6e4004430 comp rx=0 tx=0).stop 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 shutdown_connections 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe6d8069210 0x7fe6d806b6c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f00fc330 0x7fe6f0111ab0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 --2- 172.21.15.110:0/1361917285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f00fb060 0x7fe6f0065ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.905+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 >> 172.21.15.110:0/1361917285 conn(0x7fe6f00f6840 msgr2=0x7fe6f00f7f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.906+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 shutdown_connections 2024-04-25T15:46:47.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:47.906+0000 7fe6f651d640 1 -- 172.21.15.110:0/1361917285 wait complete. 2024-04-25T15:46:47.917 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T15:43:50.541493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-25T15:46:18.514536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-25T15:46:18.805544+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T15:46:47.918 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.024+0000 7fe41906e640 1 -- 172.21.15.110:0/1107006466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 msgr2=0x7fe4140fc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.024+0000 7fe41906e640 1 --2- 172.21.15.110:0/1107006466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 0x7fe4140fc4f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe404009c50 tx=0x7fe404004d10 comp rx=0 tx=0).stop 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.024+0000 7fe41906e640 1 -- 172.21.15.110:0/1107006466 shutdown_connections 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.024+0000 7fe41906e640 1 --2- 172.21.15.110:0/1107006466 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4140fd3b0 0x7fe4140fd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.024+0000 7fe41906e640 1 --2- 172.21.15.110:0/1107006466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 0x7fe4140fc4f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.024+0000 7fe41906e640 1 -- 172.21.15.110:0/1107006466 >> 172.21.15.110:0/1107006466 conn(0x7fe4140f78c0 msgr2=0x7fe4140f9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.024+0000 7fe41906e640 1 -- 172.21.15.110:0/1107006466 shutdown_connections 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.024+0000 7fe41906e640 1 -- 172.21.15.110:0/1107006466 wait complete. 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe41906e640 1 Processor -- start 2024-04-25T15:46:48.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe41906e640 1 -- start start 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe41906e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 0x7fe41410cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe41906e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4140fd3b0 0x7fe41410d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe41906e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41410d7d0 con 0x7fe4140fc0e0 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe41906e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414112190 con 0x7fe4140fd3b0 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe412d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 0x7fe41410cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe412d9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 0x7fe41410cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37646/0 (socket says 172.21.15.110:37646) 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe412d9d640 1 -- 172.21.15.110:0/1867190704 learned_addr learned my addr 172.21.15.110:0/1867190704 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe412d9d640 1 -- 172.21.15.110:0/1867190704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4140fd3b0 msgr2=0x7fe41410d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.025+0000 7fe412d9d640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4140fd3b0 0x7fe41410d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.026+0000 7fe412d9d640 1 -- 172.21.15.110:0/1867190704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4040098b0 con 0x7fe4140fc0e0 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.026+0000 7fe412d9d640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 0x7fe41410cc40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe404004a30 tx=0x7fe404004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.026+0000 7fe3fb7fe640 1 -- 172.21.15.110:0/1867190704 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe40403a070 con 0x7fe4140fc0e0 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.026+0000 7fe3fb7fe640 1 -- 172.21.15.110:0/1867190704 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe404035d30 con 0x7fe4140fc0e0 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.026+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4141122d0 con 0x7fe4140fc0e0 2024-04-25T15:46:48.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.026+0000 7fe3fb7fe640 1 -- 172.21.15.110:0/1867190704 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe40403ec70 con 0x7fe4140fc0e0 2024-04-25T15:46:48.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.026+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe414112820 con 0x7fe4140fc0e0 2024-04-25T15:46:48.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.027+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41404bcc0 con 0x7fe4140fc0e0 2024-04-25T15:46:48.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.027+0000 7fe3fb7fe640 1 -- 172.21.15.110:0/1867190704 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe404032070 con 0x7fe4140fc0e0 2024-04-25T15:46:48.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.028+0000 7fe3fb7fe640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3fc069210 0x7fe3fc06b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.028+0000 7fe3fb7fe640 1 -- 172.21.15.110:0/1867190704 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe4040a9c40 con 0x7fe4140fc0e0 2024-04-25T15:46:48.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.028+0000 7fe41259c640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3fc069210 0x7fe3fc06b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.028+0000 7fe41259c640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3fc069210 0x7fe3fc06b6c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe408004a50 tx=0x7fe408007cb0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:48.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.032+0000 7fe3fb7fe640 1 -- 172.21.15.110:0/1867190704 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe404075ef0 con 0x7fe4140fc0e0 2024-04-25T15:46:48.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.201+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe414112a40 con 0x7fe4140fc0e0 2024-04-25T15:46:48.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.201+0000 7fe3fb7fe640 1 -- 172.21.15.110:0/1867190704 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe404079510 con 0x7fe4140fc0e0 2024-04-25T15:46:48.202 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3fc069210 msgr2=0x7fe3fc06b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3fc069210 0x7fe3fc06b6c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe408004a50 tx=0x7fe408007cb0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 msgr2=0x7fe41410cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 0x7fe41410cc40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe404004a30 tx=0x7fe404004d10 comp rx=0 tx=0).stop 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 shutdown_connections 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3fc069210 0x7fe3fc06b6c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4140fd3b0 0x7fe41410d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 --2- 172.21.15.110:0/1867190704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe4140fc0e0 0x7fe41410cc40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 >> 172.21.15.110:0/1867190704 conn(0x7fe4140f78c0 msgr2=0x7fe4140f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 shutdown_connections 2024-04-25T15:46:48.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.203+0000 7fe41906e640 1 -- 172.21.15.110:0/1867190704 wait complete. 2024-04-25T15:46:48.215 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-25T15:46:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:47 smithi193 ceph-mon[33747]: pgmap v186: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:47 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/4159697351' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 -- 172.21.15.110:0/954996440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fc340 msgr2=0x7fe3d80fc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 --2- 172.21.15.110:0/954996440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fc340 0x7fe3d80fc7b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80048d0 tx=0x7fe3c8009da0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 -- 172.21.15.110:0/954996440 shutdown_connections 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 --2- 172.21.15.110:0/954996440 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fc340 0x7fe3d80fc7b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 --2- 172.21.15.110:0/954996440 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3d80fb070 0x7fe3d80fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 -- 172.21.15.110:0/954996440 >> 172.21.15.110:0/954996440 conn(0x7fe3d80f6830 msgr2=0x7fe3d80f8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 -- 172.21.15.110:0/954996440 shutdown_connections 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 -- 172.21.15.110:0/954996440 wait complete. 2024-04-25T15:46:48.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 Processor -- start 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.311+0000 7fe3dfb54640 1 -- start start 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dfb54640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fb070 0x7fe3d8065f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dfb54640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3d80fc340 0x7fe3d8111b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dfb54640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d81120a0 con 0x7fe3d80fb070 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dfb54640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d81121e0 con 0x7fe3d80fc340 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd8f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fb070 0x7fe3d8065f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd8f0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fb070 0x7fe3d8065f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37668/0 (socket says 172.21.15.110:37668) 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd8f0640 1 -- 172.21.15.110:0/4264267721 learned_addr learned my addr 172.21.15.110:0/4264267721 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd0ef640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3d80fc340 0x7fe3d8111b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd8f0640 1 -- 172.21.15.110:0/4264267721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3d80fc340 msgr2=0x7fe3d8111b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd8f0640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3d80fc340 0x7fe3d8111b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd8f0640 1 -- 172.21.15.110:0/4264267721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c8005040 con 0x7fe3d80fb070 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd0ef640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3d80fc340 0x7fe3d8111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:46:48.315 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dd8f0640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fb070 0x7fe3d8065f00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400eb70 tx=0x7fe3d400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:48.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3ce7fc640 1 -- 172.21.15.110:0/4264267721 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3d4007c30 con 0x7fe3d80fb070 2024-04-25T15:46:48.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d81123e0 con 0x7fe3d80fb070 2024-04-25T15:46:48.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3ce7fc640 1 -- 172.21.15.110:0/4264267721 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3d400f040 con 0x7fe3d80fb070 2024-04-25T15:46:48.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3ce7fc640 1 -- 172.21.15.110:0/4264267721 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3d4009680 con 0x7fe3d80fb070 2024-04-25T15:46:48.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.312+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d8112880 con 0x7fe3d80fb070 2024-04-25T15:46:48.316 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.313+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3d8105550 con 0x7fe3d80fb070 2024-04-25T15:46:48.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.314+0000 7fe3ce7fc640 1 -- 172.21.15.110:0/4264267721 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fe3d400cd40 con 0x7fe3d80fb070 2024-04-25T15:46:48.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.314+0000 7fe3ce7fc640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3c4069210 0x7fe3c406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.314+0000 7fe3ce7fc640 1 -- 172.21.15.110:0/4264267721 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fe3d4086e70 con 0x7fe3d80fb070 2024-04-25T15:46:48.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.314+0000 7fe3dd0ef640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3c4069210 0x7fe3c406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.317 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.314+0000 7fe3dd0ef640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3c4069210 0x7fe3c406b6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8065cb0 tx=0x7fe3c8009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:48.319 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.318+0000 7fe3ce7fc640 1 -- 172.21.15.110:0/4264267721 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fe3d404e250 con 0x7fe3d80fb070 2024-04-25T15:46:48.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.500+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fe3d8112aa0 con 0x7fe3d80fb070 2024-04-25T15:46:48.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.501+0000 7fe3ce7fc640 1 -- 172.21.15.110:0/4264267721 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe3d40531d0 con 0x7fe3d80fb070 2024-04-25T15:46:48.502 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3c4069210 msgr2=0x7fe3c406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3c4069210 0x7fe3c406b6c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8065cb0 tx=0x7fe3c8009da0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fb070 msgr2=0x7fe3d8065f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fb070 0x7fe3d8065f00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400eb70 tx=0x7fe3d400b040 comp rx=0 tx=0).stop 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 shutdown_connections 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fe3c4069210 0x7fe3c406b6c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3d80fc340 0x7fe3d8111b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 --2- 172.21.15.110:0/4264267721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3d80fb070 0x7fe3d8065f00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 >> 172.21.15.110:0/4264267721 conn(0x7fe3d80f6830 msgr2=0x7fe3d80ff630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:48.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 shutdown_connections 2024-04-25T15:46:48.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.503+0000 7fe3dfb54640 1 -- 172.21.15.110:0/4264267721 wait complete. 2024-04-25T15:46:48.516 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-25T15:46:48.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.617+0000 7f796eeae640 1 -- 172.21.15.110:0/1757444713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fb060 msgr2=0x7f79680fb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.617+0000 7f796eeae640 1 --2- 172.21.15.110:0/1757444713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fb060 0x7f79680fb470 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f79600048d0 tx=0x7f7960009da0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.617+0000 7f796eeae640 1 -- 172.21.15.110:0/1757444713 shutdown_connections 2024-04-25T15:46:48.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.617+0000 7f796eeae640 1 --2- 172.21.15.110:0/1757444713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79680fc330 0x7f79680fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.617+0000 7f796eeae640 1 --2- 172.21.15.110:0/1757444713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fb060 0x7f79680fb470 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.617+0000 7f796eeae640 1 -- 172.21.15.110:0/1757444713 >> 172.21.15.110:0/1757444713 conn(0x7f79680f6820 msgr2=0x7f79680f8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:48.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.617+0000 7f796eeae640 1 -- 172.21.15.110:0/1757444713 shutdown_connections 2024-04-25T15:46:48.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.617+0000 7f796eeae640 1 -- 172.21.15.110:0/1757444713 wait complete. 2024-04-25T15:46:48.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f796eeae640 1 Processor -- start 2024-04-25T15:46:48.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f796eeae640 1 -- start start 2024-04-25T15:46:48.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f796eeae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79680fb060 0x7f7968064d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f796eeae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fc330 0x7f7968111aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f796eeae640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7968111fe0 con 0x7f79680fc330 2024-04-25T15:46:48.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f796eeae640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7968112120 con 0x7f79680fb060 2024-04-25T15:46:48.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f7967fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fc330 0x7f7968111aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f7967fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fc330 0x7f7968111aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37692/0 (socket says 172.21.15.110:37692) 2024-04-25T15:46:48.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f7967fff640 1 -- 172.21.15.110:0/3842262695 learned_addr learned my addr 172.21.15.110:0/3842262695 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:48.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f796cc4a640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79680fb060 0x7f7968064d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f7967fff640 1 -- 172.21.15.110:0/3842262695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79680fb060 msgr2=0x7f7968064d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f7967fff640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79680fb060 0x7f7968064d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.618+0000 7f7967fff640 1 -- 172.21.15.110:0/3842262695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7960005040 con 0x7f79680fc330 2024-04-25T15:46:48.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.619+0000 7f7967fff640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fc330 0x7f7968111aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f795800db00 tx=0x7f7958004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:48.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.619+0000 7f79657fa640 1 -- 172.21.15.110:0/3842262695 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7958017070 con 0x7f79680fc330 2024-04-25T15:46:48.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.619+0000 7f79657fa640 1 -- 172.21.15.110:0/3842262695 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f795800f040 con 0x7f79680fc330 2024-04-25T15:46:48.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.619+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7968112380 con 0x7f79680fc330 2024-04-25T15:46:48.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.619+0000 7f79657fa640 1 -- 172.21.15.110:0/3842262695 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7958003a70 con 0x7f79680fc330 2024-04-25T15:46:48.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.619+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7968112870 con 0x7f79680fc330 2024-04-25T15:46:48.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.620+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968105560 con 0x7f79680fc330 2024-04-25T15:46:48.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.620+0000 7f79657fa640 1 -- 172.21.15.110:0/3842262695 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7958003bd0 con 0x7f79680fc330 2024-04-25T15:46:48.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.620+0000 7f79657fa640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7954069210 0x7f795406b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.621+0000 7f79657fa640 1 -- 172.21.15.110:0/3842262695 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f79580871c0 con 0x7f79680fc330 2024-04-25T15:46:48.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.621+0000 7f796cc4a640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7954069210 0x7f795406b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.621+0000 7f796cc4a640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7954069210 0x7f795406b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79600048a0 tx=0x7f7960009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:48.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.625+0000 7f79657fa640 1 -- 172.21.15.110:0/3842262695 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7958053470 con 0x7f79680fc330 2024-04-25T15:46:48.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.805+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f7968112ac0 con 0x7f79680fc330 2024-04-25T15:46:48.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.805+0000 7f79657fa640 1 -- 172.21.15.110:0/3842262695 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v45) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7958056a90 con 0x7f79680fc330 2024-04-25T15:46:48.806 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T15:46:48.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.807+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7954069210 msgr2=0x7f795406b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.807+0000 7f796eeae640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7954069210 0x7f795406b6c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f79600048a0 tx=0x7f7960009da0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.808 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.807+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fc330 msgr2=0x7f7968111aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.807+0000 7f796eeae640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fc330 0x7f7968111aa0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f795800db00 tx=0x7f7958004e60 comp rx=0 tx=0).stop 2024-04-25T15:46:48.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.807+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 shutdown_connections 2024-04-25T15:46:48.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.808+0000 7f796eeae640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7954069210 0x7f795406b6c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.808+0000 7f796eeae640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79680fc330 0x7f7968111aa0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.808+0000 7f796eeae640 1 --2- 172.21.15.110:0/3842262695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79680fb060 0x7f7968064d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.808+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 >> 172.21.15.110:0/3842262695 conn(0x7f79680f6820 msgr2=0x7f79680ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:48.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.808+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 shutdown_connections 2024-04-25T15:46:48.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.808+0000 7f796eeae640 1 -- 172.21.15.110:0/3842262695 wait complete. 2024-04-25T15:46:48.819 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T15:46:48.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 -- 172.21.15.110:0/550155548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fb090 msgr2=0x7f23340fb4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 --2- 172.21.15.110:0/550155548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fb090 0x7f23340fb4a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f231c0048d0 tx=0x7f231c009da0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.914 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 -- 172.21.15.110:0/550155548 shutdown_connections 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 --2- 172.21.15.110:0/550155548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340fc360 0x7f23340fc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 --2- 172.21.15.110:0/550155548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fb090 0x7f23340fb4a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 -- 172.21.15.110:0/550155548 >> 172.21.15.110:0/550155548 conn(0x7f23340f6870 msgr2=0x7f23340f8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 -- 172.21.15.110:0/550155548 shutdown_connections 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 -- 172.21.15.110:0/550155548 wait complete. 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 Processor -- start 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 -- start start 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.912+0000 7f233a4ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340fb090 0x7f233410bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f233a4ae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fc360 0x7f233410c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f233a4ae640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233410c720 con 0x7f23340fc360 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f233a4ae640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2334111120 con 0x7f23340fb090 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f23337fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fc360 0x7f233410c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f23337fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fc360 0x7f233410c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:37724/0 (socket says 172.21.15.110:37724) 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f23337fe640 1 -- 172.21.15.110:0/2847345269 learned_addr learned my addr 172.21.15.110:0/2847345269 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:48.915 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f2333fff640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340fb090 0x7f233410bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f23337fe640 1 -- 172.21.15.110:0/2847345269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340fb090 msgr2=0x7f233410bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f23337fe640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340fb090 0x7f233410bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f23337fe640 1 -- 172.21.15.110:0/2847345269 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c005040 con 0x7f23340fc360 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f23337fe640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fc360 0x7f233410c0d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f232400c830 tx=0x7f2324004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f2330ff9640 1 -- 172.21.15.110:0/2847345269 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f232400a850 con 0x7f23340fc360 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f2330ff9640 1 -- 172.21.15.110:0/2847345269 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f232400ae80 con 0x7f23340fc360 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23341112c0 con 0x7f23340fc360 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f2330ff9640 1 -- 172.21.15.110:0/2847345269 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2324012ea0 con 0x7f23340fc360 2024-04-25T15:46:48.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.913+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23341117b0 con 0x7f23340fc360 2024-04-25T15:46:48.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.914+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2334105570 con 0x7f23340fc360 2024-04-25T15:46:48.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.915+0000 7f2330ff9640 1 -- 172.21.15.110:0/2847345269 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2324012400 con 0x7f23340fc360 2024-04-25T15:46:48.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.915+0000 7f2330ff9640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2320069210 0x7f232006b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:48.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.915+0000 7f2330ff9640 1 -- 172.21.15.110:0/2847345269 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2324087410 con 0x7f23340fc360 2024-04-25T15:46:48.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.915+0000 7f2333fff640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2320069210 0x7f232006b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:48.917 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.916+0000 7f2333fff640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2320069210 0x7f232006b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f231c0317e0 tx=0x7f231c009da0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:48.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:48.919+0000 7f2330ff9640 1 -- 172.21.15.110:0/2847345269 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2324053740 con 0x7f23340fc360 2024-04-25T15:46:48.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:48 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1361917285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:46:48.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:48 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1867190704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:46:48.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:48 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/4264267721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T15:46:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:48 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:48 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:48 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:48 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:46:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:48 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:48 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1361917285' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T15:46:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:48 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1867190704' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T15:46:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:48 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/4264267721' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T15:46:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:48 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:48 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:48 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:46:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:48 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:46:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:48 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:46:49.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.120+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2334103be0 con 0x7f23340fc360 2024-04-25T15:46:49.121 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.120+0000 7f2330ff9640 1 -- 172.21.15.110:0/2847345269 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4689 (secure 0 0 0) 0x7f23240126b0 con 0x7f23340fc360 2024-04-25T15:46:49.121 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:46:49.121 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.110:6829/164855346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":164855346},{"type":"v1","addr":"172.21.15.110:6829","nonce":164855346}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:46:35.394851+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14468},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14468":{"gid":14468,"name":"cephfs.smithi110.mvvbtv","rank":0,"incarnation":4,"state":"up:active","state_seq":4,"addr":"172.21.15.110:6827/123050770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":123050770},{"type":"v1","addr":"172.21.15.110:6827","nonce":123050770}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2024-04-25T15:46:49.121 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:46:49.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2320069210 msgr2=0x7f232006b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:49.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2320069210 0x7f232006b6c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f231c0317e0 tx=0x7f231c009da0 comp rx=0 tx=0).stop 2024-04-25T15:46:49.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fc360 msgr2=0x7f233410c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:49.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fc360 0x7f233410c0d0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f232400c830 tx=0x7f2324004430 comp rx=0 tx=0).stop 2024-04-25T15:46:49.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 shutdown_connections 2024-04-25T15:46:49.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2320069210 0x7f232006b6c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:49.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23340fc360 0x7f233410c0d0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:49.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 --2- 172.21.15.110:0/2847345269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340fb090 0x7f233410bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:49.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 >> 172.21.15.110:0/2847345269 conn(0x7f23340f6870 msgr2=0x7f23340ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:49.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 shutdown_connections 2024-04-25T15:46:49.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:49.122+0000 7f233a4ae640 1 -- 172.21.15.110:0/2847345269 wait complete. 2024-04-25T15:46:49.134 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 13, 'max_mds': 1, 'flags': 18} 2024-04-25T15:46:49.134 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T15:46:49.164 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T15:46:49.164 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-04-25T15:46:49.164 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T15:46:49.164 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T15:46:49.164 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:49.164 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T15:46:49.164 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T15:46:49.164 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:49.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:49.165 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-04-25T15:46:49.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:49.180 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link delete ceph-brx 2024-04-25T15:46:49.245 INFO:teuthology.orchestra.run.smithi110.stderr:Cannot find device "ceph-brx" 2024-04-25T15:46:49.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:46:49.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:49.246 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-04-25T15:46:49.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:49.265 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link delete ceph-brx 2024-04-25T15:46:49.338 INFO:teuthology.orchestra.run.smithi193.stderr:Cannot find device "ceph-brx" 2024-04-25T15:46:49.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T15:46:49.340 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T15:46:49.340 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T15:46:49.590 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:50.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:49 smithi110 ceph-mon[26344]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:50.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:49 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3842262695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T15:46:50.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:49 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2847345269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:46:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:49 smithi193 ceph-mon[33747]: pgmap v187: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:49 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3842262695' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T15:46:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:49 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2847345269' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T15:46:51.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 -- 172.21.15.110:0/3965148053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84102de0 msgr2=0x7efe841031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:51.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 --2- 172.21.15.110:0/3965148053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84102de0 0x7efe841031b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7efe80009ce0 tx=0x7efe80004d10 comp rx=0 tx=0).stop 2024-04-25T15:46:51.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 -- 172.21.15.110:0/3965148053 shutdown_connections 2024-04-25T15:46:51.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 --2- 172.21.15.110:0/3965148053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84103780 0x7efe840fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:51.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 --2- 172.21.15.110:0/3965148053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84102de0 0x7efe841031b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:51.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 -- 172.21.15.110:0/3965148053 >> 172.21.15.110:0/3965148053 conn(0x7efe840f8950 msgr2=0x7efe840fad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:51.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 -- 172.21.15.110:0/3965148053 shutdown_connections 2024-04-25T15:46:51.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 -- 172.21.15.110:0/3965148053 wait complete. 2024-04-25T15:46:51.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 Processor -- start 2024-04-25T15:46:51.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.155+0000 7efe8c7b7700 1 -- start start 2024-04-25T15:46:51.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe8c7b7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84102de0 0x7efe84113cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:51.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe8c7b7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84103780 0x7efe84114200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:51.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe8c7b7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe841148e0 con 0x7efe84103780 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe8c7b7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8410dc20 con 0x7efe84102de0 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe89d52700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84103780 0x7efe84114200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe89d52700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84103780 0x7efe84114200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53054/0 (socket says 172.21.15.110:53054) 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe89d52700 1 -- 172.21.15.110:0/2251397235 learned_addr learned my addr 172.21.15.110:0/2251397235 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe8a553700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84102de0 0x7efe84113cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe89d52700 1 -- 172.21.15.110:0/2251397235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84102de0 msgr2=0x7efe84113cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe89d52700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84102de0 0x7efe84113cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe89d52700 1 -- 172.21.15.110:0/2251397235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe80009940 con 0x7efe84103780 2024-04-25T15:46:51.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe8a553700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84102de0 0x7efe84113cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:46:51.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe89d52700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84103780 0x7efe84114200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe7400b790 tx=0x7efe74009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:51.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe7affd700 1 -- 172.21.15.110:0/2251397235 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe7400f6f0 con 0x7efe84103780 2024-04-25T15:46:51.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8410dea0 con 0x7efe84103780 2024-04-25T15:46:51.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe7affd700 1 -- 172.21.15.110:0/2251397235 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe7400fd20 con 0x7efe84103780 2024-04-25T15:46:51.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe7affd700 1 -- 172.21.15.110:0/2251397235 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe740103a0 con 0x7efe84103780 2024-04-25T15:46:51.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.156+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8410e460 con 0x7efe84103780 2024-04-25T15:46:51.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.157+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe841000f0 con 0x7efe84103780 2024-04-25T15:46:51.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.158+0000 7efe7affd700 1 -- 172.21.15.110:0/2251397235 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7efe74011400 con 0x7efe84103780 2024-04-25T15:46:51.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.158+0000 7efe7affd700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7efe700692a0 0x7efe7006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:51.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.158+0000 7efe7affd700 1 -- 172.21.15.110:0/2251397235 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7efe74087060 con 0x7efe84103780 2024-04-25T15:46:51.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.158+0000 7efe8a553700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7efe700692a0 0x7efe7006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:51.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.159+0000 7efe8a553700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7efe700692a0 0x7efe7006b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe8000b740 tx=0x7efe80004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:51.164 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.162+0000 7efe7affd700 1 -- 172.21.15.110:0/2251397235 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7efe74056da0 con 0x7efe84103780 2024-04-25T15:46:51.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.396+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efe84109f50 con 0x7efe84103780 2024-04-25T15:46:51.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.396+0000 7efe7affd700 1 -- 172.21.15.110:0/2251397235 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+83 (secure 0 0 0) 0x7efe74056930 con 0x7efe84103780 2024-04-25T15:46:51.397 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7efe700692a0 msgr2=0x7efe7006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7efe700692a0 0x7efe7006b750 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7efe8000b740 tx=0x7efe80004d10 comp rx=0 tx=0).stop 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84103780 msgr2=0x7efe84114200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84103780 0x7efe84114200 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efe7400b790 tx=0x7efe74009220 comp rx=0 tx=0).stop 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 shutdown_connections 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7efe700692a0 0x7efe7006b750 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe84102de0 0x7efe84113cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 --2- 172.21.15.110:0/2251397235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efe84103780 0x7efe84114200 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 >> 172.21.15.110:0/2251397235 conn(0x7efe840f8950 msgr2=0x7efe84107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 shutdown_connections 2024-04-25T15:46:51.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:51.398+0000 7efe8c7b7700 1 -- 172.21.15.110:0/2251397235 wait complete. 2024-04-25T15:46:52.021 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:51 smithi110 ceph-mon[26344]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:52.022 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:51 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2251397235' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:46:52.046 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T15:46:52.046 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T15:46:52.046 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi110.front.sepia.ceph.com 2024-04-25T15:46:52.046 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T15:46:52.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:52.047 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T15:46:52.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T15:46:52.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T15:46:52.047 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T15:46:52.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:52.047 DEBUG:teuthology.orchestra.run.smithi110:> ip addr 2024-04-25T15:46:52.063 INFO:teuthology.orchestra.run.smithi110.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T15:46:52.063 INFO:teuthology.orchestra.run.smithi110.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T15:46:52.063 INFO:teuthology.orchestra.run.smithi110.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T15:46:52.063 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 ::1/128 scope host 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f0 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:d8:db:15 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: altname enp7s0f1 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:34 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: link/ether 0c:c4:7a:8f:cc:35 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: inet 172.21.15.110/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft 42328sec preferred_lft 42328sec 2024-04-25T15:46:52.064 INFO:teuthology.orchestra.run.smithi110.stdout: inet6 fe80::e268:46e2:ccbd:e2c8/64 scope link noprefixroute 2024-04-25T15:46:52.065 INFO:teuthology.orchestra.run.smithi110.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:46:52.065 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T15:46:52.065 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:52.065 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T15:46:52.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add name ceph-brx type bridge 2024-04-25T15:46:52.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip addr flush dev ceph-brx 2024-04-25T15:46:52.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set ceph-brx up 2024-04-25T15:46:52.065 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T15:46:52.066 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T15:46:52.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:51 smithi193 ceph-mon[33747]: pgmap v188: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:51 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2251397235' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:46:52.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:52.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:52.258 DEBUG:teuthology.orchestra.run.smithi110:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T15:46:52.282 INFO:teuthology.orchestra.run.smithi110.stdout:1 2024-04-25T15:46:52.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:52.284 DEBUG:teuthology.orchestra.run.smithi110:> route 2024-04-25T15:46:52.350 INFO:teuthology.orchestra.run.smithi110.stdout:Kernel IP routing table 2024-04-25T15:46:52.350 INFO:teuthology.orchestra.run.smithi110.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T15:46:52.350 INFO:teuthology.orchestra.run.smithi110.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T15:46:52.350 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T15:46:52.350 INFO:teuthology.orchestra.run.smithi110.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T15:46:52.351 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:52.351 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T15:46:52.351 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T15:46:52.351 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T15:46:52.351 DEBUG:teuthology.orchestra.run.smithi110:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T15:46:52.351 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T15:46:52.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:52.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:52.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:52.647 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list 2024-04-25T15:46:52.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:52.662 DEBUG:teuthology.orchestra.run.smithi110:> ip netns list-id 2024-04-25T15:46:52.716 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:52.716 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T15:46:52.717 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:46:52.717 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T15:46:52.717 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T15:46:52.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:52.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:52.832 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T15:46:52.833 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:52.833 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T15:46:52.833 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T15:46:52.833 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T15:46:52.833 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T15:46:52.833 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T15:46:52.833 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T15:46:52.833 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T15:46:52.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:52.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:52.952 DEBUG:teuthology.orchestra.run.smithi110:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:52.953 DEBUG:teuthology.orchestra.run.smithi110:> set -e 2024-04-25T15:46:52.953 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set brx.0 up 2024-04-25T15:46:52.953 DEBUG:teuthology.orchestra.run.smithi110:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T15:46:52.953 DEBUG:teuthology.orchestra.run.smithi110:> ') 2024-04-25T15:46:53.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:53.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:53.026 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T15:46:53.026 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T15:46:53.026 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:53.041 INFO:teuthology.orchestra.run.smithi110.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T15:46:53.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T15:46:53.041 DEBUG:teuthology.orchestra.run.smithi110:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:53.095 DEBUG:teuthology.orchestra.run.smithi110:> sudo modprobe fuse 2024-04-25T15:46:53.163 DEBUG:teuthology.orchestra.run.smithi110:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T15:46:53.219 INFO:teuthology.orchestra.run.smithi110.stdout:/proc 2024-04-25T15:46:53.219 INFO:teuthology.orchestra.run.smithi110.stdout:/sys 2024-04-25T15:46:53.219 INFO:teuthology.orchestra.run.smithi110.stdout:/dev 2024-04-25T15:46:53.219 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/security 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/shm 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/pts 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/run 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/cgroup 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/pstore 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/bpf 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/ 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/selinux 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/hugepages 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/mqueue 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/debug 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/tracing 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T15:46:53.220 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/config 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/fuse/connections 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/1000 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/ceph 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/cd26ae3598d05a8d36c9ff2840c8a9f5dbe2bbc06a46ceecb357012d9fff3233/merged 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/2e3acab80807655dfc69468281daaa5623966e15bb5d65099d7461754b4aaf1c/merged 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/0 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/a32b1d4c23efb0765dc4816fbce2bcd20ce23e74329104bd3b48fd4fcdd297e0/merged 2024-04-25T15:46:53.221 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/8f5a9928807a50edc846886c23970ababaa8fde6927e5ed810cfa7147ba8b1a1/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/3e39502e90bf5ddfd1e1de82ed4cd1cf28614f92c132d13764c70f4138b356fc/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/9db524ff760fb66998358c39d605edf54939e1692f5a89bdeae1d5eaf4f06c03/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/7598ecc21abdc40ec38e79a1984d23b541cc4e8966d7a6ecf6dbf4c8333190a4/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/7d479cb915a0639d203b406360ffb135254fcee2dcd1d559802888d8e9a8827f/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/fc4ab0b21ce4e36deb79273d07ea8850a4632477598cb8610f7d8f4050866d15/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/f8b875c92b5a1064993457b93f930f59030cffa7fb96aff53cfe1b40481d11a0/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/3b3e065f184f7b0ecd6f1b44bb5eee6b2b3bd4bcde388868f9b790cc1d13a2de/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/62c4ee311946e32c7576a533860cc8e4b8777e6402d0c6c7e1942ed610211113/merged 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:46:53.222 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:46:53.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:53.223 DEBUG:teuthology.orchestra.run.smithi110:> ls /sys/fs/fuse/connections 2024-04-25T15:46:53.275 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T15:46:53.275 DEBUG:teuthology.orchestra.run.smithi110:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T15:46:53.318 DEBUG:teuthology.orchestra.run.smithi110:> sudo modprobe fuse 2024-04-25T15:46:53.341 DEBUG:teuthology.orchestra.run.smithi110:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T15:46:53.398 INFO:teuthology.orchestra.run.smithi110.stdout:/proc 2024-04-25T15:46:53.398 INFO:teuthology.orchestra.run.smithi110.stdout:/sys 2024-04-25T15:46:53.398 INFO:teuthology.orchestra.run.smithi110.stdout:/dev 2024-04-25T15:46:53.398 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/security 2024-04-25T15:46:53.398 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/shm 2024-04-25T15:46:53.398 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/pts 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/run 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/cgroup 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/pstore 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/bpf 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/ 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/selinux 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/hugepages 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/dev/mqueue 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/debug 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/tracing 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T15:46:53.399 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/kernel/config 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/sys/fs/fuse/connections 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/1000 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/ceph 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/cd26ae3598d05a8d36c9ff2840c8a9f5dbe2bbc06a46ceecb357012d9fff3233/merged 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/2e3acab80807655dfc69468281daaa5623966e15bb5d65099d7461754b4aaf1c/merged 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/run/user/0 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T15:46:53.400 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/a32b1d4c23efb0765dc4816fbce2bcd20ce23e74329104bd3b48fd4fcdd297e0/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/8f5a9928807a50edc846886c23970ababaa8fde6927e5ed810cfa7147ba8b1a1/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/3e39502e90bf5ddfd1e1de82ed4cd1cf28614f92c132d13764c70f4138b356fc/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/9db524ff760fb66998358c39d605edf54939e1692f5a89bdeae1d5eaf4f06c03/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/7598ecc21abdc40ec38e79a1984d23b541cc4e8966d7a6ecf6dbf4c8333190a4/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/7d479cb915a0639d203b406360ffb135254fcee2dcd1d559802888d8e9a8827f/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/fc4ab0b21ce4e36deb79273d07ea8850a4632477598cb8610f7d8f4050866d15/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/f8b875c92b5a1064993457b93f930f59030cffa7fb96aff53cfe1b40481d11a0/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/3b3e065f184f7b0ecd6f1b44bb5eee6b2b3bd4bcde388868f9b790cc1d13a2de/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/var/lib/containers/storage/overlay/62c4ee311946e32c7576a533860cc8e4b8777e6402d0c6c7e1942ed610211113/merged 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:46:53.401 INFO:teuthology.orchestra.run.smithi110.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T15:46:53.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:53.402 DEBUG:teuthology.orchestra.run.smithi110:> ls /sys/fs/fuse/connections 2024-04-25T15:46:53.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi110.stderr:2024-04-25T15:46:53.410+0000 7f079d168300 -1 init, newargv = 0x55db17035a40 newargc=15 2024-04-25T15:46:53.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi110.stderr:ceph-fuse[73738]: starting ceph client 2024-04-25T15:46:53.417 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi110.stderr:ceph-fuse[73738]: starting fuse 2024-04-25T15:46:53.428 INFO:teuthology.orchestra.run.smithi110.stdout:145 2024-04-25T15:46:53.428 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [145] 2024-04-25T15:46:53.428 DEBUG:teuthology.orchestra.run.smithi110:> sudo stdin-killer -- python3 -c ' 2024-04-25T15:46:53.428 DEBUG:teuthology.orchestra.run.smithi110:> import glob 2024-04-25T15:46:53.428 DEBUG:teuthology.orchestra.run.smithi110:> import re 2024-04-25T15:46:53.428 DEBUG:teuthology.orchestra.run.smithi110:> import os 2024-04-25T15:46:53.428 DEBUG:teuthology.orchestra.run.smithi110:> import subprocess 2024-04-25T15:46:53.428 DEBUG:teuthology.orchestra.run.smithi110:> 2024-04-25T15:46:53.428 DEBUG:teuthology.orchestra.run.smithi110:> def _find_admin_socket(client_name): 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> files = glob.glob(asok_path) 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> # Given a non-glob path, it better be there 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> if "*" not in asok_path: 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> assert(len(files) == 1) 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> return files[0] 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> for f in files: 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> contents = proc_f.read() 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> if mountpoint in contents: 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> return f 2024-04-25T15:46:53.429 DEBUG:teuthology.orchestra.run.smithi110:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T15:46:53.430 DEBUG:teuthology.orchestra.run.smithi110:> 2024-04-25T15:46:53.430 DEBUG:teuthology.orchestra.run.smithi110:> print(_find_admin_socket("client.0")) 2024-04-25T15:46:53.430 DEBUG:teuthology.orchestra.run.smithi110:> ' 2024-04-25T15:46:53.548 INFO:teuthology.orchestra.run.smithi110.stdout:/var/run/ceph/ceph-client.0.73738.asok 2024-04-25T15:46:53.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:53.560 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.73738.asok 2024-04-25T15:46:53.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:53.560 DEBUG:teuthology.orchestra.run.smithi110:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.73738.asok status 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "metadata": { 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_sha1": "c61472c83ce76869b0fe076c19ed2f4bcc3fb4af", 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "entity_id": "0", 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "hostname": "smithi110", 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "pid": "73738", 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "root": "/" 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:46:53.691 INFO:teuthology.orchestra.run.smithi110.stdout: "dentry_count": 0, 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "dentry_pinned_count": 0, 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "id": 14526, 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "inst": { 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "name": { 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "client", 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "num": 14526 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": { 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v1", 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "192.168.0.1:0", 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 2988371434 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": { 2024-04-25T15:46:53.692 INFO:teuthology.orchestra.run.smithi110.stdout: "type": "v1", 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "addr": "192.168.0.1:0", 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "nonce": 2988371434 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "inst_str": "client.14526 192.168.0.1:0/2988371434", 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "addr_str": "192.168.0.1:0/2988371434", 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "inode_count": 1, 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_epoch": 13, 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_epoch": 45, 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_epoch_barrier": 0, 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "blocklisted": false, 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout: "fs_name": "cephfs" 2024-04-25T15:46:53.693 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:46:53.700 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T15:46:53.961 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:46:54.059 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:53 smithi193 ceph-mon[33747]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:54.083 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:53 smithi110 ceph-mon[26344]: pgmap v189: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:46:55.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.517+0000 7f5b2a39f700 1 -- 172.21.15.110:0/2309946992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 msgr2=0x7f5b24100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:55.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.517+0000 7f5b2a39f700 1 --2- 172.21.15.110:0/2309946992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 0x7f5b24100e60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f5b14004940 tx=0x7f5b14009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:55.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.517+0000 7f5b2a39f700 1 -- 172.21.15.110:0/2309946992 shutdown_connections 2024-04-25T15:46:55.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.517+0000 7f5b2a39f700 1 --2- 172.21.15.110:0/2309946992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b240fbb60 0x7f5b240fbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:55.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.517+0000 7f5b2a39f700 1 --2- 172.21.15.110:0/2309946992 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 0x7f5b24100e60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.517+0000 7f5b2a39f700 1 -- 172.21.15.110:0/2309946992 >> 172.21.15.110:0/2309946992 conn(0x7f5b24071b40 msgr2=0x7f5b24071f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.517+0000 7f5b2a39f700 1 -- 172.21.15.110:0/2309946992 shutdown_connections 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.517+0000 7f5b2a39f700 1 -- 172.21.15.110:0/2309946992 wait complete. 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b2a39f700 1 Processor -- start 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b2a39f700 1 -- start start 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b2a39f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b240fbb60 0x7f5b240fd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b2a39f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 0x7f5b240fdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b2a39f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240fe290 con 0x7f5b24100a90 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b2a39f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b240fe3d0 con 0x7f5b240fbb60 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b28b9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 0x7f5b240fdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b28b9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 0x7f5b240fdc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53066/0 (socket says 172.21.15.110:53066) 2024-04-25T15:46:55.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b28b9c700 1 -- 172.21.15.110:0/362318689 learned_addr learned my addr 172.21.15.110:0/362318689 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b2939d700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b240fbb60 0x7f5b240fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b28b9c700 1 -- 172.21.15.110:0/362318689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b240fbb60 msgr2=0x7f5b240fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b28b9c700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b240fbb60 0x7f5b240fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.518+0000 7f5b28b9c700 1 -- 172.21.15.110:0/362318689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14005040 con 0x7f5b24100a90 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.519+0000 7f5b28b9c700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 0x7f5b240fdc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000b790 tx=0x7f5b20009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.519+0000 7f5b19ffb700 1 -- 172.21.15.110:0/362318689 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2000f6f0 con 0x7f5b24100a90 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.519+0000 7f5b19ffb700 1 -- 172.21.15.110:0/362318689 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b2000fd20 con 0x7f5b24100a90 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.519+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b24119cc0 con 0x7f5b24100a90 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.519+0000 7f5b19ffb700 1 -- 172.21.15.110:0/362318689 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b200103a0 con 0x7f5b24100a90 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.519+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2411a150 con 0x7f5b24100a90 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.520+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b240fd200 con 0x7f5b24100a90 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.520+0000 7f5b19ffb700 1 -- 172.21.15.110:0/362318689 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f5b20011400 con 0x7f5b24100a90 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.521+0000 7f5b19ffb700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5b100692a0 0x7f5b1006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:46:55.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.521+0000 7f5b2939d700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5b100692a0 0x7f5b1006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:46:55.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.521+0000 7f5b19ffb700 1 -- 172.21.15.110:0/362318689 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f5b20086e80 con 0x7f5b24100a90 2024-04-25T15:46:55.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.521+0000 7f5b2939d700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5b100692a0 0x7f5b1006b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b14004f90 tx=0x7f5b14009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:46:55.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.525+0000 7f5b19ffb700 1 -- 172.21.15.110:0/362318689 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f5b20053250 con 0x7f5b24100a90 2024-04-25T15:46:55.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.756+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5b24108cc0 con 0x7f5b24100a90 2024-04-25T15:46:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.757+0000 7f5b19ffb700 1 -- 172.21.15.110:0/362318689 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+83 (secure 0 0 0) 0x7f5b20056870 con 0x7f5b24100a90 2024-04-25T15:46:55.758 INFO:teuthology.orchestra.run.smithi110.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5b100692a0 msgr2=0x7f5b1006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5b100692a0 0x7f5b1006b750 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5b14004f90 tx=0x7f5b14009e30 comp rx=0 tx=0).stop 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 msgr2=0x7f5b240fdc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 0x7f5b240fdc70 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000b790 tx=0x7f5b20009250 comp rx=0 tx=0).stop 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 shutdown_connections 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f5b100692a0 0x7f5b1006b750 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b240fbb60 0x7f5b240fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 --2- 172.21.15.110:0/362318689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b24100a90 0x7f5b240fdc70 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 >> 172.21.15.110:0/362318689 conn(0x7f5b24071b40 msgr2=0x7f5b240655e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 shutdown_connections 2024-04-25T15:46:55.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:46:55.759+0000 7f5b2a39f700 1 -- 172.21.15.110:0/362318689 wait complete. 2024-04-25T15:46:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:55 smithi110 ceph-mon[26344]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-25T15:46:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:46:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:55 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/362318689' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:46:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:55 smithi193 ceph-mon[33747]: pgmap v190: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-25T15:46:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:46:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:55 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/362318689' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi193.front.sepia.ceph.com 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T15:46:56.461 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T15:46:56.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:56.462 DEBUG:teuthology.orchestra.run.smithi193:> ip addr 2024-04-25T15:46:56.481 INFO:teuthology.orchestra.run.smithi193.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T15:46:56.482 INFO:teuthology.orchestra.run.smithi193.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T15:46:56.482 INFO:teuthology.orchestra.run.smithi193.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T15:46:56.482 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:46:56.482 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 ::1/128 scope host 2024-04-25T15:46:56.482 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:46:56.482 INFO:teuthology.orchestra.run.smithi193.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:46:56.482 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b6 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:46:56.482 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f0 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b7 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f1 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:52 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:53 brd ff:ff:ff:ff:ff:ff 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout: inet 172.21.15.193/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft 42323sec preferred_lft 42323sec 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::4273:113d:c726:3057/64 scope link noprefixroute 2024-04-25T15:46:56.483 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T15:46:56.484 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T15:46:56.484 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:56.484 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T15:46:56.484 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add name ceph-brx type bridge 2024-04-25T15:46:56.484 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip addr flush dev ceph-brx 2024-04-25T15:46:56.484 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set ceph-brx up 2024-04-25T15:46:56.484 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T15:46:56.484 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T15:46:56.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:56.754 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:56.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:56.761 DEBUG:teuthology.orchestra.run.smithi193:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T15:46:56.784 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-04-25T15:46:56.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:56.786 DEBUG:teuthology.orchestra.run.smithi193:> route 2024-04-25T15:46:56.848 INFO:teuthology.orchestra.run.smithi193.stdout:Kernel IP routing table 2024-04-25T15:46:56.848 INFO:teuthology.orchestra.run.smithi193.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T15:46:56.848 INFO:teuthology.orchestra.run.smithi193.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T15:46:56.848 INFO:teuthology.orchestra.run.smithi193.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T15:46:56.848 INFO:teuthology.orchestra.run.smithi193.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T15:46:56.848 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:56.848 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T15:46:56.848 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T15:46:56.849 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T15:46:56.849 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T15:46:56.849 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T15:46:56.938 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:57.150 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:57.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:57.157 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-04-25T15:46:57.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:57.172 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list-id 2024-04-25T15:46:57.226 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:57.226 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T15:46:57.226 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T15:46:57.226 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T15:46:57.226 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T15:46:57.313 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:57.335 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:57.341 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T15:46:57.342 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:57.342 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T15:46:57.342 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T15:46:57.342 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T15:46:57.342 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T15:46:57.342 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T15:46:57.342 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T15:46:57.342 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T15:46:57.389 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:57.449 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:57.455 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T15:46:57.455 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T15:46:57.456 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set brx.0 up 2024-04-25T15:46:57.456 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T15:46:57.456 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T15:46:57.502 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T15:46:57.523 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:57.529 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T15:46:57.529 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T15:46:57.529 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:57.543 INFO:teuthology.orchestra.run.smithi193.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T15:46:57.543 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T15:46:57.543 DEBUG:teuthology.orchestra.run.smithi193:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:57.595 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-04-25T15:46:57.658 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-04-25T15:46:57.714 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T15:46:57.715 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/0 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/18c023b65850a17f6efcef6430e964ff53025ea11c65fcd993d048155dec2b60/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/00c82bd6dd72043b3e020d86773d47565c3a9dcdaa02e46d23f5fd26cda6a8ae/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/0b8c41a77272fc063a031296578fcd6eb96feb422182955a8872a9da5e25bdec/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/91bd2fa9e2d2893b9e7f9f937f1661ef67b3ce967eff5dec95aa6e2b336c5dce/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/3117920298bc441e6f2ef02fb105e187971538e6b503a6577e35df8f68ef47cd/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/4eeb17f495becd48713224b8418d46065cddf8986fbfb8703d198c4179b76c7a/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/6c404606eaaf0a01a62164b53e588c952c705bf5cabf80f2ea1bece54daa502d/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/27c9499688b2bbb1add0e85b2abb54b9c4a6f47fbf6c5538395914fad2b7bb7f/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/8ea141a55e49d85e57340f8b90c13022f471241c256295b7b3ae7dac600619a3/merged 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T15:46:57.716 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T15:46:57.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:57.717 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-04-25T15:46:57.772 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T15:46:57.772 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T15:46:57.813 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-04-25T15:46:57.836 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-04-25T15:46:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T15:46:57.893 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/0 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/18c023b65850a17f6efcef6430e964ff53025ea11c65fcd993d048155dec2b60/merged 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/00c82bd6dd72043b3e020d86773d47565c3a9dcdaa02e46d23f5fd26cda6a8ae/merged 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/0b8c41a77272fc063a031296578fcd6eb96feb422182955a8872a9da5e25bdec/merged 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/91bd2fa9e2d2893b9e7f9f937f1661ef67b3ce967eff5dec95aa6e2b336c5dce/merged 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/3117920298bc441e6f2ef02fb105e187971538e6b503a6577e35df8f68ef47cd/merged 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/4eeb17f495becd48713224b8418d46065cddf8986fbfb8703d198c4179b76c7a/merged 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/6c404606eaaf0a01a62164b53e588c952c705bf5cabf80f2ea1bece54daa502d/merged 2024-04-25T15:46:57.894 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/27c9499688b2bbb1add0e85b2abb54b9c4a6f47fbf6c5538395914fad2b7bb7f/merged 2024-04-25T15:46:57.895 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/containers/storage/overlay/8ea141a55e49d85e57340f8b90c13022f471241c256295b7b3ae7dac600619a3/merged 2024-04-25T15:46:57.895 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-04-25T15:46:57.895 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T15:46:57.895 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T15:46:57.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:57.895 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-04-25T15:46:57.906 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:2024-04-25T15:46:57.905+0000 7f648d06c300 -1 init, newargv = 0x559ad1a048d0 newargc=15 2024-04-25T15:46:57.906 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:ceph-fuse[59490]: starting ceph client 2024-04-25T15:46:57.910 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:ceph-fuse[59490]: starting fuse 2024-04-25T15:46:57.921 INFO:teuthology.orchestra.run.smithi193.stdout:112 2024-04-25T15:46:57.921 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [112] 2024-04-25T15:46:57.921 DEBUG:teuthology.orchestra.run.smithi193:> sudo stdin-killer -- python3 -c ' 2024-04-25T15:46:57.921 DEBUG:teuthology.orchestra.run.smithi193:> import glob 2024-04-25T15:46:57.921 DEBUG:teuthology.orchestra.run.smithi193:> import re 2024-04-25T15:46:57.921 DEBUG:teuthology.orchestra.run.smithi193:> import os 2024-04-25T15:46:57.921 DEBUG:teuthology.orchestra.run.smithi193:> import subprocess 2024-04-25T15:46:57.921 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T15:46:57.921 DEBUG:teuthology.orchestra.run.smithi193:> def _find_admin_socket(client_name): 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> files = glob.glob(asok_path) 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> # Given a non-glob path, it better be there 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> if "*" not in asok_path: 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> assert(len(files) == 1) 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> return files[0] 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> for f in files: 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> contents = proc_f.read() 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> if mountpoint in contents: 2024-04-25T15:46:57.922 DEBUG:teuthology.orchestra.run.smithi193:> return f 2024-04-25T15:46:57.923 DEBUG:teuthology.orchestra.run.smithi193:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T15:46:57.923 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T15:46:57.923 DEBUG:teuthology.orchestra.run.smithi193:> print(_find_admin_socket("client.1")) 2024-04-25T15:46:57.923 DEBUG:teuthology.orchestra.run.smithi193:> ' 2024-04-25T15:46:57.981 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:57 smithi193 ceph-mon[33747]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-25T15:46:58.038 INFO:teuthology.orchestra.run.smithi193.stdout:/var/run/ceph/ceph-client.1.59490.asok 2024-04-25T15:46:58.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T15:46:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T15:46:58.048 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.59490.asok 2024-04-25T15:46:58.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:58.048 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.59490.asok status 2024-04-25T15:46:58.112 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:57 smithi110 ceph-mon[26344]: pgmap v191: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-25T15:46:58.177 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T15:46:58.177 INFO:teuthology.orchestra.run.smithi193.stdout: "metadata": { 2024-04-25T15:46:58.177 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_sha1": "c61472c83ce76869b0fe076c19ed2f4bcc3fb4af", 2024-04-25T15:46:58.177 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)", 2024-04-25T15:46:58.177 INFO:teuthology.orchestra.run.smithi193.stdout: "entity_id": "1", 2024-04-25T15:46:58.177 INFO:teuthology.orchestra.run.smithi193.stdout: "hostname": "smithi193", 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "pid": "59490", 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "root": "/" 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_count": 0, 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_pinned_count": 0, 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "id": 24297, 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "inst": { 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "name": { 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "client", 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "num": 24297 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-04-25T15:46:58.178 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.1:0", 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 4279727202 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.1:0", 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 4279727202 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "inst_str": "client.24297 192.168.0.1:0/4279727202", 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "addr_str": "192.168.0.1:0/4279727202", 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "inode_count": 1, 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_epoch": 13, 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch": 45, 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch_barrier": 0, 2024-04-25T15:46:58.179 INFO:teuthology.orchestra.run.smithi193.stdout: "blocklisted": false, 2024-04-25T15:46:58.180 INFO:teuthology.orchestra.run.smithi193.stdout: "fs_name": "cephfs" 2024-04-25T15:46:58.180 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T15:46:58.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:58.185 DEBUG:teuthology.orchestra.run.smithi110:> stat --file-system '--printf=%T 2024-04-25T15:46:58.185 DEBUG:teuthology.orchestra.run.smithi110:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:58.205 INFO:teuthology.orchestra.run.smithi110.stdout:fuseblk 2024-04-25T15:46:58.206 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:58.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:58.206 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:58.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:58.278 DEBUG:teuthology.orchestra.run.smithi193:> stat --file-system '--printf=%T 2024-04-25T15:46:58.278 DEBUG:teuthology.orchestra.run.smithi193:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:58.293 INFO:teuthology.orchestra.run.smithi193.stdout:fuseblk 2024-04-25T15:46:58.294 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:58.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T15:46:58.294 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:58.360 INFO:teuthology.run_tasks:Running task print... 2024-04-25T15:46:58.370 INFO:teuthology.task.print:**** done client 2024-04-25T15:46:58.370 INFO:teuthology.run_tasks:Running task parallel... 2024-04-25T15:46:58.381 INFO:teuthology.task.parallel:starting parallel... 2024-04-25T15:46:58.381 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-25T15:46:58.382 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-25T15:46:58.382 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:46:58.383 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs false || true' 2024-04-25T15:46:58.383 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-25T15:46:58.383 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-25T15:46:58.387 INFO:tasks.workunit:Pulling workunits from ref 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:46:58.387 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T15:46:58.387 INFO:tasks.workunit:timeout=3h 2024-04-25T15:46:58.388 INFO:tasks.workunit:cleanup=True 2024-04-25T15:46:58.388 DEBUG:teuthology.orchestra.run.smithi110:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:58.406 INFO:teuthology.orchestra.run.smithi110.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:58.406 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T15:46:58.406 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 91h/145d Inode: 1 Links: 2 2024-04-25T15:46:58.406 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:46:58.406 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T15:46:58.407 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T15:46:58.407 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-04-25 15:46:23.824882662 +0000 2024-04-25T15:46:58.407 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-04-25 15:46:58.275896745 +0000 2024-04-25T15:46:58.407 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: - 2024-04-25T15:46:58.407 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T15:46:58.407 DEBUG:teuthology.orchestra.run.smithi110:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T15:46:58.475 DEBUG:teuthology.orchestra.run.smithi193:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:58.491 INFO:teuthology.orchestra.run.smithi193.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:58.491 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-04-25T15:46:58.491 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 70h/112d Inode: 1 Links: 3 2024-04-25T15:46:58.491 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T15:46:58.492 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:fusefs_t:s0 2024-04-25T15:46:58.492 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-25T15:46:58.492 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 15:46:58.470476405 +0000 2024-04-25T15:46:58.492 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 15:46:58.470476405 +0000 2024-04-25T15:46:58.492 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: - 2024-04-25T15:46:58.492 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-25T15:46:58.492 DEBUG:teuthology.orchestra.run.smithi193:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-25T15:46:58.560 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:46:58.560 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:46:58.578 INFO:tasks.workunit.client.0.smithi110.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T15:46:58.615 INFO:tasks.workunit.client.1.smithi193.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-25T15:46:58.654 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:47:00.048 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:46:59 smithi110 ceph-mon[26344]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-25T15:47:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:46:59 smithi193 ceph-mon[33747]: pgmap v192: 65 pgs: 65 active+clean; 579 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-25T15:47:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.293+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/2276763345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8f80fcf10 msgr2=0x7fb8f80fd380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:00.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.293+0000 7fb8fd2d5700 1 --2- 172.21.15.110:0/2276763345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8f80fcf10 0x7fb8f80fd380 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec004940 tx=0x7fb8ec009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.293+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/2276763345 shutdown_connections 2024-04-25T15:47:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.293+0000 7fb8fd2d5700 1 --2- 172.21.15.110:0/2276763345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8f80fcf10 0x7fb8f80fd380 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.293+0000 7fb8fd2d5700 1 --2- 172.21.15.110:0/2276763345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8f80fbc60 0x7fb8f80fc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.293+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/2276763345 >> 172.21.15.110:0/2276763345 conn(0x7fb8f80f73e0 msgr2=0x7fb8f80f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.293+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/2276763345 shutdown_connections 2024-04-25T15:47:00.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.293+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/2276763345 wait complete. 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.294+0000 7fb8fd2d5700 1 Processor -- start 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.294+0000 7fb8fd2d5700 1 -- start start 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.294+0000 7fb8fd2d5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8f80fbc60 0x7fb8f81106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.294+0000 7fb8fd2d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8f80fcf10 0x7fb8f8112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.294+0000 7fb8fd2d5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f8066e90 con 0x7fb8f80fbc60 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.294+0000 7fb8fd2d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f8066fd0 con 0x7fb8f80fcf10 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.294+0000 7fb8f6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8f80fbc60 0x7fb8f81106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.294+0000 7fb8f6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8f80fbc60 0x7fb8f81106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39044/0 (socket says 172.21.15.110:39044) 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8f6ffd700 1 -- 172.21.15.110:0/1130621722 learned_addr learned my addr 172.21.15.110:0/1130621722 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8f67fc700 1 --2- 172.21.15.110:0/1130621722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8f80fcf10 0x7fb8f8112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8f6ffd700 1 -- 172.21.15.110:0/1130621722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8f80fcf10 msgr2=0x7fb8f8112c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8f6ffd700 1 --2- 172.21.15.110:0/1130621722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8f80fcf10 0x7fb8f8112c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8f6ffd700 1 -- 172.21.15.110:0/1130621722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec005040 con 0x7fb8f80fbc60 2024-04-25T15:47:00.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8f6ffd700 1 --2- 172.21.15.110:0/1130621722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8f80fbc60 0x7fb8f81106c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb8e800b790 tx=0x7fb8e8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:00.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8df7fe700 1 -- 172.21.15.110:0/1130621722 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e800f6f0 con 0x7fb8f80fbc60 2024-04-25T15:47:00.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/1130621722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f8113270 con 0x7fb8f80fbc60 2024-04-25T15:47:00.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8df7fe700 1 -- 172.21.15.110:0/1130621722 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8e800fd20 con 0x7fb8f80fbc60 2024-04-25T15:47:00.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8df7fe700 1 -- 172.21.15.110:0/1130621722 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8e80103a0 con 0x7fb8f80fbc60 2024-04-25T15:47:00.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.295+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/1130621722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f8113800 con 0x7fb8f80fbc60 2024-04-25T15:47:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.296+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/1130621722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f80fb060 con 0x7fb8f80fbc60 2024-04-25T15:47:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.297+0000 7fb8df7fe700 1 -- 172.21.15.110:0/1130621722 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb8e8011400 con 0x7fb8f80fbc60 2024-04-25T15:47:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.297+0000 7fb8df7fe700 1 --2- 172.21.15.110:0/1130621722 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8e00692a0 0x7fb8e006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.297+0000 7fb8df7fe700 1 -- 172.21.15.110:0/1130621722 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb8e8086f80 con 0x7fb8f80fbc60 2024-04-25T15:47:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.297+0000 7fb8f67fc700 1 --2- 172.21.15.110:0/1130621722 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8e00692a0 0x7fb8e006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:00.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.298+0000 7fb8f67fc700 1 --2- 172.21.15.110:0/1130621722 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8e00692a0 0x7fb8e006b750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00b970 tx=0x7fb8ec009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:00.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.302+0000 7fb8df7fe700 1 -- 172.21.15.110:0/1130621722 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb8e80532b0 con 0x7fb8f80fbc60 2024-04-25T15:47:00.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.486+0000 7fb8fd2d5700 1 -- 172.21.15.110:0/1130621722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fb8f804bed0 con 0x7fb8f80fbc60 2024-04-25T15:47:00.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:00.487+0000 7fb8df7fe700 1 -- 172.21.15.110:0/1130621722 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21)=-22 unrecognized config option 'mgr/orchestrator/fail_fs' v21) v1 ==== 179+0+0 (secure 0 0 0) 0x7fb8e80568d0 con 0x7fb8f80fbc60 2024-04-25T15:47:00.489 INFO:teuthology.orchestra.run.smithi110.stderr:Error EINVAL: unrecognized config option 'mgr/orchestrator/fail_fs' 2024-04-25T15:47:01.173 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-25T15:47:01.173 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:47:01.174 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-25T15:47:01.467 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:47:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:01 smithi110 ceph-mon[26344]: pgmap v193: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-25T15:47:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:01 smithi193 ceph-mon[33747]: pgmap v193: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-25T15:47:03.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.034+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2586339842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 msgr2=0x7f3d580fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:03.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.034+0000 7f3d5df9f700 1 --2- 172.21.15.110:0/2586339842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 0x7f3d580fd270 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3d48004940 tx=0x7f3d48009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:03.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.034+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2586339842 shutdown_connections 2024-04-25T15:47:03.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.034+0000 7f3d5df9f700 1 --2- 172.21.15.110:0/2586339842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d580fdfd0 0x7f3d580fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:03.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.034+0000 7f3d5df9f700 1 --2- 172.21.15.110:0/2586339842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 0x7f3d580fd270 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:03.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.034+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2586339842 >> 172.21.15.110:0/2586339842 conn(0x7f3d580f8520 msgr2=0x7f3d580fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:03.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.034+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2586339842 shutdown_connections 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.034+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2586339842 wait complete. 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d5df9f700 1 Processor -- start 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d5df9f700 1 -- start start 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d5df9f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 0x7f3d5810d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d5df9f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d580fdfd0 0x7f3d5810dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d5df9f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d5810e530 con 0x7f3d580fce60 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d5df9f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d58112ed0 con 0x7f3d580fdfd0 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d577fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 0x7f3d5810d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d577fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 0x7f3d5810d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39062/0 (socket says 172.21.15.110:39062) 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d577fe700 1 -- 172.21.15.110:0/2810203095 learned_addr learned my addr 172.21.15.110:0/2810203095 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:03.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d56ffd700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d580fdfd0 0x7f3d5810dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d577fe700 1 -- 172.21.15.110:0/2810203095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d580fdfd0 msgr2=0x7f3d5810dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d577fe700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d580fdfd0 0x7f3d5810dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d577fe700 1 -- 172.21.15.110:0/2810203095 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d48005040 con 0x7f3d580fce60 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.035+0000 7f3d56ffd700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d580fdfd0 0x7f3d5810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.036+0000 7f3d577fe700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 0x7f3d5810d960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800b3b0 tx=0x7f3d48009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.036+0000 7f3d3ffff700 1 -- 172.21.15.110:0/2810203095 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d4800e040 con 0x7f3d580fce60 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.036+0000 7f3d3ffff700 1 -- 172.21.15.110:0/2810203095 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d4800bcc0 con 0x7f3d580fce60 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.036+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d581130d0 con 0x7f3d580fce60 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.036+0000 7f3d3ffff700 1 -- 172.21.15.110:0/2810203095 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d4801c630 con 0x7f3d580fce60 2024-04-25T15:47:03.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.036+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d58113640 con 0x7f3d580fce60 2024-04-25T15:47:03.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.037+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d5804c710 con 0x7f3d580fce60 2024-04-25T15:47:03.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.037+0000 7f3d3ffff700 1 -- 172.21.15.110:0/2810203095 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3d48013980 con 0x7f3d580fce60 2024-04-25T15:47:03.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.038+0000 7f3d3ffff700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3d400692a0 0x7f3d4006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:03.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.038+0000 7f3d3ffff700 1 -- 172.21.15.110:0/2810203095 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3d48087fd0 con 0x7f3d580fce60 2024-04-25T15:47:03.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.038+0000 7f3d56ffd700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3d400692a0 0x7f3d4006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:03.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.038+0000 7f3d56ffd700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3d400692a0 0x7f3d4006b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009b10 tx=0x7f3d4c008040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:03.043 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.042+0000 7f3d3ffff700 1 -- 172.21.15.110:0/2810203095 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3d48057d10 con 0x7f3d580fce60 2024-04-25T15:47:03.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.230+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f3d58105940 con 0x7f3d580fce60 2024-04-25T15:47:03.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.230+0000 7f3d3ffff700 1 -- 172.21.15.110:0/2810203095 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f3d480578a0 con 0x7f3d580fce60 2024-04-25T15:47:03.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3d400692a0 msgr2=0x7f3d4006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:03.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3d400692a0 0x7f3d4006b750 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c009b10 tx=0x7f3d4c008040 comp rx=0 tx=0).stop 2024-04-25T15:47:03.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 msgr2=0x7f3d5810d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:03.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 0x7f3d5810d960 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3d4800b3b0 tx=0x7f3d48009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:03.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 shutdown_connections 2024-04-25T15:47:03.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d580fce60 0x7f3d5810d960 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:03.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3d400692a0 0x7f3d4006b750 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:03.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 --2- 172.21.15.110:0/2810203095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d580fdfd0 0x7f3d5810dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:03.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 >> 172.21.15.110:0/2810203095 conn(0x7f3d580f8520 msgr2=0x7f3d580f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:03.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 shutdown_connections 2024-04-25T15:47:03.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:03.232+0000 7f3d5df9f700 1 -- 172.21.15.110:0/2810203095 wait complete. 2024-04-25T15:47:03.995 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:03 smithi110 ceph-mon[26344]: pgmap v194: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-25T15:47:04.019 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-25T15:47:04.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:03 smithi193 ceph-mon[33747]: pgmap v194: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-25T15:47:04.271 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:47:05.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.899+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1911146695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 msgr2=0x7f2ad40fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:05.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.899+0000 7f2adbc6a700 1 --2- 172.21.15.110:0/1911146695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 0x7f2ad40fd3e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4004940 tx=0x7f2ac4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:05.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.899+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1911146695 shutdown_connections 2024-04-25T15:47:05.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.899+0000 7f2adbc6a700 1 --2- 172.21.15.110:0/1911146695 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 0x7f2ad40fd3e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:05.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.899+0000 7f2adbc6a700 1 --2- 172.21.15.110:0/1911146695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ad40fbcc0 0x7f2ad40fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:05.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.899+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1911146695 >> 172.21.15.110:0/1911146695 conn(0x7f2ad40f7420 msgr2=0x7f2ad40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:05.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.899+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1911146695 shutdown_connections 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.899+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1911146695 wait complete. 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2adbc6a700 1 Processor -- start 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2adbc6a700 1 -- start start 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2adbc6a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ad40fbcc0 0x7f2ad4066d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2adbc6a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 0x7f2ad4114b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2adbc6a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad410e4f0 con 0x7f2ad40fcf70 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2adbc6a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad4115050 con 0x7f2ad40fbcc0 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2ad9205700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 0x7f2ad4114b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2ad9205700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 0x7f2ad4114b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39092/0 (socket says 172.21.15.110:39092) 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2ad9205700 1 -- 172.21.15.110:0/1415134574 learned_addr learned my addr 172.21.15.110:0/1415134574 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2ad9205700 1 -- 172.21.15.110:0/1415134574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ad40fbcc0 msgr2=0x7f2ad4066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2ad9205700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ad40fbcc0 0x7f2ad4066d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2ad9205700 1 -- 172.21.15.110:0/1415134574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac4005040 con 0x7f2ad40fcf70 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.900+0000 7f2ad9205700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 0x7f2ad4114b10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009380 tx=0x7f2ac4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:05.903 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.901+0000 7f2aca7fc700 1 -- 172.21.15.110:0/1415134574 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ac400e040 con 0x7f2ad40fcf70 2024-04-25T15:47:05.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.901+0000 7f2aca7fc700 1 -- 172.21.15.110:0/1415134574 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ac400bcc0 con 0x7f2ad40fcf70 2024-04-25T15:47:05.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.901+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad4115250 con 0x7f2ad40fcf70 2024-04-25T15:47:05.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.901+0000 7f2aca7fc700 1 -- 172.21.15.110:0/1415134574 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ac401c5f0 con 0x7f2ad40fcf70 2024-04-25T15:47:05.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.901+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad4115810 con 0x7f2ad40fcf70 2024-04-25T15:47:05.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.901+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad404c710 con 0x7f2ad40fcf70 2024-04-25T15:47:05.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.902+0000 7f2aca7fc700 1 -- 172.21.15.110:0/1415134574 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f2ac4013a80 con 0x7f2ad40fcf70 2024-04-25T15:47:05.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.902+0000 7f2aca7fc700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2ac00692a0 0x7f2ac006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:05.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.902+0000 7f2aca7fc700 1 -- 172.21.15.110:0/1415134574 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f2ac4087fe0 con 0x7f2ad40fcf70 2024-04-25T15:47:05.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.902+0000 7f2ad9a06700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2ac00692a0 0x7f2ac006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:05.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.903+0000 7f2ad9a06700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2ac00692a0 0x7f2ac006b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0006fd0 tx=0x7f2ad0006db0 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:05.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:05.907+0000 7f2aca7fc700 1 -- 172.21.15.110:0/1415134574 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f2ac4057d20 con 0x7f2ad40fcf70 2024-04-25T15:47:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.093+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f2ad4104830 con 0x7f2ad40fcf70 2024-04-25T15:47:06.094 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:05 smithi110 ceph-mon[26344]: pgmap v195: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-25T15:47:06.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.093+0000 7f2aca7fc700 1 -- 172.21.15.110:0/1415134574 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f2ac40578b0 con 0x7f2ad40fcf70 2024-04-25T15:47:06.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2ac00692a0 msgr2=0x7f2ac006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:06.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2ac00692a0 0x7f2ac006b750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0006fd0 tx=0x7f2ad0006db0 comp rx=0 tx=0).stop 2024-04-25T15:47:06.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 msgr2=0x7f2ad4114b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:06.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 0x7f2ad4114b10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2ac4009380 tx=0x7f2ac4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:06.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 shutdown_connections 2024-04-25T15:47:06.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f2ac00692a0 0x7f2ac006b750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:06.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ad40fbcc0 0x7f2ad4066d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:06.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 --2- 172.21.15.110:0/1415134574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2ad40fcf70 0x7f2ad4114b10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:06.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 >> 172.21.15.110:0/1415134574 conn(0x7f2ad40f7420 msgr2=0x7f2ad40f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:06.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 shutdown_connections 2024-04-25T15:47:06.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:06.096+0000 7f2adbc6a700 1 -- 172.21.15.110:0/1415134574 wait complete. 2024-04-25T15:47:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:05 smithi193 ceph-mon[33747]: pgmap v195: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 767 B/s wr, 0 op/s 2024-04-25T15:47:06.702 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-25T15:47:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:47:08.098 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:07 smithi110 ceph-mon[26344]: pgmap v196: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-25T15:47:08.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:07 smithi193 ceph-mon[33747]: pgmap v196: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-25T15:47:08.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.581+0000 7fd3cd822700 1 -- 172.21.15.110:0/4273979169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fb1b0 msgr2=0x7fd3c80fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:08.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.581+0000 7fd3cd822700 1 --2- 172.21.15.110:0/4273979169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fb1b0 0x7fd3c80fb5c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8004940 tx=0x7fd3b8009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:08.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.581+0000 7fd3cd822700 1 -- 172.21.15.110:0/4273979169 shutdown_connections 2024-04-25T15:47:08.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.581+0000 7fd3cd822700 1 --2- 172.21.15.110:0/4273979169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c80fbb00 0x7fd3c80fe360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.581+0000 7fd3cd822700 1 --2- 172.21.15.110:0/4273979169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fb1b0 0x7fd3c80fb5c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.581+0000 7fd3cd822700 1 -- 172.21.15.110:0/4273979169 >> 172.21.15.110:0/4273979169 conn(0x7fd3c80f6bb0 msgr2=0x7fd3c80f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.581+0000 7fd3cd822700 1 -- 172.21.15.110:0/4273979169 shutdown_connections 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3cd822700 1 -- 172.21.15.110:0/4273979169 wait complete. 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3cd822700 1 Processor -- start 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3cd822700 1 -- start start 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3cd822700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c80fb1b0 0x7fd3c810db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3cd822700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fbb00 0x7fd3c810e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3cd822700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c810e6f0 con 0x7fd3c80fbb00 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3cd822700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c8113090 con 0x7fd3c80fb1b0 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fbb00 0x7fd3c810e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3bffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fbb00 0x7fd3c810e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39116/0 (socket says 172.21.15.110:39116) 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.582+0000 7fd3bffff700 1 -- 172.21.15.110:0/2152342968 learned_addr learned my addr 172.21.15.110:0/2152342968 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:08.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3c6ffd700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c80fb1b0 0x7fd3c810db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3bffff700 1 -- 172.21.15.110:0/2152342968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c80fb1b0 msgr2=0x7fd3c810db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3bffff700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c80fb1b0 0x7fd3c810db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3bffff700 1 -- 172.21.15.110:0/2152342968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8005040 con 0x7fd3c80fbb00 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3c6ffd700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c80fb1b0 0x7fd3c810db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3bffff700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fbb00 0x7fd3c810e060 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000b790 tx=0x7fd3b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3bf7fe700 1 -- 172.21.15.110:0/2152342968 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3b000f770 con 0x7fd3c80fbb00 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3bf7fe700 1 -- 172.21.15.110:0/2152342968 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3b000fda0 con 0x7fd3c80fbb00 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c81132f0 con 0x7fd3c80fbb00 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3bf7fe700 1 -- 172.21.15.110:0/2152342968 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3b000d310 con 0x7fd3c80fbb00 2024-04-25T15:47:08.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.583+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c8113860 con 0x7fd3c80fbb00 2024-04-25T15:47:08.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.584+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c804c710 con 0x7fd3c80fbb00 2024-04-25T15:47:08.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.585+0000 7fd3bf7fe700 1 -- 172.21.15.110:0/2152342968 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd3b0010420 con 0x7fd3c80fbb00 2024-04-25T15:47:08.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.585+0000 7fd3bf7fe700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b40692a0 0x7fd3b406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:08.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.585+0000 7fd3bf7fe700 1 -- 172.21.15.110:0/2152342968 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fd3b00871c0 con 0x7fd3c80fbb00 2024-04-25T15:47:08.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.585+0000 7fd3c6ffd700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b40692a0 0x7fd3b406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:08.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.586+0000 7fd3c6ffd700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b40692a0 0x7fd3b406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b970 tx=0x7fd3b8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:08.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.590+0000 7fd3bf7fe700 1 -- 172.21.15.110:0/2152342968 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd3b0056f80 con 0x7fd3c80fbb00 2024-04-25T15:47:08.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.780+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fd3c8105b40 con 0x7fd3c80fbb00 2024-04-25T15:47:08.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.780+0000 7fd3bf7fe700 1 -- 172.21.15.110:0/2152342968 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd3b0056b10 con 0x7fd3c80fbb00 2024-04-25T15:47:08.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b40692a0 msgr2=0x7fd3b406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b40692a0 0x7fd3b406b750 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b970 tx=0x7fd3b8009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fbb00 msgr2=0x7fd3c810e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fbb00 0x7fd3c810e060 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd3b000b790 tx=0x7fd3b0009250 comp rx=0 tx=0).stop 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 shutdown_connections 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b40692a0 0x7fd3b406b750 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c80fb1b0 0x7fd3c810db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 --2- 172.21.15.110:0/2152342968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c80fbb00 0x7fd3c810e060 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 >> 172.21.15.110:0/2152342968 conn(0x7fd3c80f6bb0 msgr2=0x7fd3c80f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 shutdown_connections 2024-04-25T15:47:08.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:08.782+0000 7fd3cd822700 1 -- 172.21.15.110:0/2152342968 wait complete. 2024-04-25T15:47:09.433 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-25T15:47:09.689 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:47:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:09 smithi110 ceph-mon[26344]: pgmap v197: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-25T15:47:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:09 smithi193 ceph-mon[33747]: pgmap v197: 65 pgs: 65 active+clean; 585 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-25T15:47:11.080 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:10 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:47:11.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:10 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:47:11.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/349718711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4100030 msgr2=0x7fd3c4102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:11.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 --2- 172.21.15.110:0/349718711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4100030 0x7fd3c4102410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4004940 tx=0x7fd3b4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:11.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/349718711 shutdown_connections 2024-04-25T15:47:11.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 --2- 172.21.15.110:0/349718711 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4100030 0x7fd3c4102410 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 --2- 172.21.15.110:0/349718711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c4066a90 0x7fd3c40ffaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/349718711 >> 172.21.15.110:0/349718711 conn(0x7fd3c40f74c0 msgr2=0x7fd3c40f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/349718711 shutdown_connections 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/349718711 wait complete. 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 Processor -- start 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 -- start start 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4066a90 0x7fd3c410daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c4100030 0x7fd3c410e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c410e6c0 con 0x7fd3c4066a90 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.293+0000 7fd3cbdf9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3c410ee60 con 0x7fd3c4100030 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3c9b95700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4066a90 0x7fd3c410daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3c9b95700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4066a90 0x7fd3c410daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39532/0 (socket says 172.21.15.110:39532) 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3c9b95700 1 -- 172.21.15.110:0/3702932259 learned_addr learned my addr 172.21.15.110:0/3702932259 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3c9394700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c4100030 0x7fd3c410e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:11.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3c9b95700 1 -- 172.21.15.110:0/3702932259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c4100030 msgr2=0x7fd3c410e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:11.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3c9b95700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c4100030 0x7fd3c410e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:11.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3c9b95700 1 -- 172.21.15.110:0/3702932259 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b4005040 con 0x7fd3c4066a90 2024-04-25T15:47:11.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3c9b95700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4066a90 0x7fd3c410daf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000bae0 tx=0x7fd3c0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:11.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3ba7fc700 1 -- 172.21.15.110:0/3702932259 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3c000c8c0 con 0x7fd3c4066a90 2024-04-25T15:47:11.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3ba7fc700 1 -- 172.21.15.110:0/3702932259 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd3c0014410 con 0x7fd3c4066a90 2024-04-25T15:47:11.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3ba7fc700 1 -- 172.21.15.110:0/3702932259 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3c0007610 con 0x7fd3c4066a90 2024-04-25T15:47:11.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3c410f140 con 0x7fd3c4066a90 2024-04-25T15:47:11.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.294+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3c410f700 con 0x7fd3c4066a90 2024-04-25T15:47:11.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.295+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3c405f5c0 con 0x7fd3c4066a90 2024-04-25T15:47:11.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.296+0000 7fd3ba7fc700 1 -- 172.21.15.110:0/3702932259 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fd3c0014580 con 0x7fd3c4066a90 2024-04-25T15:47:11.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.296+0000 7fd3ba7fc700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b00692a0 0x7fd3b006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:11.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.296+0000 7fd3ba7fc700 1 -- 172.21.15.110:0/3702932259 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fd3c0087270 con 0x7fd3c4066a90 2024-04-25T15:47:11.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.296+0000 7fd3c9394700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b00692a0 0x7fd3b006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:11.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.296+0000 7fd3c9394700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b00692a0 0x7fd3b006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400b970 tx=0x7fd3b4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:11.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.300+0000 7fd3ba7fc700 1 -- 172.21.15.110:0/3702932259 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fd3c004d250 con 0x7fd3c4066a90 2024-04-25T15:47:11.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.504+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3c410fa80 con 0x7fd3b00692a0 2024-04-25T15:47:11.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.509+0000 7fd3ba7fc700 1 -- 172.21.15.110:0/3702932259 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fd3c410fa80 con 0x7fd3b00692a0 2024-04-25T15:47:11.510 INFO:teuthology.orchestra.run.smithi110.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:47:11.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b00692a0 msgr2=0x7fd3b006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:11.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b00692a0 0x7fd3b006b750 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fd3b400b970 tx=0x7fd3b4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:11.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4066a90 msgr2=0x7fd3c410daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:11.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4066a90 0x7fd3c410daf0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd3c000bae0 tx=0x7fd3c0009810 comp rx=0 tx=0).stop 2024-04-25T15:47:11.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 shutdown_connections 2024-04-25T15:47:11.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd3c4066a90 0x7fd3c410daf0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:11.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fd3b00692a0 0x7fd3b006b750 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:11.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 --2- 172.21.15.110:0/3702932259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3c4100030 0x7fd3c410e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:11.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 >> 172.21.15.110:0/3702932259 conn(0x7fd3c40f74c0 msgr2=0x7fd3c40f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:11.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 shutdown_connections 2024-04-25T15:47:11.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:11.511+0000 7fd3cbdf9700 1 -- 172.21.15.110:0/3702932259 wait complete. 2024-04-25T15:47:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:11 smithi110 ceph-mon[26344]: pgmap v198: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-25T15:47:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:47:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:11 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:47:12.235 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-25T15:47:12.236 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T15:47:12.236 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-25T15:47:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:11 smithi193 ceph-mon[33747]: pgmap v198: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-25T15:47:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:47:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:11 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:47:12.497 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:47:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:12 smithi110 ceph-mon[26344]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:12 smithi110 ceph-mon[26344]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:47:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:12 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:47:12.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:12 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:47:12.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:12 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:47:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:12 smithi193 ceph-mon[33747]: from='client.14554 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:12 smithi193 ceph-mon[33747]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:47:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:12 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:47:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:12 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:47:13.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:12 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:47:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:13 smithi110 ceph-mon[26344]: pgmap v199: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:47:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:13 smithi110 ceph-mon[26344]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:47:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:13 smithi193 ceph-mon[33747]: pgmap v199: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:47:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:13 smithi193 ceph-mon[33747]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T15:47:14.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.258+0000 7f8f638b9700 1 -- 172.21.15.110:0/4031462665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fe090 msgr2=0x7f8f5c0fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.258+0000 7f8f638b9700 1 --2- 172.21.15.110:0/4031462665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fe090 0x7f8f5c0fe500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c004940 tx=0x7f8f4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:14.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.258+0000 7f8f638b9700 1 -- 172.21.15.110:0/4031462665 shutdown_connections 2024-04-25T15:47:14.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.258+0000 7f8f638b9700 1 --2- 172.21.15.110:0/4031462665 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fe090 0x7f8f5c0fe500 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.258+0000 7f8f638b9700 1 --2- 172.21.15.110:0/4031462665 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f5c0fce90 0x7f8f5c0fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.258+0000 7f8f638b9700 1 -- 172.21.15.110:0/4031462665 >> 172.21.15.110:0/4031462665 conn(0x7f8f5c0f85c0 msgr2=0x7f8f5c0fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:14.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.258+0000 7f8f638b9700 1 -- 172.21.15.110:0/4031462665 shutdown_connections 2024-04-25T15:47:14.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.258+0000 7f8f638b9700 1 -- 172.21.15.110:0/4031462665 wait complete. 2024-04-25T15:47:14.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f638b9700 1 Processor -- start 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f638b9700 1 -- start start 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f638b9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fce90 0x7f8f5c071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f638b9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f5c0fe090 0x7f8f5c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f638b9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c070860 con 0x7f8f5c0fce90 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f638b9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0709d0 con 0x7f8f5c0fe090 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f61655700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fce90 0x7f8f5c071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f61655700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fce90 0x7f8f5c071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39556/0 (socket says 172.21.15.110:39556) 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.259+0000 7f8f61655700 1 -- 172.21.15.110:0/748439039 learned_addr learned my addr 172.21.15.110:0/748439039 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:14.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f60e54700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f5c0fe090 0x7f8f5c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f61655700 1 -- 172.21.15.110:0/748439039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f5c0fe090 msgr2=0x7f8f5c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f61655700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f5c0fe090 0x7f8f5c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f61655700 1 -- 172.21.15.110:0/748439039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c005040 con 0x7f8f5c0fce90 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f60e54700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f5c0fe090 0x7f8f5c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f61655700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fce90 0x7f8f5c071de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8f58002a80 tx=0x7f8f58004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f51ffb700 1 -- 172.21.15.110:0/748439039 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f58007cc0 con 0x7f8f5c0fce90 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5c070cb0 con 0x7f8f5c0fce90 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f51ffb700 1 -- 172.21.15.110:0/748439039 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f5800f040 con 0x7f8f5c0fce90 2024-04-25T15:47:14.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f51ffb700 1 -- 172.21.15.110:0/748439039 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f5800dad0 con 0x7f8f5c0fce90 2024-04-25T15:47:14.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.260+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c115a50 con 0x7f8f5c0fce90 2024-04-25T15:47:14.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.261+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5c05f5c0 con 0x7f8f5c0fce90 2024-04-25T15:47:14.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.262+0000 7f8f51ffb700 1 -- 172.21.15.110:0/748439039 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f8f5800dc30 con 0x7f8f5c0fce90 2024-04-25T15:47:14.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.262+0000 7f8f51ffb700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8f480692a0 0x7f8f4806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.262+0000 7f8f51ffb700 1 -- 172.21.15.110:0/748439039 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f8f58086f50 con 0x7f8f5c0fce90 2024-04-25T15:47:14.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.262+0000 7f8f60e54700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8f480692a0 0x7f8f4806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.263+0000 7f8f60e54700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8f480692a0 0x7f8f4806b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00b970 tx=0x7f8f4c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:14.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.267+0000 7f8f51ffb700 1 -- 172.21.15.110:0/748439039 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f8f58053200 con 0x7f8f5c0fce90 2024-04-25T15:47:14.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.494+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f5c102400 con 0x7f8f480692a0 2024-04-25T15:47:14.496 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.496+0000 7f8f51ffb700 1 -- 172.21.15.110:0/748439039 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f8f5c102400 con 0x7f8f480692a0 2024-04-25T15:47:14.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8f480692a0 msgr2=0x7f8f4806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8f480692a0 0x7f8f4806b750 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00b970 tx=0x7f8f4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fce90 msgr2=0x7f8f5c071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fce90 0x7f8f5c071de0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8f58002a80 tx=0x7f8f58004600 comp rx=0 tx=0).stop 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 shutdown_connections 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f5c0fce90 0x7f8f5c071de0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f8f480692a0 0x7f8f4806b750 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 --2- 172.21.15.110:0/748439039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f5c0fe090 0x7f8f5c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.498+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 >> 172.21.15.110:0/748439039 conn(0x7f8f5c0f85c0 msgr2=0x7f8f5c0f9ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.499+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 shutdown_connections 2024-04-25T15:47:14.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.499+0000 7f8f638b9700 1 -- 172.21.15.110:0/748439039 wait complete. 2024-04-25T15:47:14.509 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.603+0000 7f3e722db700 1 -- 172.21.15.110:0/2430599982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 msgr2=0x7f3e6c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.603+0000 7f3e722db700 1 --2- 172.21.15.110:0/2430599982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 0x7f3e6c102300 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3e60004940 tx=0x7f3e60009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.603+0000 7f3e722db700 1 -- 172.21.15.110:0/2430599982 shutdown_connections 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.603+0000 7f3e722db700 1 --2- 172.21.15.110:0/2430599982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 0x7f3e6c102300 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.603+0000 7f3e722db700 1 --2- 172.21.15.110:0/2430599982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e6c0662c0 0x7f3e6c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.603+0000 7f3e722db700 1 -- 172.21.15.110:0/2430599982 >> 172.21.15.110:0/2430599982 conn(0x7f3e6c0f7430 msgr2=0x7f3e6c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.604+0000 7f3e722db700 1 -- 172.21.15.110:0/2430599982 shutdown_connections 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.604+0000 7f3e722db700 1 -- 172.21.15.110:0/2430599982 wait complete. 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.604+0000 7f3e722db700 1 Processor -- start 2024-04-25T15:47:14.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.604+0000 7f3e722db700 1 -- start start 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.604+0000 7f3e722db700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e6c0662c0 0x7f3e6c0fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.604+0000 7f3e722db700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 0x7f3e6c0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.604+0000 7f3e722db700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0fbe50 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.604+0000 7f3e722db700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c0fbfc0 con 0x7f3e6c0662c0 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 0x7f3e6c0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 0x7f3e6c0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39562/0 (socket says 172.21.15.110:39562) 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6b7fe700 1 -- 172.21.15.110:0/2289016832 learned_addr learned my addr 172.21.15.110:0/2289016832 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6bfff700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e6c0662c0 0x7f3e6c0fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6b7fe700 1 -- 172.21.15.110:0/2289016832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e6c0662c0 msgr2=0x7f3e6c0fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6b7fe700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e6c0662c0 0x7f3e6c0fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6b7fe700 1 -- 172.21.15.110:0/2289016832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e60005040 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6bfff700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e6c0662c0 0x7f3e6c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e6b7fe700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 0x7f3e6c0fb8a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000b3b0 tx=0x7f3e60009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:14.607 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e68ff9700 1 -- 172.21.15.110:0/2289016832 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6000e040 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e68ff9700 1 -- 172.21.15.110:0/2289016832 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e6000bcc0 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e6c0fc240 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.608 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e68ff9700 1 -- 172.21.15.110:0/2289016832 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e6001c630 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.605+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e6c115a70 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.607+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e6c05f5c0 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.607+0000 7f3e68ff9700 1 -- 172.21.15.110:0/2289016832 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3e60013980 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.607+0000 7f3e68ff9700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3e540692a0 0x7f3e5406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.607+0000 7f3e68ff9700 1 -- 172.21.15.110:0/2289016832 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3e60087f10 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.607+0000 7f3e6bfff700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3e540692a0 0x7f3e5406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.608+0000 7f3e6bfff700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3e540692a0 0x7f3e5406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fd0 tx=0x7f3e5c009f00 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:14.612 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.612+0000 7f3e68ff9700 1 -- 172.21.15.110:0/2289016832 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3e60054240 con 0x7f3e6c0ffee0 2024-04-25T15:47:14.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.839+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e6c0f8f20 con 0x7f3e540692a0 2024-04-25T15:47:14.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.840+0000 7f3e68ff9700 1 -- 172.21.15.110:0/2289016832 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f3e6c0f8f20 con 0x7f3e540692a0 2024-04-25T15:47:14.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3e540692a0 msgr2=0x7f3e5406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3e540692a0 0x7f3e5406b750 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009fd0 tx=0x7f3e5c009f00 comp rx=0 tx=0).stop 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 msgr2=0x7f3e6c0fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 0x7f3e6c0fb8a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3e6000b3b0 tx=0x7f3e60009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 shutdown_connections 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3e540692a0 0x7f3e5406b750 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e6c0662c0 0x7f3e6c0fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 --2- 172.21.15.110:0/2289016832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e6c0ffee0 0x7f3e6c0fb8a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 >> 172.21.15.110:0/2289016832 conn(0x7f3e6c0f7430 msgr2=0x7f3e6c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 shutdown_connections 2024-04-25T15:47:14.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.843+0000 7f3e722db700 1 -- 172.21.15.110:0/2289016832 wait complete. 2024-04-25T15:47:14.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.949+0000 7f7dcf387700 1 -- 172.21.15.110:0/4065588872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fcf70 msgr2=0x7f7dc80fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.949+0000 7f7dcf387700 1 --2- 172.21.15.110:0/4065588872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fcf70 0x7f7dc80fd3e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7db8004940 tx=0x7f7db8009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:14.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.949+0000 7f7dcf387700 1 -- 172.21.15.110:0/4065588872 shutdown_connections 2024-04-25T15:47:14.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.949+0000 7f7dcf387700 1 --2- 172.21.15.110:0/4065588872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fcf70 0x7f7dc80fd3e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.949+0000 7f7dcf387700 1 --2- 172.21.15.110:0/4065588872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dc80fbcc0 0x7f7dc80fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.949+0000 7f7dcf387700 1 -- 172.21.15.110:0/4065588872 >> 172.21.15.110:0/4065588872 conn(0x7f7dc80f7420 msgr2=0x7f7dc80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:14.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.949+0000 7f7dcf387700 1 -- 172.21.15.110:0/4065588872 shutdown_connections 2024-04-25T15:47:14.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.949+0000 7f7dcf387700 1 -- 172.21.15.110:0/4065588872 wait complete. 2024-04-25T15:47:14.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcf387700 1 Processor -- start 2024-04-25T15:47:14.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcf387700 1 -- start start 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcf387700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fbcc0 0x7f7dc8110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcf387700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dc80fcf70 0x7f7dc8112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcf387700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8066bc0 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcf387700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8066d00 con 0x7f7dc80fcf70 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcd123700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fbcc0 0x7f7dc8110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcd123700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fbcc0 0x7f7dc8110710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39580/0 (socket says 172.21.15.110:39580) 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcd123700 1 -- 172.21.15.110:0/3259193876 learned_addr learned my addr 172.21.15.110:0/3259193876 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcc922700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dc80fcf70 0x7f7dc8112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcd123700 1 -- 172.21.15.110:0/3259193876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dc80fcf70 msgr2=0x7f7dc8112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcd123700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dc80fcf70 0x7f7dc8112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.950+0000 7f7dcd123700 1 -- 172.21.15.110:0/3259193876 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db8005040 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.951+0000 7f7dcd123700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fbcc0 0x7f7dc8110710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c830 tx=0x7f7dc4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.951+0000 7f7dbdffb700 1 -- 172.21.15.110:0/3259193876 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dc400a760 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.951+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc8113200 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.951+0000 7f7dbdffb700 1 -- 172.21.15.110:0/3259193876 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dc400ad90 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.951+0000 7f7dbdffb700 1 -- 172.21.15.110:0/3259193876 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dc40113a0 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.951+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc81137c0 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.952+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc805f5c0 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.952+0000 7f7dbdffb700 1 -- 172.21.15.110:0/3259193876 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f7dc4011610 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.953+0000 7f7dbdffb700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7db40692a0 0x7f7db406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:14.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.953+0000 7f7dbdffb700 1 -- 172.21.15.110:0/3259193876 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f7dc4087290 con 0x7f7dc80fbcc0 2024-04-25T15:47:14.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.953+0000 7f7dcc922700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7db40692a0 0x7f7db406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:14.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.953+0000 7f7dcc922700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7db40692a0 0x7f7db406b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b3b0 tx=0x7f7db8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:14.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:14.957+0000 7f7dbdffb700 1 -- 172.21.15.110:0/3259193876 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f7dc4053590 con 0x7f7dc80fbcc0 2024-04-25T15:47:15.170 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.169+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7dc8101280 con 0x7f7db40692a0 2024-04-25T15:47:15.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.179+0000 7f7dbdffb700 1 -- 172.21.15.110:0/3259193876 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f7dc8101280 con 0x7f7db40692a0 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (4m) 26s ago 6m 24.8M - 0.25.0 c8568f914cd2 b995d416435c 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (6m) 26s ago 6m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (5m) 33s ago 5m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (4m) 26s ago 5m 87.2M - 9.4.7 954c08fa6188 6cf7f080ae79 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (46s) 26s ago 46s 15.5M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (52s) 26s ago 51s 16.1M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (49s) 33s ago 49s 11.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (44s) 33s ago 43s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:9283 running (7m) 26s ago 7m 481M - 17.2.7-902-gc61472c8 0de407c1b3af c623b709ef43 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (5m) 33s ago 5m 431M - 17.2.7-902-gc61472c8 0de407c1b3af 30a2f136ed24 2024-04-25T15:47:15.180 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (7m) 26s ago 7m 46.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (5m) 33s ago 5m 38.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (6m) 26s ago 6m 15.6M - 1.5.0 0da6a335fe13 5f8f351cfd11 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (5m) 33s ago 5m 15.6M - 1.5.0 0da6a335fe13 13afa4d53352 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (4m) 26s ago 4m 45.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (3m) 26s ago 3m 48.2M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (3m) 26s ago 3m 44.5M 4288M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (3m) 33s ago 3m 46.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (2m) 33s ago 2m 44.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (2m) 33s ago 2m 44.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:47:15.181 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (4m) 26s ago 5m 52.5M - 2.43.0 a07b618ecd1d ce7fd55a0368 2024-04-25T15:47:15.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7db40692a0 msgr2=0x7f7db406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:15.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7db40692a0 0x7f7db406b750 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b3b0 tx=0x7f7db8009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fbcc0 msgr2=0x7f7dc8110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fbcc0 0x7f7dc8110710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c830 tx=0x7f7dc4004470 comp rx=0 tx=0).stop 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 shutdown_connections 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7dc80fbcc0 0x7f7dc8110710 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f7db40692a0 0x7f7db406b750 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 --2- 172.21.15.110:0/3259193876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dc80fcf70 0x7f7dc8112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 >> 172.21.15.110:0/3259193876 conn(0x7f7dc80f7420 msgr2=0x7f7dc80f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 shutdown_connections 2024-04-25T15:47:15.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.181+0000 7f7dcf387700 1 -- 172.21.15.110:0/3259193876 wait complete. 2024-04-25T15:47:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.296+0000 7f0528e94700 1 -- 172.21.15.110:0/845968053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524071dc0 msgr2=0x7f05240702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.296+0000 7f0528e94700 1 --2- 172.21.15.110:0/845968053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524071dc0 0x7f05240702d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0514004940 tx=0x7f0514009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.296+0000 7f0528e94700 1 -- 172.21.15.110:0/845968053 shutdown_connections 2024-04-25T15:47:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.296+0000 7f0528e94700 1 --2- 172.21.15.110:0/845968053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0524070810 0x7f0524070c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.296+0000 7f0528e94700 1 --2- 172.21.15.110:0/845968053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524071dc0 0x7f05240702d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.296+0000 7f0528e94700 1 -- 172.21.15.110:0/845968053 >> 172.21.15.110:0/845968053 conn(0x7f05240f85c0 msgr2=0x7f05240fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.296+0000 7f0528e94700 1 -- 172.21.15.110:0/845968053 shutdown_connections 2024-04-25T15:47:15.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.296+0000 7f0528e94700 1 -- 172.21.15.110:0/845968053 wait complete. 2024-04-25T15:47:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f0528e94700 1 Processor -- start 2024-04-25T15:47:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f0528e94700 1 -- start start 2024-04-25T15:47:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f0528e94700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524070810 0x7f05240fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f0528e94700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0524071dc0 0x7f05240fd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f0528e94700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240fd910 con 0x7f0524070810 2024-04-25T15:47:15.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f0528e94700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05240fe160 con 0x7f0524071dc0 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f051a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524070810 0x7f05240fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f051a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524070810 0x7f05240fcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39606/0 (socket says 172.21.15.110:39606) 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f051a59c700 1 -- 172.21.15.110:0/2465049191 learned_addr learned my addr 172.21.15.110:0/2465049191 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f052259c700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0524071dc0 0x7f05240fd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f051a59c700 1 -- 172.21.15.110:0/2465049191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0524071dc0 msgr2=0x7f05240fd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f051a59c700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0524071dc0 0x7f05240fd2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f051a59c700 1 -- 172.21.15.110:0/2465049191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0514005040 con 0x7f0524070810 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.297+0000 7f051a59c700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524070810 0x7f05240fcd80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0514000c00 tx=0x7f0514009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.298+0000 7f051b7fe700 1 -- 172.21.15.110:0/2465049191 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f051400e040 con 0x7f0524070810 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.298+0000 7f051b7fe700 1 -- 172.21.15.110:0/2465049191 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f051400bcc0 con 0x7f0524070810 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.298+0000 7f051b7fe700 1 -- 172.21.15.110:0/2465049191 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f051401c630 con 0x7f0524070810 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.298+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05240fe3e0 con 0x7f0524070810 2024-04-25T15:47:15.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.298+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05240fe9a0 con 0x7f0524070810 2024-04-25T15:47:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.299+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052405f5c0 con 0x7f0524070810 2024-04-25T15:47:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.299+0000 7f051b7fe700 1 -- 172.21.15.110:0/2465049191 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0514013980 con 0x7f0524070810 2024-04-25T15:47:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.300+0000 7f051b7fe700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f05100692a0 0x7f051006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.300+0000 7f051b7fe700 1 -- 172.21.15.110:0/2465049191 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0514088530 con 0x7f0524070810 2024-04-25T15:47:15.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.300+0000 7f052259c700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f05100692a0 0x7f051006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:15.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.300+0000 7f052259c700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f05100692a0 0x7f051006b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f050c009b80 tx=0x7f050c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:15.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.304+0000 7f051b7fe700 1 -- 172.21.15.110:0/2465049191 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0514054880 con 0x7f0524070810 2024-04-25T15:47:15.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.613+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f05240fec80 con 0x7f0524070810 2024-04-25T15:47:15.614 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.614+0000 7f051b7fe700 1 -- 172.21.15.110:0/2465049191 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f0514057ea0 con 0x7f0524070810 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:47:15.615 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:47:15.616 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:47:15.616 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:47:15.616 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-25T15:47:15.616 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:47:15.616 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:47:15.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f05100692a0 msgr2=0x7f051006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:15.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f05100692a0 0x7f051006b750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f050c009b80 tx=0x7f050c00b040 comp rx=0 tx=0).stop 2024-04-25T15:47:15.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524070810 msgr2=0x7f05240fcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:15.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524070810 0x7f05240fcd80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0514000c00 tx=0x7f0514009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:15.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 shutdown_connections 2024-04-25T15:47:15.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0524070810 0x7f05240fcd80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f05100692a0 0x7f051006b750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.618 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 --2- 172.21.15.110:0/2465049191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0524071dc0 0x7f05240fd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 >> 172.21.15.110:0/2465049191 conn(0x7f05240f85c0 msgr2=0x7f05240f9c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:15.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 shutdown_connections 2024-04-25T15:47:15.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.617+0000 7f0528e94700 1 -- 172.21.15.110:0/2465049191 wait complete. 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 -- 172.21.15.110:0/1213678132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 msgr2=0x7f69480fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 --2- 172.21.15.110:0/1213678132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 0x7f69480fc0c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6944004940 tx=0x7f6944009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 -- 172.21.15.110:0/1213678132 shutdown_connections 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 --2- 172.21.15.110:0/1213678132 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69480fcf60 0x7f69480fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 --2- 172.21.15.110:0/1213678132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 0x7f69480fc0c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 -- 172.21.15.110:0/1213678132 >> 172.21.15.110:0/1213678132 conn(0x7f69480f7430 msgr2=0x7f69480f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 -- 172.21.15.110:0/1213678132 shutdown_connections 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 -- 172.21.15.110:0/1213678132 wait complete. 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 Processor -- start 2024-04-25T15:47:15.731 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 -- start start 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.729+0000 7f6950331700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 0x7f694810dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f6950331700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69480fcf60 0x7f694810dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f6950331700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694810e5f0 con 0x7f69480fbcb0 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f6950331700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694810eda0 con 0x7f69480fcf60 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f694e0cd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 0x7f694810dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f694e0cd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 0x7f694810dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39628/0 (socket says 172.21.15.110:39628) 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f694e0cd700 1 -- 172.21.15.110:0/2990266949 learned_addr learned my addr 172.21.15.110:0/2990266949 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f694d8cc700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69480fcf60 0x7f694810dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f694e0cd700 1 -- 172.21.15.110:0/2990266949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69480fcf60 msgr2=0x7f694810dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f694e0cd700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69480fcf60 0x7f694810dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f694e0cd700 1 -- 172.21.15.110:0/2990266949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6944005040 con 0x7f69480fbcb0 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f694e0cd700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 0x7f694810dab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f694400b3b0 tx=0x7f6944009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:15.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f693effd700 1 -- 172.21.15.110:0/2990266949 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694400e040 con 0x7f69480fbcb0 2024-04-25T15:47:15.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f693effd700 1 -- 172.21.15.110:0/2990266949 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f694400bbf0 con 0x7f69480fbcb0 2024-04-25T15:47:15.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694810f020 con 0x7f69480fbcb0 2024-04-25T15:47:15.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f693effd700 1 -- 172.21.15.110:0/2990266949 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694401c630 con 0x7f69480fbcb0 2024-04-25T15:47:15.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.730+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694810f560 con 0x7f69480fbcb0 2024-04-25T15:47:15.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.732+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6948101af0 con 0x7f69480fbcb0 2024-04-25T15:47:15.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.732+0000 7f693effd700 1 -- 172.21.15.110:0/2990266949 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f694400bd60 con 0x7f69480fbcb0 2024-04-25T15:47:15.734 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.732+0000 7f693effd700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f69340692a0 0x7f693406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:15.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.732+0000 7f693effd700 1 -- 172.21.15.110:0/2990266949 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6944087ec0 con 0x7f69480fbcb0 2024-04-25T15:47:15.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.732+0000 7f694d8cc700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f69340692a0 0x7f693406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:15.735 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.733+0000 7f694d8cc700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f69340692a0 0x7f693406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6938004f40 tx=0x7f693800b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:15.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:15.738+0000 7f693effd700 1 -- 172.21.15.110:0/2990266949 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6944057c80 con 0x7f69480fbcb0 2024-04-25T15:47:16.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.013+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6948104930 con 0x7f69480fbcb0 2024-04-25T15:47:16.014 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:15 smithi110 ceph-mon[26344]: pgmap v200: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-25T15:47:16.014 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:15 smithi110 ceph-mon[26344]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:16.014 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:15 smithi110 ceph-mon[26344]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:16.014 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:15 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2465049191' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:47:16.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.013+0000 7f693effd700 1 -- 172.21.15.110:0/2990266949 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f6948104930 con 0x7f69480fbcb0 2024-04-25T15:47:16.015 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:47:16.015 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:47:16.015 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:47:16.015 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:47:16.016 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:47:16.017 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:47:16.017 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:47:16.017 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:47:16.017 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:47:16.017 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:47:16.017 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:47:16.018 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:47:16.018 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:47:16.018 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:47:16.018 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:47:16.018 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:47:16.018 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:47:16.019 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:47:16.019 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:47:16.019 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:47:16.019 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:47:16.019 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:47:16.019 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:47:16.020 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:47:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:47:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:47:16.020 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:47:16.020 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:47:16.020 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:47:16.021 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:47:16.021 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:47:16.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f69340692a0 msgr2=0x7f693406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f69340692a0 0x7f693406b750 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6938004f40 tx=0x7f693800b040 comp rx=0 tx=0).stop 2024-04-25T15:47:16.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 msgr2=0x7f694810dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 0x7f694810dab0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f694400b3b0 tx=0x7f6944009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:16.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 shutdown_connections 2024-04-25T15:47:16.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f69480fbcb0 0x7f694810dab0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f69340692a0 0x7f693406b750 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 --2- 172.21.15.110:0/2990266949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69480fcf60 0x7f694810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 >> 172.21.15.110:0/2990266949 conn(0x7f69480f7430 msgr2=0x7f6948065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:16.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 shutdown_connections 2024-04-25T15:47:16.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.016+0000 7f6950331700 1 -- 172.21.15.110:0/2990266949 wait complete. 2024-04-25T15:47:16.024 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:47:16.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.125+0000 7f35bf76f700 1 -- 172.21.15.110:0/516866311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 msgr2=0x7f35b81006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.125+0000 7f35bf76f700 1 --2- 172.21.15.110:0/516866311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 0x7f35b81006f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f35a8004940 tx=0x7f35a8009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:16.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.125+0000 7f35bf76f700 1 -- 172.21.15.110:0/516866311 shutdown_connections 2024-04-25T15:47:16.128 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.125+0000 7f35bf76f700 1 --2- 172.21.15.110:0/516866311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 0x7f35b81006f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.125+0000 7f35bf76f700 1 --2- 172.21.15.110:0/516866311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b80fefd0 0x7f35b80ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.125+0000 7f35bf76f700 1 -- 172.21.15.110:0/516866311 >> 172.21.15.110:0/516866311 conn(0x7f35b80fa8f0 msgr2=0x7f35b80fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:16.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bf76f700 1 -- 172.21.15.110:0/516866311 shutdown_connections 2024-04-25T15:47:16.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bf76f700 1 -- 172.21.15.110:0/516866311 wait complete. 2024-04-25T15:47:16.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bf76f700 1 Processor -- start 2024-04-25T15:47:16.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bf76f700 1 -- start start 2024-04-25T15:47:16.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bf76f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b80fefd0 0x7f35b810fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:16.129 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bf76f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 0x7f35b8110170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bf76f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8110770 con 0x7f35b8100280 2024-04-25T15:47:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bf76f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b8110fc0 con 0x7f35b80fefd0 2024-04-25T15:47:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bcd0a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 0x7f35b8110170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bcd0a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 0x7f35b8110170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39648/0 (socket says 172.21.15.110:39648) 2024-04-25T15:47:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.126+0000 7f35bcd0a700 1 -- 172.21.15.110:0/2499988304 learned_addr learned my addr 172.21.15.110:0/2499988304 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35bd50b700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b80fefd0 0x7f35b810fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35bcd0a700 1 -- 172.21.15.110:0/2499988304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b80fefd0 msgr2=0x7f35b810fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35bcd0a700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b80fefd0 0x7f35b810fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35bcd0a700 1 -- 172.21.15.110:0/2499988304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a8005040 con 0x7f35b8100280 2024-04-25T15:47:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35bd50b700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b80fefd0 0x7f35b810fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:47:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35bcd0a700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 0x7f35b8110170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f35a800b3b0 tx=0x7f35a8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35adffb700 1 -- 172.21.15.110:0/2499988304 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a800e040 con 0x7f35b8100280 2024-04-25T15:47:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35adffb700 1 -- 172.21.15.110:0/2499988304 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f35a800bcc0 con 0x7f35b8100280 2024-04-25T15:47:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b8111240 con 0x7f35b8100280 2024-04-25T15:47:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35adffb700 1 -- 172.21.15.110:0/2499988304 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a801c630 con 0x7f35b8100280 2024-04-25T15:47:16.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.127+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b8111780 con 0x7f35b8100280 2024-04-25T15:47:16.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.129+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b805f5c0 con 0x7f35b8100280 2024-04-25T15:47:16.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.129+0000 7f35adffb700 1 -- 172.21.15.110:0/2499988304 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f35a8013980 con 0x7f35b8100280 2024-04-25T15:47:16.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.129+0000 7f35adffb700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35a40692a0 0x7f35a406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:16.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.129+0000 7f35adffb700 1 -- 172.21.15.110:0/2499988304 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f35a8087f10 con 0x7f35b8100280 2024-04-25T15:47:16.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.129+0000 7f35bd50b700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35a40692a0 0x7f35a406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:16.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.130+0000 7f35bd50b700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35a40692a0 0x7f35a406b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35b40041d0 tx=0x7f35b4009040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:16.137 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.136+0000 7f35adffb700 1 -- 172.21.15.110:0/2499988304 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f35a8057cd0 con 0x7f35b8100280 2024-04-25T15:47:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:15 smithi193 ceph-mon[33747]: pgmap v200: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-25T15:47:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:15 smithi193 ceph-mon[33747]: from='client.14558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:15 smithi193 ceph-mon[33747]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:15 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2465049191' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:47:16.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.362+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f35b8111b00 con 0x7f35a40692a0 2024-04-25T15:47:16.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.364+0000 7f35adffb700 1 -- 172.21.15.110:0/2499988304 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f35b8111b00 con 0x7f35a40692a0 2024-04-25T15:47:16.364 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:47:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T15:47:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:47:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:47:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-04-25T15:47:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "", 2024-04-25T15:47:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Doing first pull of quay.ceph.io/ceph-ci/ceph:994e2f20102a20fa9a27966c14b625cc01884df8 image", 2024-04-25T15:47:16.365 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:47:16.365 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35a40692a0 msgr2=0x7f35a406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35a40692a0 0x7f35a406b750 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f35b40041d0 tx=0x7f35b4009040 comp rx=0 tx=0).stop 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 msgr2=0x7f35b8110170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 0x7f35b8110170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f35a800b3b0 tx=0x7f35a8009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 shutdown_connections 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f35a40692a0 0x7f35a406b750 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35b80fefd0 0x7f35b810fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 --2- 172.21.15.110:0/2499988304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35b8100280 0x7f35b8110170 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 >> 172.21.15.110:0/2499988304 conn(0x7f35b80fa8f0 msgr2=0x7f35b80fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:16.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 shutdown_connections 2024-04-25T15:47:16.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.366+0000 7f35bf76f700 1 -- 172.21.15.110:0/2499988304 wait complete. 2024-04-25T15:47:16.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.473+0000 7ffbdde77700 1 -- 172.21.15.110:0/2293497651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fd310 msgr2=0x7ffbd80fd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.473+0000 7ffbdde77700 1 --2- 172.21.15.110:0/2293497651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fd310 0x7ffbd80fd720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8004940 tx=0x7ffbc8009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.473+0000 7ffbdde77700 1 -- 172.21.15.110:0/2293497651 shutdown_connections 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.473+0000 7ffbdde77700 1 --2- 172.21.15.110:0/2293497651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd80fdc60 0x7ffbd81045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.473+0000 7ffbdde77700 1 --2- 172.21.15.110:0/2293497651 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fd310 0x7ffbd80fd720 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.473+0000 7ffbdde77700 1 -- 172.21.15.110:0/2293497651 >> 172.21.15.110:0/2293497651 conn(0x7ffbd80f6b80 msgr2=0x7ffbd80f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.473+0000 7ffbdde77700 1 -- 172.21.15.110:0/2293497651 shutdown_connections 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.474+0000 7ffbdde77700 1 -- 172.21.15.110:0/2293497651 wait complete. 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.474+0000 7ffbdde77700 1 Processor -- start 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.474+0000 7ffbdde77700 1 -- start start 2024-04-25T15:47:16.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.474+0000 7ffbdde77700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd80fd310 0x7ffbd810fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.474+0000 7ffbdde77700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fdc60 0x7ffbd8110160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.474+0000 7ffbdde77700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8110760 con 0x7ffbd80fdc60 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.474+0000 7ffbdde77700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd8110fb0 con 0x7ffbd80fd310 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.474+0000 7ffbd6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fdc60 0x7ffbd8110160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbd6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fdc60 0x7ffbd8110160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39670/0 (socket says 172.21.15.110:39670) 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbd6ffd700 1 -- 172.21.15.110:0/581089551 learned_addr learned my addr 172.21.15.110:0/581089551 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbd77fe700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd80fd310 0x7ffbd810fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbd6ffd700 1 -- 172.21.15.110:0/581089551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd80fd310 msgr2=0x7ffbd810fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbd6ffd700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd80fd310 0x7ffbd810fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbd6ffd700 1 -- 172.21.15.110:0/581089551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc8005040 con 0x7ffbd80fdc60 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbd77fe700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd80fd310 0x7ffbd810fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:47:16.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbd6ffd700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fdc60 0x7ffbd8110160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c830 tx=0x7ffbcc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:16.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbbffff700 1 -- 172.21.15.110:0/581089551 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbcc00a760 con 0x7ffbd80fdc60 2024-04-25T15:47:16.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd8111290 con 0x7ffbd80fdc60 2024-04-25T15:47:16.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbbffff700 1 -- 172.21.15.110:0/581089551 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffbcc00ad90 con 0x7ffbd80fdc60 2024-04-25T15:47:16.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbbffff700 1 -- 172.21.15.110:0/581089551 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbcc0113a0 con 0x7ffbd80fdc60 2024-04-25T15:47:16.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.475+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd81117d0 con 0x7ffbd80fdc60 2024-04-25T15:47:16.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.476+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd805f5c0 con 0x7ffbd80fdc60 2024-04-25T15:47:16.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.477+0000 7ffbbffff700 1 -- 172.21.15.110:0/581089551 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ffbcc0115e0 con 0x7ffbd80fdc60 2024-04-25T15:47:16.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.477+0000 7ffbbffff700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ffbc00692a0 0x7ffbc006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:16.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.477+0000 7ffbbffff700 1 -- 172.21.15.110:0/581089551 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ffbcc087120 con 0x7ffbd80fdc60 2024-04-25T15:47:16.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.477+0000 7ffbd77fe700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ffbc00692a0 0x7ffbc006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:16.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.478+0000 7ffbd77fe700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ffbc00692a0 0x7ffbc006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800b3b0 tx=0x7ffbc8009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:16.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.484+0000 7ffbbffff700 1 -- 172.21.15.110:0/581089551 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ffbcc056f80 con 0x7ffbd80fdc60 2024-04-25T15:47:16.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.801+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ffbd804bed0 con 0x7ffbd80fdc60 2024-04-25T15:47:16.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.802+0000 7ffbbffff700 1 -- 172.21.15.110:0/581089551 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7ffbcc012400 con 0x7ffbd80fdc60 2024-04-25T15:47:16.803 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:47:16.803 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:47:16.803 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:47:16.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ffbc00692a0 msgr2=0x7ffbc006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ffbc00692a0 0x7ffbc006b750 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800b3b0 tx=0x7ffbc8009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:16.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fdc60 msgr2=0x7ffbd8110160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:16.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fdc60 0x7ffbd8110160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c830 tx=0x7ffbcc0043c0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 shutdown_connections 2024-04-25T15:47:16.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ffbc00692a0 0x7ffbc006b750 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd80fd310 0x7ffbd810fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 --2- 172.21.15.110:0/581089551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ffbd80fdc60 0x7ffbd8110160 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:16.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.804+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 >> 172.21.15.110:0/581089551 conn(0x7ffbd80f6b80 msgr2=0x7ffbd80f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:16.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.805+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 shutdown_connections 2024-04-25T15:47:16.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:16.805+0000 7ffbdde77700 1 -- 172.21.15.110:0/581089551 wait complete. 2024-04-25T15:47:17.112 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:16 smithi110 ceph-mon[26344]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:16 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2990266949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:47:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:16 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/581089551' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:47:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:16 smithi193 ceph-mon[33747]: from='client.14566 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:16 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2990266949' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:47:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:16 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/581089551' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:47:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:17 smithi110 ceph-mon[26344]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:47:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:17 smithi110 ceph-mon[26344]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:17 smithi193 ceph-mon[33747]: pgmap v201: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:47:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:17 smithi193 ceph-mon[33747]: from='client.14578 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:19 smithi110 ceph-mon[26344]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:47:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:19 smithi193 ceph-mon[33747]: pgmap v202: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:47:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:21 smithi110 ceph-mon[26344]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:47:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:21 smithi193 ceph-mon[33747]: pgmap v203: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:47:24.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:23 smithi110 ceph-mon[26344]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-25T15:47:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:23 smithi193 ceph-mon[33747]: pgmap v204: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-25T15:47:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:25 smithi110 ceph-mon[26344]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-25T15:47:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:47:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:25 smithi193 ceph-mon[33747]: pgmap v205: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-25T15:47:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:47:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:27 smithi110 ceph-mon[26344]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:28.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:27 smithi193 ceph-mon[33747]: pgmap v206: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:29 smithi193 ceph-mon[33747]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:30.362 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:29 smithi110 ceph-mon[26344]: pgmap v207: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:31 smithi193 ceph-mon[33747]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:31 smithi110 ceph-mon[26344]: pgmap v208: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:33 smithi193 ceph-mon[33747]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:34.362 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:33 smithi110 ceph-mon[26344]: pgmap v209: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:35 smithi110 ceph-mon[26344]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:35 smithi193 ceph-mon[33747]: pgmap v210: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:37 smithi193 ceph-mon[33747]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:38.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:37 smithi110 ceph-mon[26344]: pgmap v211: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:39 smithi193 ceph-mon[33747]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:40.280 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:39 smithi110 ceph-mon[26344]: pgmap v212: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:40.945 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:40 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:47:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:40 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:47:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:41 smithi110 ceph-mon[26344]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:47:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:47:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:41 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:47:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:41 smithi193 ceph-mon[33747]: pgmap v213: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:47:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:47:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:41 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:47:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:42 smithi193 ceph-mon[33747]: Upgrade: Target is version 19.0.0-3291-g994e2f20 (unknown) 2024-04-25T15:47:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:42 smithi193 ceph-mon[33747]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, digests ['quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385'] 2024-04-25T15:47:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:42 smithi193 ceph-mon[33747]: Upgrade: Need to upgrade myself (mgr.smithi110.lvgbkf) 2024-04-25T15:47:43.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:42 smithi110 ceph-mon[26344]: Upgrade: Target is version 19.0.0-3291-g994e2f20 (unknown) 2024-04-25T15:47:43.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:42 smithi110 ceph-mon[26344]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, digests ['quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385'] 2024-04-25T15:47:43.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:42 smithi110 ceph-mon[26344]: Upgrade: Need to upgrade myself (mgr.smithi110.lvgbkf) 2024-04-25T15:47:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:43 smithi193 ceph-mon[33747]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 on smithi193 2024-04-25T15:47:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:43 smithi193 ceph-mon[33747]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:44.362 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:43 smithi110 ceph-mon[26344]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 on smithi193 2024-04-25T15:47:44.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:43 smithi110 ceph-mon[26344]: pgmap v214: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:45 smithi193 ceph-mon[33747]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:46.362 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:45 smithi110 ceph-mon[26344]: pgmap v215: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:46.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.913+0000 7fa050c32700 1 -- 172.21.15.110:0/2061501387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 msgr2=0x7fa04c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.913+0000 7fa050c32700 1 --2- 172.21.15.110:0/2061501387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 0x7fa04c102300 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa040004940 tx=0x7fa040009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 -- 172.21.15.110:0/2061501387 shutdown_connections 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 --2- 172.21.15.110:0/2061501387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 0x7fa04c102300 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 --2- 172.21.15.110:0/2061501387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa04c0662c0 0x7fa04c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 -- 172.21.15.110:0/2061501387 >> 172.21.15.110:0/2061501387 conn(0x7fa04c0f7430 msgr2=0x7fa04c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 -- 172.21.15.110:0/2061501387 shutdown_connections 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 -- 172.21.15.110:0/2061501387 wait complete. 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 Processor -- start 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 -- start start 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa04c0662c0 0x7fa04c10da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 0x7fa04c10dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:46.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c10e5a0 con 0x7fa04c0ffee0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa050c32700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa04c10ed50 con 0x7fa04c0662c0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa049d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 0x7fa04c10dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa049d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 0x7fa04c10dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48372/0 (socket says 172.21.15.110:48372) 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa049d9b700 1 -- 172.21.15.110:0/1734760206 learned_addr learned my addr 172.21.15.110:0/1734760206 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.914+0000 7fa04a59c700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa04c0662c0 0x7fa04c10da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa049d9b700 1 -- 172.21.15.110:0/1734760206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa04c0662c0 msgr2=0x7fa04c10da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa049d9b700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa04c0662c0 0x7fa04c10da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa049d9b700 1 -- 172.21.15.110:0/1734760206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040005040 con 0x7fa04c0ffee0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa049d9b700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 0x7fa04c10dfa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa040013870 tx=0x7fa040009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa03affd700 1 -- 172.21.15.110:0/1734760206 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa040021070 con 0x7fa04c0ffee0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa04c10efd0 con 0x7fa04c0ffee0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa03affd700 1 -- 172.21.15.110:0/1734760206 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa040009430 con 0x7fa04c0ffee0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa03affd700 1 -- 172.21.15.110:0/1734760206 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04000e070 con 0x7fa04c0ffee0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.915+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa04c10f510 con 0x7fa04c0ffee0 2024-04-25T15:47:46.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.916+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa04c05f5c0 con 0x7fa04c0ffee0 2024-04-25T15:47:46.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.917+0000 7fa03affd700 1 -- 172.21.15.110:0/1734760206 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa04001c750 con 0x7fa04c0ffee0 2024-04-25T15:47:46.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.917+0000 7fa03affd700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa034069230 0x7fa03406b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:46.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.917+0000 7fa03affd700 1 -- 172.21.15.110:0/1734760206 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa040088460 con 0x7fa04c0ffee0 2024-04-25T15:47:46.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.917+0000 7fa04a59c700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa034069230 0x7fa03406b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:46.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.917+0000 7fa04a59c700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa034069230 0x7fa03406b6e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009b80 tx=0x7fa03c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:46.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:46.925+0000 7fa03affd700 1 -- 172.21.15.110:0/1734760206 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa040058220 con 0x7fa04c0ffee0 2024-04-25T15:47:47.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.137+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa04c10f780 con 0x7fa034069230 2024-04-25T15:47:47.138 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.138+0000 7fa03affd700 1 -- 172.21.15.110:0/1734760206 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7fa04c10f780 con 0x7fa034069230 2024-04-25T15:47:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.140+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa034069230 msgr2=0x7fa03406b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.140+0000 7fa050c32700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa034069230 0x7fa03406b6e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa03c009b80 tx=0x7fa03c00b040 comp rx=0 tx=0).stop 2024-04-25T15:47:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.140+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 msgr2=0x7fa04c10dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.140+0000 7fa050c32700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 0x7fa04c10dfa0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa040013870 tx=0x7fa040009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.141+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 shutdown_connections 2024-04-25T15:47:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.141+0000 7fa050c32700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa034069230 0x7fa03406b6e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.141+0000 7fa050c32700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa04c0662c0 0x7fa04c10da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.141+0000 7fa050c32700 1 --2- 172.21.15.110:0/1734760206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa04c0ffee0 0x7fa04c10dfa0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.141+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 >> 172.21.15.110:0/1734760206 conn(0x7fa04c0f7430 msgr2=0x7fa04c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.141+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 shutdown_connections 2024-04-25T15:47:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.141+0000 7fa050c32700 1 -- 172.21.15.110:0/1734760206 wait complete. 2024-04-25T15:47:47.151 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:47:47.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.252+0000 7f63a6f6f700 1 -- 172.21.15.110:0/3142104851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fcf60 msgr2=0x7f63a00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.252+0000 7f63a6f6f700 1 --2- 172.21.15.110:0/3142104851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fcf60 0x7f63a00fd3d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f638c004940 tx=0x7f638c009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:47.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.252+0000 7f63a6f6f700 1 -- 172.21.15.110:0/3142104851 shutdown_connections 2024-04-25T15:47:47.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.252+0000 7f63a6f6f700 1 --2- 172.21.15.110:0/3142104851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fcf60 0x7f63a00fd3d0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.252+0000 7f63a6f6f700 1 --2- 172.21.15.110:0/3142104851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a00fbcb0 0x7f63a00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.252+0000 7f63a6f6f700 1 -- 172.21.15.110:0/3142104851 >> 172.21.15.110:0/3142104851 conn(0x7f63a00f7430 msgr2=0x7f63a00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:47.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.252+0000 7f63a6f6f700 1 -- 172.21.15.110:0/3142104851 shutdown_connections 2024-04-25T15:47:47.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.252+0000 7f63a6f6f700 1 -- 172.21.15.110:0/3142104851 wait complete. 2024-04-25T15:47:47.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a6f6f700 1 Processor -- start 2024-04-25T15:47:47.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a6f6f700 1 -- start start 2024-04-25T15:47:47.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a6f6f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fbcb0 0x7f63a01106e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a6f6f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a00fcf60 0x7f63a0112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a6f6f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a00662e0 con 0x7f63a00fbcb0 2024-04-25T15:47:47.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a6f6f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a0066420 con 0x7f63a00fcf60 2024-04-25T15:47:47.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a4d0b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fbcb0 0x7f63a01106e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a4d0b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fbcb0 0x7f63a01106e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48392/0 (socket says 172.21.15.110:48392) 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f63a4d0b700 1 -- 172.21.15.110:0/2096700038 learned_addr learned my addr 172.21.15.110:0/2096700038 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.253+0000 7f639ffff700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a00fcf60 0x7f63a0112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f63a4d0b700 1 -- 172.21.15.110:0/2096700038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a00fcf60 msgr2=0x7f63a0112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f63a4d0b700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a00fcf60 0x7f63a0112c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f63a4d0b700 1 -- 172.21.15.110:0/2096700038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f638c005040 con 0x7f63a00fbcb0 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f63a4d0b700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fbcb0 0x7f63a01106e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f639400eb40 tx=0x7f639400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f639d7fa700 1 -- 172.21.15.110:0/2096700038 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6394007cc0 con 0x7f63a00fbcb0 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f639d7fa700 1 -- 172.21.15.110:0/2096700038 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f639400f040 con 0x7f63a00fbcb0 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a0113170 con 0x7f63a00fbcb0 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f639d7fa700 1 -- 172.21.15.110:0/2096700038 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639400ca10 con 0x7f63a00fbcb0 2024-04-25T15:47:47.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.254+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a0113660 con 0x7f63a00fbcb0 2024-04-25T15:47:47.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.255+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a005f5c0 con 0x7f63a00fbcb0 2024-04-25T15:47:47.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.255+0000 7f639d7fa700 1 -- 172.21.15.110:0/2096700038 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f639400cc80 con 0x7f63a00fbcb0 2024-04-25T15:47:47.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.256+0000 7f639d7fa700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f63900692a0 0x7f639006b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.256+0000 7f639d7fa700 1 -- 172.21.15.110:0/2096700038 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6394086db0 con 0x7f63a00fbcb0 2024-04-25T15:47:47.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.256+0000 7f639ffff700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f63900692a0 0x7f639006b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.257+0000 7f639ffff700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f63900692a0 0x7f639006b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f638c009b30 tx=0x7f638c009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:47.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.260+0000 7f639d7fa700 1 -- 172.21.15.110:0/2096700038 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f639404e250 con 0x7f63a00fbcb0 2024-04-25T15:47:47.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.484+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63a0101250 con 0x7f63900692a0 2024-04-25T15:47:47.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.485+0000 7f639d7fa700 1 -- 172.21.15.110:0/2096700038 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f63a0101250 con 0x7f63900692a0 2024-04-25T15:47:47.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.487+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f63900692a0 msgr2=0x7f639006b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.487+0000 7f63a6f6f700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f63900692a0 0x7f639006b750 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f638c009b30 tx=0x7f638c009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.487+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fbcb0 msgr2=0x7f63a01106e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.487+0000 7f63a6f6f700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fbcb0 0x7f63a01106e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f639400eb40 tx=0x7f639400b040 comp rx=0 tx=0).stop 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.488+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 shutdown_connections 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.488+0000 7f63a6f6f700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63a00fbcb0 0x7f63a01106e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.488+0000 7f63a6f6f700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f63900692a0 0x7f639006b750 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.488+0000 7f63a6f6f700 1 --2- 172.21.15.110:0/2096700038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63a00fcf60 0x7f63a0112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.488+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 >> 172.21.15.110:0/2096700038 conn(0x7f63a00f7430 msgr2=0x7f63a00f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.488+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 shutdown_connections 2024-04-25T15:47:47.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.488+0000 7f63a6f6f700 1 -- 172.21.15.110:0/2096700038 wait complete. 2024-04-25T15:47:47.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.596+0000 7f0db54b4700 1 -- 172.21.15.110:0/1586431271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fed80 msgr2=0x7f0db00faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.596+0000 7f0db54b4700 1 --2- 172.21.15.110:0/1586431271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fed80 0x7f0db00faec0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0da4004940 tx=0x7f0da4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:47.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.596+0000 7f0db54b4700 1 -- 172.21.15.110:0/1586431271 shutdown_connections 2024-04-25T15:47:47.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.596+0000 7f0db54b4700 1 --2- 172.21.15.110:0/1586431271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fed80 0x7f0db00faec0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.596+0000 7f0db54b4700 1 --2- 172.21.15.110:0/1586431271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db00fe430 0x7f0db00fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.596+0000 7f0db54b4700 1 -- 172.21.15.110:0/1586431271 >> 172.21.15.110:0/1586431271 conn(0x7f0db00f6bb0 msgr2=0x7f0db00f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:47.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.596+0000 7f0db54b4700 1 -- 172.21.15.110:0/1586431271 shutdown_connections 2024-04-25T15:47:47.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.596+0000 7f0db54b4700 1 -- 172.21.15.110:0/1586431271 wait complete. 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.597+0000 7f0db54b4700 1 Processor -- start 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.597+0000 7f0db54b4700 1 -- start start 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.597+0000 7f0db54b4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fe430 0x7f0db010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.597+0000 7f0db54b4700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db00fed80 0x7f0db010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.597+0000 7f0db54b4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db010e460 con 0x7f0db00fe430 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.597+0000 7f0db54b4700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db010e5d0 con 0x7f0db00fed80 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0daeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fe430 0x7f0db010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0daeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fe430 0x7f0db010d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48404/0 (socket says 172.21.15.110:48404) 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0daeffd700 1 -- 172.21.15.110:0/2427943174 learned_addr learned my addr 172.21.15.110:0/2427943174 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0dae7fc700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db00fed80 0x7f0db010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0daeffd700 1 -- 172.21.15.110:0/2427943174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db00fed80 msgr2=0x7f0db010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0daeffd700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db00fed80 0x7f0db010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0daeffd700 1 -- 172.21.15.110:0/2427943174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4005040 con 0x7f0db00fe430 2024-04-25T15:47:47.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0daeffd700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fe430 0x7f0db010d920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0da000eb40 tx=0x7f0da000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:47.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0d977fe700 1 -- 172.21.15.110:0/2427943174 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0da0007cc0 con 0x7f0db00fe430 2024-04-25T15:47:47.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db0112f80 con 0x7f0db00fe430 2024-04-25T15:47:47.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0d977fe700 1 -- 172.21.15.110:0/2427943174 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0da000f040 con 0x7f0db00fe430 2024-04-25T15:47:47.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0d977fe700 1 -- 172.21.15.110:0/2427943174 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0da000ca10 con 0x7f0db00fe430 2024-04-25T15:47:47.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.598+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db0113430 con 0x7f0db00fe430 2024-04-25T15:47:47.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.600+0000 7f0d977fe700 1 -- 172.21.15.110:0/2427943174 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f0da000cc80 con 0x7f0db00fe430 2024-04-25T15:47:47.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.600+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db004c710 con 0x7f0db00fe430 2024-04-25T15:47:47.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.600+0000 7f0d977fe700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0d980692a0 0x7f0d9806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.600+0000 7f0d977fe700 1 -- 172.21.15.110:0/2427943174 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f0da0086d50 con 0x7f0db00fe430 2024-04-25T15:47:47.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.600+0000 7f0dae7fc700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0d980692a0 0x7f0d9806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.601+0000 7f0dae7fc700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0d980692a0 0x7f0d9806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0da4013870 tx=0x7f0da4009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:47.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.605+0000 7f0d977fe700 1 -- 172.21.15.110:0/2427943174 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f0da004e250 con 0x7f0db00fe430 2024-04-25T15:47:47.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.827+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0db0113620 con 0x7f0d980692a0 2024-04-25T15:47:47.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.845+0000 7f0d977fe700 1 -- 172.21.15.110:0/2427943174 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3366 (secure 0 0 0) 0x7f0db0113620 con 0x7f0d980692a0 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (5m) 59s ago 6m 24.8M - 0.25.0 c8568f914cd2 b995d416435c 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (6m) 59s ago 6m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (5m) 66s ago 5m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (5m) 59s ago 6m 87.2M - 9.4.7 954c08fa6188 6cf7f080ae79 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (79s) 59s ago 79s 15.5M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (85s) 59s ago 84s 16.1M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (82s) 66s ago 81s 11.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (77s) 66s ago 76s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:47:47.846 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:9283 running (8m) 59s ago 8m 481M - 17.2.7-902-gc61472c8 0de407c1b3af c623b709ef43 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (5m) 66s ago 5m 431M - 17.2.7-902-gc61472c8 0de407c1b3af 30a2f136ed24 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (8m) 59s ago 8m 46.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (5m) 66s ago 5m 38.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (6m) 59s ago 6m 15.6M - 1.5.0 0da6a335fe13 5f8f351cfd11 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (5m) 66s ago 5m 15.6M - 1.5.0 0da6a335fe13 13afa4d53352 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (4m) 59s ago 4m 45.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (4m) 59s ago 4m 48.2M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (4m) 59s ago 4m 44.5M 4288M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (3m) 66s ago 3m 46.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (3m) 66s ago 3m 44.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (2m) 66s ago 2m 44.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:47:47.847 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (5m) 59s ago 6m 52.5M - 2.43.0 a07b618ecd1d ce7fd55a0368 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.847+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0d980692a0 msgr2=0x7f0d9806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.847+0000 7f0db54b4700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0d980692a0 0x7f0d9806b750 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0da4013870 tx=0x7f0da4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.847+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fe430 msgr2=0x7f0db010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.847+0000 7f0db54b4700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fe430 0x7f0db010d920 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0da000eb40 tx=0x7f0da000b040 comp rx=0 tx=0).stop 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.848+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 shutdown_connections 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.848+0000 7f0db54b4700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0db00fe430 0x7f0db010d920 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.848+0000 7f0db54b4700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f0d980692a0 0x7f0d9806b750 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.848+0000 7f0db54b4700 1 --2- 172.21.15.110:0/2427943174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0db00fed80 0x7f0db010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.848+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 >> 172.21.15.110:0/2427943174 conn(0x7f0db00f6bb0 msgr2=0x7f0db00f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.848+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 shutdown_connections 2024-04-25T15:47:47.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.848+0000 7f0db54b4700 1 -- 172.21.15.110:0/2427943174 wait complete. 2024-04-25T15:47:47.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 -- 172.21.15.110:0/1654233245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 msgr2=0x7f9a5c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 --2- 172.21.15.110:0/1654233245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 0x7f9a5c0fc150 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c004940 tx=0x7f9a4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 -- 172.21.15.110:0/1654233245 shutdown_connections 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 --2- 172.21.15.110:0/1654233245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a5c0fcff0 0x7f9a5c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 --2- 172.21.15.110:0/1654233245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 0x7f9a5c0fc150 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 -- 172.21.15.110:0/1654233245 >> 172.21.15.110:0/1654233245 conn(0x7f9a5c0f74c0 msgr2=0x7f9a5c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 -- 172.21.15.110:0/1654233245 shutdown_connections 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 -- 172.21.15.110:0/1654233245 wait complete. 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.951+0000 7f9a60c78700 1 Processor -- start 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a60c78700 1 -- start start 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a60c78700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 0x7f9a5c1106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a60c78700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a5c0fcff0 0x7f9a5c112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a60c78700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c066bc0 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a60c78700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c066d00 con 0x7f9a5c0fcff0 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a5a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 0x7f9a5c1106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a5a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 0x7f9a5c1106b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48422/0 (socket says 172.21.15.110:48422) 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a5a59c700 1 -- 172.21.15.110:0/2109923158 learned_addr learned my addr 172.21.15.110:0/2109923158 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a59d9b700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a5c0fcff0 0x7f9a5c112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a5a59c700 1 -- 172.21.15.110:0/2109923158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a5c0fcff0 msgr2=0x7f9a5c112c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a5a59c700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a5c0fcff0 0x7f9a5c112c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a5a59c700 1 -- 172.21.15.110:0/2109923158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c005040 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a5a59c700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 0x7f9a5c1106b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c013870 tx=0x7f9a4c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.952+0000 7f9a4affd700 1 -- 172.21.15.110:0/2109923158 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4c021070 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.953+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c113140 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.953+0000 7f9a4affd700 1 -- 172.21.15.110:0/2109923158 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a4c009430 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.953+0000 7f9a4affd700 1 -- 172.21.15.110:0/2109923158 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a4c00e070 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.953+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c113680 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.954+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a5c05f5c0 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.954+0000 7f9a4affd700 1 -- 172.21.15.110:0/2109923158 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f9a4c01c750 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.955+0000 7f9a4affd700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9a440692a0 0x7f9a4406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:47.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.955+0000 7f9a4affd700 1 -- 172.21.15.110:0/2109923158 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f9a4c088460 con 0x7f9a5c0fbd40 2024-04-25T15:47:47.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.955+0000 7f9a59d9b700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9a440692a0 0x7f9a4406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:47.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.955+0000 7f9a59d9b700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9a440692a0 0x7f9a4406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009b80 tx=0x7f9a5000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:47.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:47.959+0000 7f9a4affd700 1 -- 172.21.15.110:0/2109923158 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f9a4c054770 con 0x7f9a5c0fbd40 2024-04-25T15:47:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:47 smithi110 ceph-mon[26344]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:47 smithi193 ceph-mon[33747]: pgmap v216: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:48.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.269+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9a5c113ac0 con 0x7f9a5c0fbd40 2024-04-25T15:47:48.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.270+0000 7f9a4affd700 1 -- 172.21.15.110:0/2109923158 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7f9a4c004490 con 0x7f9a5c0fbd40 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:47:48.271 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:47:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:47:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:47:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:47:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-25T15:47:48.272 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:47:48.272 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:47:48.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.272+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9a440692a0 msgr2=0x7f9a4406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.272+0000 7f9a60c78700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9a440692a0 0x7f9a4406b750 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009b80 tx=0x7f9a5000b040 comp rx=0 tx=0).stop 2024-04-25T15:47:48.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.272+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 msgr2=0x7f9a5c1106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.272+0000 7f9a60c78700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 0x7f9a5c1106b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c013870 tx=0x7f9a4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:48.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.273+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 shutdown_connections 2024-04-25T15:47:48.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.273+0000 7f9a60c78700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a5c0fbd40 0x7f9a5c1106b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.273+0000 7f9a60c78700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f9a440692a0 0x7f9a4406b750 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.273+0000 7f9a60c78700 1 --2- 172.21.15.110:0/2109923158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a5c0fcff0 0x7f9a5c112c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.273+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 >> 172.21.15.110:0/2109923158 conn(0x7f9a5c0f74c0 msgr2=0x7f9a5c0f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:48.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.273+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 shutdown_connections 2024-04-25T15:47:48.274 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.273+0000 7f9a60c78700 1 -- 172.21.15.110:0/2109923158 wait complete. 2024-04-25T15:47:48.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.376+0000 7f3af65b5700 1 -- 172.21.15.110:0/2805054872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 msgr2=0x7f3af00fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.376+0000 7f3af65b5700 1 --2- 172.21.15.110:0/2805054872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 0x7f3af00fe4f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4004940 tx=0x7f3ae4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:48.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.376+0000 7f3af65b5700 1 -- 172.21.15.110:0/2805054872 shutdown_connections 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.376+0000 7f3af65b5700 1 --2- 172.21.15.110:0/2805054872 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 0x7f3af00fe4f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.376+0000 7f3af65b5700 1 --2- 172.21.15.110:0/2805054872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3af00fcf10 0x7f3af00fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.376+0000 7f3af65b5700 1 -- 172.21.15.110:0/2805054872 >> 172.21.15.110:0/2805054872 conn(0x7f3af00f8560 msgr2=0x7f3af00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.377+0000 7f3af65b5700 1 -- 172.21.15.110:0/2805054872 shutdown_connections 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.377+0000 7f3af65b5700 1 -- 172.21.15.110:0/2805054872 wait complete. 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.377+0000 7f3af65b5700 1 Processor -- start 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.377+0000 7f3af65b5700 1 -- start start 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.377+0000 7f3af65b5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3af00fcf10 0x7f3af0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.377+0000 7f3af65b5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 0x7f3af00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.377+0000 7f3af65b5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af0070860 con 0x7f3af00fe080 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.377+0000 7f3af65b5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3af00709d0 con 0x7f3af00fcf10 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aef7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 0x7f3af00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aef7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 0x7f3af00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48438/0 (socket says 172.21.15.110:48438) 2024-04-25T15:47:48.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aef7fe700 1 -- 172.21.15.110:0/181561877 learned_addr learned my addr 172.21.15.110:0/181561877 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:48.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aeffff700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3af00fcf10 0x7f3af0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:48.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aef7fe700 1 -- 172.21.15.110:0/181561877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3af00fcf10 msgr2=0x7f3af0071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aef7fe700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3af00fcf10 0x7f3af0071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aef7fe700 1 -- 172.21.15.110:0/181561877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae4005040 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aef7fe700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 0x7f3af00702b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4013870 tx=0x7f3ae4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aecff9700 1 -- 172.21.15.110:0/181561877 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ae4021070 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aecff9700 1 -- 172.21.15.110:0/181561877 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ae4009430 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3af0070c50 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3aecff9700 1 -- 172.21.15.110:0/181561877 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ae400e070 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.378+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3af01159b0 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.379+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3af005f5c0 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.380+0000 7f3aecff9700 1 -- 172.21.15.110:0/181561877 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3ae401c750 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.380+0000 7f3aecff9700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3ad8069230 0x7f3ad806b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.380+0000 7f3aecff9700 1 -- 172.21.15.110:0/181561877 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3ae4088460 con 0x7f3af00fe080 2024-04-25T15:47:48.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.380+0000 7f3aeffff700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3ad8069230 0x7f3ad806b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:48.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.381+0000 7f3aeffff700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3ad8069230 0x7f3ad806b6e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3af0071b70 tx=0x7f3ae000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:48.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.385+0000 7f3aecff9700 1 -- 172.21.15.110:0/181561877 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3ae4054790 con 0x7f3af00fe080 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.646+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3af004bed0 con 0x7f3af00fe080 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.646+0000 7f3aecff9700 1 -- 172.21.15.110:0/181561877 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f3ae4057db0 con 0x7f3af00fe080 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:47:48.647 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:47:48.648 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:47:48.649 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:47:48.650 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:47:48.650 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:47:48.650 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:47:48.650 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:47:48.650 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:47:48.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3ad8069230 msgr2=0x7f3ad806b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3ad8069230 0x7f3ad806b6e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3af0071b70 tx=0x7f3ae000b040 comp rx=0 tx=0).stop 2024-04-25T15:47:48.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 msgr2=0x7f3af00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 0x7f3af00702b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3ae4013870 tx=0x7f3ae4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:48.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 shutdown_connections 2024-04-25T15:47:48.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3ad8069230 0x7f3ad806b6e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3af00fcf10 0x7f3af0071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 --2- 172.21.15.110:0/181561877 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3af00fe080 0x7f3af00702b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 >> 172.21.15.110:0/181561877 conn(0x7f3af00f8560 msgr2=0x7f3af00f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:48.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 shutdown_connections 2024-04-25T15:47:48.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.648+0000 7f3af65b5700 1 -- 172.21.15.110:0/181561877 wait complete. 2024-04-25T15:47:48.652 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:47:48.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.755+0000 7f04c53ea700 1 -- 172.21.15.110:0/2934905799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 msgr2=0x7f04c00fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.755+0000 7f04c53ea700 1 --2- 172.21.15.110:0/2934905799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 0x7f04c00fe4f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f04b4004940 tx=0x7f04b4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.755+0000 7f04c53ea700 1 -- 172.21.15.110:0/2934905799 shutdown_connections 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 --2- 172.21.15.110:0/2934905799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 0x7f04c00fe4f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 --2- 172.21.15.110:0/2934905799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c00fcf10 0x7f04c00fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 -- 172.21.15.110:0/2934905799 >> 172.21.15.110:0/2934905799 conn(0x7f04c00f8560 msgr2=0x7f04c00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 -- 172.21.15.110:0/2934905799 shutdown_connections 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 -- 172.21.15.110:0/2934905799 wait complete. 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 Processor -- start 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 -- start start 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c00fcf10 0x7f04c010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 0x7f04c010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:48.759 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c010e4f0 con 0x7f04c00fe080 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.756+0000 7f04c53ea700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c010ecf0 con 0x7f04c00fcf10 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 0x7f04c010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04be7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 0x7f04c010def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48454/0 (socket says 172.21.15.110:48454) 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04be7fc700 1 -- 172.21.15.110:0/3433075729 learned_addr learned my addr 172.21.15.110:0/3433075729 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04beffd700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c00fcf10 0x7f04c010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04be7fc700 1 -- 172.21.15.110:0/3433075729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c00fcf10 msgr2=0x7f04c010d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04be7fc700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c00fcf10 0x7f04c010d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04be7fc700 1 -- 172.21.15.110:0/3433075729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b4005040 con 0x7f04c00fe080 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04be7fc700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 0x7f04c010def0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04b4013870 tx=0x7f04b4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:48.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04a77fe700 1 -- 172.21.15.110:0/3433075729 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04b4021070 con 0x7f04c00fe080 2024-04-25T15:47:48.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04a77fe700 1 -- 172.21.15.110:0/3433075729 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04b4009430 con 0x7f04c00fe080 2024-04-25T15:47:48.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c010ef70 con 0x7f04c00fe080 2024-04-25T15:47:48.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04a77fe700 1 -- 172.21.15.110:0/3433075729 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04b400e070 con 0x7f04c00fe080 2024-04-25T15:47:48.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.757+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c010f4b0 con 0x7f04c00fe080 2024-04-25T15:47:48.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.759+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c005f5c0 con 0x7f04c00fe080 2024-04-25T15:47:48.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.759+0000 7f04a77fe700 1 -- 172.21.15.110:0/3433075729 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f04b401c750 con 0x7f04c00fe080 2024-04-25T15:47:48.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.759+0000 7f04a77fe700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f04a8069230 0x7f04a806b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:48.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.759+0000 7f04a77fe700 1 -- 172.21.15.110:0/3433075729 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f04b4088460 con 0x7f04c00fe080 2024-04-25T15:47:48.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.759+0000 7f04beffd700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f04a8069230 0x7f04a806b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:48.762 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.760+0000 7f04beffd700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f04a8069230 0x7f04a806b6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009b80 tx=0x7f04b000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:48.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.764+0000 7f04a77fe700 1 -- 172.21.15.110:0/3433075729 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f04b4058220 con 0x7f04c00fe080 2024-04-25T15:47:48.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.984+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04c010f760 con 0x7f04a8069230 2024-04-25T15:47:48.985 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:48 smithi110 ceph-mon[26344]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:48.985 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:48 smithi110 ceph-mon[26344]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:48.985 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:48 smithi110 ceph-mon[26344]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:48.985 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:48 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2109923158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:47:48.985 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:48 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/181561877' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:47:48.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.985+0000 7f04a77fe700 1 -- 172.21.15.110:0/3433075729 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+447 (secure 0 0 0) 0x7f04c010f760 con 0x7f04a8069230 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "0/21 daemons upgraded", 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 image on host smithi193", 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:47:48.986 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f04a8069230 msgr2=0x7f04a806b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f04a8069230 0x7f04a806b6e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009b80 tx=0x7f04b000b040 comp rx=0 tx=0).stop 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 msgr2=0x7f04c010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 0x7f04c010def0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f04b4013870 tx=0x7f04b4009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 shutdown_connections 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f04a8069230 0x7f04a806b6e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c00fcf10 0x7f04c010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 --2- 172.21.15.110:0/3433075729 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f04c00fe080 0x7f04c010def0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 >> 172.21.15.110:0/3433075729 conn(0x7f04c00f8560 msgr2=0x7f04c00f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 shutdown_connections 2024-04-25T15:47:48.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:48.987+0000 7f04c53ea700 1 -- 172.21.15.110:0/3433075729 wait complete. 2024-04-25T15:47:49.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.101+0000 7fce1543a700 1 -- 172.21.15.110:0/2147423946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fbcb0 msgr2=0x7fce100fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:49.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.101+0000 7fce1543a700 1 --2- 172.21.15.110:0/2147423946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fbcb0 0x7fce100fc0c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fce00004940 tx=0x7fce00009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:49.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.101+0000 7fce1543a700 1 -- 172.21.15.110:0/2147423946 shutdown_connections 2024-04-25T15:47:49.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.101+0000 7fce1543a700 1 --2- 172.21.15.110:0/2147423946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce100fcf60 0x7fce100fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:49.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.101+0000 7fce1543a700 1 --2- 172.21.15.110:0/2147423946 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fbcb0 0x7fce100fc0c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:49.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.101+0000 7fce1543a700 1 -- 172.21.15.110:0/2147423946 >> 172.21.15.110:0/2147423946 conn(0x7fce100f7430 msgr2=0x7fce100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:49.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.101+0000 7fce1543a700 1 -- 172.21.15.110:0/2147423946 shutdown_connections 2024-04-25T15:47:49.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.101+0000 7fce1543a700 1 -- 172.21.15.110:0/2147423946 wait complete. 2024-04-25T15:47:49.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce1543a700 1 Processor -- start 2024-04-25T15:47:49.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce1543a700 1 -- start start 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce1543a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce100fbcb0 0x7fce1010c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce1543a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fcf60 0x7fce1010cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce1543a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1010d3c0 con 0x7fce100fcf60 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce1543a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1010dc10 con 0x7fce100fbcb0 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce0e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fcf60 0x7fce1010cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce0e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fcf60 0x7fce1010cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48462/0 (socket says 172.21.15.110:48462) 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce0effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce100fbcb0 0x7fce1010c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.102+0000 7fce0e7fc700 1 -- 172.21.15.110:0/2256160104 learned_addr learned my addr 172.21.15.110:0/2256160104 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fce0e7fc700 1 -- 172.21.15.110:0/2256160104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce100fbcb0 msgr2=0x7fce1010c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fce0e7fc700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce100fbcb0 0x7fce1010c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fce0e7fc700 1 -- 172.21.15.110:0/2256160104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce00005040 con 0x7fce100fcf60 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fce0e7fc700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fcf60 0x7fce1010cdc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce04010d50 tx=0x7fce0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:49.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fcdf77fe700 1 -- 172.21.15.110:0/2256160104 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce0400e040 con 0x7fce100fcf60 2024-04-25T15:47:49.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1010de90 con 0x7fce100fcf60 2024-04-25T15:47:49.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fcdf77fe700 1 -- 172.21.15.110:0/2256160104 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce0400ce10 con 0x7fce100fcf60 2024-04-25T15:47:49.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fcdf77fe700 1 -- 172.21.15.110:0/2256160104 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce04009860 con 0x7fce100fcf60 2024-04-25T15:47:49.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.103+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1010e3d0 con 0x7fce100fcf60 2024-04-25T15:47:49.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.104+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1005f5c0 con 0x7fce100fcf60 2024-04-25T15:47:49.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.105+0000 7fcdf77fe700 1 -- 172.21.15.110:0/2256160104 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fce0400c940 con 0x7fce100fcf60 2024-04-25T15:47:49.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.105+0000 7fcdf77fe700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcdf80692b0 0x7fcdf806b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:47:49.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.105+0000 7fcdf77fe700 1 -- 172.21.15.110:0/2256160104 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fce04087770 con 0x7fce100fcf60 2024-04-25T15:47:49.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.105+0000 7fce0effd700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcdf80692b0 0x7fcdf806b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:47:49.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.106+0000 7fce0effd700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcdf80692b0 0x7fcdf806b760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce0000b0d0 tx=0x7fce00009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:47:49.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.109+0000 7fcdf77fe700 1 -- 172.21.15.110:0/2256160104 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fce04057530 con 0x7fce100fcf60 2024-04-25T15:47:49.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:48 smithi193 ceph-mon[33747]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:49.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:48 smithi193 ceph-mon[33747]: from='client.14590 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:49.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:48 smithi193 ceph-mon[33747]: from='client.14594 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:49.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:48 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2109923158' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:47:49.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:48 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/181561877' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:47:49.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.386+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fce1010e6e0 con 0x7fce100fcf60 2024-04-25T15:47:49.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.387+0000 7fcdf77fe700 1 -- 172.21.15.110:0/2256160104 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fce040570c0 con 0x7fce100fcf60 2024-04-25T15:47:49.388 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:47:49.388 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:47:49.388 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:47:49.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcdf80692b0 msgr2=0x7fcdf806b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:49.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcdf80692b0 0x7fcdf806b760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fce0000b0d0 tx=0x7fce00009e30 comp rx=0 tx=0).stop 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fcf60 msgr2=0x7fce1010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fcf60 0x7fce1010cdc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fce04010d50 tx=0x7fce0400b040 comp rx=0 tx=0).stop 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 shutdown_connections 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fcdf80692b0 0x7fcdf806b760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce100fbcb0 0x7fce1010c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 --2- 172.21.15.110:0/2256160104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fce100fcf60 0x7fce1010cdc0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 >> 172.21.15.110:0/2256160104 conn(0x7fce100f7430 msgr2=0x7fce100f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 shutdown_connections 2024-04-25T15:47:49.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:47:49.389+0000 7fce1543a700 1 -- 172.21.15.110:0/2256160104 wait complete. 2024-04-25T15:47:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:49 smithi193 ceph-mon[33747]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:49 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2256160104' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:47:50.362 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:49 smithi110 ceph-mon[26344]: pgmap v217: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:50.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:49 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2256160104' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:47:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:50 smithi193 ceph-mon[33747]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:50 smithi110 ceph-mon[26344]: from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:47:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:51 smithi193 ceph-mon[33747]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:52.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:51 smithi110 ceph-mon[26344]: pgmap v218: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:52 smithi193 ceph-mon[33747]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:53.263 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:52 smithi110 ceph-mon[26344]: pgmap v219: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:55.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:55 smithi110 ceph-mon[26344]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:55.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:55 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:47:55.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:55 smithi193 ceph-mon[33747]: pgmap v220: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:55.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:55 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:47:57.612 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:57 smithi110 ceph-mon[26344]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:57.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:57 smithi193 ceph-mon[33747]: pgmap v221: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:47:59 smithi110 ceph-mon[26344]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:47:59.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:47:59 smithi193 ceph-mon[33747]: pgmap v222: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:01.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:01 smithi110 ceph-mon[26344]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:01.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:01 smithi193 ceph-mon[33747]: pgmap v223: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:03.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:03 smithi110 ceph-mon[26344]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:03 smithi193 ceph-mon[33747]: pgmap v224: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:05.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:05 smithi110 ceph-mon[26344]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:05 smithi193 ceph-mon[33747]: pgmap v225: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:07 smithi193 ceph-mon[33747]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:07.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:07 smithi110 ceph-mon[26344]: pgmap v226: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:09.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:09 smithi193 ceph-mon[33747]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:09.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:09 smithi110 ceph-mon[26344]: pgmap v227: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:10.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:10 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:48:10.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:10 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:48:11.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:11 smithi193 ceph-mon[33747]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:11.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:11 smithi110 ceph-mon[26344]: pgmap v228: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:13 smithi193 ceph-mon[33747]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:13 smithi110 ceph-mon[26344]: pgmap v229: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:15 smithi193 ceph-mon[33747]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:15 smithi193 ceph-mon[33747]: Upgrade: Updating mgr.smithi193.mgoteh 2024-04-25T15:48:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:15 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:15 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:48:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:15 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:48:15.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:15 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:48:15.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:15 smithi193 ceph-mon[33747]: Deploying daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:48:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:15 smithi110 ceph-mon[26344]: pgmap v230: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:15 smithi110 ceph-mon[26344]: Upgrade: Updating mgr.smithi193.mgoteh 2024-04-25T15:48:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:15 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:15 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:48:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:15 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:48:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:15 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:48:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:15 smithi110 ceph-mon[26344]: Deploying daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:48:17.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:17 smithi193 ceph-mon[33747]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:17.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:17 smithi110 ceph-mon[26344]: pgmap v231: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:19.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.496+0000 7ff15a52d700 1 -- 172.21.15.110:0/2414372229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 msgr2=0x7ff1540fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:19.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.496+0000 7ff15a52d700 1 --2- 172.21.15.110:0/2414372229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 0x7ff1540fd400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff148004940 tx=0x7ff148009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:19.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.496+0000 7ff15a52d700 1 -- 172.21.15.110:0/2414372229 shutdown_connections 2024-04-25T15:48:19.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.496+0000 7ff15a52d700 1 --2- 172.21.15.110:0/2414372229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 0x7ff1540fd400 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.496+0000 7ff15a52d700 1 --2- 172.21.15.110:0/2414372229 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1540fbce0 0x7ff1540fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.499 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.496+0000 7ff15a52d700 1 -- 172.21.15.110:0/2414372229 >> 172.21.15.110:0/2414372229 conn(0x7ff1540f7440 msgr2=0x7ff1540f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.496+0000 7ff15a52d700 1 -- 172.21.15.110:0/2414372229 shutdown_connections 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.496+0000 7ff15a52d700 1 -- 172.21.15.110:0/2414372229 wait complete. 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff15a52d700 1 Processor -- start 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff15a52d700 1 -- start start 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff15a52d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1540fbce0 0x7ff1541106b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff15a52d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 0x7ff154112c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff15a52d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff154066bd0 con 0x7ff1540fcf90 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff15a52d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff154066d10 con 0x7ff1540fbce0 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff1537fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 0x7ff154112c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff153fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1540fbce0 0x7ff1541106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff1537fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 0x7ff154112c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47858/0 (socket says 172.21.15.110:47858) 2024-04-25T15:48:19.500 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff1537fe700 1 -- 172.21.15.110:0/888652436 learned_addr learned my addr 172.21.15.110:0/888652436 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff1537fe700 1 -- 172.21.15.110:0/888652436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1540fbce0 msgr2=0x7ff1541106b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff1537fe700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1540fbce0 0x7ff1541106b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.497+0000 7ff1537fe700 1 -- 172.21.15.110:0/888652436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff148005040 con 0x7ff1540fcf90 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.498+0000 7ff1537fe700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 0x7ff154112c00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff148009150 tx=0x7ff148009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.498+0000 7ff150ff9700 1 -- 172.21.15.110:0/888652436 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14800e040 con 0x7ff1540fcf90 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.498+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15410c490 con 0x7ff1540fcf90 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.498+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff15410ca00 con 0x7ff1540fcf90 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.498+0000 7ff150ff9700 1 -- 172.21.15.110:0/888652436 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff148004290 con 0x7ff1540fcf90 2024-04-25T15:48:19.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.498+0000 7ff150ff9700 1 -- 172.21.15.110:0/888652436 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff14801c630 con 0x7ff1540fcf90 2024-04-25T15:48:19.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.499+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff15405f5c0 con 0x7ff1540fcf90 2024-04-25T15:48:19.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.499+0000 7ff150ff9700 1 -- 172.21.15.110:0/888652436 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7ff148013420 con 0x7ff1540fcf90 2024-04-25T15:48:19.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.500+0000 7ff150ff9700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ff13c0692a0 0x7ff13c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:19.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.500+0000 7ff150ff9700 1 -- 172.21.15.110:0/888652436 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7ff148087d90 con 0x7ff1540fcf90 2024-04-25T15:48:19.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.500+0000 7ff153fff700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ff13c0692a0 0x7ff13c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:19.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.500+0000 7ff153fff700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ff13c0692a0 0x7ff13c06b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff154066850 tx=0x7ff144011040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:19.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.504+0000 7ff150ff9700 1 -- 172.21.15.110:0/888652436 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7ff148054160 con 0x7ff1540fcf90 2024-04-25T15:48:19.716 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.715+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff154101280 con 0x7ff13c0692a0 2024-04-25T15:48:19.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.716+0000 7ff150ff9700 1 -- 172.21.15.110:0/888652436 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7ff154101280 con 0x7ff13c0692a0 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ff13c0692a0 msgr2=0x7ff13c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ff13c0692a0 0x7ff13c06b750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff154066850 tx=0x7ff144011040 comp rx=0 tx=0).stop 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 msgr2=0x7ff154112c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 0x7ff154112c00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff148009150 tx=0x7ff148009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 shutdown_connections 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7ff13c0692a0 0x7ff13c06b750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1540fbce0 0x7ff1541106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 --2- 172.21.15.110:0/888652436 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1540fcf90 0x7ff154112c00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.719 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 >> 172.21.15.110:0/888652436 conn(0x7ff1540f7440 msgr2=0x7ff1540f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:19.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 shutdown_connections 2024-04-25T15:48:19.720 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.718+0000 7ff15a52d700 1 -- 172.21.15.110:0/888652436 wait complete. 2024-04-25T15:48:19.729 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:48:19.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:19 smithi193 ceph-mon[33747]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.820+0000 7f52f17fb700 1 -- 172.21.15.110:0/1351913298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fbcc0 msgr2=0x7f52ec0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.820+0000 7f52f17fb700 1 --2- 172.21.15.110:0/1351913298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fbcc0 0x7f52ec0fc0d0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f52e4009ce0 tx=0x7f52e4004d10 comp rx=0 tx=0).stop 2024-04-25T15:48:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.820+0000 7f52f17fb700 1 -- 172.21.15.110:0/1351913298 shutdown_connections 2024-04-25T15:48:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.820+0000 7f52f17fb700 1 --2- 172.21.15.110:0/1351913298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52ec0fcf70 0x7f52ec0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.820+0000 7f52f17fb700 1 --2- 172.21.15.110:0/1351913298 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fbcc0 0x7f52ec0fc0d0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.820+0000 7f52f17fb700 1 -- 172.21.15.110:0/1351913298 >> 172.21.15.110:0/1351913298 conn(0x7f52ec0f7440 msgr2=0x7f52ec0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:19.822 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:19 smithi110 ceph-mon[26344]: pgmap v232: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:19.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.820+0000 7f52f17fb700 1 -- 172.21.15.110:0/1351913298 shutdown_connections 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.820+0000 7f52f17fb700 1 -- 172.21.15.110:0/1351913298 wait complete. 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52f17fb700 1 Processor -- start 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52f17fb700 1 -- start start 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52f17fb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52ec0fbcc0 0x7f52ec10db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52f17fb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fcf70 0x7f52ec10e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52f17fb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec10e6c0 con 0x7f52ec0fcf70 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52f17fb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ec10e830 con 0x7f52ec0fbcc0 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52ea7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fcf70 0x7f52ec10e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52ea7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fcf70 0x7f52ec10e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47868/0 (socket says 172.21.15.110:47868) 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52ea7fc700 1 -- 172.21.15.110:0/2720569764 learned_addr learned my addr 172.21.15.110:0/2720569764 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52eaffd700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52ec0fbcc0 0x7f52ec10db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:19.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52ea7fc700 1 -- 172.21.15.110:0/2720569764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52ec0fbcc0 msgr2=0x7f52ec10db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52ea7fc700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52ec0fbcc0 0x7f52ec10db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.821+0000 7f52ea7fc700 1 -- 172.21.15.110:0/2720569764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e4009940 con 0x7f52ec0fcf70 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.822+0000 7f52ea7fc700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fcf70 0x7f52ec10e0c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f52d4002a50 tx=0x7f52d40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.822+0000 7f52e37fe700 1 -- 172.21.15.110:0/2720569764 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d4007cc0 con 0x7f52ec0fcf70 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.822+0000 7f52e37fe700 1 -- 172.21.15.110:0/2720569764 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f52d400f040 con 0x7f52ec0fcf70 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.822+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec1131f0 con 0x7f52ec0fcf70 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.822+0000 7f52e37fe700 1 -- 172.21.15.110:0/2720569764 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d400da30 con 0x7f52ec0fcf70 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.822+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ec1137b0 con 0x7f52ec0fcf70 2024-04-25T15:48:19.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.823+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ec101b00 con 0x7f52ec0fcf70 2024-04-25T15:48:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.823+0000 7f52e37fe700 1 -- 172.21.15.110:0/2720569764 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f52d400dbb0 con 0x7f52ec0fcf70 2024-04-25T15:48:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.824+0000 7f52e37fe700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52d80692a0 0x7f52d806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.824+0000 7f52e37fe700 1 -- 172.21.15.110:0/2720569764 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f52d4086c40 con 0x7f52ec0fcf70 2024-04-25T15:48:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.824+0000 7f52eaffd700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52d80692a0 0x7f52d806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:19.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.824+0000 7f52eaffd700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52d80692a0 0x7f52d806b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f52e400b740 tx=0x7f52e4004d10 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:19.829 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:19.828+0000 7f52e37fe700 1 -- 172.21.15.110:0/2720569764 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f52d4056770 con 0x7f52ec0fcf70 2024-04-25T15:48:20.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.040+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52ec02b2f0 con 0x7f52d80692a0 2024-04-25T15:48:20.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.041+0000 7f52e37fe700 1 -- 172.21.15.110:0/2720569764 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f52ec02b2f0 con 0x7f52d80692a0 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52d80692a0 msgr2=0x7f52d806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52d80692a0 0x7f52d806b750 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f52e400b740 tx=0x7f52e4004d10 comp rx=0 tx=0).stop 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fcf70 msgr2=0x7f52ec10e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fcf70 0x7f52ec10e0c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f52d4002a50 tx=0x7f52d40046b0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 shutdown_connections 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f52d80692a0 0x7f52d806b750 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f52ec0fbcc0 0x7f52ec10db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 --2- 172.21.15.110:0/2720569764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f52ec0fcf70 0x7f52ec10e0c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 >> 172.21.15.110:0/2720569764 conn(0x7f52ec0f7440 msgr2=0x7f52ec066860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 shutdown_connections 2024-04-25T15:48:20.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.043+0000 7f52f17fb700 1 -- 172.21.15.110:0/2720569764 wait complete. 2024-04-25T15:48:20.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.151+0000 7febd0f64700 1 -- 172.21.15.110:0/4028827480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fdbf0 msgr2=0x7febcc0fc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.151+0000 7febd0f64700 1 --2- 172.21.15.110:0/4028827480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fdbf0 0x7febcc0fc000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7febc0004940 tx=0x7febc0009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.151+0000 7febd0f64700 1 -- 172.21.15.110:0/4028827480 shutdown_connections 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.151+0000 7febd0f64700 1 --2- 172.21.15.110:0/4028827480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fdbf0 0x7febcc0fc000 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.151+0000 7febd0f64700 1 --2- 172.21.15.110:0/4028827480 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febcc0fd2a0 0x7febcc0fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.151+0000 7febd0f64700 1 -- 172.21.15.110:0/4028827480 >> 172.21.15.110:0/4028827480 conn(0x7febcc0f6b80 msgr2=0x7febcc0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.151+0000 7febd0f64700 1 -- 172.21.15.110:0/4028827480 shutdown_connections 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.151+0000 7febd0f64700 1 -- 172.21.15.110:0/4028827480 wait complete. 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febd0f64700 1 Processor -- start 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febd0f64700 1 -- start start 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febd0f64700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fd2a0 0x7febcc110760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febd0f64700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febcc0fdbf0 0x7febcc112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febd0f64700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc0fbc60 con 0x7febcc0fd2a0 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febd0f64700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febcc0fbda0 con 0x7febcc0fdbf0 2024-04-25T15:48:20.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febca59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fd2a0 0x7febcc110760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febca59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fd2a0 0x7febcc110760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47872/0 (socket says 172.21.15.110:47872) 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febca59c700 1 -- 172.21.15.110:0/1156039599 learned_addr learned my addr 172.21.15.110:0/1156039599 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febc9d9b700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febcc0fdbf0 0x7febcc112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febca59c700 1 -- 172.21.15.110:0/1156039599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febcc0fdbf0 msgr2=0x7febcc112cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febca59c700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febcc0fdbf0 0x7febcc112cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febca59c700 1 -- 172.21.15.110:0/1156039599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0005040 con 0x7febcc0fd2a0 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.152+0000 7febca59c700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fd2a0 0x7febcc110760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7febbc002a50 tx=0x7febbc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.153+0000 7febbaffd700 1 -- 172.21.15.110:0/1156039599 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febbc007cc0 con 0x7febcc0fd2a0 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.153+0000 7febbaffd700 1 -- 172.21.15.110:0/1156039599 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febbc00f040 con 0x7febcc0fd2a0 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.153+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febcc1132b0 con 0x7febcc0fd2a0 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.153+0000 7febbaffd700 1 -- 172.21.15.110:0/1156039599 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febbc00da30 con 0x7febcc0fd2a0 2024-04-25T15:48:20.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.153+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febcc113840 con 0x7febcc0fd2a0 2024-04-25T15:48:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.154+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febcc05f5c0 con 0x7febcc0fd2a0 2024-04-25T15:48:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.154+0000 7febbaffd700 1 -- 172.21.15.110:0/1156039599 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7febbc00dbb0 con 0x7febcc0fd2a0 2024-04-25T15:48:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.155+0000 7febbaffd700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7febb40692b0 0x7febb406b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.155+0000 7febbaffd700 1 -- 172.21.15.110:0/1156039599 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7febbc086f20 con 0x7febcc0fd2a0 2024-04-25T15:48:20.156 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.155+0000 7febc9d9b700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7febb40692b0 0x7febb406b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.157 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.155+0000 7febc9d9b700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7febb40692b0 0x7febb406b760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febc0000c00 tx=0x7febc0009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:20.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.159+0000 7febbaffd700 1 -- 172.21.15.110:0/1156039599 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7febbc0531d0 con 0x7febcc0fd2a0 2024-04-25T15:48:20.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.359+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7febcc1013c0 con 0x7febb40692b0 2024-04-25T15:48:20.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.369+0000 7febbaffd700 1 -- 172.21.15.110:0/1156039599 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3410 (secure 0 0 0) 0x7febcc1013c0 con 0x7febb40692b0 2024-04-25T15:48:20.370 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:48:20.370 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (5m) 91s ago 7m 24.8M - 0.25.0 c8568f914cd2 b995d416435c 2024-04-25T15:48:20.370 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (7m) 91s ago 7m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:48:20.370 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (6m) 98s ago 6m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:48:20.370 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (5m) 91s ago 6m 87.2M - 9.4.7 954c08fa6188 6cf7f080ae79 2024-04-25T15:48:20.370 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (112s) 91s ago 111s 15.5M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:48:20.370 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (117s) 91s ago 116s 16.1M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (114s) 98s ago 114s 11.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (109s) 98s ago 109s 13.5M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:9283 running (8m) 91s ago 8m 481M - 17.2.7-902-gc61472c8 0de407c1b3af c623b709ef43 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (6m) 98s ago 6m 431M - 17.2.7-902-gc61472c8 0de407c1b3af 30a2f136ed24 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (8m) 91s ago 8m 46.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (6m) 98s ago 6m 38.1M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (7m) 91s ago 7m 15.6M - 1.5.0 0da6a335fe13 5f8f351cfd11 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 98s ago 6m 15.6M - 1.5.0 0da6a335fe13 13afa4d53352 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (5m) 91s ago 5m 45.3M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (5m) 91s ago 5m 48.2M 4288M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (4m) 91s ago 4m 44.5M 4288M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (4m) 98s ago 4m 46.2M 5312M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (3m) 98s ago 3m 44.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (3m) 98s ago 3m 44.1M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:48:20.371 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (5m) 91s ago 6m 52.5M - 2.43.0 a07b618ecd1d ce7fd55a0368 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7febb40692b0 msgr2=0x7febb406b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7febb40692b0 0x7febb406b760 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7febc0000c00 tx=0x7febc0009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fd2a0 msgr2=0x7febcc110760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fd2a0 0x7febcc110760 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7febbc002a50 tx=0x7febbc0046b0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 shutdown_connections 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febcc0fd2a0 0x7febcc110760 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7febb40692b0 0x7febb406b760 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 --2- 172.21.15.110:0/1156039599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7febcc0fdbf0 0x7febcc112cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 >> 172.21.15.110:0/1156039599 conn(0x7febcc0f6b80 msgr2=0x7febcc0f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 shutdown_connections 2024-04-25T15:48:20.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.372+0000 7febd0f64700 1 -- 172.21.15.110:0/1156039599 wait complete. 2024-04-25T15:48:20.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 -- 172.21.15.110:0/1964805971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 msgr2=0x7fa4580fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 --2- 172.21.15.110:0/1964805971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 0x7fa4580fd3d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa444004940 tx=0x7fa444009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:20.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 -- 172.21.15.110:0/1964805971 shutdown_connections 2024-04-25T15:48:20.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 --2- 172.21.15.110:0/1964805971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 0x7fa4580fd3d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 --2- 172.21.15.110:0/1964805971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4580fbcb0 0x7fa4580fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 -- 172.21.15.110:0/1964805971 >> 172.21.15.110:0/1964805971 conn(0x7fa4580f7430 msgr2=0x7fa4580f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:20.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 -- 172.21.15.110:0/1964805971 shutdown_connections 2024-04-25T15:48:20.476 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 -- 172.21.15.110:0/1964805971 wait complete. 2024-04-25T15:48:20.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 Processor -- start 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.475+0000 7fa45ec3b700 1 -- start start 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa45ec3b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4580fbcb0 0x7fa45810fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa45ec3b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 0x7fa458110270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa45ec3b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa458110870 con 0x7fa4580fcf60 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa45ec3b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa458110fb0 con 0x7fa4580fbcb0 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa457fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 0x7fa458110270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa457fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 0x7fa458110270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47888/0 (socket says 172.21.15.110:47888) 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa457fff700 1 -- 172.21.15.110:0/491980164 learned_addr learned my addr 172.21.15.110:0/491980164 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa45c9d7700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4580fbcb0 0x7fa45810fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa457fff700 1 -- 172.21.15.110:0/491980164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4580fbcb0 msgr2=0x7fa45810fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa457fff700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4580fbcb0 0x7fa45810fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa457fff700 1 -- 172.21.15.110:0/491980164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa444005040 con 0x7fa4580fcf60 2024-04-25T15:48:20.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa457fff700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 0x7fa458110270 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b3b0 tx=0x7fa444009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:20.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.476+0000 7fa4557fa700 1 -- 172.21.15.110:0/491980164 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44400e040 con 0x7fa4580fcf60 2024-04-25T15:48:20.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.477+0000 7fa4557fa700 1 -- 172.21.15.110:0/491980164 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa44400bbf0 con 0x7fa4580fcf60 2024-04-25T15:48:20.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.477+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa458111230 con 0x7fa4580fcf60 2024-04-25T15:48:20.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.477+0000 7fa4557fa700 1 -- 172.21.15.110:0/491980164 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa44401c630 con 0x7fa4580fcf60 2024-04-25T15:48:20.479 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.477+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4581117f0 con 0x7fa4580fcf60 2024-04-25T15:48:20.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.478+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4581018f0 con 0x7fa4580fcf60 2024-04-25T15:48:20.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.478+0000 7fa4557fa700 1 -- 172.21.15.110:0/491980164 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fa44400bd60 con 0x7fa4580fcf60 2024-04-25T15:48:20.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.479+0000 7fa4557fa700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa4480692a0 0x7fa44806b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.479+0000 7fa45c9d7700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa4480692a0 0x7fa44806b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.479+0000 7fa4557fa700 1 -- 172.21.15.110:0/491980164 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fa444087da0 con 0x7fa4580fcf60 2024-04-25T15:48:20.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.479+0000 7fa45c9d7700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa4480692a0 0x7fa44806b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa44c004f40 tx=0x7fa44c00b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:20.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.483+0000 7fa4557fa700 1 -- 172.21.15.110:0/491980164 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fa444057c00 con 0x7fa4580fcf60 2024-04-25T15:48:20.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.782+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa458104930 con 0x7fa4580fcf60 2024-04-25T15:48:20.784 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:20 smithi110 ceph-mon[26344]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:20.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.783+0000 7fa4557fa700 1 -- 172.21.15.110:0/491980164 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+638 (secure 0 0 0) 0x7fa458104930 con 0x7fa4580fcf60 2024-04-25T15:48:20.785 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:48:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:48:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:48:20.785 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:48:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:48:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:48:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:48:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:48:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:48:20.786 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:48:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:48:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:48:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:48:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:48:20.787 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 14 2024-04-25T15:48:20.788 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:48:20.788 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:48:20.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa4480692a0 msgr2=0x7fa44806b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa4480692a0 0x7fa44806b750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa44c004f40 tx=0x7fa44c00b040 comp rx=0 tx=0).stop 2024-04-25T15:48:20.789 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 msgr2=0x7fa458110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 0x7fa458110270 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa44400b3b0 tx=0x7fa444009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:20.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 shutdown_connections 2024-04-25T15:48:20.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fa4480692a0 0x7fa44806b750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4580fbcb0 0x7fa45810fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 --2- 172.21.15.110:0/491980164 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4580fcf60 0x7fa458110270 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 >> 172.21.15.110:0/491980164 conn(0x7fa4580f7430 msgr2=0x7fa458065cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:20.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 shutdown_connections 2024-04-25T15:48:20.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.785+0000 7fa45ec3b700 1 -- 172.21.15.110:0/491980164 wait complete. 2024-04-25T15:48:20.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.888+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/4218961053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fcf60 msgr2=0x7fb8c80fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.888+0000 7fb8cdfaa700 1 --2- 172.21.15.110:0/4218961053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fcf60 0x7fb8c80fd3d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc004940 tx=0x7fb8bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.888+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/4218961053 shutdown_connections 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.888+0000 7fb8cdfaa700 1 --2- 172.21.15.110:0/4218961053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fcf60 0x7fb8c80fd3d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.888+0000 7fb8cdfaa700 1 --2- 172.21.15.110:0/4218961053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8c80fbcb0 0x7fb8c80fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.888+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/4218961053 >> 172.21.15.110:0/4218961053 conn(0x7fb8c80f7430 msgr2=0x7fb8c80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.888+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/4218961053 shutdown_connections 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.888+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/4218961053 wait complete. 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8cdfaa700 1 Processor -- start 2024-04-25T15:48:20.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8cdfaa700 1 -- start start 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8cdfaa700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fbcb0 0x7fb8c810c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8cdfaa700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8c80fcf60 0x7fb8c810ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8cdfaa700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c810d470 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8cdfaa700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c810dcc0 con 0x7fb8c80fcf60 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8c77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fbcb0 0x7fb8c810c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8c77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fbcb0 0x7fb8c810c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47896/0 (socket says 172.21.15.110:47896) 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8c77fe700 1 -- 172.21.15.110:0/2634910448 learned_addr learned my addr 172.21.15.110:0/2634910448 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.889+0000 7fb8c6ffd700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8c80fcf60 0x7fb8c810ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8c77fe700 1 -- 172.21.15.110:0/2634910448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8c80fcf60 msgr2=0x7fb8c810ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8c77fe700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8c80fcf60 0x7fb8c810ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:20.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8c77fe700 1 -- 172.21.15.110:0/2634910448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc005040 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8c77fe700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fbcb0 0x7fb8c810c930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800ed20 tx=0x7fb8b8004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8affff700 1 -- 172.21.15.110:0/2634910448 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b800cd10 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c810df40 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8affff700 1 -- 172.21.15.110:0/2634910448 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8b800ce70 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8affff700 1 -- 172.21.15.110:0/2634910448 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b8010870 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.890+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c810e500 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.891+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c805f5c0 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.892+0000 7fb8affff700 1 -- 172.21.15.110:0/2634910448 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7fb8b8010b00 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.892+0000 7fb8affff700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8b00692b0 0x7fb8b006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.892+0000 7fb8affff700 1 -- 172.21.15.110:0/2634910448 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7fb8b8089a80 con 0x7fb8c80fbcb0 2024-04-25T15:48:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.892+0000 7fb8c6ffd700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8b00692b0 0x7fb8b006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:20.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.893+0000 7fb8c6ffd700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8b00692b0 0x7fb8b006b760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00b3b0 tx=0x7fb8bc009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:20.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:20.896+0000 7fb8affff700 1 -- 172.21.15.110:0/2634910448 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7fb8b8059840 con 0x7fb8c80fbcb0 2024-04-25T15:48:21.000 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:20 smithi193 ceph-mon[33747]: from='client.14614 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.146+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb8c810e7e0 con 0x7fb8c80fbcb0 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.146+0000 7fb8affff700 1 -- 172.21.15.110:0/2634910448 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7fb8b8015720 con 0x7fb8c80fbcb0 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:48:21.147 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:48:21.148 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:48:21.149 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:48:21.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8b00692b0 msgr2=0x7fb8b006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8b00692b0 0x7fb8b006b760 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc00b3b0 tx=0x7fb8bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:21.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fbcb0 msgr2=0x7fb8c810c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fbcb0 0x7fb8c810c930 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800ed20 tx=0x7fb8b8004570 comp rx=0 tx=0).stop 2024-04-25T15:48:21.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 shutdown_connections 2024-04-25T15:48:21.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8c80fbcb0 0x7fb8c810c930 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7fb8b00692b0 0x7fb8b006b760 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 --2- 172.21.15.110:0/2634910448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8c80fcf60 0x7fb8c810ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 >> 172.21.15.110:0/2634910448 conn(0x7fb8c80f7430 msgr2=0x7fb8c80f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:21.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 shutdown_connections 2024-04-25T15:48:21.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.149+0000 7fb8cdfaa700 1 -- 172.21.15.110:0/2634910448 wait complete. 2024-04-25T15:48:21.151 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:48:21.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 -- 172.21.15.110:0/2956421255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 msgr2=0x7f3cdc1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 --2- 172.21.15.110:0/2956421255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 0x7f3cdc1023b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc004940 tx=0x7f3ccc009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:21.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 -- 172.21.15.110:0/2956421255 shutdown_connections 2024-04-25T15:48:21.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 --2- 172.21.15.110:0/2956421255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 0x7f3cdc1023b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 --2- 172.21.15.110:0/2956421255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cdc066af0 0x7f3cdc0ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 -- 172.21.15.110:0/2956421255 >> 172.21.15.110:0/2956421255 conn(0x7f3cdc0f74c0 msgr2=0x7f3cdc0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 -- 172.21.15.110:0/2956421255 shutdown_connections 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 -- 172.21.15.110:0/2956421255 wait complete. 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 Processor -- start 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.255+0000 7f3ce16ba700 1 -- start start 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3ce16ba700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cdc066af0 0x7f3cdc10da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3ce16ba700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 0x7f3cdc10df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3ce16ba700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc10e580 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3ce16ba700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc10edd0 con 0x7f3cdc066af0 2024-04-25T15:48:21.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cda7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 0x7f3cdc10df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cda7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 0x7f3cdc10df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47908/0 (socket says 172.21.15.110:47908) 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cda7fc700 1 -- 172.21.15.110:0/3520331800 learned_addr learned my addr 172.21.15.110:0/3520331800 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cdaffd700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cdc066af0 0x7f3cdc10da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cda7fc700 1 -- 172.21.15.110:0/3520331800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cdc066af0 msgr2=0x7f3cdc10da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cda7fc700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cdc066af0 0x7f3cdc10da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cda7fc700 1 -- 172.21.15.110:0/3520331800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ccc005040 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cda7fc700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 0x7f3cdc10df80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b3b0 tx=0x7f3ccc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cc37fe700 1 -- 172.21.15.110:0/3520331800 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ccc00e040 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cdc10f050 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cc37fe700 1 -- 172.21.15.110:0/3520331800 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ccc00bbf0 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3cc37fe700 1 -- 172.21.15.110:0/3520331800 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ccc01c630 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.256+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cdc10f590 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.258+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cdc05f5c0 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.258+0000 7f3cc37fe700 1 -- 172.21.15.110:0/3520331800 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f3ccc00bd60 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.258+0000 7f3cc37fe700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3cc40692a0 0x7f3cc406b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:21.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.259+0000 7f3cc37fe700 1 -- 172.21.15.110:0/3520331800 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f3ccc087ec0 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.259+0000 7f3cdaffd700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3cc40692a0 0x7f3cc406b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:21.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.259+0000 7f3cdaffd700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3cc40692a0 0x7f3cc406b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc066830 tx=0x7f3cd000b040 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:21.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.263+0000 7f3cc37fe700 1 -- 172.21.15.110:0/3520331800 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f3ccc057c80 con 0x7f3cdc0fffd0 2024-04-25T15:48:21.473 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.472+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 --> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cdc10f880 con 0x7f3cc40692a0 2024-04-25T15:48:21.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.474+0000 7f3cc37fe700 1 -- 172.21.15.110:0/3520331800 <== mgr.14210 v2:172.21.15.110:6800/1619647096 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+349 (secure 0 0 0) 0x7f3cdc10f880 con 0x7f3cc40692a0 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "0/21 daemons upgraded", 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mgr daemons", 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:48:21.475 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3cc40692a0 msgr2=0x7f3cc406b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3cc40692a0 0x7f3cc406b750 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc066830 tx=0x7f3cd000b040 comp rx=0 tx=0).stop 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 msgr2=0x7f3cdc10df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 0x7f3cdc10df80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00b3b0 tx=0x7f3ccc009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 shutdown_connections 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f3cc40692a0 0x7f3cc406b750 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3cdc066af0 0x7f3cdc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 --2- 172.21.15.110:0/3520331800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3cdc0fffd0 0x7f3cdc10df80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.477 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 >> 172.21.15.110:0/3520331800 conn(0x7f3cdc0f74c0 msgr2=0x7f3cdc0f7890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:21.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 shutdown_connections 2024-04-25T15:48:21.478 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.476+0000 7f3ce16ba700 1 -- 172.21.15.110:0/3520331800 wait complete. 2024-04-25T15:48:21.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.581+0000 7f6d77f25700 1 -- 172.21.15.110:0/89409356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fd2a0 msgr2=0x7f6d700fd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.581+0000 7f6d77f25700 1 --2- 172.21.15.110:0/89409356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fd2a0 0x7f6d700fd6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6d60004940 tx=0x7f6d60009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:21.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.581+0000 7f6d77f25700 1 -- 172.21.15.110:0/89409356 shutdown_connections 2024-04-25T15:48:21.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.581+0000 7f6d77f25700 1 --2- 172.21.15.110:0/89409356 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d700fdbf0 0x7f6d700fc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.581+0000 7f6d77f25700 1 --2- 172.21.15.110:0/89409356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fd2a0 0x7f6d700fd6b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.581+0000 7f6d77f25700 1 -- 172.21.15.110:0/89409356 >> 172.21.15.110:0/89409356 conn(0x7f6d700f6b80 msgr2=0x7f6d700f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:21.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.582+0000 7f6d77f25700 1 -- 172.21.15.110:0/89409356 shutdown_connections 2024-04-25T15:48:21.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.582+0000 7f6d77f25700 1 -- 172.21.15.110:0/89409356 wait complete. 2024-04-25T15:48:21.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.582+0000 7f6d77f25700 1 Processor -- start 2024-04-25T15:48:21.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.582+0000 7f6d77f25700 1 -- start start 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.582+0000 7f6d77f25700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d700fd2a0 0x7f6d701106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.582+0000 7f6d77f25700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fdbf0 0x7f6d70112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.582+0000 7f6d77f25700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d700fbc40 con 0x7f6d700fdbf0 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.582+0000 7f6d77f25700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d700fbd80 con 0x7f6d700fd2a0 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d754c0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fdbf0 0x7f6d70112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d75cc1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d700fd2a0 0x7f6d701106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d754c0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fdbf0 0x7f6d70112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47928/0 (socket says 172.21.15.110:47928) 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d754c0700 1 -- 172.21.15.110:0/855380825 learned_addr learned my addr 172.21.15.110:0/855380825 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d754c0700 1 -- 172.21.15.110:0/855380825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d700fd2a0 msgr2=0x7f6d701106f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d754c0700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d700fd2a0 0x7f6d701106f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d754c0700 1 -- 172.21.15.110:0/855380825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d60005040 con 0x7f6d700fdbf0 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d754c0700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fdbf0 0x7f6d70112c40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00eb40 tx=0x7f6d6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d667fc700 1 -- 172.21.15.110:0/855380825 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6c007cc0 con 0x7f6d700fdbf0 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d667fc700 1 -- 172.21.15.110:0/855380825 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d6c00f040 con 0x7f6d700fdbf0 2024-04-25T15:48:21.585 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d667fc700 1 -- 172.21.15.110:0/855380825 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d6c00c9d0 con 0x7f6d700fdbf0 2024-04-25T15:48:21.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d70113180 con 0x7f6d700fdbf0 2024-04-25T15:48:21.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.583+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d70113740 con 0x7f6d700fdbf0 2024-04-25T15:48:21.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.584+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d7005f5c0 con 0x7f6d700fdbf0 2024-04-25T15:48:21.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.585+0000 7f6d667fc700 1 -- 172.21.15.110:0/855380825 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) v1 ==== 86631+0+0 (secure 0 0 0) 0x7f6d6c00cb30 con 0x7f6d700fdbf0 2024-04-25T15:48:21.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.585+0000 7f6d667fc700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f6d5c0692a0 0x7f6d5c06b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:21.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.586+0000 7f6d667fc700 1 -- 172.21.15.110:0/855380825 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5464+0+0 (secure 0 0 0) 0x7f6d6c086ac0 con 0x7f6d700fdbf0 2024-04-25T15:48:21.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.586+0000 7f6d75cc1700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f6d5c0692a0 0x7f6d5c06b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:21.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.586+0000 7f6d75cc1700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f6d5c0692a0 0x7f6d5c06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6d60013870 tx=0x7f6d60009e30 comp rx=0 tx=0).ready entity=mgr.14210 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:21.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.590+0000 7f6d667fc700 1 -- 172.21.15.110:0/855380825 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176702 (secure 0 0 0) 0x7f6d6c04e250 con 0x7f6d700fdbf0 2024-04-25T15:48:21.705 INFO:tasks.workunit.client.0.smithi110.stderr:Updating files: 94% (11650/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-25T15:48:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:21 smithi110 ceph-mon[26344]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:21 smithi110 ceph-mon[26344]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:21 smithi110 ceph-mon[26344]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:21 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/491980164' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:48:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:21 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2634910448' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:48:21.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.870+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6d7004bed0 con 0x7f6d700fdbf0 2024-04-25T15:48:21.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.870+0000 7f6d667fc700 1 -- 172.21.15.110:0/855380825 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f6d6c052ea0 con 0x7f6d700fdbf0 2024-04-25T15:48:21.871 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:48:21.871 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:48:21.871 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:48:21.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.872+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f6d5c0692a0 msgr2=0x7f6d5c06b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.872+0000 7f6d77f25700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f6d5c0692a0 0x7f6d5c06b750 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6d60013870 tx=0x7f6d60009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.872+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fdbf0 msgr2=0x7f6d70112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.872+0000 7f6d77f25700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fdbf0 0x7f6d70112c40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c00eb40 tx=0x7f6d6c00b040 comp rx=0 tx=0).stop 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.872+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 shutdown_connections 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.873+0000 7f6d77f25700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:6800/1619647096,v1:172.21.15.110:6801/1619647096] conn(0x7f6d5c0692a0 0x7f6d5c06b750 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.873+0000 7f6d77f25700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d700fd2a0 0x7f6d701106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.873+0000 7f6d77f25700 1 --2- 172.21.15.110:0/855380825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6d700fdbf0 0x7f6d70112c40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.873+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 >> 172.21.15.110:0/855380825 conn(0x7f6d700f6b80 msgr2=0x7f6d700f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.873+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 shutdown_connections 2024-04-25T15:48:21.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:21.873+0000 7f6d77f25700 1 -- 172.21.15.110:0/855380825 wait complete. 2024-04-25T15:48:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:21 smithi193 ceph-mon[33747]: from='client.14618 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:21 smithi193 ceph-mon[33747]: pgmap v233: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:21 smithi193 ceph-mon[33747]: from='client.14622 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:21 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/491980164' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:48:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:21 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2634910448' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr: git switch -c 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr:Or undo this operation with: 2024-04-25T15:48:22.467 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-04-25T15:48:22.468 INFO:tasks.workunit.client.0.smithi110.stderr: git switch - 2024-04-25T15:48:22.468 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-04-25T15:48:22.468 INFO:tasks.workunit.client.0.smithi110.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T15:48:22.468 INFO:tasks.workunit.client.0.smithi110.stderr: 2024-04-25T15:48:22.468 INFO:tasks.workunit.client.0.smithi110.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T15:48:22.473 DEBUG:teuthology.orchestra.run.smithi110:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T15:48:22.604 INFO:tasks.workunit.client.0.smithi110.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T15:48:22.607 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T15:48:22.607 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T15:48:22.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:22 smithi193 ceph-mon[33747]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:22.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:22 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/855380825' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:48:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:22 smithi110 ceph-mon[26344]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:22 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/855380825' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:48:23.242 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T15:48:23.297 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T15:48:23.337 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T15:48:23.338 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T15:48:23.338 INFO:tasks.workunit.client.0.smithi110.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T15:48:23.376 INFO:tasks.workunit.client.0.smithi110.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T15:48:23.379 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T15:48:23.380 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T15:48:23.395 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-25T15:48:23.396 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-25T15:48:23.396 DEBUG:teuthology.orchestra.run.smithi110:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-25T15:48:23.459 INFO:tasks.workunit.client.0.smithi110.stderr:+ mkdir -p fsstress 2024-04-25T15:48:23.461 INFO:tasks.workunit.client.0.smithi110.stderr:+ pushd fsstress 2024-04-25T15:48:23.462 INFO:tasks.workunit.client.0.smithi110.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T15:48:23.462 INFO:tasks.workunit.client.0.smithi110.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-25T15:48:23.753 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:23 smithi193 ceph-mon[33747]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:23.763 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:23 smithi110 ceph-mon[26344]: pgmap v234: 65 pgs: 65 active+clean; 588 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:23.993 INFO:tasks.workunit.client.0.smithi110.stderr:+ tar xzf ltp-full.tgz 2024-04-25T15:48:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:25 smithi110 ceph-mon[26344]: pgmap v235: 65 pgs: 65 active+clean; 595 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-25T15:48:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:48:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:25 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:48:25.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:25 smithi193 ceph-mon[33747]: pgmap v235: 65 pgs: 65 active+clean; 595 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 682 B/s wr, 0 op/s 2024-04-25T15:48:25.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:25.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:25.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:48:25.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:25 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:48:27.041 INFO:tasks.workunit.client.1.smithi193.stderr:Updating files: 84% (10389/12271) Updating files: 85% (10431/12271) Updating files: 86% (10554/12271) Updating files: 87% (10676/12271) Updating files: 88% (10799/12271) Updating files: 89% (10922/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-04-25T15:48:27.356 INFO:tasks.workunit.client.1.smithi193.stderr:Note: switching to '994e2f20102a20fa9a27966c14b625cc01884df8'. 2024-04-25T15:48:27.356 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr: git switch -c 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr:Or undo this operation with: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr: git switch - 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr: 2024-04-25T15:48:27.357 INFO:tasks.workunit.client.1.smithi193.stderr:HEAD is now at 994e2f20102 Merge PR #56810 into wip-pdonnell-testing-20240425.015853-debug 2024-04-25T15:48:27.363 DEBUG:teuthology.orchestra.run.smithi193:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-25T15:48:27.422 INFO:tasks.workunit.client.1.smithi193.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T15:48:27.424 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-25T15:48:27.425 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T15:48:27.849 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:27 smithi193 ceph-mon[33747]: pgmap v236: 65 pgs: 65 active+clean; 5.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 388 KiB/s wr, 8 op/s 2024-04-25T15:48:27.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:27 smithi110 ceph-mon[26344]: pgmap v236: 65 pgs: 65 active+clean; 5.1 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 388 KiB/s wr, 8 op/s 2024-04-25T15:48:28.109 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T15:48:28.170 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T15:48:28.212 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-25T15:48:28.214 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-25T15:48:28.214 INFO:tasks.workunit.client.1.smithi193.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T15:48:28.253 INFO:tasks.workunit.client.1.smithi193.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-25T15:48:28.256 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T15:48:28.256 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-25T15:48:28.312 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-25T15:48:28.313 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-25T15:48:28.313 DEBUG:teuthology.orchestra.run.smithi193:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=994e2f20102a20fa9a27966c14b625cc01884df8 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-25T15:48:28.376 INFO:tasks.workunit.client.1.smithi193.stderr:+ mkdir -p fsstress 2024-04-25T15:48:28.379 INFO:tasks.workunit.client.1.smithi193.stderr:+ pushd fsstress 2024-04-25T15:48:28.380 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-25T15:48:28.380 INFO:tasks.workunit.client.1.smithi193.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-25T15:48:28.998 INFO:tasks.workunit.client.1.smithi193.stderr:+ tar xzf ltp-full.tgz 2024-04-25T15:48:29.006 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:28 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:29.006 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:28 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:29.006 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:28 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:29.006 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:28 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:28 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:28 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:28 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:28 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:29.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:29 smithi193 ceph-mon[33747]: pgmap v237: 65 pgs: 65 active+clean; 9.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 792 KiB/s wr, 21 op/s 2024-04-25T15:48:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:29 smithi110 ceph-mon[26344]: pgmap v237: 65 pgs: 65 active+clean; 9.8 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 792 KiB/s wr, 21 op/s 2024-04-25T15:48:32.066 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:31 smithi193 ceph-mon[33747]: pgmap v238: 65 pgs: 65 active+clean; 17 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 43 op/s 2024-04-25T15:48:32.112 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:31 smithi110 ceph-mon[26344]: pgmap v238: 65 pgs: 65 active+clean; 17 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1.4 MiB/s wr, 43 op/s 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr fail", "who": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: Activating manager daemon smithi193.mgoteh 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: osdmap e46: 6 total, 6 up, 6 in 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "mgr fail", "who": "smithi110.lvgbkf"}]': finished 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: mgrmap e20: smithi193.mgoteh(active, starting, since 0.00769978s) 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: Active manager daemon smithi193.mgoteh restarted 2024-04-25T15:48:33.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: Activating manager daemon smithi193.mgoteh 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: osdmap e47: 6 total, 6 up, 6 in 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: mgrmap e21: smithi193.mgoteh(active, starting, since 0.0062633s) 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mgoteh", "id": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:48:33.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: Manager daemon smithi193.mgoteh is now available 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mgoteh/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mgoteh/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mgoteh/trash_purge_schedule"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mgoteh/trash_purge_schedule"}]: dispatch 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:33.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:33 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:33.510 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:33.510 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:33.510 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:48:33.510 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr fail", "who": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: Activating manager daemon smithi193.mgoteh 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: osdmap e46: 6 total, 6 up, 6 in 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.14210 172.21.15.110:0/3439568488' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "mgr fail", "who": "smithi110.lvgbkf"}]': finished 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: mgrmap e20: smithi193.mgoteh(active, starting, since 0.00769978s) 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: Active manager daemon smithi193.mgoteh restarted 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: Activating manager daemon smithi193.mgoteh 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: osdmap e47: 6 total, 6 up, 6 in 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: mgrmap e21: smithi193.mgoteh(active, starting, since 0.0062633s) 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:48:33.511 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mgoteh", "id": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:48:33.512 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: Manager daemon smithi193.mgoteh is now available 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mgoteh/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mgoteh/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mgoteh/trash_purge_schedule"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi193.mgoteh/trash_purge_schedule"}]: dispatch 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:33.513 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:33 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:34.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:34 smithi193 ceph-mon[33747]: Deploying cephadm binary to smithi193 2024-04-25T15:48:34.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:34 smithi193 ceph-mon[33747]: Deploying cephadm binary to smithi110 2024-04-25T15:48:34.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:34 smithi193 ceph-mon[33747]: [25/Apr/2024:15:48:33] ENGINE Bus STARTING 2024-04-25T15:48:34.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:34 smithi193 ceph-mon[33747]: [25/Apr/2024:15:48:33] ENGINE Serving on http://172.21.15.193:8765 2024-04-25T15:48:34.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:34 smithi193 ceph-mon[33747]: mgrmap e22: smithi193.mgoteh(active, since 1.01114s) 2024-04-25T15:48:34.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:34 smithi193 ceph-mon[33747]: pgmap v3: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:34.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:34 smithi110 ceph-mon[26344]: Deploying cephadm binary to smithi193 2024-04-25T15:48:34.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:34 smithi110 ceph-mon[26344]: Deploying cephadm binary to smithi110 2024-04-25T15:48:34.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:34 smithi110 ceph-mon[26344]: [25/Apr/2024:15:48:33] ENGINE Bus STARTING 2024-04-25T15:48:34.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:34 smithi110 ceph-mon[26344]: [25/Apr/2024:15:48:33] ENGINE Serving on http://172.21.15.193:8765 2024-04-25T15:48:34.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:34 smithi110 ceph-mon[26344]: mgrmap e22: smithi193.mgoteh(active, since 1.01114s) 2024-04-25T15:48:34.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:34 smithi110 ceph-mon[26344]: pgmap v3: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:35 smithi110 ceph-mon[26344]: [25/Apr/2024:15:48:33] ENGINE Serving on https://172.21.15.193:7150 2024-04-25T15:48:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:35 smithi110 ceph-mon[26344]: [25/Apr/2024:15:48:33] ENGINE Bus STARTED 2024-04-25T15:48:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:35 smithi110 ceph-mon[26344]: pgmap v4: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:35.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:35 smithi110 ceph-mon[26344]: mgrmap e23: smithi193.mgoteh(active, since 2s) 2024-04-25T15:48:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:35 smithi193 ceph-mon[33747]: [25/Apr/2024:15:48:33] ENGINE Serving on https://172.21.15.193:7150 2024-04-25T15:48:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:35 smithi193 ceph-mon[33747]: [25/Apr/2024:15:48:33] ENGINE Bus STARTED 2024-04-25T15:48:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:35 smithi193 ceph-mon[33747]: pgmap v4: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:35.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:35 smithi193 ceph-mon[33747]: mgrmap e23: smithi193.mgoteh(active, since 2s) 2024-04-25T15:48:37.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:37 smithi193 ceph-mon[33747]: pgmap v5: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:37.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:37 smithi193 ceph-mon[33747]: mgrmap e24: smithi193.mgoteh(active, since 4s) 2024-04-25T15:48:37.764 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:37 smithi110 ceph-mon[26344]: pgmap v5: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:37.764 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:37 smithi110 ceph-mon[26344]: mgrmap e24: smithi193.mgoteh(active, since 4s) 2024-04-25T15:48:39.602 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:39 smithi110 ceph-mon[26344]: pgmap v6: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:39.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:39 smithi193 ceph-mon[33747]: pgmap v6: 65 pgs: 65 active+clean; 44 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:48:41.528 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:41 smithi110 ceph-mon[26344]: pgmap v7: 65 pgs: 65 active+clean; 65 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-04-25T15:48:41.529 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:41 smithi110 ceph-mon[26344]: Standby manager daemon smithi110.lvgbkf started 2024-04-25T15:48:41.529 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:41 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.110:0/2322915925' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.lvgbkf/crt"}]: dispatch 2024-04-25T15:48:41.529 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:41 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.110:0/2322915925' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:48:41.529 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:41 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.110:0/2322915925' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.lvgbkf/key"}]: dispatch 2024-04-25T15:48:41.529 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:41 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.110:0/2322915925' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:48:41.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:41 smithi193 ceph-mon[33747]: pgmap v7: 65 pgs: 65 active+clean; 65 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 3.1 MiB/s wr, 159 op/s 2024-04-25T15:48:41.816 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:41 smithi193 ceph-mon[33747]: Standby manager daemon smithi110.lvgbkf started 2024-04-25T15:48:41.816 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:41 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.110:0/2322915925' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.lvgbkf/crt"}]: dispatch 2024-04-25T15:48:41.816 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:41 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.110:0/2322915925' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:48:41.816 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:41 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.110:0/2322915925' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.lvgbkf/key"}]: dispatch 2024-04-25T15:48:41.816 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:41 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.110:0/2322915925' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:48:42.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:42 smithi193 ceph-mon[33747]: mgrmap e25: smithi193.mgoteh(active, since 9s), standbys: smithi110.lvgbkf 2024-04-25T15:48:42.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:42 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:48:42.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:42 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:42.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:42 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:42.837 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:42 smithi110 ceph-mon[26344]: mgrmap e25: smithi193.mgoteh(active, since 9s), standbys: smithi110.lvgbkf 2024-04-25T15:48:42.837 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:42 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:48:42.837 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:42 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:42.837 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:42 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:43 smithi193 ceph-mon[33747]: pgmap v8: 65 pgs: 65 active+clean; 65 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 2.4 MiB/s wr, 124 op/s 2024-04-25T15:48:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:43 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:43.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:43 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:43 smithi110 ceph-mon[26344]: pgmap v8: 65 pgs: 65 active+clean; 65 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 2.4 MiB/s wr, 124 op/s 2024-04-25T15:48:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:43 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:43 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:44 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:44 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:44 smithi193 ceph-mon[33747]: pgmap v9: 65 pgs: 65 active+clean; 87 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 165 op/s 2024-04-25T15:48:45.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:44 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:45.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:44 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:45.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:44 smithi110 ceph-mon[26344]: pgmap v9: 65 pgs: 65 active+clean; 87 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 4.0 MiB/s wr, 165 op/s 2024-04-25T15:48:46.366 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: Adjusting osd_memory_target on smithi110 to 1557M 2024-04-25T15:48:46.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:46 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:46.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:46.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:46.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:46.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: Adjusting osd_memory_target on smithi110 to 1557M 2024-04-25T15:48:46.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:46 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:47.362 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:47 smithi110 ceph-mon[26344]: pgmap v10: 65 pgs: 65 active+clean; 87 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 151 op/s 2024-04-25T15:48:47.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:47 smithi193 ceph-mon[33747]: pgmap v10: 65 pgs: 65 active+clean; 87 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 151 op/s 2024-04-25T15:48:48.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:48 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:48:48.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:48 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:48:49.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:49 smithi110 ceph-mon[26344]: pgmap v11: 65 pgs: 65 active+clean; 87 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 151 op/s 2024-04-25T15:48:49.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:49 smithi193 ceph-mon[33747]: pgmap v11: 65 pgs: 65 active+clean; 87 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 3.7 MiB/s wr, 151 op/s 2024-04-25T15:48:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:51 smithi193 ceph-mon[33747]: pgmap v12: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 235 op/s 2024-04-25T15:48:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:51 smithi110 ceph-mon[26344]: pgmap v12: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 5.4 MiB/s wr, 235 op/s 2024-04-25T15:48:52.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.013+0000 7fdf8432b700 1 -- 172.21.15.110:0/2846863575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c0fefd0 msgr2=0x7fdf7c0ff3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.013+0000 7fdf8432b700 1 --2- 172.21.15.110:0/2846863575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c0fefd0 0x7fdf7c0ff3e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdf78004940 tx=0x7fdf78009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:52.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.014+0000 7fdf8432b700 1 -- 172.21.15.110:0/2846863575 shutdown_connections 2024-04-25T15:48:52.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.014+0000 7fdf8432b700 1 --2- 172.21.15.110:0/2846863575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c100280 0x7fdf7c1006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.014+0000 7fdf8432b700 1 --2- 172.21.15.110:0/2846863575 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c0fefd0 0x7fdf7c0ff3e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.014+0000 7fdf8432b700 1 -- 172.21.15.110:0/2846863575 >> 172.21.15.110:0/2846863575 conn(0x7fdf7c0faab0 msgr2=0x7fdf7c0fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:52.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.014+0000 7fdf8432b700 1 -- 172.21.15.110:0/2846863575 shutdown_connections 2024-04-25T15:48:52.015 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.014+0000 7fdf8432b700 1 -- 172.21.15.110:0/2846863575 wait complete. 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.014+0000 7fdf8432b700 1 Processor -- start 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf8432b700 1 -- start start 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf8432b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0fefd0 0x7fdf7c10fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf8432b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c100280 0x7fdf7c1100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf8432b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c1106d0 con 0x7fdf7c100280 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf8432b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7c110f20 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf818c6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c100280 0x7fdf7c1100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf820c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0fefd0 0x7fdf7c10fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:52.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf820c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0fefd0 0x7fdf7c10fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:34680/0 (socket says 172.21.15.110:34680) 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf818c6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c100280 0x7fdf7c1100d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54150/0 (socket says 172.21.15.110:54150) 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf820c7700 1 -- 172.21.15.110:0/3315242531 learned_addr learned my addr 172.21.15.110:0/3315242531 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf820c7700 1 -- 172.21.15.110:0/3315242531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c100280 msgr2=0x7fdf7c1100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf820c7700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c100280 0x7fdf7c1100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf820c7700 1 -- 172.21.15.110:0/3315242531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf78005040 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.015+0000 7fdf818c6700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c100280 0x7fdf7c1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.016+0000 7fdf820c7700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0fefd0 0x7fdf7c10fb90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf78013870 tx=0x7fdf78009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.016+0000 7fdf72ffd700 1 -- 172.21.15.110:0/3315242531 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf78021070 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.016+0000 7fdf72ffd700 1 -- 172.21.15.110:0/3315242531 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf78009430 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.016+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf7c1111a0 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.016+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7c1116e0 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.018 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.016+0000 7fdf72ffd700 1 -- 172.21.15.110:0/3315242531 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf7800e070 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.017+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf600052e0 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.017+0000 7fdf72ffd700 1 -- 172.21.15.110:0/3315242531 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7fdf78013af0 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.018+0000 7fdf72ffd700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fdf6806fc50 0x7fdf68072100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.018+0000 7fdf72ffd700 1 -- 172.21.15.110:0/3315242531 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fdf78090870 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.018+0000 7fdf818c6700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fdf6806fc50 0x7fdf68072100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:52.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.019+0000 7fdf818c6700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fdf6806fc50 0x7fdf68072100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009b80 tx=0x7fdf6c00b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:52.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.023+0000 7fdf72ffd700 1 -- 172.21.15.110:0/3315242531 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fdf78059cd0 con 0x7fdf7c0fefd0 2024-04-25T15:48:52.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.350+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf60000bf0 con 0x7fdf6806fc50 2024-04-25T15:48:52.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.352+0000 7fdf72ffd700 1 -- 172.21.15.110:0/3315242531 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fdf60000bf0 con 0x7fdf6806fc50 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fdf6806fc50 msgr2=0x7fdf68072100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fdf6806fc50 0x7fdf68072100 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c009b80 tx=0x7fdf6c00b040 comp rx=0 tx=0).stop 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0fefd0 msgr2=0x7fdf7c10fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0fefd0 0x7fdf7c10fb90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf78013870 tx=0x7fdf78009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 shutdown_connections 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fdf6806fc50 0x7fdf68072100 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf7c0fefd0 0x7fdf7c10fb90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 --2- 172.21.15.110:0/3315242531 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf7c100280 0x7fdf7c1100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 >> 172.21.15.110:0/3315242531 conn(0x7fdf7c0faab0 msgr2=0x7fdf7c0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:52.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 shutdown_connections 2024-04-25T15:48:52.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.355+0000 7fdf8432b700 1 -- 172.21.15.110:0/3315242531 wait complete. 2024-04-25T15:48:52.372 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:48:52.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.487+0000 7f1485871700 1 -- 172.21.15.110:0/2037314370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 msgr2=0x7f14800fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.487+0000 7f1485871700 1 --2- 172.21.15.110:0/2037314370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 0x7f14800fc0c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1470004940 tx=0x7f1470009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:52.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.487+0000 7f1485871700 1 -- 172.21.15.110:0/2037314370 shutdown_connections 2024-04-25T15:48:52.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.487+0000 7f1485871700 1 --2- 172.21.15.110:0/2037314370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14800fcf60 0x7f14800fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.487+0000 7f1485871700 1 --2- 172.21.15.110:0/2037314370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 0x7f14800fc0c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.487+0000 7f1485871700 1 -- 172.21.15.110:0/2037314370 >> 172.21.15.110:0/2037314370 conn(0x7f14800f7430 msgr2=0x7f14800f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:52.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.488+0000 7f1485871700 1 -- 172.21.15.110:0/2037314370 shutdown_connections 2024-04-25T15:48:52.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.488+0000 7f1485871700 1 -- 172.21.15.110:0/2037314370 wait complete. 2024-04-25T15:48:52.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.489+0000 7f1485871700 1 Processor -- start 2024-04-25T15:48:52.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.489+0000 7f1485871700 1 -- start start 2024-04-25T15:48:52.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.489+0000 7f1485871700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 0x7f148010fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.489+0000 7f1485871700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14800fcf60 0x7f14801100b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.489+0000 7f1485871700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14801106b0 con 0x7f14800fbcb0 2024-04-25T15:48:52.491 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.489+0000 7f1485871700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1480110f00 con 0x7f14800fcf60 2024-04-25T15:48:52.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.490+0000 7f147effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 0x7f148010fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:52.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.490+0000 7f147effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 0x7f148010fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54168/0 (socket says 172.21.15.110:54168) 2024-04-25T15:48:52.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.490+0000 7f147effd700 1 -- 172.21.15.110:0/2530582988 learned_addr learned my addr 172.21.15.110:0/2530582988 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:52.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.490+0000 7f147effd700 1 -- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14800fcf60 msgr2=0x7f14801100b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T15:48:52.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.490+0000 7f147effd700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14800fcf60 0x7f14801100b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.490+0000 7f147effd700 1 -- 172.21.15.110:0/2530582988 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1470005040 con 0x7f14800fbcb0 2024-04-25T15:48:52.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.491+0000 7f147effd700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 0x7f148010fb70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1470013870 tx=0x7f1470009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:52.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.492+0000 7f14677fe700 1 -- 172.21.15.110:0/2530582988 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1470021070 con 0x7f14800fbcb0 2024-04-25T15:48:52.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.492+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1480111100 con 0x7f14800fbcb0 2024-04-25T15:48:52.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.492+0000 7f14677fe700 1 -- 172.21.15.110:0/2530582988 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f147001c400 con 0x7f14800fbcb0 2024-04-25T15:48:52.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.492+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1480111640 con 0x7f14800fbcb0 2024-04-25T15:48:52.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.492+0000 7f14677fe700 1 -- 172.21.15.110:0/2530582988 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f147000e070 con 0x7f14800fbcb0 2024-04-25T15:48:52.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.494+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0052e0 con 0x7f14800fbcb0 2024-04-25T15:48:52.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.495+0000 7f14677fe700 1 -- 172.21.15.110:0/2530582988 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f147001ca20 con 0x7f14800fbcb0 2024-04-25T15:48:52.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.495+0000 7f14677fe700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f146806fd80 0x7f1468072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.496+0000 7f14677fe700 1 -- 172.21.15.110:0/2530582988 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f1470091080 con 0x7f14800fbcb0 2024-04-25T15:48:52.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.496+0000 7f147e7fc700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f146806fd80 0x7f1468072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:52.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.497+0000 7f147e7fc700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f146806fd80 0x7f1468072230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1474009fd0 tx=0x7f147400b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:52.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.501+0000 7f14677fe700 1 -- 172.21.15.110:0/2530582988 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f147005a580 con 0x7f14800fbcb0 2024-04-25T15:48:52.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.851+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f146c000bf0 con 0x7f146806fd80 2024-04-25T15:48:52.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.852+0000 7f14677fe700 1 -- 172.21.15.110:0/2530582988 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f146c000bf0 con 0x7f146806fd80 2024-04-25T15:48:52.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f146806fd80 msgr2=0x7f1468072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f146806fd80 0x7f1468072230 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1474009fd0 tx=0x7f147400b040 comp rx=0 tx=0).stop 2024-04-25T15:48:52.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 msgr2=0x7f148010fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 0x7f148010fb70 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1470013870 tx=0x7f1470009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:52.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 shutdown_connections 2024-04-25T15:48:52.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f146806fd80 0x7f1468072230 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14800fbcb0 0x7f148010fb70 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 --2- 172.21.15.110:0/2530582988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14800fcf60 0x7f14801100b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.855+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 >> 172.21.15.110:0/2530582988 conn(0x7f14800f7430 msgr2=0x7f1480065a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:52.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.856+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 shutdown_connections 2024-04-25T15:48:52.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.856+0000 7f1485871700 1 -- 172.21.15.110:0/2530582988 wait complete. 2024-04-25T15:48:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.971+0000 7f92cd638700 1 -- 172.21.15.110:0/2743920867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c80fff10 msgr2=0x7f92c8102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.971+0000 7f92cd638700 1 --2- 172.21.15.110:0/2743920867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c80fff10 0x7f92c8102330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f92bc004940 tx=0x7f92bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.971+0000 7f92cd638700 1 -- 172.21.15.110:0/2743920867 shutdown_connections 2024-04-25T15:48:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.971+0000 7f92cd638700 1 --2- 172.21.15.110:0/2743920867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c80fff10 0x7f92c8102330 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.971+0000 7f92cd638700 1 --2- 172.21.15.110:0/2743920867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c8066b50 0x7f92c80ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.971+0000 7f92cd638700 1 -- 172.21.15.110:0/2743920867 >> 172.21.15.110:0/2743920867 conn(0x7f92c80f7440 msgr2=0x7f92c80f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.971+0000 7f92cd638700 1 -- 172.21.15.110:0/2743920867 shutdown_connections 2024-04-25T15:48:52.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.971+0000 7f92cd638700 1 -- 172.21.15.110:0/2743920867 wait complete. 2024-04-25T15:48:52.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92cd638700 1 Processor -- start 2024-04-25T15:48:52.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92cd638700 1 -- start start 2024-04-25T15:48:52.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92cd638700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c8066b50 0x7f92c810fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92cd638700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c80fff10 0x7f92c8110100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92cd638700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c8110700 con 0x7f92c8066b50 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92cd638700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c8110f50 con 0x7f92c80fff10 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92c6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c8066b50 0x7f92c810fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92c6ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c8066b50 0x7f92c810fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54188/0 (socket says 172.21.15.110:54188) 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92c6ffd700 1 -- 172.21.15.110:0/973089456 learned_addr learned my addr 172.21.15.110:0/973089456 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.972+0000 7f92c67fc700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c80fff10 0x7f92c8110100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92c6ffd700 1 -- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c80fff10 msgr2=0x7f92c8110100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92c6ffd700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c80fff10 0x7f92c8110100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92c6ffd700 1 -- 172.21.15.110:0/973089456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc005040 con 0x7f92c8066b50 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92c6ffd700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c8066b50 0x7f92c810fbc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f92b800eb40 tx=0x7f92b800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92af7fe700 1 -- 172.21.15.110:0/973089456 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92b8007cc0 con 0x7f92c8066b50 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c81111d0 con 0x7f92c8066b50 2024-04-25T15:48:52.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92af7fe700 1 -- 172.21.15.110:0/973089456 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92b800f040 con 0x7f92c8066b50 2024-04-25T15:48:52.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92af7fe700 1 -- 172.21.15.110:0/973089456 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92b800c9d0 con 0x7f92c8066b50 2024-04-25T15:48:52.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.973+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c8111710 con 0x7f92c8066b50 2024-04-25T15:48:52.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.974+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c805f5c0 con 0x7f92c8066b50 2024-04-25T15:48:52.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.975+0000 7f92af7fe700 1 -- 172.21.15.110:0/973089456 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f92b80090d0 con 0x7f92c8066b50 2024-04-25T15:48:52.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.975+0000 7f92af7fe700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f92b006fcb0 0x7f92b0072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:52.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.975+0000 7f92af7fe700 1 -- 172.21.15.110:0/973089456 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f92b808f140 con 0x7f92c8066b50 2024-04-25T15:48:52.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.975+0000 7f92c67fc700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f92b006fcb0 0x7f92b0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:52.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.976+0000 7f92c67fc700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f92b006fcb0 0x7f92b0072160 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009b30 tx=0x7f92bc009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:52.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:52.980+0000 7f92af7fe700 1 -- 172.21.15.110:0/973089456 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f92b8050930 con 0x7f92c8066b50 2024-04-25T15:48:53.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.291+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92c81119d0 con 0x7f92b006fcb0 2024-04-25T15:48:53.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.304+0000 7f92af7fe700 1 -- 172.21.15.110:0/973089456 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f92c81119d0 con 0x7f92b006fcb0 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (6m) 9s ago 7m 24.3M - 0.25.0 c8568f914cd2 b995d416435c 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (8m) 9s ago 8m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (7m) 11s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (6m) 9s ago 7m 85.9M - 9.4.7 954c08fa6188 6cf7f080ae79 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (2m) 9s ago 2m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (2m) 9s ago 2m 86.6M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (2m) 11s ago 2m 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (2m) 11s ago 2m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:9283 running (9m) 9s ago 9m 363M - 17.2.7-902-gc61472c8 0de407c1b3af c623b709ef43 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (29s) 11s ago 6m 558M - 19.0.0-3291-g994e2f20 d53c96779a29 82852af5489a 2024-04-25T15:48:53.305 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (9m) 9s ago 9m 49.0M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (6m) 11s ago 6m 40.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (7m) 9s ago 7m 15.5M - 1.5.0 0da6a335fe13 5f8f351cfd11 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 11s ago 6m 15.3M - 1.5.0 0da6a335fe13 13afa4d53352 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (6m) 9s ago 6m 61.7M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (5m) 9s ago 5m 66.0M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (5m) 9s ago 5m 56.3M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (4m) 11s ago 4m 76.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (4m) 11s ago 4m 64.8M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (3m) 11s ago 3m 69.0M 5312M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:48:53.306 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (6m) 9s ago 7m 54.9M - 2.43.0 a07b618ecd1d ce7fd55a0368 2024-04-25T15:48:53.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f92b006fcb0 msgr2=0x7f92b0072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:53.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f92b006fcb0 0x7f92b0072160 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009b30 tx=0x7f92bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:53.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c8066b50 msgr2=0x7f92c810fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:53.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c8066b50 0x7f92c810fbc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f92b800eb40 tx=0x7f92b800b040 comp rx=0 tx=0).stop 2024-04-25T15:48:53.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 shutdown_connections 2024-04-25T15:48:53.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f92b006fcb0 0x7f92b0072160 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92c8066b50 0x7f92c810fbc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 --2- 172.21.15.110:0/973089456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92c80fff10 0x7f92c8110100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.307+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 >> 172.21.15.110:0/973089456 conn(0x7f92c80f7440 msgr2=0x7f92c81002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:53.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.308+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 shutdown_connections 2024-04-25T15:48:53.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.308+0000 7f92cd638700 1 -- 172.21.15.110:0/973089456 wait complete. 2024-04-25T15:48:53.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.452+0000 7f351d642700 1 -- 172.21.15.110:0/542878012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 msgr2=0x7f35180fcb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:53.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.452+0000 7f351d642700 1 --2- 172.21.15.110:0/542878012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 0x7f35180fcb40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f350c004940 tx=0x7f350c009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:53.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.452+0000 7f351d642700 1 -- 172.21.15.110:0/542878012 shutdown_connections 2024-04-25T15:48:53.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.452+0000 7f351d642700 1 --2- 172.21.15.110:0/542878012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 0x7f35180fcb40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.452+0000 7f351d642700 1 --2- 172.21.15.110:0/542878012 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35180fb420 0x7f35180fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.452+0000 7f351d642700 1 -- 172.21.15.110:0/542878012 >> 172.21.15.110:0/542878012 conn(0x7f35180f6b80 msgr2=0x7f35180f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:53.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.452+0000 7f351d642700 1 -- 172.21.15.110:0/542878012 shutdown_connections 2024-04-25T15:48:53.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.452+0000 7f351d642700 1 -- 172.21.15.110:0/542878012 wait complete. 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.453+0000 7f351d642700 1 Processor -- start 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.453+0000 7f351d642700 1 -- start start 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.453+0000 7f351d642700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35180fb420 0x7f351810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.453+0000 7f351d642700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 0x7f351810def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.453+0000 7f351d642700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351810e4f0 con 0x7f35180fc6d0 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.453+0000 7f351d642700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351810ed40 con 0x7f35180fb420 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f35167fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 0x7f351810def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f3516ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35180fb420 0x7f351810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f35167fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 0x7f351810def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54196/0 (socket says 172.21.15.110:54196) 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f3516ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35180fb420 0x7f351810d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:34742/0 (socket says 172.21.15.110:34742) 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f35167fc700 1 -- 172.21.15.110:0/2892638841 learned_addr learned my addr 172.21.15.110:0/2892638841 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:53.455 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f3516ffd700 1 -- 172.21.15.110:0/2892638841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 msgr2=0x7f351810def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:53.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f3516ffd700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 0x7f351810def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f3516ffd700 1 -- 172.21.15.110:0/2892638841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c005040 con 0x7f35180fb420 2024-04-25T15:48:53.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.454+0000 7f3516ffd700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35180fb420 0x7f351810d9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f350800afd0 tx=0x7f350800c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:53.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.455+0000 7f34ff7fe700 1 -- 172.21.15.110:0/2892638841 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3508007ab0 con 0x7f35180fb420 2024-04-25T15:48:53.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.455+0000 7f34ff7fe700 1 -- 172.21.15.110:0/2892638841 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3508010040 con 0x7f35180fb420 2024-04-25T15:48:53.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.455+0000 7f34ff7fe700 1 -- 172.21.15.110:0/2892638841 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f350800e900 con 0x7f35180fb420 2024-04-25T15:48:53.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.455+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351810efc0 con 0x7f35180fb420 2024-04-25T15:48:53.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.455+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351810f440 con 0x7f35180fb420 2024-04-25T15:48:53.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.456+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351805f5c0 con 0x7f35180fb420 2024-04-25T15:48:53.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.456+0000 7f34ff7fe700 1 -- 172.21.15.110:0/2892638841 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f350800ea60 con 0x7f35180fb420 2024-04-25T15:48:53.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.457+0000 7f34ff7fe700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f350006fcb0 0x7f3500072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:53.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.457+0000 7f34ff7fe700 1 -- 172.21.15.110:0/2892638841 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f350808efd0 con 0x7f35180fb420 2024-04-25T15:48:53.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.457+0000 7f35167fc700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f350006fcb0 0x7f3500072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:53.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.458+0000 7f35167fc700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f350006fcb0 0x7f3500072160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f350c009b30 tx=0x7f350c01c040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:53.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.462+0000 7f34ff7fe700 1 -- 172.21.15.110:0/2892638841 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f3508059260 con 0x7f35180fb420 2024-04-25T15:48:53.730 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:53 smithi110 ceph-mon[26344]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:53.730 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:53 smithi110 ceph-mon[26344]: pgmap v13: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 142 op/s 2024-04-25T15:48:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:53 smithi193 ceph-mon[33747]: from='client.24409 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:53 smithi193 ceph-mon[33747]: pgmap v13: 65 pgs: 65 active+clean; 108 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.6 MiB/s wr, 142 op/s 2024-04-25T15:48:53.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.936+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f351810f870 con 0x7f35180fb420 2024-04-25T15:48:53.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.937+0000 7f34ff7fe700 1 -- 172.21.15.110:0/2892638841 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f3508058c00 con 0x7f35180fb420 2024-04-25T15:48:53.938 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:48:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:48:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:48:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:48:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:48:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T15:48:53.938 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:48:53.939 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:48:53.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f350006fcb0 msgr2=0x7f3500072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:53.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f350006fcb0 0x7f3500072160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f350c009b30 tx=0x7f350c01c040 comp rx=0 tx=0).stop 2024-04-25T15:48:53.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35180fb420 msgr2=0x7f351810d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:53.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35180fb420 0x7f351810d9b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f350800afd0 tx=0x7f350800c040 comp rx=0 tx=0).stop 2024-04-25T15:48:53.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 shutdown_connections 2024-04-25T15:48:53.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f350006fcb0 0x7f3500072160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35180fb420 0x7f351810d9b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 --2- 172.21.15.110:0/2892638841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35180fc6d0 0x7f351810def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 >> 172.21.15.110:0/2892638841 conn(0x7f35180f6b80 msgr2=0x7f35180f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.939+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 shutdown_connections 2024-04-25T15:48:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:53.940+0000 7f351d642700 1 -- 172.21.15.110:0/2892638841 wait complete. 2024-04-25T15:48:54.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.056+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3655241648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 msgr2=0x7fa8ac104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:54.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.056+0000 7fa8b1eab700 1 --2- 172.21.15.110:0/3655241648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 0x7fa8ac104740 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0004940 tx=0x7fa8a0009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:54.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.056+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3655241648 shutdown_connections 2024-04-25T15:48:54.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.056+0000 7fa8b1eab700 1 --2- 172.21.15.110:0/3655241648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 0x7fa8ac104740 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.056+0000 7fa8b1eab700 1 --2- 172.21.15.110:0/3655241648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8ac066240 0x7fa8ac101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.056+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3655241648 >> 172.21.15.110:0/3655241648 conn(0x7fa8ac0f7430 msgr2=0x7fa8ac0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:54.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.057+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3655241648 shutdown_connections 2024-04-25T15:48:54.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.057+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3655241648 wait complete. 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.057+0000 7fa8b1eab700 1 Processor -- start 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.057+0000 7fa8b1eab700 1 -- start start 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8b1eab700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8ac066240 0x7fa8ac10fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8b1eab700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 0x7fa8ac110270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8b1eab700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac110870 con 0x7fa8ac102360 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8b1eab700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac110fb0 con 0x7fa8ac066240 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8aaffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 0x7fa8ac110270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8ab7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8ac066240 0x7fa8ac10fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8aaffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 0x7fa8ac110270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54220/0 (socket says 172.21.15.110:54220) 2024-04-25T15:48:54.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8aaffd700 1 -- 172.21.15.110:0/3444804931 learned_addr learned my addr 172.21.15.110:0/3444804931 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:54.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8aaffd700 1 -- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8ac066240 msgr2=0x7fa8ac10fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:54.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8aaffd700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8ac066240 0x7fa8ac10fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8aaffd700 1 -- 172.21.15.110:0/3444804931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a0005040 con 0x7fa8ac102360 2024-04-25T15:48:54.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.058+0000 7fa8aaffd700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 0x7fa8ac110270 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000b3b0 tx=0x7fa8a0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:54.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.059+0000 7fa893fff700 1 -- 172.21.15.110:0/3444804931 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8a000e040 con 0x7fa8ac102360 2024-04-25T15:48:54.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.059+0000 7fa893fff700 1 -- 172.21.15.110:0/3444804931 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa8a0009af0 con 0x7fa8ac102360 2024-04-25T15:48:54.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.059+0000 7fa893fff700 1 -- 172.21.15.110:0/3444804931 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8a001c630 con 0x7fa8ac102360 2024-04-25T15:48:54.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.059+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8ac111230 con 0x7fa8ac102360 2024-04-25T15:48:54.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.059+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8ac1117a0 con 0x7fa8ac102360 2024-04-25T15:48:54.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.060+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8ac05f5c0 con 0x7fa8ac102360 2024-04-25T15:48:54.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.061+0000 7fa893fff700 1 -- 172.21.15.110:0/3444804931 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7fa8a000bb90 con 0x7fa8ac102360 2024-04-25T15:48:54.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.061+0000 7fa893fff700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fa89406fcb0 0x7fa894072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:54.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.061+0000 7fa893fff700 1 -- 172.21.15.110:0/3444804931 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fa8a0090b80 con 0x7fa8ac102360 2024-04-25T15:48:54.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.061+0000 7fa8ab7fe700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fa89406fcb0 0x7fa894072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:54.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.062+0000 7fa8ab7fe700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fa89406fcb0 0x7fa894072160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac065f80 tx=0x7fa89c009040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:54.067 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.067+0000 7fa893fff700 1 -- 172.21.15.110:0/3444804931 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fa8a0059e60 con 0x7fa8ac102360 2024-04-25T15:48:54.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.424+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa8ac107ab0 con 0x7fa8ac102360 2024-04-25T15:48:54.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.424+0000 7fa893fff700 1 -- 172.21.15.110:0/3444804931 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7fa8a0059800 con 0x7fa8ac102360 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:48:54.426 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:48:54.427 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:48:54.428 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:48:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fa89406fcb0 msgr2=0x7fa894072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fa89406fcb0 0x7fa894072160 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac065f80 tx=0x7fa89c009040 comp rx=0 tx=0).stop 2024-04-25T15:48:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 msgr2=0x7fa8ac110270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 0x7fa8ac110270 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa8a000b3b0 tx=0x7fa8a0009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 shutdown_connections 2024-04-25T15:48:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fa89406fcb0 0x7fa894072160 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8ac066240 0x7fa8ac10fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 --2- 172.21.15.110:0/3444804931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8ac102360 0x7fa8ac110270 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 >> 172.21.15.110:0/3444804931 conn(0x7fa8ac0f7430 msgr2=0x7fa8ac0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 shutdown_connections 2024-04-25T15:48:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.428+0000 7fa8b1eab700 1 -- 172.21.15.110:0/3444804931 wait complete. 2024-04-25T15:48:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:48:54.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/39773893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09c0fc270 msgr2=0x7fd09c0fc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:54.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 --2- 172.21.15.110:0/39773893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09c0fc270 0x7fd09c0fc680 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd08c004940 tx=0x7fd08c009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:54.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/39773893 shutdown_connections 2024-04-25T15:48:54.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 --2- 172.21.15.110:0/39773893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0fcbc0 0x7fd09c0fd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 --2- 172.21.15.110:0/39773893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09c0fc270 0x7fd09c0fc680 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/39773893 >> 172.21.15.110:0/39773893 conn(0x7fd09c0f6b80 msgr2=0x7fd09c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:54.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/39773893 shutdown_connections 2024-04-25T15:48:54.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/39773893 wait complete. 2024-04-25T15:48:54.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.570+0000 7fd0a1ae8700 1 Processor -- start 2024-04-25T15:48:54.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd0a1ae8700 1 -- start start 2024-04-25T15:48:54.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd0a1ae8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09c0fc270 0x7fd09c114040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:54.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd0a1ae8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0fcbc0 0x7fd09c114580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:54.573 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd0a1ae8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c114b80 con 0x7fd09c0fc270 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd0a1ae8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c1153d0 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd09affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0fcbc0 0x7fd09c114580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd09affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0fcbc0 0x7fd09c114580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:34774/0 (socket says 172.21.15.110:34774) 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd09affd700 1 -- 172.21.15.110:0/981806264 learned_addr learned my addr 172.21.15.110:0/981806264 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd09b7fe700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09c0fc270 0x7fd09c114040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd09affd700 1 -- 172.21.15.110:0/981806264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09c0fc270 msgr2=0x7fd09c114040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd09affd700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09c0fc270 0x7fd09c114040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.571+0000 7fd09affd700 1 -- 172.21.15.110:0/981806264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c005040 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.572+0000 7fd09affd700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0fcbc0 0x7fd09c114580 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd090002a50 tx=0x7fd0900046b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.572+0000 7fd083fff700 1 -- 172.21.15.110:0/981806264 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd090007cc0 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.572+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09c1156b0 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.572+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c115bf0 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.572+0000 7fd083fff700 1 -- 172.21.15.110:0/981806264 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd09000f040 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.574 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.572+0000 7fd083fff700 1 -- 172.21.15.110:0/981806264 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd09000da30 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.573+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09c1019c0 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.574+0000 7fd083fff700 1 -- 172.21.15.110:0/981806264 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7fd09000db90 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.574+0000 7fd083fff700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fd08406fcb0 0x7fd084072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:54.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.574+0000 7fd083fff700 1 -- 172.21.15.110:0/981806264 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fd09008f5a0 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.574+0000 7fd09b7fe700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fd08406fcb0 0x7fd084072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:54.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.575+0000 7fd09b7fe700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fd08406fcb0 0x7fd084072160 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00b3b0 tx=0x7fd08c009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:54.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.579+0000 7fd083fff700 1 -- 172.21.15.110:0/981806264 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fd090058880 con 0x7fd09c0fcbc0 2024-04-25T15:48:54.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:54 smithi193 ceph-mon[33747]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:54.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:54 smithi193 ceph-mon[33747]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:54.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:54 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2892638841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:48:54.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:54 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3444804931' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:48:54.787 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:54 smithi110 ceph-mon[26344]: from='client.14678 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:54.787 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:54 smithi110 ceph-mon[26344]: from='client.14682 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:54.787 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:54 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2892638841' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:48:54.787 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:54 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3444804931' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:48:54.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.935+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd09c115e60 con 0x7fd08406fcb0 2024-04-25T15:48:54.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.937+0000 7fd083fff700 1 -- 172.21.15.110:0/981806264 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7fd09c115e60 con 0x7fd08406fcb0 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "1/21 daemons upgraded", 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:48:54.938 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:48:54.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fd08406fcb0 msgr2=0x7fd084072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:54.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fd08406fcb0 0x7fd084072160 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00b3b0 tx=0x7fd08c009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:54.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0fcbc0 msgr2=0x7fd09c114580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:54.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0fcbc0 0x7fd09c114580 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd090002a50 tx=0x7fd0900046b0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 shutdown_connections 2024-04-25T15:48:54.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fd08406fcb0 0x7fd084072160 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd09c0fc270 0x7fd09c114040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 --2- 172.21.15.110:0/981806264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0fcbc0 0x7fd09c114580 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:54.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 >> 172.21.15.110:0/981806264 conn(0x7fd09c0f6b80 msgr2=0x7fd09c0fb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:54.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 shutdown_connections 2024-04-25T15:48:54.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:54.940+0000 7fd0a1ae8700 1 -- 172.21.15.110:0/981806264 wait complete. 2024-04-25T15:48:55.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.055+0000 7f3d116f6700 1 -- 172.21.15.110:0/3309222306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fbd40 msgr2=0x7f3d0c0fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:55.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.055+0000 7f3d116f6700 1 --2- 172.21.15.110:0/3309222306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fbd40 0x7f3d0c0fc150 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc004940 tx=0x7f3cfc009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:55.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.055+0000 7f3d116f6700 1 -- 172.21.15.110:0/3309222306 shutdown_connections 2024-04-25T15:48:55.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.055+0000 7f3d116f6700 1 --2- 172.21.15.110:0/3309222306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c0fcff0 0x7f3d0c0fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:55.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.055+0000 7f3d116f6700 1 --2- 172.21.15.110:0/3309222306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fbd40 0x7f3d0c0fc150 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:55.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.055+0000 7f3d116f6700 1 -- 172.21.15.110:0/3309222306 >> 172.21.15.110:0/3309222306 conn(0x7f3d0c0f74c0 msgr2=0x7f3d0c0f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:55.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.056+0000 7f3d116f6700 1 -- 172.21.15.110:0/3309222306 shutdown_connections 2024-04-25T15:48:55.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.056+0000 7f3d116f6700 1 -- 172.21.15.110:0/3309222306 wait complete. 2024-04-25T15:48:55.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.056+0000 7f3d116f6700 1 Processor -- start 2024-04-25T15:48:55.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d116f6700 1 -- start start 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d116f6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c0fbd40 0x7f3d0c10d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d116f6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fcff0 0x7f3d0c10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d116f6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c10e4c0 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d116f6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0c10ed10 con 0x7f3d0c0fbd40 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c0fbd40 0x7f3d0c10d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fcff0 0x7f3d0c10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0affd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c0fbd40 0x7f3d0c10d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:34792/0 (socket says 172.21.15.110:34792) 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0affd700 1 -- 172.21.15.110:0/1396799997 learned_addr learned my addr 172.21.15.110:0/1396799997 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0a7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fcff0 0x7f3d0c10dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54262/0 (socket says 172.21.15.110:54262) 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0a7fc700 1 -- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c0fbd40 msgr2=0x7f3d0c10d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0a7fc700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c0fbd40 0x7f3d0c10d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0a7fc700 1 -- 172.21.15.110:0/1396799997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cfc005040 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.057+0000 7f3d0affd700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c0fbd40 0x7f3d0c10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:48:55.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.058+0000 7f3d0a7fc700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fcff0 0x7f3d0c10dec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009fd0 tx=0x7f3d0000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:55.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.058+0000 7f3cf37fe700 1 -- 172.21.15.110:0/1396799997 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d00004e90 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.058+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d0c10ef90 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.058+0000 7f3cf37fe700 1 -- 172.21.15.110:0/1396799997 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d00004e90 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.058+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0c10f4d0 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.058+0000 7f3cf37fe700 1 -- 172.21.15.110:0/1396799997 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d0000c7d0 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.059+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d0c101b80 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.060+0000 7f3cf37fe700 1 -- 172.21.15.110:0/1396799997 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f3d00007cc0 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.061+0000 7f3cf37fe700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3cf406fd80 0x7f3cf4072230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:48:55.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.061+0000 7f3cf37fe700 1 -- 172.21.15.110:0/1396799997 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f3d0008f050 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.061+0000 7f3d0affd700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3cf406fd80 0x7f3cf4072230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:48:55.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.061+0000 7f3d0affd700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3cf406fd80 0x7f3cf4072230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00b0d0 tx=0x7f3cfc009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:48:55.066 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.066+0000 7f3cf37fe700 1 -- 172.21.15.110:0/1396799997 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f3d00050930 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.480+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3d0c104860 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.481+0000 7f3cf37fe700 1 -- 172.21.15.110:0/1396799997 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3d000583c0 con 0x7f3d0c0fcff0 2024-04-25T15:48:55.482 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:48:55.482 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:48:55.482 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:48:55.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3cf406fd80 msgr2=0x7f3cf4072230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:55.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3cf406fd80 0x7f3cf4072230 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc00b0d0 tx=0x7f3cfc009e30 comp rx=0 tx=0).stop 2024-04-25T15:48:55.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fcff0 msgr2=0x7f3d0c10dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:48:55.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fcff0 0x7f3d0c10dec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009fd0 tx=0x7f3d0000b040 comp rx=0 tx=0).stop 2024-04-25T15:48:55.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 shutdown_connections 2024-04-25T15:48:55.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3cf406fd80 0x7f3cf4072230 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:55.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3d0c0fbd40 0x7f3d0c10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:55.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 --2- 172.21.15.110:0/1396799997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d0c0fcff0 0x7f3d0c10dec0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:48:55.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 >> 172.21.15.110:0/1396799997 conn(0x7f3d0c0f74c0 msgr2=0x7f3d0c0f8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:48:55.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.483+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 shutdown_connections 2024-04-25T15:48:55.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:48:55.484+0000 7f3d116f6700 1 -- 172.21.15.110:0/1396799997 wait complete. 2024-04-25T15:48:55.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: pgmap v14: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.3 MiB/s wr, 250 op/s 2024-04-25T15:48:55.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:55.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:55.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: Adjusting osd_memory_target on smithi193 to 2581M 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:48:55.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:55 smithi193 ceph-mon[33747]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:48:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: pgmap v14: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 5.3 MiB/s wr, 250 op/s 2024-04-25T15:48:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:55.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T15:48:55.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: Adjusting osd_memory_target on smithi193 to 2581M 2024-04-25T15:48:55.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: from='client.24425 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:48:55.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:55 smithi110 ceph-mon[26344]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:48:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1396799997' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:48:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:48:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:48:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: pgmap v15: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 191 op/s 2024-04-25T15:48:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:56 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1396799997' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: pgmap v15: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 191 op/s 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:56 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:57 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:57 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:57 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:57 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.778 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:57 smithi193 ceph-mon[33747]: Reconfiguring node-exporter.smithi110 (dependencies changed)... 2024-04-25T15:48:57.779 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:57 smithi193 ceph-mon[33747]: Reconfiguring daemon node-exporter.smithi110 on smithi110 2024-04-25T15:48:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:57 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:57 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:57 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:57 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:48:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:57 smithi110 ceph-mon[26344]: Reconfiguring node-exporter.smithi110 (dependencies changed)... 2024-04-25T15:48:57.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:57 smithi110 ceph-mon[26344]: Reconfiguring daemon node-exporter.smithi110 on smithi110 2024-04-25T15:48:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:48:58 smithi110 ceph-mon[26344]: pgmap v16: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 191 op/s 2024-04-25T15:48:58.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:48:58 smithi193 ceph-mon[33747]: pgmap v16: 65 pgs: 65 active+clean; 128 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.5 MiB/s wr, 191 op/s 2024-04-25T15:49:01.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:01 smithi193 ceph-mon[33747]: pgmap v17: 65 pgs: 65 active+clean; 143 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 261 op/s 2024-04-25T15:49:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:01 smithi110 ceph-mon[26344]: pgmap v17: 65 pgs: 65 active+clean; 143 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.7 MiB/s wr, 261 op/s 2024-04-25T15:49:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:03 smithi193 ceph-mon[33747]: pgmap v18: 65 pgs: 65 active+clean; 143 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 177 op/s 2024-04-25T15:49:03.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:03 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:49:03.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:03 smithi110 ceph-mon[26344]: pgmap v18: 65 pgs: 65 active+clean; 143 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.0 MiB/s wr, 177 op/s 2024-04-25T15:49:03.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:03 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:49:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:05 smithi193 ceph-mon[33747]: pgmap v19: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.3 MiB/s wr, 305 op/s 2024-04-25T15:49:05.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:05 smithi110 ceph-mon[26344]: pgmap v19: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.3 MiB/s wr, 305 op/s 2024-04-25T15:49:06.702 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:06 smithi110 ceph-mon[26344]: pgmap v20: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2.6 MiB/s wr, 198 op/s 2024-04-25T15:49:06.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:06 smithi193 ceph-mon[33747]: pgmap v20: 65 pgs: 65 active+clean; 158 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 2.6 MiB/s wr, 198 op/s 2024-04-25T15:49:09.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:09 smithi193 ceph-mon[33747]: pgmap v21: 65 pgs: 65 active+clean; 175 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 279 op/s 2024-04-25T15:49:09.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:09 smithi110 ceph-mon[26344]: pgmap v21: 65 pgs: 65 active+clean; 175 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.0 MiB/s wr, 279 op/s 2024-04-25T15:49:10.814 INFO:tasks.workunit.client.0.smithi110.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-25T15:49:10.820 INFO:tasks.workunit.client.0.smithi110.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T15:49:10.820 INFO:tasks.workunit.client.0.smithi110.stderr:+ make 2024-04-25T15:49:11.690 INFO:tasks.workunit.client.0.smithi110.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-25T15:49:11.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:11 smithi193 ceph-mon[33747]: pgmap v22: 65 pgs: 65 active+clean; 180 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 330 op/s 2024-04-25T15:49:11.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:11 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:11.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:11 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:11.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:11 smithi110 ceph-mon[26344]: pgmap v22: 65 pgs: 65 active+clean; 180 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 4.5 MiB/s wr, 330 op/s 2024-04-25T15:49:11.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:11 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:11.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:11 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:12.094 INFO:tasks.workunit.client.0.smithi110.stderr:++ readlink -f fsstress 2024-04-25T15:49:12.096 INFO:tasks.workunit.client.0.smithi110.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-25T15:49:12.096 INFO:tasks.workunit.client.0.smithi110.stderr:+ popd 2024-04-25T15:49:12.097 INFO:tasks.workunit.client.0.smithi110.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-25T15:49:12.097 INFO:tasks.workunit.client.0.smithi110.stderr:+ popd 2024-04-25T15:49:12.097 INFO:tasks.workunit.client.0.smithi110.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-25T15:49:12.098 INFO:tasks.workunit.client.0.smithi110.stderr:++ mktemp -d -p . 2024-04-25T15:49:12.101 INFO:tasks.workunit.client.0.smithi110.stderr:+ T=./tmp.2GmEzzzUTX 2024-04-25T15:49:12.101 INFO:tasks.workunit.client.0.smithi110.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.2GmEzzzUTX -l 1 -n 1000 -p 10 -v 2024-04-25T15:49:12.105 INFO:tasks.workunit.client.0.smithi110.stdout:seed = 1714153403 2024-04-25T15:49:12.108 INFO:tasks.workunit.client.0.smithi110.stdout:0/0: rename - no filename 2024-04-25T15:49:12.108 INFO:tasks.workunit.client.0.smithi110.stdout:0/1: dread - no filename 2024-04-25T15:49:12.109 INFO:tasks.workunit.client.0.smithi110.stdout:0/2: rmdir - no directory 2024-04-25T15:49:12.109 INFO:tasks.workunit.client.0.smithi110.stdout:0/3: truncate - no filename 2024-04-25T15:49:12.110 INFO:tasks.workunit.client.0.smithi110.stdout:1/0: dread - no filename 2024-04-25T15:49:12.110 INFO:tasks.workunit.client.0.smithi110.stdout:0/4: symlink l0 0 2024-04-25T15:49:12.112 INFO:tasks.workunit.client.0.smithi110.stdout:2/0: write - no filename 2024-04-25T15:49:12.112 INFO:tasks.workunit.client.0.smithi110.stdout:2/1: chown . 44638 1 2024-04-25T15:49:12.112 INFO:tasks.workunit.client.0.smithi110.stdout:2/2: rename - no filename 2024-04-25T15:49:12.112 INFO:tasks.workunit.client.0.smithi110.stdout:0/5: mkdir d1 0 2024-04-25T15:49:12.114 INFO:tasks.workunit.client.0.smithi110.stdout:1/1: mkdir d0 0 2024-04-25T15:49:12.114 INFO:tasks.workunit.client.0.smithi110.stdout:1/2: dread - no filename 2024-04-25T15:49:12.114 INFO:tasks.workunit.client.0.smithi110.stdout:1/3: truncate - no filename 2024-04-25T15:49:12.114 INFO:tasks.workunit.client.0.smithi110.stdout:1/4: dwrite - no filename 2024-04-25T15:49:12.116 INFO:tasks.workunit.client.0.smithi110.stdout:2/3: symlink l0 0 2024-04-25T15:49:12.117 INFO:tasks.workunit.client.0.smithi110.stdout:0/6: mkdir d1/d2 0 2024-04-25T15:49:12.117 INFO:tasks.workunit.client.0.smithi110.stdout:0/7: dread - no filename 2024-04-25T15:49:12.117 INFO:tasks.workunit.client.0.smithi110.stdout:0/8: read - no filename 2024-04-25T15:49:12.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/0: dwrite - no filename 2024-04-25T15:49:12.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/1: chown . 967 1 2024-04-25T15:49:12.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/2: rename - no filename 2024-04-25T15:49:12.119 INFO:tasks.workunit.client.0.smithi110.stdout:4/3: dwrite - no filename 2024-04-25T15:49:12.119 INFO:tasks.workunit.client.0.smithi110.stdout:1/5: creat d0/f1 x:0 0 0 2024-04-25T15:49:12.120 INFO:tasks.workunit.client.0.smithi110.stdout:1/6: write d0/f1 [278982,120421] 0 2024-04-25T15:49:12.121 INFO:tasks.workunit.client.0.smithi110.stdout:2/4: creat f1 x:0 0 0 2024-04-25T15:49:12.123 INFO:tasks.workunit.client.0.smithi110.stdout:0/9: creat d1/f3 x:0 0 0 2024-04-25T15:49:12.124 INFO:tasks.workunit.client.0.smithi110.stdout:5/0: dwrite - no filename 2024-04-25T15:49:12.126 INFO:tasks.workunit.client.0.smithi110.stdout:3/0: getdents . 0 2024-04-25T15:49:12.126 INFO:tasks.workunit.client.0.smithi110.stdout:3/1: rename - no filename 2024-04-25T15:49:12.145 INFO:tasks.workunit.client.0.smithi110.stdout:1/7: dread d0/f1 [0,4194304] 0 2024-04-25T15:49:12.146 INFO:tasks.workunit.client.0.smithi110.stdout:4/4: symlink l0 0 2024-04-25T15:49:12.147 INFO:tasks.workunit.client.0.smithi110.stdout:0/10: creat d1/f4 x:0 0 0 2024-04-25T15:49:12.149 INFO:tasks.workunit.client.0.smithi110.stdout:4/5: chown l0 107 1 2024-04-25T15:49:12.149 INFO:tasks.workunit.client.0.smithi110.stdout:4/6: dwrite - no filename 2024-04-25T15:49:12.149 INFO:tasks.workunit.client.0.smithi110.stdout:4/7: dread - no filename 2024-04-25T15:49:12.149 INFO:tasks.workunit.client.0.smithi110.stdout:4/8: write - no filename 2024-04-25T15:49:12.153 INFO:tasks.workunit.client.0.smithi110.stdout:2/5: dwrite f1 [0,4194304] 0 2024-04-25T15:49:12.158 INFO:tasks.workunit.client.0.smithi110.stdout:0/11: dwrite d1/f4 [0,4194304] 0 2024-04-25T15:49:12.160 INFO:tasks.workunit.client.0.smithi110.stdout:2/6: dread f1 [0,4194304] 0 2024-04-25T15:49:12.171 INFO:tasks.workunit.client.0.smithi110.stdout:2/7: dwrite f1 [0,4194304] 0 2024-04-25T15:49:12.305 INFO:tasks.workunit.client.0.smithi110.stdout:5/1: creat f0 x:0 0 0 2024-04-25T15:49:12.307 INFO:tasks.workunit.client.0.smithi110.stdout:7/0: dwrite - no filename 2024-04-25T15:49:12.307 INFO:tasks.workunit.client.0.smithi110.stdout:7/1: dread - no filename 2024-04-25T15:49:12.307 INFO:tasks.workunit.client.0.smithi110.stdout:7/2: rename - no filename 2024-04-25T15:49:12.307 INFO:tasks.workunit.client.0.smithi110.stdout:7/3: dread - no filename 2024-04-25T15:49:12.307 INFO:tasks.workunit.client.0.smithi110.stdout:7/4: dwrite - no filename 2024-04-25T15:49:12.307 INFO:tasks.workunit.client.0.smithi110.stdout:7/5: dread - no filename 2024-04-25T15:49:12.307 INFO:tasks.workunit.client.0.smithi110.stdout:7/6: chown . 24 1 2024-04-25T15:49:12.310 INFO:tasks.workunit.client.0.smithi110.stdout:4/9: rename l0 to l1 0 2024-04-25T15:49:12.310 INFO:tasks.workunit.client.0.smithi110.stdout:4/10: write - no filename 2024-04-25T15:49:12.314 INFO:tasks.workunit.client.0.smithi110.stdout:3/2: mkdir d0 0 2024-04-25T15:49:12.314 INFO:tasks.workunit.client.0.smithi110.stdout:3/3: link - no file 2024-04-25T15:49:12.314 INFO:tasks.workunit.client.0.smithi110.stdout:3/4: readlink - no filename 2024-04-25T15:49:12.314 INFO:tasks.workunit.client.0.smithi110.stdout:5/2: creat f1 x:0 0 0 2024-04-25T15:49:12.315 INFO:tasks.workunit.client.0.smithi110.stdout:6/0: creat f0 x:0 0 0 2024-04-25T15:49:12.316 INFO:tasks.workunit.client.0.smithi110.stdout:8/0: readlink - no filename 2024-04-25T15:49:12.316 INFO:tasks.workunit.client.0.smithi110.stdout:8/1: write - no filename 2024-04-25T15:49:12.317 INFO:tasks.workunit.client.0.smithi110.stdout:0/12: creat d1/d2/f5 x:0 0 0 2024-04-25T15:49:12.318 INFO:tasks.workunit.client.0.smithi110.stdout:0/13: chown d1/f4 0 1 2024-04-25T15:49:12.318 INFO:tasks.workunit.client.0.smithi110.stdout:0/14: readlink l0 0 2024-04-25T15:49:12.321 INFO:tasks.workunit.client.0.smithi110.stdout:5/3: unlink f1 0 2024-04-25T15:49:12.321 INFO:tasks.workunit.client.0.smithi110.stdout:7/7: creat f0 x:0 0 0 2024-04-25T15:49:12.323 INFO:tasks.workunit.client.0.smithi110.stdout:3/5: mknod d0/c1 0 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:6/1: dwrite f0 [0,4194304] 0 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:9/0: truncate - no filename 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:9/1: truncate - no filename 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:9/2: truncate - no filename 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:9/3: link - no file 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:9/4: dread - no filename 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:9/5: write - no filename 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:9/6: link - no file 2024-04-25T15:49:12.324 INFO:tasks.workunit.client.0.smithi110.stdout:9/7: fsync - no filename 2024-04-25T15:49:12.328 INFO:tasks.workunit.client.0.smithi110.stdout:8/2: creat f0 x:0 0 0 2024-04-25T15:49:12.329 INFO:tasks.workunit.client.0.smithi110.stdout:7/8: rename f0 to f1 0 2024-04-25T15:49:12.331 INFO:tasks.workunit.client.0.smithi110.stdout:8/3: write f0 [801374,67971] 0 2024-04-25T15:49:12.332 INFO:tasks.workunit.client.0.smithi110.stdout:0/15: dwrite d1/f3 [0,4194304] 0 2024-04-25T15:49:12.337 INFO:tasks.workunit.client.0.smithi110.stdout:6/2: dwrite f0 [0,4194304] 0 2024-04-25T15:49:12.339 INFO:tasks.workunit.client.0.smithi110.stdout:9/8: creat f0 x:0 0 0 2024-04-25T15:49:12.342 INFO:tasks.workunit.client.0.smithi110.stdout:8/4: mknod c1 0 2024-04-25T15:49:12.345 INFO:tasks.workunit.client.0.smithi110.stdout:0/16: unlink d1/f3 0 2024-04-25T15:49:12.349 INFO:tasks.workunit.client.0.smithi110.stdout:9/9: creat f1 x:0 0 0 2024-04-25T15:49:12.350 INFO:tasks.workunit.client.0.smithi110.stdout:6/3: dwrite f0 [0,4194304] 0 2024-04-25T15:49:12.350 INFO:tasks.workunit.client.0.smithi110.stdout:8/5: dread f0 [0,4194304] 0 2024-04-25T15:49:12.353 INFO:tasks.workunit.client.0.smithi110.stdout:7/9: dwrite f1 [0,4194304] 0 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:6/4: mkdir d1 0 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:9/10: mknod c2 0 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:0/17: dwrite d1/d2/f5 [0,4194304] 0 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:9/11: write f0 [916280,81001] 0 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:9/12: rmdir - no directory 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:8/6: rename f0 to f2 0 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:9/13: dread - f1 zero size 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:8/7: dread f2 [0,4194304] 0 2024-04-25T15:49:12.370 INFO:tasks.workunit.client.0.smithi110.stdout:7/10: creat f2 x:0 0 0 2024-04-25T15:49:12.371 INFO:tasks.workunit.client.0.smithi110.stdout:7/11: write f1 [3371407,67144] 0 2024-04-25T15:49:12.371 INFO:tasks.workunit.client.0.smithi110.stdout:8/8: chown f2 2868537 1 2024-04-25T15:49:12.372 INFO:tasks.workunit.client.0.smithi110.stdout:6/5: dwrite f0 [0,4194304] 0 2024-04-25T15:49:12.373 INFO:tasks.workunit.client.0.smithi110.stdout:7/12: dread - f2 zero size 2024-04-25T15:49:12.374 INFO:tasks.workunit.client.0.smithi110.stdout:7/13: chown f2 0 1 2024-04-25T15:49:12.374 INFO:tasks.workunit.client.0.smithi110.stdout:0/18: dread d1/f4 [0,4194304] 0 2024-04-25T15:49:12.375 INFO:tasks.workunit.client.0.smithi110.stdout:9/14: creat f3 x:0 0 0 2024-04-25T15:49:12.375 INFO:tasks.workunit.client.0.smithi110.stdout:7/14: chown f2 458990587 1 2024-04-25T15:49:12.375 INFO:tasks.workunit.client.0.smithi110.stdout:7/15: readlink - no filename 2024-04-25T15:49:12.375 INFO:tasks.workunit.client.0.smithi110.stdout:9/15: fdatasync f1 0 2024-04-25T15:49:12.376 INFO:tasks.workunit.client.0.smithi110.stdout:8/9: dread f2 [0,4194304] 0 2024-04-25T15:49:12.377 INFO:tasks.workunit.client.0.smithi110.stdout:0/19: symlink d1/d2/l6 0 2024-04-25T15:49:12.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/10: dread f2 [0,4194304] 0 2024-04-25T15:49:12.378 INFO:tasks.workunit.client.0.smithi110.stdout:8/11: readlink - no filename 2024-04-25T15:49:12.379 INFO:tasks.workunit.client.0.smithi110.stdout:7/16: symlink l3 0 2024-04-25T15:49:12.379 INFO:tasks.workunit.client.0.smithi110.stdout:7/17: truncate f2 563970 0 2024-04-25T15:49:12.380 INFO:tasks.workunit.client.0.smithi110.stdout:9/16: rename c2 to c4 0 2024-04-25T15:49:12.381 INFO:tasks.workunit.client.0.smithi110.stdout:0/20: fdatasync d1/f4 0 2024-04-25T15:49:12.382 INFO:tasks.workunit.client.0.smithi110.stdout:0/21: write d1/d2/f5 [2485343,37736] 0 2024-04-25T15:49:12.384 INFO:tasks.workunit.client.0.smithi110.stdout:7/18: creat f4 x:0 0 0 2024-04-25T15:49:12.384 INFO:tasks.workunit.client.0.smithi110.stdout:0/22: write d1/f4 [2862386,49117] 0 2024-04-25T15:49:12.389 INFO:tasks.workunit.client.0.smithi110.stdout:7/19: mknod c5 0 2024-04-25T15:49:12.390 INFO:tasks.workunit.client.0.smithi110.stdout:7/20: write f4 [286297,99806] 0 2024-04-25T15:49:12.393 INFO:tasks.workunit.client.0.smithi110.stdout:7/21: symlink l6 0 2024-04-25T15:49:12.394 INFO:tasks.workunit.client.0.smithi110.stdout:7/22: chown l3 361345869 1 2024-04-25T15:49:12.397 INFO:tasks.workunit.client.0.smithi110.stdout:7/23: symlink l7 0 2024-04-25T15:49:12.404 INFO:tasks.workunit.client.0.smithi110.stdout:7/24: dwrite f4 [0,4194304] 0 2024-04-25T15:49:12.404 INFO:tasks.workunit.client.0.smithi110.stdout:7/25: stat c5 0 2024-04-25T15:49:12.412 INFO:tasks.workunit.client.0.smithi110.stdout:7/26: dwrite f2 [0,4194304] 0 2024-04-25T15:49:12.413 INFO:tasks.workunit.client.0.smithi110.stdout:7/27: write f1 [2951948,46939] 0 2024-04-25T15:49:12.417 INFO:tasks.workunit.client.0.smithi110.stdout:7/28: dread f1 [0,4194304] 0 2024-04-25T15:49:12.419 INFO:tasks.workunit.client.0.smithi110.stdout:7/29: mkdir d8 0 2024-04-25T15:49:12.423 INFO:tasks.workunit.client.0.smithi110.stdout:7/30: rmdir d8 0 2024-04-25T15:49:12.424 INFO:tasks.workunit.client.0.smithi110.stdout:7/31: chown l6 2 1 2024-04-25T15:49:12.424 INFO:tasks.workunit.client.0.smithi110.stdout:7/32: chown c5 2135627 1 2024-04-25T15:49:12.426 INFO:tasks.workunit.client.0.smithi110.stdout:7/33: mknod c9 0 2024-04-25T15:49:12.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:12 smithi110 ceph-mon[26344]: Reconfiguring alertmanager.smithi110 (dependencies changed)... 2024-04-25T15:49:12.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:12 smithi193 ceph-mon[33747]: Reconfiguring alertmanager.smithi110 (dependencies changed)... 2024-04-25T15:49:12.892 INFO:tasks.workunit.client.0.smithi110.stdout:2/8: write f1 [4163587,107388] 0 2024-04-25T15:49:12.895 INFO:tasks.workunit.client.0.smithi110.stdout:4/11: chown l1 2852 1 2024-04-25T15:49:12.896 INFO:tasks.workunit.client.0.smithi110.stdout:4/12: dread - no filename 2024-04-25T15:49:12.896 INFO:tasks.workunit.client.0.smithi110.stdout:4/13: dwrite - no filename 2024-04-25T15:49:12.896 INFO:tasks.workunit.client.0.smithi110.stdout:4/14: dread - no filename 2024-04-25T15:49:12.899 INFO:tasks.workunit.client.0.smithi110.stdout:2/9: dwrite f1 [0,4194304] 0 2024-04-25T15:49:12.901 INFO:tasks.workunit.client.0.smithi110.stdout:5/4: getdents . 0 2024-04-25T15:49:12.904 INFO:tasks.workunit.client.0.smithi110.stdout:2/10: dread f1 [0,4194304] 0 2024-04-25T15:49:12.904 INFO:tasks.workunit.client.0.smithi110.stdout:2/11: chown f1 7 1 2024-04-25T15:49:12.905 INFO:tasks.workunit.client.0.smithi110.stdout:5/5: creat f2 x:0 0 0 2024-04-25T15:49:12.909 INFO:tasks.workunit.client.0.smithi110.stdout:5/6: mknod c3 0 2024-04-25T15:49:12.911 INFO:tasks.workunit.client.0.smithi110.stdout:7/34: fsync f1 0 2024-04-25T15:49:12.912 INFO:tasks.workunit.client.0.smithi110.stdout:5/7: mkdir d4 0 2024-04-25T15:49:12.917 INFO:tasks.workunit.client.0.smithi110.stdout:7/35: rename l6 to la 0 2024-04-25T15:49:12.918 INFO:tasks.workunit.client.0.smithi110.stdout:5/8: rename c3 to d4/c5 0 2024-04-25T15:49:12.920 INFO:tasks.workunit.client.0.smithi110.stdout:2/12: getdents . 0 2024-04-25T15:49:12.921 INFO:tasks.workunit.client.0.smithi110.stdout:7/36: dread f2 [0,4194304] 0 2024-04-25T15:49:12.924 INFO:tasks.workunit.client.0.smithi110.stdout:5/9: creat d4/f6 x:0 0 0 2024-04-25T15:49:12.924 INFO:tasks.workunit.client.0.smithi110.stdout:5/10: readlink - no filename 2024-04-25T15:49:12.925 INFO:tasks.workunit.client.0.smithi110.stdout:2/13: creat f2 x:0 0 0 2024-04-25T15:49:12.926 INFO:tasks.workunit.client.0.smithi110.stdout:2/14: truncate f1 4560751 0 2024-04-25T15:49:12.926 INFO:tasks.workunit.client.0.smithi110.stdout:7/37: creat fb x:0 0 0 2024-04-25T15:49:12.926 INFO:tasks.workunit.client.0.smithi110.stdout:7/38: rmdir - no directory 2024-04-25T15:49:12.927 INFO:tasks.workunit.client.0.smithi110.stdout:7/39: chown c9 484 1 2024-04-25T15:49:12.929 INFO:tasks.workunit.client.0.smithi110.stdout:5/11: mknod d4/c7 0 2024-04-25T15:49:12.929 INFO:tasks.workunit.client.0.smithi110.stdout:5/12: chown d4 7 1 2024-04-25T15:49:12.929 INFO:tasks.workunit.client.0.smithi110.stdout:5/13: read - d4/f6 zero size 2024-04-25T15:49:12.931 INFO:tasks.workunit.client.0.smithi110.stdout:7/40: rename c9 to cc 0 2024-04-25T15:49:12.931 INFO:tasks.workunit.client.0.smithi110.stdout:7/41: stat c5 0 2024-04-25T15:49:12.934 INFO:tasks.workunit.client.0.smithi110.stdout:7/42: fdatasync f2 0 2024-04-25T15:49:12.936 INFO:tasks.workunit.client.0.smithi110.stdout:7/43: creat fd x:0 0 0 2024-04-25T15:49:12.937 INFO:tasks.workunit.client.0.smithi110.stdout:7/44: chown fb 5613 1 2024-04-25T15:49:12.940 INFO:tasks.workunit.client.0.smithi110.stdout:9/17: getdents . 0 2024-04-25T15:49:12.942 INFO:tasks.workunit.client.0.smithi110.stdout:9/18: mknod c5 0 2024-04-25T15:49:12.942 INFO:tasks.workunit.client.0.smithi110.stdout:9/19: rmdir - no directory 2024-04-25T15:49:12.942 INFO:tasks.workunit.client.0.smithi110.stdout:8/12: truncate f2 770933 0 2024-04-25T15:49:12.944 INFO:tasks.workunit.client.0.smithi110.stdout:9/20: creat f6 x:0 0 0 2024-04-25T15:49:12.945 INFO:tasks.workunit.client.0.smithi110.stdout:9/21: fsync f0 0 2024-04-25T15:49:12.945 INFO:tasks.workunit.client.0.smithi110.stdout:9/22: fsync f0 0 2024-04-25T15:49:12.946 INFO:tasks.workunit.client.0.smithi110.stdout:9/23: chown f3 345 1 2024-04-25T15:49:12.947 INFO:tasks.workunit.client.0.smithi110.stdout:9/24: dread - f1 zero size 2024-04-25T15:49:12.949 INFO:tasks.workunit.client.0.smithi110.stdout:9/25: rename c4 to c7 0 2024-04-25T15:49:12.950 INFO:tasks.workunit.client.0.smithi110.stdout:0/23: truncate d1/f4 3808661 0 2024-04-25T15:49:12.951 INFO:tasks.workunit.client.0.smithi110.stdout:9/26: write f1 [15197,108691] 0 2024-04-25T15:49:12.953 INFO:tasks.workunit.client.0.smithi110.stdout:0/24: symlink d1/d2/l7 0 2024-04-25T15:49:12.954 INFO:tasks.workunit.client.0.smithi110.stdout:0/25: truncate d1/d2/f5 5071471 0 2024-04-25T15:49:12.957 INFO:tasks.workunit.client.0.smithi110.stdout:9/27: link f6 f8 0 2024-04-25T15:49:12.957 INFO:tasks.workunit.client.0.smithi110.stdout:9/28: chown c5 31766 1 2024-04-25T15:49:12.959 INFO:tasks.workunit.client.0.smithi110.stdout:0/26: creat d1/f8 x:0 0 0 2024-04-25T15:49:12.961 INFO:tasks.workunit.client.0.smithi110.stdout:9/29: mkdir d9 0 2024-04-25T15:49:12.962 INFO:tasks.workunit.client.0.smithi110.stdout:0/27: mknod d1/c9 0 2024-04-25T15:49:12.965 INFO:tasks.workunit.client.0.smithi110.stdout:0/28: creat d1/fa x:0 0 0 2024-04-25T15:49:12.969 INFO:tasks.workunit.client.0.smithi110.stdout:9/30: dwrite f0 [0,4194304] 0 2024-04-25T15:49:12.974 INFO:tasks.workunit.client.0.smithi110.stdout:9/31: symlink d9/la 0 2024-04-25T15:49:12.976 INFO:tasks.workunit.client.0.smithi110.stdout:9/32: mkdir d9/db 0 2024-04-25T15:49:12.976 INFO:tasks.workunit.client.0.smithi110.stdout:9/33: chown d9/db 990745 1 2024-04-25T15:49:12.977 INFO:tasks.workunit.client.0.smithi110.stdout:9/34: read f1 [17989,118325] 0 2024-04-25T15:49:12.979 INFO:tasks.workunit.client.0.smithi110.stdout:9/35: mknod d9/cc 0 2024-04-25T15:49:12.982 INFO:tasks.workunit.client.0.smithi110.stdout:9/36: dread f1 [0,4194304] 0 2024-04-25T15:49:12.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/37: dwrite f8 [0,4194304] 0 2024-04-25T15:49:12.992 INFO:tasks.workunit.client.0.smithi110.stdout:9/38: getdents d9/db 0 2024-04-25T15:49:12.996 INFO:tasks.workunit.client.0.smithi110.stdout:9/39: link f6 d9/fd 0 2024-04-25T15:49:13.165 INFO:tasks.workunit.client.0.smithi110.stdout:2/15: getdents . 0 2024-04-25T15:49:13.168 INFO:tasks.workunit.client.0.smithi110.stdout:5/14: rmdir d4 39 2024-04-25T15:49:13.175 INFO:tasks.workunit.client.0.smithi110.stdout:7/45: write f4 [4890950,122937] 0 2024-04-25T15:49:13.176 INFO:tasks.workunit.client.0.smithi110.stdout:7/46: chown f2 489819 1 2024-04-25T15:49:13.179 INFO:tasks.workunit.client.0.smithi110.stdout:8/13: truncate f2 1428247 0 2024-04-25T15:49:13.206 INFO:tasks.workunit.client.0.smithi110.stdout:9/40: write f0 [5105130,42863] 0 2024-04-25T15:49:13.251 INFO:tasks.workunit.client.0.smithi110.stdout:0/29: write d1/f4 [4636947,3590] 0 2024-04-25T15:49:13.368 INFO:tasks.workunit.client.0.smithi110.stdout:2/16: unlink l0 0 2024-04-25T15:49:13.370 INFO:tasks.workunit.client.0.smithi110.stdout:7/47: symlink le 0 2024-04-25T15:49:13.374 INFO:tasks.workunit.client.0.smithi110.stdout:7/48: dread f2 [0,4194304] 0 2024-04-25T15:49:13.374 INFO:tasks.workunit.client.0.smithi110.stdout:9/41: write f6 [2823640,12182] 0 2024-04-25T15:49:13.374 INFO:tasks.workunit.client.0.smithi110.stdout:9/42: fsync f3 0 2024-04-25T15:49:13.375 INFO:tasks.workunit.client.0.smithi110.stdout:7/49: write f1 [4029047,34677] 0 2024-04-25T15:49:13.377 INFO:tasks.workunit.client.0.smithi110.stdout:0/30: rename d1/d2/l7 to d1/lb 0 2024-04-25T15:49:13.379 INFO:tasks.workunit.client.0.smithi110.stdout:0/31: fsync d1/f4 0 2024-04-25T15:49:13.383 INFO:tasks.workunit.client.0.smithi110.stdout:7/50: symlink lf 0 2024-04-25T15:49:13.384 INFO:tasks.workunit.client.0.smithi110.stdout:9/43: dwrite d9/fd [4194304,4194304] 0 2024-04-25T15:49:13.386 INFO:tasks.workunit.client.0.smithi110.stdout:2/17: link f1 f3 0 2024-04-25T15:49:13.387 INFO:tasks.workunit.client.0.smithi110.stdout:2/18: chown f2 561 1 2024-04-25T15:49:13.391 INFO:tasks.workunit.client.0.smithi110.stdout:0/32: rename l0 to d1/d2/lc 0 2024-04-25T15:49:13.394 INFO:tasks.workunit.client.0.smithi110.stdout:7/51: mknod c10 0 2024-04-25T15:49:13.394 INFO:tasks.workunit.client.0.smithi110.stdout:7/52: read f4 [2849849,56961] 0 2024-04-25T15:49:13.395 INFO:tasks.workunit.client.0.smithi110.stdout:7/53: fsync f2 0 2024-04-25T15:49:13.395 INFO:tasks.workunit.client.0.smithi110.stdout:7/54: chown l7 4 1 2024-04-25T15:49:13.400 INFO:tasks.workunit.client.0.smithi110.stdout:0/33: creat d1/fd x:0 0 0 2024-04-25T15:49:13.401 INFO:tasks.workunit.client.0.smithi110.stdout:0/34: stat d1/fd 0 2024-04-25T15:49:13.405 INFO:tasks.workunit.client.0.smithi110.stdout:7/55: rename f1 to f11 0 2024-04-25T15:49:13.405 INFO:tasks.workunit.client.0.smithi110.stdout:7/56: chown f2 564 1 2024-04-25T15:49:13.406 INFO:tasks.workunit.client.0.smithi110.stdout:2/19: link f1 f4 0 2024-04-25T15:49:13.407 INFO:tasks.workunit.client.0.smithi110.stdout:5/15: getdents d4 0 2024-04-25T15:49:13.458 INFO:tasks.workunit.client.0.smithi110.stdout:9/44: link d9/cc d9/ce 0 2024-04-25T15:49:13.462 INFO:tasks.workunit.client.0.smithi110.stdout:9/45: dread f6 [4194304,4194304] 0 2024-04-25T15:49:13.462 INFO:tasks.workunit.client.0.smithi110.stdout:2/20: fdatasync f4 0 2024-04-25T15:49:13.474 INFO:tasks.workunit.client.0.smithi110.stdout:5/16: rename f2 to d4/f8 0 2024-04-25T15:49:13.480 INFO:tasks.workunit.client.0.smithi110.stdout:5/17: creat d4/f9 x:0 0 0 2024-04-25T15:49:13.485 INFO:tasks.workunit.client.0.smithi110.stdout:9/46: mkdir d9/db/df 0 2024-04-25T15:49:13.487 INFO:tasks.workunit.client.0.smithi110.stdout:5/18: creat d4/fa x:0 0 0 2024-04-25T15:49:13.488 INFO:tasks.workunit.client.0.smithi110.stdout:5/19: chown d4/c5 7054 1 2024-04-25T15:49:13.488 INFO:tasks.workunit.client.0.smithi110.stdout:5/20: read - d4/f8 zero size 2024-04-25T15:49:13.489 INFO:tasks.workunit.client.0.smithi110.stdout:9/47: symlink d9/l10 0 2024-04-25T15:49:13.494 INFO:tasks.workunit.client.0.smithi110.stdout:5/21: link d4/f6 d4/fb 0 2024-04-25T15:49:13.498 INFO:tasks.workunit.client.0.smithi110.stdout:9/48: dwrite f8 [0,4194304] 0 2024-04-25T15:49:13.500 INFO:tasks.workunit.client.0.smithi110.stdout:9/49: write f3 [919449,21675] 0 2024-04-25T15:49:13.503 INFO:tasks.workunit.client.0.smithi110.stdout:5/22: dwrite f0 [0,4194304] 0 2024-04-25T15:49:13.509 INFO:tasks.workunit.client.0.smithi110.stdout:5/23: symlink d4/lc 0 2024-04-25T15:49:13.513 INFO:tasks.workunit.client.0.smithi110.stdout:5/24: creat d4/fd x:0 0 0 2024-04-25T15:49:13.513 INFO:tasks.workunit.client.0.smithi110.stdout:5/25: write d4/fa [820272,11552] 0 2024-04-25T15:49:13.514 INFO:tasks.workunit.client.0.smithi110.stdout:5/26: chown d4/fb 29 1 2024-04-25T15:49:13.516 INFO:tasks.workunit.client.0.smithi110.stdout:5/27: mkdir d4/de 0 2024-04-25T15:49:13.518 INFO:tasks.workunit.client.0.smithi110.stdout:5/28: symlink d4/lf 0 2024-04-25T15:49:13.519 INFO:tasks.workunit.client.0.smithi110.stdout:5/29: creat d4/f10 x:0 0 0 2024-04-25T15:49:13.521 INFO:tasks.workunit.client.0.smithi110.stdout:5/30: symlink d4/de/l11 0 2024-04-25T15:49:13.651 INFO:tasks.workunit.client.0.smithi110.stdout:8/14: dread f2 [0,4194304] 0 2024-04-25T15:49:13.657 INFO:tasks.workunit.client.0.smithi110.stdout:0/35: getdents d1/d2 0 2024-04-25T15:49:13.658 INFO:tasks.workunit.client.0.smithi110.stdout:0/36: chown d1/fd 4 1 2024-04-25T15:49:13.661 INFO:tasks.workunit.client.0.smithi110.stdout:0/37: rename d1/d2/l6 to d1/le 0 2024-04-25T15:49:13.664 INFO:tasks.workunit.client.0.smithi110.stdout:0/38: creat d1/d2/ff x:0 0 0 2024-04-25T15:49:13.666 INFO:tasks.workunit.client.0.smithi110.stdout:7/57: getdents . 0 2024-04-25T15:49:13.666 INFO:tasks.workunit.client.0.smithi110.stdout:7/58: rmdir - no directory 2024-04-25T15:49:13.669 INFO:tasks.workunit.client.0.smithi110.stdout:7/59: mknod c12 0 2024-04-25T15:49:13.671 INFO:tasks.workunit.client.0.smithi110.stdout:7/60: mknod c13 0 2024-04-25T15:49:13.674 INFO:tasks.workunit.client.0.smithi110.stdout:7/61: unlink c13 0 2024-04-25T15:49:13.675 INFO:tasks.workunit.client.0.smithi110.stdout:7/62: truncate fd 276711 0 2024-04-25T15:49:13.675 INFO:tasks.workunit.client.0.smithi110.stdout:7/63: dread - fb zero size 2024-04-25T15:49:13.678 INFO:tasks.workunit.client.0.smithi110.stdout:7/64: creat f14 x:0 0 0 2024-04-25T15:49:13.678 INFO:tasks.workunit.client.0.smithi110.stdout:9/50: getdents d9/db 0 2024-04-25T15:49:13.681 INFO:tasks.workunit.client.0.smithi110.stdout:7/65: mkdir d15 0 2024-04-25T15:49:13.683 INFO:tasks.workunit.client.0.smithi110.stdout:9/51: creat d9/db/f11 x:0 0 0 2024-04-25T15:49:13.686 INFO:tasks.workunit.client.0.smithi110.stdout:9/52: dread f8 [4194304,4194304] 0 2024-04-25T15:49:13.690 INFO:tasks.workunit.client.0.smithi110.stdout:9/53: chown d9/cc 89 1 2024-04-25T15:49:13.694 INFO:tasks.workunit.client.0.smithi110.stdout:7/66: rmdir d15 0 2024-04-25T15:49:13.694 INFO:tasks.workunit.client.0.smithi110.stdout:5/31: getdents d4 0 2024-04-25T15:49:13.696 INFO:tasks.workunit.client.0.smithi110.stdout:9/54: fsync f8 0 2024-04-25T15:49:13.698 INFO:tasks.workunit.client.0.smithi110.stdout:7/67: symlink l16 0 2024-04-25T15:49:13.699 INFO:tasks.workunit.client.0.smithi110.stdout:5/32: symlink d4/de/l12 0 2024-04-25T15:49:13.700 INFO:tasks.workunit.client.0.smithi110.stdout:5/33: fsync d4/f10 0 2024-04-25T15:49:13.703 INFO:tasks.workunit.client.0.smithi110.stdout:5/34: dread d4/fa [0,4194304] 0 2024-04-25T15:49:13.709 INFO:tasks.workunit.client.0.smithi110.stdout:5/35: rename f0 to d4/de/f13 0 2024-04-25T15:49:13.714 INFO:tasks.workunit.client.0.smithi110.stdout:5/36: symlink d4/de/l14 0 2024-04-25T15:49:13.715 INFO:tasks.workunit.client.0.smithi110.stdout:9/55: link c7 d9/db/c12 0 2024-04-25T15:49:13.716 INFO:tasks.workunit.client.0.smithi110.stdout:9/56: write d9/fd [2505776,69274] 0 2024-04-25T15:49:13.717 INFO:tasks.workunit.client.0.smithi110.stdout:9/57: readlink d9/la 0 2024-04-25T15:49:13.719 INFO:tasks.workunit.client.0.smithi110.stdout:5/37: mkdir d4/d15 0 2024-04-25T15:49:13.720 INFO:tasks.workunit.client.0.smithi110.stdout:9/58: mkdir d9/d13 0 2024-04-25T15:49:13.721 INFO:tasks.workunit.client.0.smithi110.stdout:9/59: write d9/db/f11 [455069,47080] 0 2024-04-25T15:49:13.722 INFO:tasks.workunit.client.0.smithi110.stdout:5/38: fsync d4/f8 0 2024-04-25T15:49:13.723 INFO:tasks.workunit.client.0.smithi110.stdout:5/39: truncate d4/fd 911643 0 2024-04-25T15:49:13.723 INFO:tasks.workunit.client.0.smithi110.stdout:5/40: chown d4/de/l14 878602 1 2024-04-25T15:49:13.724 INFO:tasks.workunit.client.0.smithi110.stdout:5/41: truncate d4/fb 369295 0 2024-04-25T15:49:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:13 smithi193 ceph-mon[33747]: Reconfiguring daemon alertmanager.smithi110 on smithi110 2024-04-25T15:49:13.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:13 smithi193 ceph-mon[33747]: pgmap v23: 65 pgs: 65 active+clean; 180 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 260 op/s 2024-04-25T15:49:13.834 INFO:tasks.workunit.client.0.smithi110.stdout:0/39: rmdir d1/d2 39 2024-04-25T15:49:13.836 INFO:tasks.workunit.client.0.smithi110.stdout:8/15: dwrite f2 [0,4194304] 0 2024-04-25T15:49:13.837 INFO:tasks.workunit.client.0.smithi110.stdout:8/16: write f2 [2282894,129334] 0 2024-04-25T15:49:13.838 INFO:tasks.workunit.client.0.smithi110.stdout:8/17: chown f2 5848 1 2024-04-25T15:49:13.843 INFO:tasks.workunit.client.0.smithi110.stdout:8/18: creat f3 x:0 0 0 2024-04-25T15:49:13.845 INFO:tasks.workunit.client.0.smithi110.stdout:0/40: creat d1/f10 x:0 0 0 2024-04-25T15:49:13.848 INFO:tasks.workunit.client.0.smithi110.stdout:0/41: creat d1/d2/f11 x:0 0 0 2024-04-25T15:49:13.849 INFO:tasks.workunit.client.0.smithi110.stdout:0/42: chown d1/f10 46 1 2024-04-25T15:49:13.852 INFO:tasks.workunit.client.0.smithi110.stdout:9/60: fsync d9/db/f11 0 2024-04-25T15:49:13.857 INFO:tasks.workunit.client.0.smithi110.stdout:0/43: dwrite d1/f8 [0,4194304] 0 2024-04-25T15:49:13.857 INFO:tasks.workunit.client.0.smithi110.stdout:9/61: mknod d9/db/df/c14 0 2024-04-25T15:49:13.860 INFO:tasks.workunit.client.0.smithi110.stdout:0/44: creat d1/d2/f12 x:0 0 0 2024-04-25T15:49:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:13 smithi110 ceph-mon[26344]: Reconfiguring daemon alertmanager.smithi110 on smithi110 2024-04-25T15:49:13.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:13 smithi110 ceph-mon[26344]: pgmap v23: 65 pgs: 65 active+clean; 180 MiB data, 2.3 GiB used, 534 GiB / 536 GiB avail; 0 B/s rd, 3.2 MiB/s wr, 260 op/s 2024-04-25T15:49:13.863 INFO:tasks.workunit.client.0.smithi110.stdout:0/45: mknod d1/c13 0 2024-04-25T15:49:13.864 INFO:tasks.workunit.client.0.smithi110.stdout:0/46: read d1/d2/f5 [1002679,68680] 0 2024-04-25T15:49:13.865 INFO:tasks.workunit.client.0.smithi110.stdout:0/47: chown d1 162 1 2024-04-25T15:49:13.866 INFO:tasks.workunit.client.0.smithi110.stdout:9/62: dwrite d9/fd [4194304,4194304] 0 2024-04-25T15:49:13.867 INFO:tasks.workunit.client.0.smithi110.stdout:9/63: write d9/db/f11 [664449,126966] 0 2024-04-25T15:49:13.897 INFO:tasks.workunit.client.0.smithi110.stdout:9/64: creat d9/db/f15 x:0 0 0 2024-04-25T15:49:13.899 INFO:tasks.workunit.client.0.smithi110.stdout:0/48: symlink d1/l14 0 2024-04-25T15:49:13.903 INFO:tasks.workunit.client.0.smithi110.stdout:9/65: dwrite f3 [0,4194304] 0 2024-04-25T15:49:13.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/49: dread d1/f8 [0,4194304] 0 2024-04-25T15:49:13.994 INFO:tasks.workunit.client.0.smithi110.stdout:0/50: dread - d1/d2/ff zero size 2024-04-25T15:49:13.996 INFO:tasks.workunit.client.0.smithi110.stdout:5/42: rename d4/de/f13 to d4/f16 0 2024-04-25T15:49:13.998 INFO:tasks.workunit.client.0.smithi110.stdout:5/43: truncate d4/f6 1159132 0 2024-04-25T15:49:14.000 INFO:tasks.workunit.client.0.smithi110.stdout:5/44: chown d4/de/l11 101860616 1 2024-04-25T15:49:14.002 INFO:tasks.workunit.client.0.smithi110.stdout:0/51: dwrite d1/f10 [0,4194304] 0 2024-04-25T15:49:14.002 INFO:tasks.workunit.client.0.smithi110.stdout:5/45: write d4/fd [849052,86103] 0 2024-04-25T15:49:14.002 INFO:tasks.workunit.client.0.smithi110.stdout:9/66: rename d9/la to d9/db/df/l16 0 2024-04-25T15:49:14.003 INFO:tasks.workunit.client.0.smithi110.stdout:0/52: readlink d1/d2/lc 0 2024-04-25T15:49:14.012 INFO:tasks.workunit.client.0.smithi110.stdout:5/46: dwrite d4/fd [0,4194304] 0 2024-04-25T15:49:14.015 INFO:tasks.workunit.client.0.smithi110.stdout:9/67: unlink f3 0 2024-04-25T15:49:14.017 INFO:tasks.workunit.client.0.smithi110.stdout:9/68: stat d9/db/f11 0 2024-04-25T15:49:14.023 INFO:tasks.workunit.client.0.smithi110.stdout:9/69: creat d9/db/df/f17 x:0 0 0 2024-04-25T15:49:14.024 INFO:tasks.workunit.client.0.smithi110.stdout:9/70: dread - d9/db/df/f17 zero size 2024-04-25T15:49:14.024 INFO:tasks.workunit.client.0.smithi110.stdout:9/71: chown c5 5525 1 2024-04-25T15:49:14.085 INFO:tasks.workunit.client.0.smithi110.stdout:0/53: getdents d1/d2 0 2024-04-25T15:49:14.091 INFO:tasks.workunit.client.0.smithi110.stdout:0/54: rename d1/d2/f5 to d1/f15 0 2024-04-25T15:49:14.092 INFO:tasks.workunit.client.0.smithi110.stdout:0/55: write d1/f4 [2464808,57088] 0 2024-04-25T15:49:14.104 INFO:tasks.workunit.client.0.smithi110.stdout:9/72: write f1 [486606,82620] 0 2024-04-25T15:49:14.109 INFO:tasks.workunit.client.0.smithi110.stdout:5/47: dwrite d4/fb [0,4194304] 0 2024-04-25T15:49:14.115 INFO:tasks.workunit.client.0.smithi110.stdout:9/73: dwrite f1 [0,4194304] 0 2024-04-25T15:49:14.116 INFO:tasks.workunit.client.0.smithi110.stdout:9/74: chown f0 2766825 1 2024-04-25T15:49:14.116 INFO:tasks.workunit.client.0.smithi110.stdout:5/48: getdents d4 0 2024-04-25T15:49:14.119 INFO:tasks.workunit.client.0.smithi110.stdout:9/75: symlink d9/d13/l18 0 2024-04-25T15:49:14.124 INFO:tasks.workunit.client.0.smithi110.stdout:0/56: getdents d1/d2 0 2024-04-25T15:49:14.129 INFO:tasks.workunit.client.0.smithi110.stdout:9/76: dwrite d9/db/f11 [0,4194304] 0 2024-04-25T15:49:14.131 INFO:tasks.workunit.client.0.smithi110.stdout:0/57: rename d1/f4 to d1/f16 0 2024-04-25T15:49:14.133 INFO:tasks.workunit.client.0.smithi110.stdout:0/58: creat d1/d2/f17 x:0 0 0 2024-04-25T15:49:14.135 INFO:tasks.workunit.client.0.smithi110.stdout:0/59: rename d1/l14 to d1/l18 0 2024-04-25T15:49:14.173 INFO:tasks.workunit.client.0.smithi110.stdout:0/60: dwrite d1/f15 [4194304,4194304] 0 2024-04-25T15:49:14.173 INFO:tasks.workunit.client.0.smithi110.stdout:0/61: stat d1/f15 0 2024-04-25T15:49:14.443 INFO:tasks.workunit.client.0.smithi110.stdout:0/62: creat d1/f19 x:0 0 0 2024-04-25T15:49:14.443 INFO:tasks.workunit.client.0.smithi110.stdout:0/63: write d1/f19 [160046,72747] 0 2024-04-25T15:49:14.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:14 smithi193 ceph-mon[33747]: pgmap v24: 65 pgs: 65 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 89 KiB/s rd, 10 MiB/s wr, 394 op/s 2024-04-25T15:49:14.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:14 smithi110 ceph-mon[26344]: pgmap v24: 65 pgs: 65 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 89 KiB/s rd, 10 MiB/s wr, 394 op/s 2024-04-25T15:49:15.561 INFO:tasks.workunit.client.0.smithi110.stdout:1/8: sync 2024-04-25T15:49:15.566 INFO:tasks.workunit.client.0.smithi110.stdout:1/9: unlink d0/f1 0 2024-04-25T15:49:15.568 INFO:tasks.workunit.client.0.smithi110.stdout:1/10: creat d0/f2 x:0 0 0 2024-04-25T15:49:15.569 INFO:tasks.workunit.client.0.smithi110.stdout:1/11: fdatasync d0/f2 0 2024-04-25T15:49:15.572 INFO:tasks.workunit.client.0.smithi110.stdout:1/12: unlink d0/f2 0 2024-04-25T15:49:15.574 INFO:tasks.workunit.client.0.smithi110.stdout:1/13: mkdir d0/d3 0 2024-04-25T15:49:15.574 INFO:tasks.workunit.client.0.smithi110.stdout:1/14: dwrite - no filename 2024-04-25T15:49:15.578 INFO:tasks.workunit.client.0.smithi110.stdout:1/15: symlink d0/d3/l4 0 2024-04-25T15:49:15.826 INFO:tasks.workunit.client.0.smithi110.stdout:6/6: sync 2024-04-25T15:49:15.826 INFO:tasks.workunit.client.0.smithi110.stdout:3/6: sync 2024-04-25T15:49:15.827 INFO:tasks.workunit.client.0.smithi110.stdout:6/7: stat f0 0 2024-04-25T15:49:15.829 INFO:tasks.workunit.client.0.smithi110.stdout:6/8: mknod d1/c2 0 2024-04-25T15:49:15.830 INFO:tasks.workunit.client.0.smithi110.stdout:6/9: read f0 [4054890,8332] 0 2024-04-25T15:49:15.833 INFO:tasks.workunit.client.0.smithi110.stdout:6/10: creat d1/f3 x:0 0 0 2024-04-25T15:49:15.833 INFO:tasks.workunit.client.0.smithi110.stdout:6/11: chown f0 7 1 2024-04-25T15:49:15.834 INFO:tasks.workunit.client.0.smithi110.stdout:6/12: fdatasync d1/f3 0 2024-04-25T15:49:15.834 INFO:tasks.workunit.client.0.smithi110.stdout:3/7: getdents d0 0 2024-04-25T15:49:15.838 INFO:tasks.workunit.client.0.smithi110.stdout:6/13: symlink d1/l4 0 2024-04-25T15:49:15.838 INFO:tasks.workunit.client.0.smithi110.stdout:6/14: rename d1 to d1/d5 22 2024-04-25T15:49:15.839 INFO:tasks.workunit.client.0.smithi110.stdout:3/8: rmdir d0 39 2024-04-25T15:49:15.842 INFO:tasks.workunit.client.0.smithi110.stdout:6/15: write f0 [1662380,59934] 0 2024-04-25T15:49:15.843 INFO:tasks.workunit.client.0.smithi110.stdout:6/16: truncate d1/f3 1030902 0 2024-04-25T15:49:15.843 INFO:tasks.workunit.client.0.smithi110.stdout:3/9: rename d0 to d0/d2 22 2024-04-25T15:49:15.843 INFO:tasks.workunit.client.0.smithi110.stdout:3/10: dread - no filename 2024-04-25T15:49:15.843 INFO:tasks.workunit.client.0.smithi110.stdout:3/11: fsync - no filename 2024-04-25T15:49:15.846 INFO:tasks.workunit.client.0.smithi110.stdout:3/12: creat d0/f3 x:0 0 0 2024-04-25T15:49:15.847 INFO:tasks.workunit.client.0.smithi110.stdout:6/17: rename d1/l4 to d1/l6 0 2024-04-25T15:49:15.849 INFO:tasks.workunit.client.0.smithi110.stdout:3/13: creat d0/f4 x:0 0 0 2024-04-25T15:49:15.850 INFO:tasks.workunit.client.0.smithi110.stdout:3/14: chown d0/f4 5102650 1 2024-04-25T15:49:15.856 INFO:tasks.workunit.client.0.smithi110.stdout:3/15: dwrite d0/f4 [0,4194304] 0 2024-04-25T15:49:15.860 INFO:tasks.workunit.client.0.smithi110.stdout:3/16: symlink d0/l5 0 2024-04-25T15:49:15.863 INFO:tasks.workunit.client.0.smithi110.stdout:4/15: sync 2024-04-25T15:49:15.863 INFO:tasks.workunit.client.0.smithi110.stdout:2/21: sync 2024-04-25T15:49:15.865 INFO:tasks.workunit.client.0.smithi110.stdout:2/22: write f1 [1986063,58036] 0 2024-04-25T15:49:15.866 INFO:tasks.workunit.client.0.smithi110.stdout:4/16: creat f2 x:0 0 0 2024-04-25T15:49:15.866 INFO:tasks.workunit.client.0.smithi110.stdout:2/23: dread - f2 zero size 2024-04-25T15:49:15.868 INFO:tasks.workunit.client.0.smithi110.stdout:2/24: dread - f2 zero size 2024-04-25T15:49:15.868 INFO:tasks.workunit.client.0.smithi110.stdout:2/25: readlink - no filename 2024-04-25T15:49:15.869 INFO:tasks.workunit.client.0.smithi110.stdout:3/17: dwrite d0/f4 [0,4194304] 0 2024-04-25T15:49:15.871 INFO:tasks.workunit.client.0.smithi110.stdout:3/18: write d0/f4 [1880700,129278] 0 2024-04-25T15:49:15.871 INFO:tasks.workunit.client.0.smithi110.stdout:2/26: rename f1 to f5 0 2024-04-25T15:49:15.872 INFO:tasks.workunit.client.0.smithi110.stdout:2/27: dread - f2 zero size 2024-04-25T15:49:15.877 INFO:tasks.workunit.client.0.smithi110.stdout:3/19: creat d0/f6 x:0 0 0 2024-04-25T15:49:15.880 INFO:tasks.workunit.client.0.smithi110.stdout:3/20: mkdir d0/d7 0 2024-04-25T15:49:15.881 INFO:tasks.workunit.client.0.smithi110.stdout:3/21: dread - d0/f6 zero size 2024-04-25T15:49:15.882 INFO:tasks.workunit.client.0.smithi110.stdout:3/22: write d0/f6 [793604,110375] 0 2024-04-25T15:49:15.883 INFO:tasks.workunit.client.0.smithi110.stdout:2/28: dwrite f3 [0,4194304] 0 2024-04-25T15:49:15.884 INFO:tasks.workunit.client.0.smithi110.stdout:3/23: truncate d0/f6 1759747 0 2024-04-25T15:49:15.885 INFO:tasks.workunit.client.0.smithi110.stdout:2/29: write f5 [8812,81139] 0 2024-04-25T15:49:15.886 INFO:tasks.workunit.client.0.smithi110.stdout:2/30: dread - f2 zero size 2024-04-25T15:49:15.888 INFO:tasks.workunit.client.0.smithi110.stdout:3/24: mknod d0/c8 0 2024-04-25T15:49:15.889 INFO:tasks.workunit.client.0.smithi110.stdout:3/25: dread - d0/f3 zero size 2024-04-25T15:49:15.892 INFO:tasks.workunit.client.0.smithi110.stdout:3/26: mknod d0/c9 0 2024-04-25T15:49:15.893 INFO:tasks.workunit.client.0.smithi110.stdout:2/31: dwrite f3 [0,4194304] 0 2024-04-25T15:49:15.896 INFO:tasks.workunit.client.0.smithi110.stdout:2/32: rename f2 to f6 0 2024-04-25T15:49:15.897 INFO:tasks.workunit.client.0.smithi110.stdout:2/33: readlink - no filename 2024-04-25T15:49:15.900 INFO:tasks.workunit.client.0.smithi110.stdout:2/34: rename f3 to f7 0 2024-04-25T15:49:15.904 INFO:tasks.workunit.client.0.smithi110.stdout:3/27: dwrite d0/f3 [0,4194304] 0 2024-04-25T15:49:15.908 INFO:tasks.workunit.client.0.smithi110.stdout:2/35: write f6 [890335,72184] 0 2024-04-25T15:49:15.910 INFO:tasks.workunit.client.0.smithi110.stdout:3/28: dread d0/f3 [0,4194304] 0 2024-04-25T15:49:15.911 INFO:tasks.workunit.client.0.smithi110.stdout:3/29: stat d0/l5 0 2024-04-25T15:49:15.914 INFO:tasks.workunit.client.0.smithi110.stdout:2/36: write f4 [2153763,43859] 0 2024-04-25T15:49:15.922 INFO:tasks.workunit.client.0.smithi110.stdout:2/37: dwrite f5 [4194304,4194304] 0 2024-04-25T15:49:15.924 INFO:tasks.workunit.client.0.smithi110.stdout:3/30: dwrite d0/f3 [0,4194304] 0 2024-04-25T15:49:15.931 INFO:tasks.workunit.client.0.smithi110.stdout:3/31: getdents d0/d7 0 2024-04-25T15:49:15.932 INFO:tasks.workunit.client.0.smithi110.stdout:3/32: read d0/f3 [364370,115638] 0 2024-04-25T15:49:15.936 INFO:tasks.workunit.client.0.smithi110.stdout:3/33: link d0/f4 d0/d7/fa 0 2024-04-25T15:49:15.944 INFO:tasks.workunit.client.0.smithi110.stdout:3/34: fdatasync d0/f3 0 2024-04-25T15:49:15.949 INFO:tasks.workunit.client.0.smithi110.stdout:3/35: dread d0/f6 [0,4194304] 0 2024-04-25T15:49:15.950 INFO:tasks.workunit.client.0.smithi110.stdout:3/36: write d0/f3 [556777,113743] 0 2024-04-25T15:49:15.958 INFO:tasks.workunit.client.0.smithi110.stdout:3/37: dwrite d0/f6 [0,4194304] 0 2024-04-25T15:49:15.959 INFO:tasks.workunit.client.0.smithi110.stdout:3/38: read d0/d7/fa [3060738,66380] 0 2024-04-25T15:49:15.964 INFO:tasks.workunit.client.0.smithi110.stdout:3/39: dread d0/f4 [0,4194304] 0 2024-04-25T15:49:15.966 INFO:tasks.workunit.client.0.smithi110.stdout:3/40: rmdir d0/d7 39 2024-04-25T15:49:15.973 INFO:tasks.workunit.client.0.smithi110.stdout:3/41: dwrite d0/f6 [0,4194304] 0 2024-04-25T15:49:15.979 INFO:tasks.workunit.client.0.smithi110.stdout:3/42: mknod d0/cb 0 2024-04-25T15:49:15.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/43: dwrite d0/f6 [0,4194304] 0 2024-04-25T15:49:15.988 INFO:tasks.workunit.client.0.smithi110.stdout:3/44: creat d0/d7/fc x:0 0 0 2024-04-25T15:49:15.992 INFO:tasks.workunit.client.0.smithi110.stdout:3/45: dread d0/f4 [0,4194304] 0 2024-04-25T15:49:15.993 INFO:tasks.workunit.client.0.smithi110.stdout:3/46: readlink d0/l5 0 2024-04-25T15:49:15.997 INFO:tasks.workunit.client.0.smithi110.stdout:3/47: creat d0/fd x:0 0 0 2024-04-25T15:49:15.999 INFO:tasks.workunit.client.0.smithi110.stdout:3/48: creat d0/d7/fe x:0 0 0 2024-04-25T15:49:16.003 INFO:tasks.workunit.client.0.smithi110.stdout:3/49: creat d0/ff x:0 0 0 2024-04-25T15:49:16.004 INFO:tasks.workunit.client.0.smithi110.stdout:3/50: dread - d0/d7/fe zero size 2024-04-25T15:49:16.005 INFO:tasks.workunit.client.0.smithi110.stdout:3/51: write d0/ff [100208,92886] 0 2024-04-25T15:49:16.005 INFO:tasks.workunit.client.0.smithi110.stdout:3/52: chown d0/c8 0 1 2024-04-25T15:49:16.008 INFO:tasks.workunit.client.0.smithi110.stdout:3/53: symlink d0/l10 0 2024-04-25T15:49:16.008 INFO:tasks.workunit.client.0.smithi110.stdout:3/54: rename d0 to d0/d11 22 2024-04-25T15:49:16.011 INFO:tasks.workunit.client.0.smithi110.stdout:3/55: creat d0/f12 x:0 0 0 2024-04-25T15:49:16.011 INFO:tasks.workunit.client.0.smithi110.stdout:3/56: chown d0 28327891 1 2024-04-25T15:49:16.012 INFO:tasks.workunit.client.0.smithi110.stdout:3/57: chown d0/d7 31 1 2024-04-25T15:49:16.013 INFO:tasks.workunit.client.0.smithi110.stdout:3/58: mknod d0/c13 0 2024-04-25T15:49:16.015 INFO:tasks.workunit.client.0.smithi110.stdout:3/59: fdatasync d0/d7/fa 0 2024-04-25T15:49:16.017 INFO:tasks.workunit.client.0.smithi110.stdout:3/60: mknod d0/d7/c14 0 2024-04-25T15:49:16.027 INFO:tasks.workunit.client.0.smithi110.stdout:9/77: sync 2024-04-25T15:49:16.027 INFO:tasks.workunit.client.0.smithi110.stdout:8/19: sync 2024-04-25T15:49:16.027 INFO:tasks.workunit.client.0.smithi110.stdout:7/68: sync 2024-04-25T15:49:16.027 INFO:tasks.workunit.client.0.smithi110.stdout:0/64: sync 2024-04-25T15:49:16.027 INFO:tasks.workunit.client.0.smithi110.stdout:5/49: sync 2024-04-25T15:49:16.027 INFO:tasks.workunit.client.0.smithi110.stdout:5/50: stat d4 0 2024-04-25T15:49:16.027 INFO:tasks.workunit.client.0.smithi110.stdout:9/78: fsync f0 0 2024-04-25T15:49:16.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/69: chown le 4009437 1 2024-04-25T15:49:16.028 INFO:tasks.workunit.client.0.smithi110.stdout:0/65: dread - d1/d2/f12 zero size 2024-04-25T15:49:16.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/70: chown fd 0 1 2024-04-25T15:49:16.029 INFO:tasks.workunit.client.0.smithi110.stdout:8/20: mknod c4 0 2024-04-25T15:49:16.029 INFO:tasks.workunit.client.0.smithi110.stdout:8/21: stat f2 0 2024-04-25T15:49:16.031 INFO:tasks.workunit.client.0.smithi110.stdout:0/66: truncate d1/f8 2320774 0 2024-04-25T15:49:16.032 INFO:tasks.workunit.client.0.smithi110.stdout:8/22: creat f5 x:0 0 0 2024-04-25T15:49:16.063 INFO:tasks.workunit.client.0.smithi110.stdout:0/67: mkdir d1/d1a 0 2024-04-25T15:49:16.064 INFO:tasks.workunit.client.0.smithi110.stdout:8/23: link f3 f6 0 2024-04-25T15:49:16.067 INFO:tasks.workunit.client.0.smithi110.stdout:8/24: mknod c7 0 2024-04-25T15:49:16.068 INFO:tasks.workunit.client.0.smithi110.stdout:8/25: read - f3 zero size 2024-04-25T15:49:16.069 INFO:tasks.workunit.client.0.smithi110.stdout:8/26: stat f3 0 2024-04-25T15:49:16.069 INFO:tasks.workunit.client.0.smithi110.stdout:0/68: dwrite d1/f10 [0,4194304] 0 2024-04-25T15:49:16.070 INFO:tasks.workunit.client.0.smithi110.stdout:8/27: chown c4 1 1 2024-04-25T15:49:16.070 INFO:tasks.workunit.client.0.smithi110.stdout:8/28: chown f5 951926 1 2024-04-25T15:49:16.070 INFO:tasks.workunit.client.0.smithi110.stdout:0/69: dread - d1/d2/f11 zero size 2024-04-25T15:49:16.072 INFO:tasks.workunit.client.0.smithi110.stdout:8/29: write f5 [911723,120725] 0 2024-04-25T15:49:16.073 INFO:tasks.workunit.client.0.smithi110.stdout:8/30: write f2 [508720,33519] 0 2024-04-25T15:49:16.073 INFO:tasks.workunit.client.0.smithi110.stdout:8/31: chown f5 1 1 2024-04-25T15:49:16.075 INFO:tasks.workunit.client.0.smithi110.stdout:0/70: rename d1/d2/f17 to d1/d2/f1b 0 2024-04-25T15:49:16.080 INFO:tasks.workunit.client.0.smithi110.stdout:8/32: fdatasync f6 0 2024-04-25T15:49:16.080 INFO:tasks.workunit.client.0.smithi110.stdout:8/33: chown f2 26908 1 2024-04-25T15:49:16.081 INFO:tasks.workunit.client.0.smithi110.stdout:8/34: chown c1 6588 1 2024-04-25T15:49:16.083 INFO:tasks.workunit.client.0.smithi110.stdout:8/35: symlink l8 0 2024-04-25T15:49:16.086 INFO:tasks.workunit.client.0.smithi110.stdout:8/36: symlink l9 0 2024-04-25T15:49:16.086 INFO:tasks.workunit.client.0.smithi110.stdout:0/71: dwrite d1/f16 [0,4194304] 0 2024-04-25T15:49:16.087 INFO:tasks.workunit.client.0.smithi110.stdout:8/37: read f2 [760733,124684] 0 2024-04-25T15:49:16.087 INFO:tasks.workunit.client.0.smithi110.stdout:8/38: chown l9 1216039 1 2024-04-25T15:49:16.095 INFO:tasks.workunit.client.0.smithi110.stdout:8/39: dwrite f5 [0,4194304] 0 2024-04-25T15:49:16.096 INFO:tasks.workunit.client.0.smithi110.stdout:8/40: dread - f3 zero size 2024-04-25T15:49:16.096 INFO:tasks.workunit.client.0.smithi110.stdout:8/41: readlink l9 0 2024-04-25T15:49:16.102 INFO:tasks.workunit.client.0.smithi110.stdout:0/72: link d1/f10 d1/f1c 0 2024-04-25T15:49:16.103 INFO:tasks.workunit.client.0.smithi110.stdout:8/42: dwrite f5 [0,4194304] 0 2024-04-25T15:49:16.104 INFO:tasks.workunit.client.0.smithi110.stdout:8/43: chown f6 6314964 1 2024-04-25T15:49:16.106 INFO:tasks.workunit.client.0.smithi110.stdout:0/73: mknod d1/d2/c1d 0 2024-04-25T15:49:16.110 INFO:tasks.workunit.client.0.smithi110.stdout:0/74: symlink d1/l1e 0 2024-04-25T15:49:16.111 INFO:tasks.workunit.client.0.smithi110.stdout:0/75: stat d1/l1e 0 2024-04-25T15:49:16.112 INFO:tasks.workunit.client.0.smithi110.stdout:0/76: stat d1/d2/f11 0 2024-04-25T15:49:16.112 INFO:tasks.workunit.client.0.smithi110.stdout:8/44: dwrite f3 [0,4194304] 0 2024-04-25T15:49:16.115 INFO:tasks.workunit.client.0.smithi110.stdout:0/77: mknod d1/d2/c1f 0 2024-04-25T15:49:16.120 INFO:tasks.workunit.client.0.smithi110.stdout:0/78: mkdir d1/d20 0 2024-04-25T15:49:16.121 INFO:tasks.workunit.client.0.smithi110.stdout:8/45: dwrite f5 [0,4194304] 0 2024-04-25T15:49:16.124 INFO:tasks.workunit.client.0.smithi110.stdout:0/79: symlink d1/d20/l21 0 2024-04-25T15:49:16.172 INFO:tasks.workunit.client.0.smithi110.stdout:1/16: sync 2024-04-25T15:49:16.173 INFO:tasks.workunit.client.0.smithi110.stdout:4/17: sync 2024-04-25T15:49:16.173 INFO:tasks.workunit.client.0.smithi110.stdout:1/17: dread - no filename 2024-04-25T15:49:16.173 INFO:tasks.workunit.client.0.smithi110.stdout:1/18: fdatasync - no filename 2024-04-25T15:49:16.173 INFO:tasks.workunit.client.0.smithi110.stdout:1/19: dread - no filename 2024-04-25T15:49:16.173 INFO:tasks.workunit.client.0.smithi110.stdout:1/20: rename d0 to d0/d3/d5 22 2024-04-25T15:49:16.173 INFO:tasks.workunit.client.0.smithi110.stdout:1/21: dread - no filename 2024-04-25T15:49:16.177 INFO:tasks.workunit.client.0.smithi110.stdout:1/22: creat d0/d3/f6 x:0 0 0 2024-04-25T15:49:16.179 INFO:tasks.workunit.client.0.smithi110.stdout:1/23: write d0/d3/f6 [863668,31206] 0 2024-04-25T15:49:16.180 INFO:tasks.workunit.client.0.smithi110.stdout:4/18: dwrite f2 [0,4194304] 0 2024-04-25T15:49:16.182 INFO:tasks.workunit.client.0.smithi110.stdout:1/24: creat d0/d3/f7 x:0 0 0 2024-04-25T15:49:16.182 INFO:tasks.workunit.client.0.smithi110.stdout:4/19: symlink l3 0 2024-04-25T15:49:16.195 INFO:tasks.workunit.client.0.smithi110.stdout:1/25: mknod d0/c8 0 2024-04-25T15:49:16.195 INFO:tasks.workunit.client.0.smithi110.stdout:4/20: fdatasync f2 0 2024-04-25T15:49:16.201 INFO:tasks.workunit.client.0.smithi110.stdout:4/21: dwrite f2 [0,4194304] 0 2024-04-25T15:49:16.213 INFO:tasks.workunit.client.0.smithi110.stdout:4/22: fdatasync f2 0 2024-04-25T15:49:16.330 INFO:tasks.workunit.client.0.smithi110.stdout:9/79: sync 2024-04-25T15:49:16.331 INFO:tasks.workunit.client.0.smithi110.stdout:9/80: dread - d9/db/df/f17 zero size 2024-04-25T15:49:16.334 INFO:tasks.workunit.client.0.smithi110.stdout:9/81: dread f6 [4194304,4194304] 0 2024-04-25T15:49:16.340 INFO:tasks.workunit.client.0.smithi110.stdout:9/82: dwrite d9/db/f11 [0,4194304] 0 2024-04-25T15:49:16.347 INFO:tasks.workunit.client.0.smithi110.stdout:9/83: dwrite d9/db/df/f17 [0,4194304] 0 2024-04-25T15:49:16.368 INFO:tasks.workunit.client.0.smithi110.stdout:7/71: sync 2024-04-25T15:49:16.371 INFO:tasks.workunit.client.0.smithi110.stdout:7/72: creat f17 x:0 0 0 2024-04-25T15:49:16.371 INFO:tasks.workunit.client.0.smithi110.stdout:7/73: chown le 88667324 1 2024-04-25T15:49:16.373 INFO:tasks.workunit.client.0.smithi110.stdout:7/74: symlink l18 0 2024-04-25T15:49:16.375 INFO:tasks.workunit.client.0.smithi110.stdout:7/75: creat f19 x:0 0 0 2024-04-25T15:49:16.377 INFO:tasks.workunit.client.0.smithi110.stdout:7/76: creat f1a x:0 0 0 2024-04-25T15:49:16.380 INFO:tasks.workunit.client.0.smithi110.stdout:7/77: mkdir d1b 0 2024-04-25T15:49:16.380 INFO:tasks.workunit.client.0.smithi110.stdout:7/78: chown lf 357409853 1 2024-04-25T15:49:16.445 INFO:tasks.workunit.client.0.smithi110.stdout:6/18: chown d1/l6 273093 1 2024-04-25T15:49:16.449 INFO:tasks.workunit.client.0.smithi110.stdout:6/19: getdents d1 0 2024-04-25T15:49:16.449 INFO:tasks.workunit.client.0.smithi110.stdout:4/23: fsync f2 0 2024-04-25T15:49:16.449 INFO:tasks.workunit.client.0.smithi110.stdout:6/20: read d1/f3 [83795,60773] 0 2024-04-25T15:49:16.452 INFO:tasks.workunit.client.0.smithi110.stdout:2/38: rename f5 to f8 0 2024-04-25T15:49:16.452 INFO:tasks.workunit.client.0.smithi110.stdout:2/39: stat f4 0 2024-04-25T15:49:16.453 INFO:tasks.workunit.client.0.smithi110.stdout:4/24: mknod c4 0 2024-04-25T15:49:16.458 INFO:tasks.workunit.client.0.smithi110.stdout:4/25: symlink l5 0 2024-04-25T15:49:16.462 INFO:tasks.workunit.client.0.smithi110.stdout:6/21: link d1/l6 d1/l7 0 2024-04-25T15:49:16.462 INFO:tasks.workunit.client.0.smithi110.stdout:2/40: dwrite f4 [0,4194304] 0 2024-04-25T15:49:16.463 INFO:tasks.workunit.client.0.smithi110.stdout:6/22: chown d1/c2 7765491 1 2024-04-25T15:49:16.464 INFO:tasks.workunit.client.0.smithi110.stdout:4/26: dread f2 [0,4194304] 0 2024-04-25T15:49:16.465 INFO:tasks.workunit.client.0.smithi110.stdout:2/41: mknod c9 0 2024-04-25T15:49:16.470 INFO:tasks.workunit.client.0.smithi110.stdout:4/27: dwrite f2 [0,4194304] 0 2024-04-25T15:49:16.472 INFO:tasks.workunit.client.0.smithi110.stdout:4/28: truncate f2 5031968 0 2024-04-25T15:49:16.473 INFO:tasks.workunit.client.0.smithi110.stdout:2/42: link f8 fa 0 2024-04-25T15:49:16.478 INFO:tasks.workunit.client.0.smithi110.stdout:4/29: unlink l1 0 2024-04-25T15:49:16.479 INFO:tasks.workunit.client.0.smithi110.stdout:4/30: stat c4 0 2024-04-25T15:49:16.480 INFO:tasks.workunit.client.0.smithi110.stdout:6/23: link d1/l7 d1/l8 0 2024-04-25T15:49:16.485 INFO:tasks.workunit.client.0.smithi110.stdout:6/24: creat d1/f9 x:0 0 0 2024-04-25T15:49:16.487 INFO:tasks.workunit.client.0.smithi110.stdout:2/43: fdatasync f4 0 2024-04-25T15:49:16.491 INFO:tasks.workunit.client.0.smithi110.stdout:6/25: chown d1/l6 6334 1 2024-04-25T15:49:16.493 INFO:tasks.workunit.client.0.smithi110.stdout:2/44: dwrite f6 [0,4194304] 0 2024-04-25T15:49:16.494 INFO:tasks.workunit.client.0.smithi110.stdout:2/45: chown f7 105602 1 2024-04-25T15:49:16.495 INFO:tasks.workunit.client.0.smithi110.stdout:6/26: dread d1/f3 [0,4194304] 0 2024-04-25T15:49:16.495 INFO:tasks.workunit.client.0.smithi110.stdout:2/46: read f4 [4614170,48616] 0 2024-04-25T15:49:16.496 INFO:tasks.workunit.client.0.smithi110.stdout:2/47: chown f8 17340868 1 2024-04-25T15:49:16.496 INFO:tasks.workunit.client.0.smithi110.stdout:6/27: truncate d1/f9 466802 0 2024-04-25T15:49:16.498 INFO:tasks.workunit.client.0.smithi110.stdout:2/48: write f7 [4377458,76479] 0 2024-04-25T15:49:16.499 INFO:tasks.workunit.client.0.smithi110.stdout:6/28: mknod d1/ca 0 2024-04-25T15:49:16.503 INFO:tasks.workunit.client.0.smithi110.stdout:6/29: mkdir d1/db 0 2024-04-25T15:49:16.505 INFO:tasks.workunit.client.0.smithi110.stdout:2/49: dwrite fa [4194304,4194304] 0 2024-04-25T15:49:16.506 INFO:tasks.workunit.client.0.smithi110.stdout:2/50: write f8 [2581069,84990] 0 2024-04-25T15:49:16.506 INFO:tasks.workunit.client.0.smithi110.stdout:6/30: unlink d1/c2 0 2024-04-25T15:49:16.507 INFO:tasks.workunit.client.0.smithi110.stdout:6/31: truncate f0 4800472 0 2024-04-25T15:49:16.507 INFO:tasks.workunit.client.0.smithi110.stdout:2/51: write f4 [961079,17300] 0 2024-04-25T15:49:16.510 INFO:tasks.workunit.client.0.smithi110.stdout:1/26: sync 2024-04-25T15:49:16.514 INFO:tasks.workunit.client.0.smithi110.stdout:6/32: stat d1/l6 0 2024-04-25T15:49:16.517 INFO:tasks.workunit.client.0.smithi110.stdout:2/52: dwrite f8 [0,4194304] 0 2024-04-25T15:49:16.520 INFO:tasks.workunit.client.0.smithi110.stdout:3/61: truncate d0/d7/fa 1777405 0 2024-04-25T15:49:16.522 INFO:tasks.workunit.client.0.smithi110.stdout:2/53: rename f7 to fb 0 2024-04-25T15:49:16.524 INFO:tasks.workunit.client.0.smithi110.stdout:1/27: link d0/d3/l4 d0/d3/l9 0 2024-04-25T15:49:16.528 INFO:tasks.workunit.client.0.smithi110.stdout:6/33: dwrite d1/f3 [0,4194304] 0 2024-04-25T15:49:16.528 INFO:tasks.workunit.client.0.smithi110.stdout:2/54: unlink fa 0 2024-04-25T15:49:16.533 INFO:tasks.workunit.client.0.smithi110.stdout:3/62: dwrite d0/fd [0,4194304] 0 2024-04-25T15:49:16.538 INFO:tasks.workunit.client.0.smithi110.stdout:2/55: read f4 [6017426,127703] 0 2024-04-25T15:49:16.540 INFO:tasks.workunit.client.0.smithi110.stdout:1/28: dwrite d0/d3/f6 [0,4194304] 0 2024-04-25T15:49:16.541 INFO:tasks.workunit.client.0.smithi110.stdout:5/51: truncate d4/f16 2854142 0 2024-04-25T15:49:16.545 INFO:tasks.workunit.client.0.smithi110.stdout:1/29: creat d0/fa x:0 0 0 2024-04-25T15:49:16.547 INFO:tasks.workunit.client.0.smithi110.stdout:5/52: rmdir d4/de 39 2024-04-25T15:49:16.550 INFO:tasks.workunit.client.0.smithi110.stdout:3/63: creat d0/f15 x:0 0 0 2024-04-25T15:49:16.551 INFO:tasks.workunit.client.0.smithi110.stdout:1/30: mkdir d0/db 0 2024-04-25T15:49:16.553 INFO:tasks.workunit.client.0.smithi110.stdout:1/31: write d0/d3/f7 [500662,105541] 0 2024-04-25T15:49:16.561 INFO:tasks.workunit.client.0.smithi110.stdout:1/32: creat d0/db/fc x:0 0 0 2024-04-25T15:49:16.564 INFO:tasks.workunit.client.0.smithi110.stdout:8/46: truncate f2 2573788 0 2024-04-25T15:49:16.565 INFO:tasks.workunit.client.0.smithi110.stdout:1/33: creat d0/db/fd x:0 0 0 2024-04-25T15:49:16.566 INFO:tasks.workunit.client.0.smithi110.stdout:1/34: write d0/db/fc [632381,20780] 0 2024-04-25T15:49:16.567 INFO:tasks.workunit.client.0.smithi110.stdout:8/47: symlink la 0 2024-04-25T15:49:16.570 INFO:tasks.workunit.client.0.smithi110.stdout:1/35: dread d0/d3/f6 [0,4194304] 0 2024-04-25T15:49:16.571 INFO:tasks.workunit.client.0.smithi110.stdout:8/48: creat fb x:0 0 0 2024-04-25T15:49:16.573 INFO:tasks.workunit.client.0.smithi110.stdout:1/36: readlink d0/d3/l9 0 2024-04-25T15:49:16.576 INFO:tasks.workunit.client.0.smithi110.stdout:8/49: creat fc x:0 0 0 2024-04-25T15:49:16.578 INFO:tasks.workunit.client.0.smithi110.stdout:0/80: rmdir d1 39 2024-04-25T15:49:16.579 INFO:tasks.workunit.client.0.smithi110.stdout:8/50: symlink ld 0 2024-04-25T15:49:16.581 INFO:tasks.workunit.client.0.smithi110.stdout:0/81: dread - d1/d2/f11 zero size 2024-04-25T15:49:16.583 INFO:tasks.workunit.client.0.smithi110.stdout:8/51: creat fe x:0 0 0 2024-04-25T15:49:16.589 INFO:tasks.workunit.client.0.smithi110.stdout:8/52: dwrite f5 [4194304,4194304] 0 2024-04-25T15:49:16.597 INFO:tasks.workunit.client.0.smithi110.stdout:8/53: dwrite f5 [0,4194304] 0 2024-04-25T15:49:16.602 INFO:tasks.workunit.client.0.smithi110.stdout:8/54: creat ff x:0 0 0 2024-04-25T15:49:16.606 INFO:tasks.workunit.client.0.smithi110.stdout:8/55: mknod c10 0 2024-04-25T15:49:16.609 INFO:tasks.workunit.client.0.smithi110.stdout:8/56: creat f11 x:0 0 0 2024-04-25T15:49:16.615 INFO:tasks.workunit.client.0.smithi110.stdout:8/57: dwrite fc [0,4194304] 0 2024-04-25T15:49:16.620 INFO:tasks.workunit.client.0.smithi110.stdout:8/58: symlink l12 0 2024-04-25T15:49:16.623 INFO:tasks.workunit.client.0.smithi110.stdout:8/59: mknod c13 0 2024-04-25T15:49:16.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/60: dwrite fb [0,4194304] 0 2024-04-25T15:49:16.630 INFO:tasks.workunit.client.0.smithi110.stdout:8/61: stat la 0 2024-04-25T15:49:16.633 INFO:tasks.workunit.client.0.smithi110.stdout:7/79: sync 2024-04-25T15:49:16.755 INFO:tasks.workunit.client.0.smithi110.stdout:9/84: truncate d9/db/f11 2779820 0 2024-04-25T15:49:16.779 INFO:tasks.workunit.client.0.smithi110.stdout:4/31: truncate f2 4891409 0 2024-04-25T15:49:16.784 INFO:tasks.workunit.client.0.smithi110.stdout:6/34: rmdir d1 39 2024-04-25T15:49:16.788 INFO:tasks.workunit.client.0.smithi110.stdout:6/35: symlink d1/db/lc 0 2024-04-25T15:49:16.788 INFO:tasks.workunit.client.0.smithi110.stdout:6/36: write f0 [118947,80320] 0 2024-04-25T15:49:16.790 INFO:tasks.workunit.client.0.smithi110.stdout:6/37: write d1/f3 [1035981,117101] 0 2024-04-25T15:49:16.791 INFO:tasks.workunit.client.0.smithi110.stdout:6/38: fsync f0 0 2024-04-25T15:49:16.792 INFO:tasks.workunit.client.0.smithi110.stdout:6/39: chown f0 20 1 2024-04-25T15:49:16.799 INFO:tasks.workunit.client.0.smithi110.stdout:6/40: dwrite d1/f9 [0,4194304] 0 2024-04-25T15:49:16.804 INFO:tasks.workunit.client.0.smithi110.stdout:6/41: mkdir d1/dd 0 2024-04-25T15:49:16.805 INFO:tasks.workunit.client.0.smithi110.stdout:6/42: fdatasync f0 0 2024-04-25T15:49:16.807 INFO:tasks.workunit.client.0.smithi110.stdout:2/56: truncate f8 6293737 0 2024-04-25T15:49:16.807 INFO:tasks.workunit.client.0.smithi110.stdout:2/57: rmdir - no directory 2024-04-25T15:49:16.809 INFO:tasks.workunit.client.0.smithi110.stdout:6/43: dread d1/f3 [0,4194304] 0 2024-04-25T15:49:16.811 INFO:tasks.workunit.client.0.smithi110.stdout:2/58: symlink lc 0 2024-04-25T15:49:16.811 INFO:tasks.workunit.client.0.smithi110.stdout:2/59: rmdir - no directory 2024-04-25T15:49:16.814 INFO:tasks.workunit.client.0.smithi110.stdout:6/44: creat d1/fe x:0 0 0 2024-04-25T15:49:16.815 INFO:tasks.workunit.client.0.smithi110.stdout:6/45: write f0 [3629700,43639] 0 2024-04-25T15:49:16.815 INFO:tasks.workunit.client.0.smithi110.stdout:6/46: chown d1 31043 1 2024-04-25T15:49:16.817 INFO:tasks.workunit.client.0.smithi110.stdout:6/47: creat d1/ff x:0 0 0 2024-04-25T15:49:16.823 INFO:tasks.workunit.client.0.smithi110.stdout:1/37: rename d0/db to d0/d3/de 0 2024-04-25T15:49:16.824 INFO:tasks.workunit.client.0.smithi110.stdout:6/48: dwrite d1/fe [0,4194304] 0 2024-04-25T15:49:16.826 INFO:tasks.workunit.client.0.smithi110.stdout:1/38: readlink d0/d3/l4 0 2024-04-25T15:49:16.826 INFO:tasks.workunit.client.0.smithi110.stdout:1/39: stat d0/d3/l9 0 2024-04-25T15:49:16.827 INFO:tasks.workunit.client.0.smithi110.stdout:6/49: write d1/f3 [969902,99891] 0 2024-04-25T15:49:16.830 INFO:tasks.workunit.client.0.smithi110.stdout:1/40: unlink d0/d3/l9 0 2024-04-25T15:49:16.833 INFO:tasks.workunit.client.0.smithi110.stdout:1/41: mknod d0/cf 0 2024-04-25T15:49:16.835 INFO:tasks.workunit.client.0.smithi110.stdout:6/50: rename d1/l6 to d1/dd/l10 0 2024-04-25T15:49:16.835 INFO:tasks.workunit.client.0.smithi110.stdout:6/51: readlink d1/db/lc 0 2024-04-25T15:49:16.838 INFO:tasks.workunit.client.0.smithi110.stdout:6/52: mknod d1/c11 0 2024-04-25T15:49:16.842 INFO:tasks.workunit.client.0.smithi110.stdout:0/82: write d1/f1c [1171495,100779] 0 2024-04-25T15:49:16.843 INFO:tasks.workunit.client.0.smithi110.stdout:0/83: readlink d1/l1e 0 2024-04-25T15:49:16.846 INFO:tasks.workunit.client.0.smithi110.stdout:0/84: truncate d1/d2/f11 565162 0 2024-04-25T15:49:16.846 INFO:tasks.workunit.client.0.smithi110.stdout:6/53: dwrite d1/ff [0,4194304] 0 2024-04-25T15:49:16.847 INFO:tasks.workunit.client.0.smithi110.stdout:0/85: fdatasync d1/d2/f12 0 2024-04-25T15:49:16.849 INFO:tasks.workunit.client.0.smithi110.stdout:6/54: rmdir d1/dd 39 2024-04-25T15:49:16.850 INFO:tasks.workunit.client.0.smithi110.stdout:0/86: symlink d1/d2/l22 0 2024-04-25T15:49:16.854 INFO:tasks.workunit.client.0.smithi110.stdout:6/55: mkdir d1/d12 0 2024-04-25T15:49:16.855 INFO:tasks.workunit.client.0.smithi110.stdout:6/56: read d1/fe [3734969,56248] 0 2024-04-25T15:49:16.855 INFO:tasks.workunit.client.0.smithi110.stdout:0/87: mkdir d1/d23 0 2024-04-25T15:49:16.861 INFO:tasks.workunit.client.0.smithi110.stdout:0/88: creat d1/d23/f24 x:0 0 0 2024-04-25T15:49:16.863 INFO:tasks.workunit.client.0.smithi110.stdout:0/89: symlink d1/d23/l25 0 2024-04-25T15:49:16.863 INFO:tasks.workunit.client.0.smithi110.stdout:0/90: write d1/f1c [963183,108288] 0 2024-04-25T15:49:16.867 INFO:tasks.workunit.client.0.smithi110.stdout:0/91: mknod d1/d1a/c26 0 2024-04-25T15:49:16.867 INFO:tasks.workunit.client.0.smithi110.stdout:0/92: rename d1/d2 to d1/d2/d27 22 2024-04-25T15:49:16.867 INFO:tasks.workunit.client.0.smithi110.stdout:7/80: sync 2024-04-25T15:49:16.870 INFO:tasks.workunit.client.0.smithi110.stdout:7/81: mknod d1b/c1c 0 2024-04-25T15:49:16.873 INFO:tasks.workunit.client.0.smithi110.stdout:0/93: creat d1/f28 x:0 0 0 2024-04-25T15:49:16.875 INFO:tasks.workunit.client.0.smithi110.stdout:0/94: write d1/d2/ff [302159,13561] 0 2024-04-25T15:49:16.878 INFO:tasks.workunit.client.0.smithi110.stdout:7/82: dwrite fd [0,4194304] 0 2024-04-25T15:49:16.881 INFO:tasks.workunit.client.0.smithi110.stdout:7/83: symlink d1b/l1d 0 2024-04-25T15:49:16.884 INFO:tasks.workunit.client.0.smithi110.stdout:7/84: symlink d1b/l1e 0 2024-04-25T15:49:16.885 INFO:tasks.workunit.client.0.smithi110.stdout:0/95: dwrite d1/f19 [0,4194304] 0 2024-04-25T15:49:16.924 INFO:tasks.workunit.client.0.smithi110.stdout:9/85: rmdir d9/db 39 2024-04-25T15:49:16.927 INFO:tasks.workunit.client.0.smithi110.stdout:9/86: mknod d9/d13/c19 0 2024-04-25T15:49:16.929 INFO:tasks.workunit.client.0.smithi110.stdout:9/87: creat d9/d13/f1a x:0 0 0 2024-04-25T15:49:16.938 INFO:tasks.workunit.client.0.smithi110.stdout:4/32: dread f2 [0,4194304] 0 2024-04-25T15:49:16.947 INFO:tasks.workunit.client.0.smithi110.stdout:3/64: dwrite d0/d7/fa [0,4194304] 0 2024-04-25T15:49:16.948 INFO:tasks.workunit.client.0.smithi110.stdout:5/53: link d4/f16 d4/de/f17 0 2024-04-25T15:49:16.948 INFO:tasks.workunit.client.0.smithi110.stdout:4/33: dwrite f2 [0,4194304] 0 2024-04-25T15:49:16.950 INFO:tasks.workunit.client.0.smithi110.stdout:4/34: read f2 [3747179,66267] 0 2024-04-25T15:49:16.952 INFO:tasks.workunit.client.0.smithi110.stdout:0/96: dread d1/f8 [0,4194304] 0 2024-04-25T15:49:16.955 INFO:tasks.workunit.client.0.smithi110.stdout:6/57: creat d1/dd/f13 x:0 0 0 2024-04-25T15:49:16.958 INFO:tasks.workunit.client.0.smithi110.stdout:4/35: dread f2 [4194304,4194304] 0 2024-04-25T15:49:16.958 INFO:tasks.workunit.client.0.smithi110.stdout:5/54: unlink d4/fa 0 2024-04-25T15:49:16.962 INFO:tasks.workunit.client.0.smithi110.stdout:2/60: dread fb [0,4194304] 0 2024-04-25T15:49:16.963 INFO:tasks.workunit.client.0.smithi110.stdout:0/97: mknod d1/d23/c29 0 2024-04-25T15:49:16.964 INFO:tasks.workunit.client.0.smithi110.stdout:3/65: dwrite d0/d7/fc [0,4194304] 0 2024-04-25T15:49:16.964 INFO:tasks.workunit.client.0.smithi110.stdout:0/98: stat d1/d23/c29 0 2024-04-25T15:49:16.964 INFO:tasks.workunit.client.0.smithi110.stdout:3/66: fdatasync d0/ff 0 2024-04-25T15:49:16.973 INFO:tasks.workunit.client.0.smithi110.stdout:6/58: dwrite d1/fe [0,4194304] 0 2024-04-25T15:49:16.975 INFO:tasks.workunit.client.0.smithi110.stdout:0/99: dwrite d1/f19 [0,4194304] 0 2024-04-25T15:49:16.975 INFO:tasks.workunit.client.0.smithi110.stdout:0/100: truncate d1/f28 169512 0 2024-04-25T15:49:16.984 INFO:tasks.workunit.client.0.smithi110.stdout:0/101: dwrite d1/f28 [0,4194304] 0 2024-04-25T15:49:16.985 INFO:tasks.workunit.client.0.smithi110.stdout:1/42: truncate d0/d3/f7 453434 0 2024-04-25T15:49:16.986 INFO:tasks.workunit.client.0.smithi110.stdout:0/102: chown d1/d2/f12 484120 1 2024-04-25T15:49:16.988 INFO:tasks.workunit.client.0.smithi110.stdout:4/36: rename f2 to f6 0 2024-04-25T15:49:16.989 INFO:tasks.workunit.client.0.smithi110.stdout:5/55: symlink d4/d15/l18 0 2024-04-25T15:49:16.990 INFO:tasks.workunit.client.0.smithi110.stdout:5/56: fsync d4/f9 0 2024-04-25T15:49:16.992 INFO:tasks.workunit.client.0.smithi110.stdout:6/59: mknod d1/db/c14 0 2024-04-25T15:49:16.992 INFO:tasks.workunit.client.0.smithi110.stdout:4/37: dread f6 [0,4194304] 0 2024-04-25T15:49:16.996 INFO:tasks.workunit.client.0.smithi110.stdout:8/62: truncate f2 1886589 0 2024-04-25T15:49:16.997 INFO:tasks.workunit.client.0.smithi110.stdout:8/63: truncate f3 4645793 0 2024-04-25T15:49:16.997 INFO:tasks.workunit.client.0.smithi110.stdout:3/67: mkdir d0/d16 0 2024-04-25T15:49:16.999 INFO:tasks.workunit.client.0.smithi110.stdout:5/57: creat d4/de/f19 x:0 0 0 2024-04-25T15:49:17.002 INFO:tasks.workunit.client.0.smithi110.stdout:5/58: fdatasync d4/f10 0 2024-04-25T15:49:17.003 INFO:tasks.workunit.client.0.smithi110.stdout:4/38: symlink l7 0 2024-04-25T15:49:17.006 INFO:tasks.workunit.client.0.smithi110.stdout:8/64: dwrite f11 [0,4194304] 0 2024-04-25T15:49:17.007 INFO:tasks.workunit.client.0.smithi110.stdout:8/65: readlink la 0 2024-04-25T15:49:17.008 INFO:tasks.workunit.client.0.smithi110.stdout:3/68: dwrite d0/f6 [0,4194304] 0 2024-04-25T15:49:17.009 INFO:tasks.workunit.client.0.smithi110.stdout:4/39: write f6 [944761,30494] 0 2024-04-25T15:49:17.009 INFO:tasks.workunit.client.0.smithi110.stdout:3/69: stat d0/d16 0 2024-04-25T15:49:17.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/40: write f6 [5095331,111697] 0 2024-04-25T15:49:17.011 INFO:tasks.workunit.client.0.smithi110.stdout:5/59: creat d4/d15/f1a x:0 0 0 2024-04-25T15:49:17.012 INFO:tasks.workunit.client.0.smithi110.stdout:4/41: write f6 [1165754,63378] 0 2024-04-25T15:49:17.013 INFO:tasks.workunit.client.0.smithi110.stdout:8/66: mkdir d14 0 2024-04-25T15:49:17.015 INFO:tasks.workunit.client.0.smithi110.stdout:8/67: write f11 [560504,638] 0 2024-04-25T15:49:17.016 INFO:tasks.workunit.client.0.smithi110.stdout:8/68: write fe [438170,65507] 0 2024-04-25T15:49:17.016 INFO:tasks.workunit.client.0.smithi110.stdout:3/70: dread d0/f4 [0,4194304] 0 2024-04-25T15:49:17.018 INFO:tasks.workunit.client.0.smithi110.stdout:7/85: write fd [4723219,22346] 0 2024-04-25T15:49:17.023 INFO:tasks.workunit.client.0.smithi110.stdout:7/86: rename l16 to d1b/l1f 0 2024-04-25T15:49:17.025 INFO:tasks.workunit.client.0.smithi110.stdout:3/71: dwrite d0/f4 [0,4194304] 0 2024-04-25T15:49:17.027 INFO:tasks.workunit.client.0.smithi110.stdout:7/87: dread f4 [0,4194304] 0 2024-04-25T15:49:17.113 INFO:tasks.workunit.client.0.smithi110.stdout:9/88: write f8 [8841861,56091] 0 2024-04-25T15:49:17.113 INFO:tasks.workunit.client.0.smithi110.stdout:3/72: write d0/d7/fa [4744546,127771] 0 2024-04-25T15:49:17.113 INFO:tasks.workunit.client.0.smithi110.stdout:2/61: write fb [3506952,71714] 0 2024-04-25T15:49:17.114 INFO:tasks.workunit.client.0.smithi110.stdout:9/89: chown f6 437 1 2024-04-25T15:49:17.115 INFO:tasks.workunit.client.0.smithi110.stdout:2/62: fdatasync f6 0 2024-04-25T15:49:17.116 INFO:tasks.workunit.client.0.smithi110.stdout:7/88: rename c10 to d1b/c20 0 2024-04-25T15:49:17.119 INFO:tasks.workunit.client.0.smithi110.stdout:9/90: fsync d9/db/df/f17 0 2024-04-25T15:49:17.120 INFO:tasks.workunit.client.0.smithi110.stdout:2/63: creat fd x:0 0 0 2024-04-25T15:49:17.121 INFO:tasks.workunit.client.0.smithi110.stdout:2/64: readlink lc 0 2024-04-25T15:49:17.121 INFO:tasks.workunit.client.0.smithi110.stdout:9/91: dread - d9/d13/f1a zero size 2024-04-25T15:49:17.123 INFO:tasks.workunit.client.0.smithi110.stdout:9/92: write f0 [1796682,97810] 0 2024-04-25T15:49:17.124 INFO:tasks.workunit.client.0.smithi110.stdout:2/65: mkdir de 0 2024-04-25T15:49:17.124 INFO:tasks.workunit.client.0.smithi110.stdout:7/89: link l18 d1b/l21 0 2024-04-25T15:49:17.124 INFO:tasks.workunit.client.0.smithi110.stdout:3/73: dwrite d0/d7/fe [0,4194304] 0 2024-04-25T15:49:17.125 INFO:tasks.workunit.client.0.smithi110.stdout:7/90: truncate f2 5170417 0 2024-04-25T15:49:17.126 INFO:tasks.workunit.client.0.smithi110.stdout:7/91: chown f19 6212 1 2024-04-25T15:49:17.127 INFO:tasks.workunit.client.0.smithi110.stdout:7/92: chown f11 0 1 2024-04-25T15:49:17.129 INFO:tasks.workunit.client.0.smithi110.stdout:2/66: dread f6 [0,4194304] 0 2024-04-25T15:49:17.129 INFO:tasks.workunit.client.0.smithi110.stdout:3/74: dread d0/fd [0,4194304] 0 2024-04-25T15:49:17.129 INFO:tasks.workunit.client.0.smithi110.stdout:9/93: dread f1 [0,4194304] 0 2024-04-25T15:49:17.130 INFO:tasks.workunit.client.0.smithi110.stdout:9/94: readlink d9/l10 0 2024-04-25T15:49:17.132 INFO:tasks.workunit.client.0.smithi110.stdout:3/75: unlink d0/f6 0 2024-04-25T15:49:17.132 INFO:tasks.workunit.client.0.smithi110.stdout:7/93: link c12 d1b/c22 0 2024-04-25T15:49:17.133 INFO:tasks.workunit.client.0.smithi110.stdout:7/94: read - fb zero size 2024-04-25T15:49:17.134 INFO:tasks.workunit.client.0.smithi110.stdout:3/76: read d0/d7/fa [887462,17434] 0 2024-04-25T15:49:17.136 INFO:tasks.workunit.client.0.smithi110.stdout:7/95: mknod d1b/c23 0 2024-04-25T15:49:17.137 INFO:tasks.workunit.client.0.smithi110.stdout:3/77: chown d0/l5 60320651 1 2024-04-25T15:49:17.139 INFO:tasks.workunit.client.0.smithi110.stdout:2/67: dwrite fd [0,4194304] 0 2024-04-25T15:49:17.141 INFO:tasks.workunit.client.0.smithi110.stdout:3/78: creat d0/d7/f17 x:0 0 0 2024-04-25T15:49:17.141 INFO:tasks.workunit.client.0.smithi110.stdout:9/95: dwrite d9/d13/f1a [0,4194304] 0 2024-04-25T15:49:17.143 INFO:tasks.workunit.client.0.smithi110.stdout:2/68: mknod de/cf 0 2024-04-25T15:49:17.144 INFO:tasks.workunit.client.0.smithi110.stdout:2/69: read fb [2654761,10388] 0 2024-04-25T15:49:17.149 INFO:tasks.workunit.client.0.smithi110.stdout:2/70: creat de/f10 x:0 0 0 2024-04-25T15:49:17.150 INFO:tasks.workunit.client.0.smithi110.stdout:7/96: dread fd [0,4194304] 0 2024-04-25T15:49:17.150 INFO:tasks.workunit.client.0.smithi110.stdout:2/71: fdatasync f6 0 2024-04-25T15:49:17.152 INFO:tasks.workunit.client.0.smithi110.stdout:7/97: write f1a [182702,34242] 0 2024-04-25T15:49:17.153 INFO:tasks.workunit.client.0.smithi110.stdout:9/96: dwrite d9/db/df/f17 [4194304,4194304] 0 2024-04-25T15:49:17.155 INFO:tasks.workunit.client.0.smithi110.stdout:7/98: creat d1b/f24 x:0 0 0 2024-04-25T15:49:17.155 INFO:tasks.workunit.client.0.smithi110.stdout:9/97: write d9/fd [2134766,128764] 0 2024-04-25T15:49:17.155 INFO:tasks.workunit.client.0.smithi110.stdout:7/99: dread - f17 zero size 2024-04-25T15:49:17.157 INFO:tasks.workunit.client.0.smithi110.stdout:9/98: chown d9/db/df/l16 0 1 2024-04-25T15:49:17.159 INFO:tasks.workunit.client.0.smithi110.stdout:9/99: chown d9/db/c12 165216 1 2024-04-25T15:49:17.159 INFO:tasks.workunit.client.0.smithi110.stdout:9/100: chown d9/d13/f1a 2039 1 2024-04-25T15:49:17.162 INFO:tasks.workunit.client.0.smithi110.stdout:9/101: rename f1 to d9/db/f1b 0 2024-04-25T15:49:17.233 INFO:tasks.workunit.client.0.smithi110.stdout:5/60: fsync d4/de/f19 0 2024-04-25T15:49:17.238 INFO:tasks.workunit.client.0.smithi110.stdout:6/60: truncate d1/fe 3568028 0 2024-04-25T15:49:17.240 INFO:tasks.workunit.client.0.smithi110.stdout:6/61: mknod d1/c15 0 2024-04-25T15:49:17.242 INFO:tasks.workunit.client.0.smithi110.stdout:6/62: symlink d1/db/l16 0 2024-04-25T15:49:17.243 INFO:tasks.workunit.client.0.smithi110.stdout:4/42: truncate f6 3551750 0 2024-04-25T15:49:17.244 INFO:tasks.workunit.client.0.smithi110.stdout:6/63: creat d1/dd/f17 x:0 0 0 2024-04-25T15:49:17.254 INFO:tasks.workunit.client.0.smithi110.stdout:3/79: getdents d0 0 2024-04-25T15:49:17.256 INFO:tasks.workunit.client.0.smithi110.stdout:3/80: chown d0/l10 0 1 2024-04-25T15:49:17.257 INFO:tasks.workunit.client.0.smithi110.stdout:3/81: write d0/f15 [267064,55014] 0 2024-04-25T15:49:17.260 INFO:tasks.workunit.client.0.smithi110.stdout:9/102: dwrite d9/db/f1b [4194304,4194304] 0 2024-04-25T15:49:17.260 INFO:tasks.workunit.client.0.smithi110.stdout:2/72: rmdir de 39 2024-04-25T15:49:17.263 INFO:tasks.workunit.client.0.smithi110.stdout:7/100: truncate fd 1245157 0 2024-04-25T15:49:17.267 INFO:tasks.workunit.client.0.smithi110.stdout:9/103: rmdir d9 39 2024-04-25T15:49:17.269 INFO:tasks.workunit.client.0.smithi110.stdout:1/43: write d0/d3/f7 [276662,129856] 0 2024-04-25T15:49:17.271 INFO:tasks.workunit.client.0.smithi110.stdout:3/82: dwrite d0/f4 [4194304,4194304] 0 2024-04-25T15:49:17.274 INFO:tasks.workunit.client.0.smithi110.stdout:7/101: dread f2 [0,4194304] 0 2024-04-25T15:49:17.274 INFO:tasks.workunit.client.0.smithi110.stdout:5/61: write d4/f16 [424088,73997] 0 2024-04-25T15:49:17.281 INFO:tasks.workunit.client.0.smithi110.stdout:9/104: dwrite f6 [4194304,4194304] 0 2024-04-25T15:49:17.282 INFO:tasks.workunit.client.0.smithi110.stdout:7/102: creat d1b/f25 x:0 0 0 2024-04-25T15:49:17.284 INFO:tasks.workunit.client.0.smithi110.stdout:1/44: link d0/fa d0/f10 0 2024-04-25T15:49:17.285 INFO:tasks.workunit.client.0.smithi110.stdout:6/64: dread d1/fe [0,4194304] 0 2024-04-25T15:49:17.286 INFO:tasks.workunit.client.0.smithi110.stdout:7/103: dread f1a [0,4194304] 0 2024-04-25T15:49:17.288 INFO:tasks.workunit.client.0.smithi110.stdout:6/65: chown d1/fe 7 1 2024-04-25T15:49:17.289 INFO:tasks.workunit.client.0.smithi110.stdout:3/83: dwrite d0/ff [0,4194304] 0 2024-04-25T15:49:17.289 INFO:tasks.workunit.client.0.smithi110.stdout:7/104: chown d1b/f25 4624 1 2024-04-25T15:49:17.294 INFO:tasks.workunit.client.0.smithi110.stdout:1/45: symlink d0/l11 0 2024-04-25T15:49:17.295 INFO:tasks.workunit.client.0.smithi110.stdout:3/84: dread d0/f4 [0,4194304] 0 2024-04-25T15:49:17.296 INFO:tasks.workunit.client.0.smithi110.stdout:3/85: write d0/d7/fa [1791825,15429] 0 2024-04-25T15:49:17.301 INFO:tasks.workunit.client.0.smithi110.stdout:1/46: dread d0/d3/f6 [0,4194304] 0 2024-04-25T15:49:17.303 INFO:tasks.workunit.client.0.smithi110.stdout:9/105: creat d9/d13/f1c x:0 0 0 2024-04-25T15:49:17.305 INFO:tasks.workunit.client.0.smithi110.stdout:2/73: dwrite f4 [4194304,4194304] 0 2024-04-25T15:49:17.305 INFO:tasks.workunit.client.0.smithi110.stdout:7/105: unlink d1b/f25 0 2024-04-25T15:49:17.306 INFO:tasks.workunit.client.0.smithi110.stdout:7/106: chown f14 0 1 2024-04-25T15:49:17.306 INFO:tasks.workunit.client.0.smithi110.stdout:2/74: write f8 [8044706,88204] 0 2024-04-25T15:49:17.307 INFO:tasks.workunit.client.0.smithi110.stdout:3/86: rmdir d0 39 2024-04-25T15:49:17.311 INFO:tasks.workunit.client.0.smithi110.stdout:1/47: mknod d0/d3/c12 0 2024-04-25T15:49:17.316 INFO:tasks.workunit.client.0.smithi110.stdout:7/107: creat d1b/f26 x:0 0 0 2024-04-25T15:49:17.318 INFO:tasks.workunit.client.0.smithi110.stdout:3/87: fsync d0/f15 0 2024-04-25T15:49:17.318 INFO:tasks.workunit.client.0.smithi110.stdout:3/88: chown d0/l10 29 1 2024-04-25T15:49:17.320 INFO:tasks.workunit.client.0.smithi110.stdout:1/48: creat d0/d3/de/f13 x:0 0 0 2024-04-25T15:49:17.321 INFO:tasks.workunit.client.0.smithi110.stdout:1/49: dread - d0/f10 zero size 2024-04-25T15:49:17.321 INFO:tasks.workunit.client.0.smithi110.stdout:2/75: mkdir de/d11 0 2024-04-25T15:49:17.322 INFO:tasks.workunit.client.0.smithi110.stdout:5/62: truncate d4/de/f17 1844348 0 2024-04-25T15:49:17.327 INFO:tasks.workunit.client.0.smithi110.stdout:5/63: dread d4/fd [0,4194304] 0 2024-04-25T15:49:17.327 INFO:tasks.workunit.client.0.smithi110.stdout:4/43: truncate f6 2085086 0 2024-04-25T15:49:17.328 INFO:tasks.workunit.client.0.smithi110.stdout:4/44: chown l7 1769838 1 2024-04-25T15:49:17.332 INFO:tasks.workunit.client.0.smithi110.stdout:2/76: symlink de/l12 0 2024-04-25T15:49:17.333 INFO:tasks.workunit.client.0.smithi110.stdout:2/77: chown f4 594 1 2024-04-25T15:49:17.334 INFO:tasks.workunit.client.0.smithi110.stdout:2/78: write f4 [4127831,14845] 0 2024-04-25T15:49:17.334 INFO:tasks.workunit.client.0.smithi110.stdout:2/79: rename de to de/d13 22 2024-04-25T15:49:17.337 INFO:tasks.workunit.client.0.smithi110.stdout:5/64: unlink d4/c5 0 2024-04-25T15:49:17.338 INFO:tasks.workunit.client.0.smithi110.stdout:5/65: readlink d4/lc 0 2024-04-25T15:49:17.339 INFO:tasks.workunit.client.0.smithi110.stdout:4/45: creat f8 x:0 0 0 2024-04-25T15:49:17.344 INFO:tasks.workunit.client.0.smithi110.stdout:2/80: symlink de/l14 0 2024-04-25T15:49:17.345 INFO:tasks.workunit.client.0.smithi110.stdout:5/66: rename d4/de/l11 to d4/de/l1b 0 2024-04-25T15:49:17.346 INFO:tasks.workunit.client.0.smithi110.stdout:4/46: mknod c9 0 2024-04-25T15:49:17.347 INFO:tasks.workunit.client.0.smithi110.stdout:2/81: dread fd [0,4194304] 0 2024-04-25T15:49:17.379 INFO:tasks.workunit.client.0.smithi110.stdout:4/47: creat fa x:0 0 0 2024-04-25T15:49:17.380 INFO:tasks.workunit.client.0.smithi110.stdout:3/89: truncate d0/ff 3150536 0 2024-04-25T15:49:17.383 INFO:tasks.workunit.client.0.smithi110.stdout:4/48: rename l7 to lb 0 2024-04-25T15:49:17.384 INFO:tasks.workunit.client.0.smithi110.stdout:4/49: read - fa zero size 2024-04-25T15:49:17.387 INFO:tasks.workunit.client.0.smithi110.stdout:3/90: creat d0/d16/f18 x:0 0 0 2024-04-25T15:49:17.387 INFO:tasks.workunit.client.0.smithi110.stdout:1/50: dwrite d0/f10 [0,4194304] 0 2024-04-25T15:49:17.388 INFO:tasks.workunit.client.0.smithi110.stdout:4/50: creat fc x:0 0 0 2024-04-25T15:49:17.389 INFO:tasks.workunit.client.0.smithi110.stdout:2/82: dwrite f6 [0,4194304] 0 2024-04-25T15:49:17.389 INFO:tasks.workunit.client.0.smithi110.stdout:1/51: chown d0/fa 1287 1 2024-04-25T15:49:17.391 INFO:tasks.workunit.client.0.smithi110.stdout:3/91: write d0/d7/fa [224245,28619] 0 2024-04-25T15:49:17.398 INFO:tasks.workunit.client.0.smithi110.stdout:3/92: symlink d0/l19 0 2024-04-25T15:49:17.399 INFO:tasks.workunit.client.0.smithi110.stdout:2/83: symlink de/d11/l15 0 2024-04-25T15:49:17.403 INFO:tasks.workunit.client.0.smithi110.stdout:2/84: mkdir de/d11/d16 0 2024-04-25T15:49:17.404 INFO:tasks.workunit.client.0.smithi110.stdout:2/85: read f4 [3987394,13716] 0 2024-04-25T15:49:17.405 INFO:tasks.workunit.client.0.smithi110.stdout:1/52: getdents d0/d3/de 0 2024-04-25T15:49:17.406 INFO:tasks.workunit.client.0.smithi110.stdout:1/53: read d0/d3/de/fc [481558,110418] 0 2024-04-25T15:49:17.406 INFO:tasks.workunit.client.0.smithi110.stdout:2/86: write fd [3038471,43123] 0 2024-04-25T15:49:17.409 INFO:tasks.workunit.client.0.smithi110.stdout:1/54: mkdir d0/d14 0 2024-04-25T15:49:17.411 INFO:tasks.workunit.client.0.smithi110.stdout:1/55: creat d0/f15 x:0 0 0 2024-04-25T15:49:17.412 INFO:tasks.workunit.client.0.smithi110.stdout:1/56: fsync d0/f10 0 2024-04-25T15:49:17.416 INFO:tasks.workunit.client.0.smithi110.stdout:2/87: dwrite f4 [0,4194304] 0 2024-04-25T15:49:17.418 INFO:tasks.workunit.client.0.smithi110.stdout:2/88: chown de/d11/l15 1397885 1 2024-04-25T15:49:17.420 INFO:tasks.workunit.client.0.smithi110.stdout:3/93: stat d0/d16 0 2024-04-25T15:49:17.423 INFO:tasks.workunit.client.0.smithi110.stdout:1/57: dwrite d0/d3/de/fc [0,4194304] 0 2024-04-25T15:49:17.424 INFO:tasks.workunit.client.0.smithi110.stdout:1/58: write d0/d3/de/fc [3657528,56251] 0 2024-04-25T15:49:17.425 INFO:tasks.workunit.client.0.smithi110.stdout:3/94: chown d0/d7/f17 448105 1 2024-04-25T15:49:17.426 INFO:tasks.workunit.client.0.smithi110.stdout:1/59: write d0/f10 [2932444,85844] 0 2024-04-25T15:49:17.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/89: rmdir de/d11/d16 0 2024-04-25T15:49:17.428 INFO:tasks.workunit.client.0.smithi110.stdout:2/90: chown de 858317556 1 2024-04-25T15:49:17.432 INFO:tasks.workunit.client.0.smithi110.stdout:2/91: mknod de/d11/c17 0 2024-04-25T15:49:17.433 INFO:tasks.workunit.client.0.smithi110.stdout:1/60: write d0/d3/de/fc [2013025,71635] 0 2024-04-25T15:49:17.435 INFO:tasks.workunit.client.0.smithi110.stdout:2/92: creat de/d11/f18 x:0 0 0 2024-04-25T15:49:17.436 INFO:tasks.workunit.client.0.smithi110.stdout:1/61: mknod d0/c16 0 2024-04-25T15:49:17.439 INFO:tasks.workunit.client.0.smithi110.stdout:2/93: dread f6 [0,4194304] 0 2024-04-25T15:49:17.444 INFO:tasks.workunit.client.0.smithi110.stdout:2/94: link f4 de/d11/f19 0 2024-04-25T15:49:17.448 INFO:tasks.workunit.client.0.smithi110.stdout:2/95: mknod de/d11/c1a 0 2024-04-25T15:49:17.448 INFO:tasks.workunit.client.0.smithi110.stdout:2/96: chown fd 0 1 2024-04-25T15:49:17.453 INFO:tasks.workunit.client.0.smithi110.stdout:2/97: dread fd [0,4194304] 0 2024-04-25T15:49:17.455 INFO:tasks.workunit.client.0.smithi110.stdout:1/62: truncate d0/d3/f7 91716 0 2024-04-25T15:49:17.458 INFO:tasks.workunit.client.0.smithi110.stdout:3/95: write d0/ff [87730,68159] 0 2024-04-25T15:49:17.459 INFO:tasks.workunit.client.0.smithi110.stdout:3/96: fsync d0/d16/f18 0 2024-04-25T15:49:17.462 INFO:tasks.workunit.client.0.smithi110.stdout:3/97: mknod d0/d7/c1a 0 2024-04-25T15:49:17.462 INFO:tasks.workunit.client.0.smithi110.stdout:1/63: dwrite d0/fa [0,4194304] 0 2024-04-25T15:49:17.462 INFO:tasks.workunit.client.0.smithi110.stdout:3/98: fdatasync d0/d16/f18 0 2024-04-25T15:49:17.465 INFO:tasks.workunit.client.0.smithi110.stdout:3/99: fsync d0/f15 0 2024-04-25T15:49:17.469 INFO:tasks.workunit.client.0.smithi110.stdout:1/64: dwrite d0/d3/de/fd [0,4194304] 0 2024-04-25T15:49:17.474 INFO:tasks.workunit.client.0.smithi110.stdout:1/65: symlink d0/d14/l17 0 2024-04-25T15:49:17.482 INFO:tasks.workunit.client.0.smithi110.stdout:1/66: rename d0/d3/f7 to d0/f18 0 2024-04-25T15:49:17.490 INFO:tasks.workunit.client.0.smithi110.stdout:1/67: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:17.494 INFO:tasks.workunit.client.0.smithi110.stdout:1/68: rename d0/l11 to d0/l19 0 2024-04-25T15:49:17.503 INFO:tasks.workunit.client.0.smithi110.stdout:1/69: dwrite d0/d3/f6 [0,4194304] 0 2024-04-25T15:49:17.504 INFO:tasks.workunit.client.0.smithi110.stdout:1/70: write d0/d3/de/fc [617583,43609] 0 2024-04-25T15:49:17.507 INFO:tasks.workunit.client.0.smithi110.stdout:1/71: dread d0/f18 [0,4194304] 0 2024-04-25T15:49:17.511 INFO:tasks.workunit.client.0.smithi110.stdout:1/72: creat d0/d14/f1a x:0 0 0 2024-04-25T15:49:17.513 INFO:tasks.workunit.client.0.smithi110.stdout:1/73: mknod d0/d3/c1b 0 2024-04-25T15:49:17.514 INFO:tasks.workunit.client.0.smithi110.stdout:1/74: dread - d0/d3/de/f13 zero size 2024-04-25T15:49:17.521 INFO:tasks.workunit.client.0.smithi110.stdout:1/75: dwrite d0/f15 [4194304,4194304] 0 2024-04-25T15:49:17.525 INFO:tasks.workunit.client.0.smithi110.stdout:1/76: rename d0/d3/de/fd to d0/d3/f1c 0 2024-04-25T15:49:17.528 INFO:tasks.workunit.client.0.smithi110.stdout:1/77: creat d0/d14/f1d x:0 0 0 2024-04-25T15:49:17.537 INFO:tasks.workunit.client.0.smithi110.stdout:1/78: dwrite d0/d14/f1a [0,4194304] 0 2024-04-25T15:49:17.537 INFO:tasks.workunit.client.0.smithi110.stdout:1/79: chown d0/d3/de 408 1 2024-04-25T15:49:17.538 INFO:tasks.workunit.client.0.smithi110.stdout:1/80: fsync d0/d3/f6 0 2024-04-25T15:49:17.539 INFO:tasks.workunit.client.0.smithi110.stdout:1/81: write d0/f15 [1048571,61424] 0 2024-04-25T15:49:17.540 INFO:tasks.workunit.client.0.smithi110.stdout:1/82: chown d0/c8 115285 1 2024-04-25T15:49:17.548 INFO:tasks.workunit.client.0.smithi110.stdout:1/83: dwrite d0/d3/de/fc [4194304,4194304] 0 2024-04-25T15:49:17.550 INFO:tasks.workunit.client.0.smithi110.stdout:1/84: mknod d0/d3/c1e 0 2024-04-25T15:49:17.554 INFO:tasks.workunit.client.0.smithi110.stdout:1/85: dread d0/f10 [0,4194304] 0 2024-04-25T15:49:17.555 INFO:tasks.workunit.client.0.smithi110.stdout:1/86: write d0/d3/de/f13 [439359,117764] 0 2024-04-25T15:49:17.557 INFO:tasks.workunit.client.0.smithi110.stdout:1/87: creat d0/d14/f1f x:0 0 0 2024-04-25T15:49:17.560 INFO:tasks.workunit.client.0.smithi110.stdout:0/103: sync 2024-04-25T15:49:17.595 INFO:tasks.workunit.client.0.smithi110.stdout:1/88: truncate d0/d14/f1a 708396 0 2024-04-25T15:49:17.596 INFO:tasks.workunit.client.0.smithi110.stdout:0/104: truncate d1/d2/f11 93722 0 2024-04-25T15:49:17.600 INFO:tasks.workunit.client.0.smithi110.stdout:1/89: unlink d0/l19 0 2024-04-25T15:49:17.600 INFO:tasks.workunit.client.0.smithi110.stdout:1/90: read d0/f15 [2943157,28604] 0 2024-04-25T15:49:17.603 INFO:tasks.workunit.client.0.smithi110.stdout:0/105: creat d1/f2a x:0 0 0 2024-04-25T15:49:17.604 INFO:tasks.workunit.client.0.smithi110.stdout:1/91: mkdir d0/d20 0 2024-04-25T15:49:17.604 INFO:tasks.workunit.client.0.smithi110.stdout:1/92: chown d0/d14 29463040 1 2024-04-25T15:49:17.605 INFO:tasks.workunit.client.0.smithi110.stdout:1/93: rename d0/d3 to d0/d3/d21 22 2024-04-25T15:49:17.607 INFO:tasks.workunit.client.0.smithi110.stdout:0/106: unlink d1/f15 0 2024-04-25T15:49:17.608 INFO:tasks.workunit.client.0.smithi110.stdout:1/94: dread d0/d3/de/f13 [0,4194304] 0 2024-04-25T15:49:17.608 INFO:tasks.workunit.client.0.smithi110.stdout:0/107: chown d1/d23/f24 21532 1 2024-04-25T15:49:17.609 INFO:tasks.workunit.client.0.smithi110.stdout:1/95: chown d0/d3/f6 245551 1 2024-04-25T15:49:17.613 INFO:tasks.workunit.client.0.smithi110.stdout:0/108: dwrite d1/f28 [0,4194304] 0 2024-04-25T15:49:17.614 INFO:tasks.workunit.client.0.smithi110.stdout:1/96: getdents d0/d20 0 2024-04-25T15:49:17.615 INFO:tasks.workunit.client.0.smithi110.stdout:1/97: chown d0/f10 6971 1 2024-04-25T15:49:17.615 INFO:tasks.workunit.client.0.smithi110.stdout:1/98: read - d0/d14/f1f zero size 2024-04-25T15:49:17.639 INFO:tasks.workunit.client.0.smithi110.stdout:0/109: write d1/f8 [2671903,127971] 0 2024-04-25T15:49:17.640 INFO:tasks.workunit.client.0.smithi110.stdout:0/110: fsync d1/f1c 0 2024-04-25T15:49:17.643 INFO:tasks.workunit.client.0.smithi110.stdout:1/99: truncate d0/fa 1722570 0 2024-04-25T15:49:17.644 INFO:tasks.workunit.client.0.smithi110.stdout:0/111: creat d1/d23/f2b x:0 0 0 2024-04-25T15:49:17.645 INFO:tasks.workunit.client.0.smithi110.stdout:0/112: chown d1/fd 255 1 2024-04-25T15:49:17.647 INFO:tasks.workunit.client.0.smithi110.stdout:1/100: dread d0/d3/de/fc [0,4194304] 0 2024-04-25T15:49:17.648 INFO:tasks.workunit.client.0.smithi110.stdout:0/113: mknod d1/d23/c2c 0 2024-04-25T15:49:17.650 INFO:tasks.workunit.client.0.smithi110.stdout:1/101: mknod d0/d3/de/c22 0 2024-04-25T15:49:17.652 INFO:tasks.workunit.client.0.smithi110.stdout:0/114: rename d1/d2/ff to d1/d1a/f2d 0 2024-04-25T15:49:17.657 INFO:tasks.workunit.client.0.smithi110.stdout:0/115: mknod d1/d1a/c2e 0 2024-04-25T15:49:17.663 INFO:tasks.workunit.client.0.smithi110.stdout:0/116: dwrite d1/f8 [0,4194304] 0 2024-04-25T15:49:17.669 INFO:tasks.workunit.client.0.smithi110.stdout:0/117: dwrite d1/d23/f2b [0,4194304] 0 2024-04-25T15:49:17.670 INFO:tasks.workunit.client.0.smithi110.stdout:0/118: dread - d1/fd zero size 2024-04-25T15:49:17.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:17 smithi193 ceph-mon[33747]: pgmap v25: 65 pgs: 65 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 89 KiB/s rd, 8.9 MiB/s wr, 266 op/s 2024-04-25T15:49:17.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:17 smithi110 ceph-mon[26344]: pgmap v25: 65 pgs: 65 active+clean; 246 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 89 KiB/s rd, 8.9 MiB/s wr, 266 op/s 2024-04-25T15:49:17.759 INFO:tasks.workunit.client.0.smithi110.stdout:8/69: sync 2024-04-25T15:49:17.762 INFO:tasks.workunit.client.0.smithi110.stdout:8/70: dread f3 [0,4194304] 0 2024-04-25T15:49:17.795 INFO:tasks.workunit.client.0.smithi110.stdout:1/102: rename d0/d3/de to d0/d23 0 2024-04-25T15:49:17.796 INFO:tasks.workunit.client.0.smithi110.stdout:1/103: read d0/d23/f13 [391675,49896] 0 2024-04-25T15:49:17.796 INFO:tasks.workunit.client.0.smithi110.stdout:1/104: fdatasync d0/d14/f1f 0 2024-04-25T15:49:17.798 INFO:tasks.workunit.client.0.smithi110.stdout:8/71: creat d14/f15 x:0 0 0 2024-04-25T15:49:17.799 INFO:tasks.workunit.client.0.smithi110.stdout:8/72: dread - d14/f15 zero size 2024-04-25T15:49:17.801 INFO:tasks.workunit.client.0.smithi110.stdout:1/105: rename d0/d3/f1c to d0/d20/f24 0 2024-04-25T15:49:17.806 INFO:tasks.workunit.client.0.smithi110.stdout:8/73: dwrite ff [0,4194304] 0 2024-04-25T15:49:17.813 INFO:tasks.workunit.client.0.smithi110.stdout:8/74: link f3 d14/f16 0 2024-04-25T15:49:17.823 INFO:tasks.workunit.client.0.smithi110.stdout:0/119: truncate d1/d1a/f2d 103756 0 2024-04-25T15:49:17.843 INFO:tasks.workunit.client.0.smithi110.stdout:1/106: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:17.844 INFO:tasks.workunit.client.0.smithi110.stdout:8/75: rmdir d14 39 2024-04-25T15:49:17.852 INFO:tasks.workunit.client.0.smithi110.stdout:8/76: creat d14/f17 x:0 0 0 2024-04-25T15:49:17.853 INFO:tasks.workunit.client.0.smithi110.stdout:1/107: dwrite d0/d23/fc [0,4194304] 0 2024-04-25T15:49:17.854 INFO:tasks.workunit.client.0.smithi110.stdout:8/77: write d14/f17 [334601,102666] 0 2024-04-25T15:49:17.855 INFO:tasks.workunit.client.0.smithi110.stdout:8/78: write d14/f17 [611525,35552] 0 2024-04-25T15:49:17.858 INFO:tasks.workunit.client.0.smithi110.stdout:1/108: readlink d0/d3/l4 0 2024-04-25T15:49:17.862 INFO:tasks.workunit.client.0.smithi110.stdout:8/79: rename fc to d14/f18 0 2024-04-25T15:49:17.869 INFO:tasks.workunit.client.0.smithi110.stdout:0/120: dwrite d1/d2/f11 [0,4194304] 0 2024-04-25T15:49:17.869 INFO:tasks.workunit.client.0.smithi110.stdout:8/80: fdatasync d14/f18 0 2024-04-25T15:49:17.870 INFO:tasks.workunit.client.0.smithi110.stdout:0/121: dread - d1/d2/f12 zero size 2024-04-25T15:49:17.871 INFO:tasks.workunit.client.0.smithi110.stdout:1/109: unlink d0/f10 0 2024-04-25T15:49:17.877 INFO:tasks.workunit.client.0.smithi110.stdout:8/81: rename l9 to d14/l19 0 2024-04-25T15:49:17.878 INFO:tasks.workunit.client.0.smithi110.stdout:0/122: dwrite d1/f19 [0,4194304] 0 2024-04-25T15:49:17.883 INFO:tasks.workunit.client.0.smithi110.stdout:1/110: rename d0/d3 to d0/d25 0 2024-04-25T15:49:17.885 INFO:tasks.workunit.client.0.smithi110.stdout:8/82: mknod d14/c1a 0 2024-04-25T15:49:17.887 INFO:tasks.workunit.client.0.smithi110.stdout:0/123: stat d1/le 0 2024-04-25T15:49:17.887 INFO:tasks.workunit.client.0.smithi110.stdout:0/124: chown d1/d23/c29 536 1 2024-04-25T15:49:17.888 INFO:tasks.workunit.client.0.smithi110.stdout:1/111: symlink d0/d14/l26 0 2024-04-25T15:49:17.890 INFO:tasks.workunit.client.0.smithi110.stdout:8/83: read f2 [1449302,77179] 0 2024-04-25T15:49:17.890 INFO:tasks.workunit.client.0.smithi110.stdout:8/84: readlink l8 0 2024-04-25T15:49:17.892 INFO:tasks.workunit.client.0.smithi110.stdout:0/125: creat d1/d23/f2f x:0 0 0 2024-04-25T15:49:17.894 INFO:tasks.workunit.client.0.smithi110.stdout:1/112: creat d0/f27 x:0 0 0 2024-04-25T15:49:17.896 INFO:tasks.workunit.client.0.smithi110.stdout:8/85: creat d14/f1b x:0 0 0 2024-04-25T15:49:17.898 INFO:tasks.workunit.client.0.smithi110.stdout:0/126: mkdir d1/d23/d30 0 2024-04-25T15:49:17.899 INFO:tasks.workunit.client.0.smithi110.stdout:1/113: dread d0/d25/f6 [0,4194304] 0 2024-04-25T15:49:17.902 INFO:tasks.workunit.client.0.smithi110.stdout:8/86: dwrite f5 [0,4194304] 0 2024-04-25T15:49:17.903 INFO:tasks.workunit.client.0.smithi110.stdout:8/87: fdatasync ff 0 2024-04-25T15:49:17.904 INFO:tasks.workunit.client.0.smithi110.stdout:1/114: dread d0/f15 [4194304,4194304] 0 2024-04-25T15:49:17.908 INFO:tasks.workunit.client.0.smithi110.stdout:1/115: dread d0/d25/f6 [0,4194304] 0 2024-04-25T15:49:17.915 INFO:tasks.workunit.client.0.smithi110.stdout:1/116: fdatasync d0/f15 0 2024-04-25T15:49:17.915 INFO:tasks.workunit.client.0.smithi110.stdout:1/117: dread - d0/f27 zero size 2024-04-25T15:49:17.918 INFO:tasks.workunit.client.0.smithi110.stdout:1/118: mknod d0/c28 0 2024-04-25T15:49:17.924 INFO:tasks.workunit.client.0.smithi110.stdout:1/119: rename d0/d14/f1a to d0/d14/f29 0 2024-04-25T15:49:17.926 INFO:tasks.workunit.client.0.smithi110.stdout:1/120: mknod d0/d25/c2a 0 2024-04-25T15:49:17.927 INFO:tasks.workunit.client.0.smithi110.stdout:8/88: rmdir d14 39 2024-04-25T15:49:17.932 INFO:tasks.workunit.client.0.smithi110.stdout:1/121: dwrite d0/d25/f6 [0,4194304] 0 2024-04-25T15:49:17.937 INFO:tasks.workunit.client.0.smithi110.stdout:8/89: rename f2 to d14/f1c 0 2024-04-25T15:49:17.939 INFO:tasks.workunit.client.0.smithi110.stdout:1/122: creat d0/f2b x:0 0 0 2024-04-25T15:49:17.941 INFO:tasks.workunit.client.0.smithi110.stdout:1/123: write d0/d14/f1f [171132,58106] 0 2024-04-25T15:49:17.944 INFO:tasks.workunit.client.0.smithi110.stdout:8/90: dwrite fe [0,4194304] 0 2024-04-25T15:49:17.946 INFO:tasks.workunit.client.0.smithi110.stdout:1/124: rename d0/f2b to d0/d14/f2c 0 2024-04-25T15:49:17.947 INFO:tasks.workunit.client.0.smithi110.stdout:1/125: read - d0/f27 zero size 2024-04-25T15:49:17.953 INFO:tasks.workunit.client.0.smithi110.stdout:1/126: mkdir d0/d23/d2d 0 2024-04-25T15:49:17.955 INFO:tasks.workunit.client.0.smithi110.stdout:1/127: fdatasync d0/d20/f24 0 2024-04-25T15:49:17.956 INFO:tasks.workunit.client.0.smithi110.stdout:1/128: chown d0/d25/c12 0 1 2024-04-25T15:49:17.961 INFO:tasks.workunit.client.0.smithi110.stdout:0/127: write d1/d1a/f2d [256940,27901] 0 2024-04-25T15:49:17.962 INFO:tasks.workunit.client.0.smithi110.stdout:1/129: dwrite d0/d20/f24 [4194304,4194304] 0 2024-04-25T15:49:17.963 INFO:tasks.workunit.client.0.smithi110.stdout:0/128: fdatasync d1/d23/f24 0 2024-04-25T15:49:17.963 INFO:tasks.workunit.client.0.smithi110.stdout:1/130: chown d0/d25/c1b 2318 1 2024-04-25T15:49:17.968 INFO:tasks.workunit.client.0.smithi110.stdout:0/129: mknod d1/d20/c31 0 2024-04-25T15:49:17.978 INFO:tasks.workunit.client.0.smithi110.stdout:8/91: dwrite f6 [0,4194304] 0 2024-04-25T15:49:17.984 INFO:tasks.workunit.client.0.smithi110.stdout:8/92: dwrite ff [4194304,4194304] 0 2024-04-25T15:49:17.989 INFO:tasks.workunit.client.0.smithi110.stdout:0/130: rmdir d1 39 2024-04-25T15:49:17.996 INFO:tasks.workunit.client.0.smithi110.stdout:0/131: creat d1/f32 x:0 0 0 2024-04-25T15:49:17.999 INFO:tasks.workunit.client.0.smithi110.stdout:0/132: dread d1/d23/f2b [0,4194304] 0 2024-04-25T15:49:18.001 INFO:tasks.workunit.client.0.smithi110.stdout:0/133: chown d1/l18 138668 1 2024-04-25T15:49:18.003 INFO:tasks.workunit.client.0.smithi110.stdout:0/134: rename d1/d1a to d1/d23/d30/d33 0 2024-04-25T15:49:18.006 INFO:tasks.workunit.client.0.smithi110.stdout:0/135: mknod d1/d23/c34 0 2024-04-25T15:49:18.007 INFO:tasks.workunit.client.0.smithi110.stdout:1/131: truncate d0/d23/fc 3260140 0 2024-04-25T15:49:18.010 INFO:tasks.workunit.client.0.smithi110.stdout:0/136: rename d1/d2 to d1/d23/d30/d33/d35 0 2024-04-25T15:49:18.012 INFO:tasks.workunit.client.0.smithi110.stdout:1/132: rename d0/d23/d2d to d0/d14/d2e 0 2024-04-25T15:49:18.017 INFO:tasks.workunit.client.0.smithi110.stdout:0/137: link d1/d23/f2f d1/d23/d30/d33/d35/f36 0 2024-04-25T15:49:18.018 INFO:tasks.workunit.client.0.smithi110.stdout:1/133: dwrite d0/f15 [4194304,4194304] 0 2024-04-25T15:49:18.020 INFO:tasks.workunit.client.0.smithi110.stdout:0/138: chown d1/lb 533 1 2024-04-25T15:49:18.021 INFO:tasks.workunit.client.0.smithi110.stdout:1/134: write d0/d23/f13 [669577,100202] 0 2024-04-25T15:49:18.022 INFO:tasks.workunit.client.0.smithi110.stdout:1/135: chown d0/d14/f2c 0 1 2024-04-25T15:49:18.026 INFO:tasks.workunit.client.0.smithi110.stdout:1/136: read d0/d14/f29 [73946,48945] 0 2024-04-25T15:49:18.028 INFO:tasks.workunit.client.0.smithi110.stdout:0/139: dwrite d1/d23/d30/d33/f2d [0,4194304] 0 2024-04-25T15:49:18.029 INFO:tasks.workunit.client.0.smithi110.stdout:0/140: write d1/f19 [226651,108067] 0 2024-04-25T15:49:18.030 INFO:tasks.workunit.client.0.smithi110.stdout:0/141: dread - d1/d23/f2f zero size 2024-04-25T15:49:18.031 INFO:tasks.workunit.client.0.smithi110.stdout:1/137: truncate d0/d14/f29 417615 0 2024-04-25T15:49:18.033 INFO:tasks.workunit.client.0.smithi110.stdout:1/138: dread d0/f18 [0,4194304] 0 2024-04-25T15:49:18.034 INFO:tasks.workunit.client.0.smithi110.stdout:1/139: write d0/f15 [7704077,83719] 0 2024-04-25T15:49:18.035 INFO:tasks.workunit.client.0.smithi110.stdout:1/140: dread - d0/d14/f2c zero size 2024-04-25T15:49:18.040 INFO:tasks.workunit.client.0.smithi110.stdout:0/142: link d1/d23/d30/d33/d35/f11 d1/d23/d30/f37 0 2024-04-25T15:49:18.041 INFO:tasks.workunit.client.0.smithi110.stdout:0/143: write d1/d23/d30/d33/d35/f12 [416567,129761] 0 2024-04-25T15:49:18.041 INFO:tasks.workunit.client.0.smithi110.stdout:1/141: symlink d0/d14/l2f 0 2024-04-25T15:49:18.043 INFO:tasks.workunit.client.0.smithi110.stdout:1/142: write d0/d14/f1d [797073,98641] 0 2024-04-25T15:49:18.043 INFO:tasks.workunit.client.0.smithi110.stdout:0/144: chown d1/d23/d30/d33/c26 1614 1 2024-04-25T15:49:18.044 INFO:tasks.workunit.client.0.smithi110.stdout:0/145: write d1/fd [357682,23779] 0 2024-04-25T15:49:18.048 INFO:tasks.workunit.client.0.smithi110.stdout:1/143: creat d0/d23/f30 x:0 0 0 2024-04-25T15:49:18.055 INFO:tasks.workunit.client.0.smithi110.stdout:0/146: creat d1/f38 x:0 0 0 2024-04-25T15:49:18.058 INFO:tasks.workunit.client.0.smithi110.stdout:0/147: rename d1/le to d1/d20/l39 0 2024-04-25T15:49:18.063 INFO:tasks.workunit.client.0.smithi110.stdout:0/148: link d1/f8 d1/d23/d30/d33/d35/f3a 0 2024-04-25T15:49:18.064 INFO:tasks.workunit.client.0.smithi110.stdout:0/149: dread - d1/d23/d30/d33/d35/f36 zero size 2024-04-25T15:49:18.066 INFO:tasks.workunit.client.0.smithi110.stdout:0/150: rmdir d1/d23/d30/d33 39 2024-04-25T15:49:18.070 INFO:tasks.workunit.client.0.smithi110.stdout:0/151: dread d1/f19 [0,4194304] 0 2024-04-25T15:49:18.072 INFO:tasks.workunit.client.0.smithi110.stdout:0/152: symlink d1/d23/d30/l3b 0 2024-04-25T15:49:18.079 INFO:tasks.workunit.client.0.smithi110.stdout:0/153: dread d1/d23/d30/d33/d35/f11 [0,4194304] 0 2024-04-25T15:49:18.083 INFO:tasks.workunit.client.0.smithi110.stdout:1/144: dwrite d0/f18 [0,4194304] 0 2024-04-25T15:49:18.088 INFO:tasks.workunit.client.0.smithi110.stdout:1/145: creat d0/d14/f31 x:0 0 0 2024-04-25T15:49:18.088 INFO:tasks.workunit.client.0.smithi110.stdout:1/146: write d0/f15 [5972329,4711] 0 2024-04-25T15:49:18.089 INFO:tasks.workunit.client.0.smithi110.stdout:1/147: fsync d0/d14/f1f 0 2024-04-25T15:49:18.093 INFO:tasks.workunit.client.0.smithi110.stdout:1/148: creat d0/d14/f32 x:0 0 0 2024-04-25T15:49:18.093 INFO:tasks.workunit.client.0.smithi110.stdout:1/149: readlink d0/d14/l26 0 2024-04-25T15:49:18.100 INFO:tasks.workunit.client.0.smithi110.stdout:1/150: getdents d0 0 2024-04-25T15:49:18.102 INFO:tasks.workunit.client.0.smithi110.stdout:0/154: write d1/d23/f2f [284632,43641] 0 2024-04-25T15:49:18.103 INFO:tasks.workunit.client.0.smithi110.stdout:0/155: dread - d1/f38 zero size 2024-04-25T15:49:18.104 INFO:tasks.workunit.client.0.smithi110.stdout:1/151: dread d0/d25/f6 [0,4194304] 0 2024-04-25T15:49:18.104 INFO:tasks.workunit.client.0.smithi110.stdout:0/156: chown d1/l1e 227729661 1 2024-04-25T15:49:18.112 INFO:tasks.workunit.client.0.smithi110.stdout:0/157: creat d1/d23/d30/f3c x:0 0 0 2024-04-25T15:49:18.116 INFO:tasks.workunit.client.0.smithi110.stdout:0/158: getdents d1/d23/d30/d33 0 2024-04-25T15:49:18.118 INFO:tasks.workunit.client.0.smithi110.stdout:0/159: rename d1/d23/d30 to d1/d23/d30/d3d 22 2024-04-25T15:49:18.121 INFO:tasks.workunit.client.0.smithi110.stdout:0/160: unlink d1/f10 0 2024-04-25T15:49:18.122 INFO:tasks.workunit.client.0.smithi110.stdout:1/152: dwrite d0/d23/fc [0,4194304] 0 2024-04-25T15:49:18.125 INFO:tasks.workunit.client.0.smithi110.stdout:1/153: rename d0/d20 to d0/d25/d33 0 2024-04-25T15:49:18.125 INFO:tasks.workunit.client.0.smithi110.stdout:1/154: dread - d0/d14/f31 zero size 2024-04-25T15:49:18.126 INFO:tasks.workunit.client.0.smithi110.stdout:0/161: mkdir d1/d3e 0 2024-04-25T15:49:18.127 INFO:tasks.workunit.client.0.smithi110.stdout:1/155: mknod d0/d25/c34 0 2024-04-25T15:49:18.129 INFO:tasks.workunit.client.0.smithi110.stdout:1/156: truncate d0/d23/f30 76843 0 2024-04-25T15:49:18.129 INFO:tasks.workunit.client.0.smithi110.stdout:0/162: mknod d1/d23/d30/c3f 0 2024-04-25T15:49:18.130 INFO:tasks.workunit.client.0.smithi110.stdout:0/163: chown d1/d23/d30/d33/d35/c1d 303380627 1 2024-04-25T15:49:18.133 INFO:tasks.workunit.client.0.smithi110.stdout:1/157: link d0/d14/l2f d0/d14/l35 0 2024-04-25T15:49:18.135 INFO:tasks.workunit.client.0.smithi110.stdout:1/158: dread d0/d23/f30 [0,4194304] 0 2024-04-25T15:49:18.138 INFO:tasks.workunit.client.0.smithi110.stdout:0/164: dwrite d1/f38 [0,4194304] 0 2024-04-25T15:49:18.139 INFO:tasks.workunit.client.0.smithi110.stdout:1/159: unlink d0/d25/c12 0 2024-04-25T15:49:18.151 INFO:tasks.workunit.client.0.smithi110.stdout:6/66: sync 2024-04-25T15:49:18.152 INFO:tasks.workunit.client.0.smithi110.stdout:9/106: sync 2024-04-25T15:49:18.152 INFO:tasks.workunit.client.0.smithi110.stdout:1/160: getdents d0/d23 0 2024-04-25T15:49:18.154 INFO:tasks.workunit.client.0.smithi110.stdout:6/67: write d1/f3 [4652882,120031] 0 2024-04-25T15:49:18.154 INFO:tasks.workunit.client.0.smithi110.stdout:1/161: dread - d0/d14/f32 zero size 2024-04-25T15:49:18.155 INFO:tasks.workunit.client.0.smithi110.stdout:6/68: fdatasync d1/f9 0 2024-04-25T15:49:18.156 INFO:tasks.workunit.client.0.smithi110.stdout:6/69: chown d1/ca 490789 1 2024-04-25T15:49:18.163 INFO:tasks.workunit.client.0.smithi110.stdout:1/162: dwrite d0/f27 [0,4194304] 0 2024-04-25T15:49:18.167 INFO:tasks.workunit.client.0.smithi110.stdout:0/165: getdents d1 0 2024-04-25T15:49:18.167 INFO:tasks.workunit.client.0.smithi110.stdout:9/107: dread f0 [0,4194304] 0 2024-04-25T15:49:18.172 INFO:tasks.workunit.client.0.smithi110.stdout:9/108: dread f6 [4194304,4194304] 0 2024-04-25T15:49:18.177 INFO:tasks.workunit.client.0.smithi110.stdout:6/70: getdents d1/db 0 2024-04-25T15:49:18.183 INFO:tasks.workunit.client.0.smithi110.stdout:0/166: dwrite d1/f32 [0,4194304] 0 2024-04-25T15:49:18.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/163: write d0/fa [266101,112428] 0 2024-04-25T15:49:18.184 INFO:tasks.workunit.client.0.smithi110.stdout:9/109: dwrite d9/db/f1b [0,4194304] 0 2024-04-25T15:49:18.187 INFO:tasks.workunit.client.0.smithi110.stdout:6/71: dread d1/f9 [0,4194304] 0 2024-04-25T15:49:18.187 INFO:tasks.workunit.client.0.smithi110.stdout:1/164: readlink d0/d14/l35 0 2024-04-25T15:49:18.188 INFO:tasks.workunit.client.0.smithi110.stdout:0/167: chown d1/d23/d30/d33/c26 1 1 2024-04-25T15:49:18.189 INFO:tasks.workunit.client.0.smithi110.stdout:1/165: write d0/fa [39125,75169] 0 2024-04-25T15:49:18.189 INFO:tasks.workunit.client.0.smithi110.stdout:9/110: fsync f8 0 2024-04-25T15:49:18.191 INFO:tasks.workunit.client.0.smithi110.stdout:1/166: write d0/f15 [1569324,119916] 0 2024-04-25T15:49:18.191 INFO:tasks.workunit.client.0.smithi110.stdout:6/72: symlink d1/dd/l18 0 2024-04-25T15:49:18.192 INFO:tasks.workunit.client.0.smithi110.stdout:6/73: dread - d1/dd/f17 zero size 2024-04-25T15:49:18.193 INFO:tasks.workunit.client.0.smithi110.stdout:6/74: fdatasync d1/ff 0 2024-04-25T15:49:18.196 INFO:tasks.workunit.client.0.smithi110.stdout:9/111: write d9/db/f11 [1286551,62669] 0 2024-04-25T15:49:18.197 INFO:tasks.workunit.client.0.smithi110.stdout:0/168: symlink d1/l40 0 2024-04-25T15:49:18.202 INFO:tasks.workunit.client.0.smithi110.stdout:9/112: dread f0 [4194304,4194304] 0 2024-04-25T15:49:18.202 INFO:tasks.workunit.client.0.smithi110.stdout:1/167: dwrite d0/d14/f32 [0,4194304] 0 2024-04-25T15:49:18.203 INFO:tasks.workunit.client.0.smithi110.stdout:6/75: link d1/fe d1/db/f19 0 2024-04-25T15:49:18.204 INFO:tasks.workunit.client.0.smithi110.stdout:0/169: readlink d1/l18 0 2024-04-25T15:49:18.211 INFO:tasks.workunit.client.0.smithi110.stdout:6/76: dwrite d1/dd/f17 [0,4194304] 0 2024-04-25T15:49:18.214 INFO:tasks.workunit.client.0.smithi110.stdout:7/108: sync 2024-04-25T15:49:18.214 INFO:tasks.workunit.client.0.smithi110.stdout:5/67: sync 2024-04-25T15:49:18.214 INFO:tasks.workunit.client.0.smithi110.stdout:2/98: sync 2024-04-25T15:49:18.214 INFO:tasks.workunit.client.0.smithi110.stdout:4/51: sync 2024-04-25T15:49:18.214 INFO:tasks.workunit.client.0.smithi110.stdout:3/100: sync 2024-04-25T15:49:18.215 INFO:tasks.workunit.client.0.smithi110.stdout:5/68: readlink d4/d15/l18 0 2024-04-25T15:49:18.216 INFO:tasks.workunit.client.0.smithi110.stdout:5/69: chown d4/de/l14 20 1 2024-04-25T15:49:18.217 INFO:tasks.workunit.client.0.smithi110.stdout:5/70: fdatasync d4/f10 0 2024-04-25T15:49:18.217 INFO:tasks.workunit.client.0.smithi110.stdout:6/77: creat d1/db/f1a x:0 0 0 2024-04-25T15:49:18.220 INFO:tasks.workunit.client.0.smithi110.stdout:2/99: write f6 [1040728,3607] 0 2024-04-25T15:49:18.221 INFO:tasks.workunit.client.0.smithi110.stdout:7/109: mkdir d1b/d27 0 2024-04-25T15:49:18.221 INFO:tasks.workunit.client.0.smithi110.stdout:7/110: read - f14 zero size 2024-04-25T15:49:18.225 INFO:tasks.workunit.client.0.smithi110.stdout:4/52: creat fd x:0 0 0 2024-04-25T15:49:18.227 INFO:tasks.workunit.client.0.smithi110.stdout:0/170: write d1/d23/d30/d33/d35/f3a [3590271,26462] 0 2024-04-25T15:49:18.228 INFO:tasks.workunit.client.0.smithi110.stdout:1/168: truncate d0/d23/fc 2497329 0 2024-04-25T15:49:18.230 INFO:tasks.workunit.client.0.smithi110.stdout:7/111: dwrite fb [0,4194304] 0 2024-04-25T15:49:18.231 INFO:tasks.workunit.client.0.smithi110.stdout:2/100: symlink de/l1b 0 2024-04-25T15:49:18.233 INFO:tasks.workunit.client.0.smithi110.stdout:3/101: symlink d0/l1b 0 2024-04-25T15:49:18.233 INFO:tasks.workunit.client.0.smithi110.stdout:9/113: dread d9/db/f11 [0,4194304] 0 2024-04-25T15:49:18.235 INFO:tasks.workunit.client.0.smithi110.stdout:4/53: chown lb 13 1 2024-04-25T15:49:18.238 INFO:tasks.workunit.client.0.smithi110.stdout:6/78: write d1/fe [2585442,53816] 0 2024-04-25T15:49:18.239 INFO:tasks.workunit.client.0.smithi110.stdout:1/169: symlink d0/d25/l36 0 2024-04-25T15:49:18.243 INFO:tasks.workunit.client.0.smithi110.stdout:0/171: creat d1/d23/d30/d33/d35/f41 x:0 0 0 2024-04-25T15:49:18.244 INFO:tasks.workunit.client.0.smithi110.stdout:6/79: dread d1/fe [0,4194304] 0 2024-04-25T15:49:18.246 INFO:tasks.workunit.client.0.smithi110.stdout:2/101: unlink de/d11/f18 0 2024-04-25T15:49:18.248 INFO:tasks.workunit.client.0.smithi110.stdout:0/172: dread d1/d23/d30/d33/d35/f36 [0,4194304] 0 2024-04-25T15:49:18.250 INFO:tasks.workunit.client.0.smithi110.stdout:1/170: dwrite d0/d14/f1d [0,4194304] 0 2024-04-25T15:49:18.251 INFO:tasks.workunit.client.0.smithi110.stdout:9/114: fsync f6 0 2024-04-25T15:49:18.255 INFO:tasks.workunit.client.0.smithi110.stdout:5/71: write d4/de/f17 [1184321,13266] 0 2024-04-25T15:49:18.256 INFO:tasks.workunit.client.0.smithi110.stdout:0/173: dread d1/f32 [0,4194304] 0 2024-04-25T15:49:18.259 INFO:tasks.workunit.client.0.smithi110.stdout:7/112: write f2 [4559108,94398] 0 2024-04-25T15:49:18.259 INFO:tasks.workunit.client.0.smithi110.stdout:7/113: chown d1b 1270752430 1 2024-04-25T15:49:18.263 INFO:tasks.workunit.client.0.smithi110.stdout:2/102: fsync fd 0 2024-04-25T15:49:18.264 INFO:tasks.workunit.client.0.smithi110.stdout:3/102: mkdir d0/d7/d1c 0 2024-04-25T15:49:18.265 INFO:tasks.workunit.client.0.smithi110.stdout:9/115: write f0 [2853064,67414] 0 2024-04-25T15:49:18.266 INFO:tasks.workunit.client.0.smithi110.stdout:4/54: fdatasync f6 0 2024-04-25T15:49:18.267 INFO:tasks.workunit.client.0.smithi110.stdout:6/80: dread d1/f3 [0,4194304] 0 2024-04-25T15:49:18.267 INFO:tasks.workunit.client.0.smithi110.stdout:5/72: truncate d4/fd 1174266 0 2024-04-25T15:49:18.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/55: dread - fc zero size 2024-04-25T15:49:18.268 INFO:tasks.workunit.client.0.smithi110.stdout:6/81: chown d1/dd 188776031 1 2024-04-25T15:49:18.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/56: fdatasync fa 0 2024-04-25T15:49:18.269 INFO:tasks.workunit.client.0.smithi110.stdout:0/174: creat d1/d23/f42 x:0 0 0 2024-04-25T15:49:18.271 INFO:tasks.workunit.client.0.smithi110.stdout:2/103: dread fb [4194304,4194304] 0 2024-04-25T15:49:18.274 INFO:tasks.workunit.client.0.smithi110.stdout:7/114: dread fd [0,4194304] 0 2024-04-25T15:49:18.275 INFO:tasks.workunit.client.0.smithi110.stdout:7/115: write d1b/f24 [731838,2331] 0 2024-04-25T15:49:18.276 INFO:tasks.workunit.client.0.smithi110.stdout:5/73: dwrite d4/f6 [0,4194304] 0 2024-04-25T15:49:18.276 INFO:tasks.workunit.client.0.smithi110.stdout:3/103: chown d0/f3 371643841 1 2024-04-25T15:49:18.276 INFO:tasks.workunit.client.0.smithi110.stdout:7/116: fsync f2 0 2024-04-25T15:49:18.278 INFO:tasks.workunit.client.0.smithi110.stdout:7/117: truncate f17 492400 0 2024-04-25T15:49:18.278 INFO:tasks.workunit.client.0.smithi110.stdout:7/118: dread - f14 zero size 2024-04-25T15:49:18.285 INFO:tasks.workunit.client.0.smithi110.stdout:6/82: write d1/f3 [5225912,81666] 0 2024-04-25T15:49:18.285 INFO:tasks.workunit.client.0.smithi110.stdout:5/74: dwrite d4/f16 [0,4194304] 0 2024-04-25T15:49:18.286 INFO:tasks.workunit.client.0.smithi110.stdout:6/83: dread - d1/dd/f13 zero size 2024-04-25T15:49:18.290 INFO:tasks.workunit.client.0.smithi110.stdout:4/57: unlink c9 0 2024-04-25T15:49:18.290 INFO:tasks.workunit.client.0.smithi110.stdout:4/58: truncate fc 503469 0 2024-04-25T15:49:18.301 INFO:tasks.workunit.client.0.smithi110.stdout:9/116: mknod d9/c1d 0 2024-04-25T15:49:18.302 INFO:tasks.workunit.client.0.smithi110.stdout:8/93: sync 2024-04-25T15:49:18.303 INFO:tasks.workunit.client.0.smithi110.stdout:1/171: write d0/d23/fc [3021973,14371] 0 2024-04-25T15:49:18.303 INFO:tasks.workunit.client.0.smithi110.stdout:3/104: dwrite d0/d16/f18 [0,4194304] 0 2024-04-25T15:49:18.305 INFO:tasks.workunit.client.0.smithi110.stdout:2/104: truncate f8 3640181 0 2024-04-25T15:49:18.311 INFO:tasks.workunit.client.0.smithi110.stdout:3/105: dwrite d0/d7/fc [0,4194304] 0 2024-04-25T15:49:18.313 INFO:tasks.workunit.client.0.smithi110.stdout:9/117: symlink d9/d13/l1e 0 2024-04-25T15:49:18.316 INFO:tasks.workunit.client.0.smithi110.stdout:8/94: creat d14/f1d x:0 0 0 2024-04-25T15:49:18.316 INFO:tasks.workunit.client.0.smithi110.stdout:8/95: fsync d14/f15 0 2024-04-25T15:49:18.320 INFO:tasks.workunit.client.0.smithi110.stdout:1/172: symlink d0/l37 0 2024-04-25T15:49:18.322 INFO:tasks.workunit.client.0.smithi110.stdout:5/75: rename d4/fb to d4/de/f1c 0 2024-04-25T15:49:18.325 INFO:tasks.workunit.client.0.smithi110.stdout:6/84: creat d1/d12/f1b x:0 0 0 2024-04-25T15:49:18.327 INFO:tasks.workunit.client.0.smithi110.stdout:7/119: rmdir d1b/d27 0 2024-04-25T15:49:18.329 INFO:tasks.workunit.client.0.smithi110.stdout:4/59: dwrite f6 [0,4194304] 0 2024-04-25T15:49:18.330 INFO:tasks.workunit.client.0.smithi110.stdout:9/118: fsync f8 0 2024-04-25T15:49:18.330 INFO:tasks.workunit.client.0.smithi110.stdout:7/120: truncate f11 4402999 0 2024-04-25T15:49:18.336 INFO:tasks.workunit.client.0.smithi110.stdout:1/173: dwrite d0/d14/f1f [0,4194304] 0 2024-04-25T15:49:18.337 INFO:tasks.workunit.client.0.smithi110.stdout:8/96: dread d14/f1c [0,4194304] 0 2024-04-25T15:49:18.338 INFO:tasks.workunit.client.0.smithi110.stdout:5/76: dwrite d4/f9 [0,4194304] 0 2024-04-25T15:49:18.339 INFO:tasks.workunit.client.0.smithi110.stdout:3/106: symlink d0/l1d 0 2024-04-25T15:49:18.340 INFO:tasks.workunit.client.0.smithi110.stdout:1/174: read d0/d23/f30 [65063,28933] 0 2024-04-25T15:49:18.344 INFO:tasks.workunit.client.0.smithi110.stdout:4/60: unlink lb 0 2024-04-25T15:49:18.345 INFO:tasks.workunit.client.0.smithi110.stdout:4/61: dread - f8 zero size 2024-04-25T15:49:18.346 INFO:tasks.workunit.client.0.smithi110.stdout:4/62: dread - f8 zero size 2024-04-25T15:49:18.346 INFO:tasks.workunit.client.0.smithi110.stdout:4/63: rmdir - no directory 2024-04-25T15:49:18.347 INFO:tasks.workunit.client.0.smithi110.stdout:5/77: dread d4/f6 [0,4194304] 0 2024-04-25T15:49:18.349 INFO:tasks.workunit.client.0.smithi110.stdout:0/175: getdents d1/d20 0 2024-04-25T15:49:18.351 INFO:tasks.workunit.client.0.smithi110.stdout:6/85: symlink d1/l1c 0 2024-04-25T15:49:18.354 INFO:tasks.workunit.client.0.smithi110.stdout:8/97: rename c1 to d14/c1e 0 2024-04-25T15:49:18.356 INFO:tasks.workunit.client.0.smithi110.stdout:3/107: rmdir d0/d7 39 2024-04-25T15:49:18.361 INFO:tasks.workunit.client.0.smithi110.stdout:1/175: dread d0/d14/f29 [0,4194304] 0 2024-04-25T15:49:18.361 INFO:tasks.workunit.client.0.smithi110.stdout:4/64: mknod ce 0 2024-04-25T15:49:18.366 INFO:tasks.workunit.client.0.smithi110.stdout:6/86: fsync d1/db/f19 0 2024-04-25T15:49:18.369 INFO:tasks.workunit.client.0.smithi110.stdout:8/98: mknod d14/c1f 0 2024-04-25T15:49:18.370 INFO:tasks.workunit.client.0.smithi110.stdout:8/99: write f3 [3022470,44313] 0 2024-04-25T15:49:18.370 INFO:tasks.workunit.client.0.smithi110.stdout:6/87: dread d1/dd/f17 [0,4194304] 0 2024-04-25T15:49:18.373 INFO:tasks.workunit.client.0.smithi110.stdout:4/65: creat ff x:0 0 0 2024-04-25T15:49:18.373 INFO:tasks.workunit.client.0.smithi110.stdout:1/176: fdatasync d0/d25/f6 0 2024-04-25T15:49:18.375 INFO:tasks.workunit.client.0.smithi110.stdout:5/78: mkdir d4/d1d 0 2024-04-25T15:49:18.377 INFO:tasks.workunit.client.0.smithi110.stdout:1/177: dread d0/d14/f1f [0,4194304] 0 2024-04-25T15:49:18.377 INFO:tasks.workunit.client.0.smithi110.stdout:1/178: readlink d0/l37 0 2024-04-25T15:49:18.382 INFO:tasks.workunit.client.0.smithi110.stdout:2/105: write fb [3567443,8974] 0 2024-04-25T15:49:18.385 INFO:tasks.workunit.client.0.smithi110.stdout:8/100: creat d14/f20 x:0 0 0 2024-04-25T15:49:18.387 INFO:tasks.workunit.client.0.smithi110.stdout:4/66: mknod c10 0 2024-04-25T15:49:18.387 INFO:tasks.workunit.client.0.smithi110.stdout:4/67: rmdir - no directory 2024-04-25T15:49:18.387 INFO:tasks.workunit.client.0.smithi110.stdout:0/176: write d1/f1c [228617,101544] 0 2024-04-25T15:49:18.388 INFO:tasks.workunit.client.0.smithi110.stdout:4/68: fdatasync f6 0 2024-04-25T15:49:18.390 INFO:tasks.workunit.client.0.smithi110.stdout:7/121: link d1b/l21 d1b/l28 0 2024-04-25T15:49:18.391 INFO:tasks.workunit.client.0.smithi110.stdout:8/101: dwrite f5 [0,4194304] 0 2024-04-25T15:49:18.393 INFO:tasks.workunit.client.0.smithi110.stdout:1/179: symlink d0/d25/d33/l38 0 2024-04-25T15:49:18.396 INFO:tasks.workunit.client.0.smithi110.stdout:2/106: creat de/f1c x:0 0 0 2024-04-25T15:49:18.400 INFO:tasks.workunit.client.0.smithi110.stdout:4/69: rename ce to c11 0 2024-04-25T15:49:18.401 INFO:tasks.workunit.client.0.smithi110.stdout:5/79: creat d4/d1d/f1e x:0 0 0 2024-04-25T15:49:18.402 INFO:tasks.workunit.client.0.smithi110.stdout:7/122: mkdir d1b/d29 0 2024-04-25T15:49:18.402 INFO:tasks.workunit.client.0.smithi110.stdout:5/80: write d4/f9 [2448735,82201] 0 2024-04-25T15:49:18.404 INFO:tasks.workunit.client.0.smithi110.stdout:8/102: mkdir d14/d21 0 2024-04-25T15:49:18.406 INFO:tasks.workunit.client.0.smithi110.stdout:3/108: rename d0/f4 to d0/f1e 0 2024-04-25T15:49:18.408 INFO:tasks.workunit.client.0.smithi110.stdout:6/88: creat d1/f1d x:0 0 0 2024-04-25T15:49:18.410 INFO:tasks.workunit.client.0.smithi110.stdout:4/70: creat f12 x:0 0 0 2024-04-25T15:49:18.410 INFO:tasks.workunit.client.0.smithi110.stdout:8/103: dwrite d14/f1b [0,4194304] 0 2024-04-25T15:49:18.414 INFO:tasks.workunit.client.0.smithi110.stdout:5/81: mkdir d4/d1d/d1f 0 2024-04-25T15:49:18.416 INFO:tasks.workunit.client.0.smithi110.stdout:5/82: write d4/f10 [305020,49824] 0 2024-04-25T15:49:18.418 INFO:tasks.workunit.client.0.smithi110.stdout:0/177: dwrite d1/f32 [4194304,4194304] 0 2024-04-25T15:49:18.421 INFO:tasks.workunit.client.0.smithi110.stdout:3/109: creat d0/d16/f1f x:0 0 0 2024-04-25T15:49:18.425 INFO:tasks.workunit.client.0.smithi110.stdout:5/83: dwrite d4/de/f19 [0,4194304] 0 2024-04-25T15:49:18.426 INFO:tasks.workunit.client.0.smithi110.stdout:5/84: fsync d4/d1d/f1e 0 2024-04-25T15:49:18.426 INFO:tasks.workunit.client.0.smithi110.stdout:4/71: mkdir d13 0 2024-04-25T15:49:18.427 INFO:tasks.workunit.client.0.smithi110.stdout:4/72: write fd [768528,57419] 0 2024-04-25T15:49:18.428 INFO:tasks.workunit.client.0.smithi110.stdout:8/104: creat d14/f22 x:0 0 0 2024-04-25T15:49:18.429 INFO:tasks.workunit.client.0.smithi110.stdout:7/123: getdents d1b/d29 0 2024-04-25T15:49:18.431 INFO:tasks.workunit.client.0.smithi110.stdout:1/180: link d0/c16 d0/d25/c39 0 2024-04-25T15:49:18.432 INFO:tasks.workunit.client.0.smithi110.stdout:1/181: chown d0/l37 1686144 1 2024-04-25T15:49:18.436 INFO:tasks.workunit.client.0.smithi110.stdout:8/105: dwrite d14/f18 [4194304,4194304] 0 2024-04-25T15:49:18.437 INFO:tasks.workunit.client.0.smithi110.stdout:8/106: chown ld 58 1 2024-04-25T15:49:18.438 INFO:tasks.workunit.client.0.smithi110.stdout:8/107: readlink ld 0 2024-04-25T15:49:18.438 INFO:tasks.workunit.client.0.smithi110.stdout:6/89: mknod d1/c1e 0 2024-04-25T15:49:18.442 INFO:tasks.workunit.client.0.smithi110.stdout:5/85: stat d4/de/l1b 0 2024-04-25T15:49:18.444 INFO:tasks.workunit.client.0.smithi110.stdout:5/86: fdatasync d4/d15/f1a 0 2024-04-25T15:49:18.444 INFO:tasks.workunit.client.0.smithi110.stdout:1/182: dwrite d0/d23/fc [0,4194304] 0 2024-04-25T15:49:18.447 INFO:tasks.workunit.client.0.smithi110.stdout:0/178: rename d1/fa to d1/d20/f43 0 2024-04-25T15:49:18.447 INFO:tasks.workunit.client.0.smithi110.stdout:5/87: truncate d4/f8 700317 0 2024-04-25T15:49:18.449 INFO:tasks.workunit.client.0.smithi110.stdout:8/108: dwrite ff [0,4194304] 0 2024-04-25T15:49:18.451 INFO:tasks.workunit.client.0.smithi110.stdout:8/109: read d14/f1b [1888678,61769] 0 2024-04-25T15:49:18.457 INFO:tasks.workunit.client.0.smithi110.stdout:6/90: creat d1/d12/f1f x:0 0 0 2024-04-25T15:49:18.458 INFO:tasks.workunit.client.0.smithi110.stdout:4/73: getdents d13 0 2024-04-25T15:49:18.459 INFO:tasks.workunit.client.0.smithi110.stdout:7/124: mknod d1b/d29/c2a 0 2024-04-25T15:49:18.460 INFO:tasks.workunit.client.0.smithi110.stdout:8/110: dwrite d14/f22 [0,4194304] 0 2024-04-25T15:49:18.461 INFO:tasks.workunit.client.0.smithi110.stdout:1/183: symlink d0/d14/l3a 0 2024-04-25T15:49:18.461 INFO:tasks.workunit.client.0.smithi110.stdout:7/125: dread - f14 zero size 2024-04-25T15:49:18.462 INFO:tasks.workunit.client.0.smithi110.stdout:8/111: truncate fb 4788550 0 2024-04-25T15:49:18.463 INFO:tasks.workunit.client.0.smithi110.stdout:0/179: creat d1/d23/f44 x:0 0 0 2024-04-25T15:49:18.465 INFO:tasks.workunit.client.0.smithi110.stdout:7/126: dread fd [0,4194304] 0 2024-04-25T15:49:18.467 INFO:tasks.workunit.client.0.smithi110.stdout:3/110: truncate d0/d16/f18 63027 0 2024-04-25T15:49:18.470 INFO:tasks.workunit.client.0.smithi110.stdout:1/184: dwrite d0/d23/fc [0,4194304] 0 2024-04-25T15:49:18.471 INFO:tasks.workunit.client.0.smithi110.stdout:5/88: creat d4/d1d/d1f/f20 x:0 0 0 2024-04-25T15:49:18.472 INFO:tasks.workunit.client.0.smithi110.stdout:1/185: chown d0/d14/l26 1 1 2024-04-25T15:49:18.474 INFO:tasks.workunit.client.0.smithi110.stdout:0/180: mknod d1/d23/d30/d33/d35/c45 0 2024-04-25T15:49:18.475 INFO:tasks.workunit.client.0.smithi110.stdout:7/127: creat d1b/d29/f2b x:0 0 0 2024-04-25T15:49:18.477 INFO:tasks.workunit.client.0.smithi110.stdout:4/74: creat d13/f14 x:0 0 0 2024-04-25T15:49:18.480 INFO:tasks.workunit.client.0.smithi110.stdout:4/75: dread fd [0,4194304] 0 2024-04-25T15:49:18.481 INFO:tasks.workunit.client.0.smithi110.stdout:8/112: mkdir d14/d21/d23 0 2024-04-25T15:49:18.481 INFO:tasks.workunit.client.0.smithi110.stdout:8/113: readlink l12 0 2024-04-25T15:49:18.484 INFO:tasks.workunit.client.0.smithi110.stdout:5/89: symlink d4/d1d/d1f/l21 0 2024-04-25T15:49:18.485 INFO:tasks.workunit.client.0.smithi110.stdout:5/90: stat d4/lc 0 2024-04-25T15:49:18.490 INFO:tasks.workunit.client.0.smithi110.stdout:1/186: creat d0/d23/f3b x:0 0 0 2024-04-25T15:49:18.490 INFO:tasks.workunit.client.0.smithi110.stdout:7/128: symlink d1b/d29/l2c 0 2024-04-25T15:49:18.491 INFO:tasks.workunit.client.0.smithi110.stdout:8/114: dwrite ff [0,4194304] 0 2024-04-25T15:49:18.492 INFO:tasks.workunit.client.0.smithi110.stdout:7/129: chown d1b/d29/c2a 7719453 1 2024-04-25T15:49:18.496 INFO:tasks.workunit.client.0.smithi110.stdout:6/91: rename d1/l1c to d1/l20 0 2024-04-25T15:49:18.497 INFO:tasks.workunit.client.0.smithi110.stdout:1/187: dwrite d0/d14/f32 [0,4194304] 0 2024-04-25T15:49:18.498 INFO:tasks.workunit.client.0.smithi110.stdout:6/92: dread - d1/dd/f13 zero size 2024-04-25T15:49:18.499 INFO:tasks.workunit.client.0.smithi110.stdout:1/188: read d0/d23/f30 [48384,42747] 0 2024-04-25T15:49:18.500 INFO:tasks.workunit.client.0.smithi110.stdout:1/189: chown d0/c8 686 1 2024-04-25T15:49:18.500 INFO:tasks.workunit.client.0.smithi110.stdout:0/181: rmdir d1/d23/d30 39 2024-04-25T15:49:18.503 INFO:tasks.workunit.client.0.smithi110.stdout:4/76: rmdir d13 39 2024-04-25T15:49:18.506 INFO:tasks.workunit.client.0.smithi110.stdout:3/111: symlink d0/d7/d1c/l20 0 2024-04-25T15:49:18.507 INFO:tasks.workunit.client.0.smithi110.stdout:7/130: symlink d1b/l2d 0 2024-04-25T15:49:18.512 INFO:tasks.workunit.client.0.smithi110.stdout:0/182: chown d1/d23/d30/d33/d35/f12 99 1 2024-04-25T15:49:18.516 INFO:tasks.workunit.client.0.smithi110.stdout:6/93: dwrite d1/dd/f17 [0,4194304] 0 2024-04-25T15:49:18.517 INFO:tasks.workunit.client.0.smithi110.stdout:6/94: chown d1/db/l16 0 1 2024-04-25T15:49:18.518 INFO:tasks.workunit.client.0.smithi110.stdout:8/115: creat d14/d21/d23/f24 x:0 0 0 2024-04-25T15:49:18.519 INFO:tasks.workunit.client.0.smithi110.stdout:3/112: mknod d0/d7/d1c/c21 0 2024-04-25T15:49:18.521 INFO:tasks.workunit.client.0.smithi110.stdout:7/131: symlink d1b/d29/l2e 0 2024-04-25T15:49:18.522 INFO:tasks.workunit.client.0.smithi110.stdout:5/91: creat d4/f22 x:0 0 0 2024-04-25T15:49:18.523 INFO:tasks.workunit.client.0.smithi110.stdout:5/92: chown d4/lc 48433 1 2024-04-25T15:49:18.525 INFO:tasks.workunit.client.0.smithi110.stdout:4/77: mknod d13/c15 0 2024-04-25T15:49:18.525 INFO:tasks.workunit.client.0.smithi110.stdout:4/78: dread - f12 zero size 2024-04-25T15:49:18.527 INFO:tasks.workunit.client.0.smithi110.stdout:8/116: creat d14/d21/d23/f25 x:0 0 0 2024-04-25T15:49:18.528 INFO:tasks.workunit.client.0.smithi110.stdout:3/113: creat d0/d7/d1c/f22 x:0 0 0 2024-04-25T15:49:18.529 INFO:tasks.workunit.client.0.smithi110.stdout:7/132: write f11 [4669129,58280] 0 2024-04-25T15:49:18.529 INFO:tasks.workunit.client.0.smithi110.stdout:3/114: readlink d0/d7/d1c/l20 0 2024-04-25T15:49:18.530 INFO:tasks.workunit.client.0.smithi110.stdout:0/183: symlink d1/d23/l46 0 2024-04-25T15:49:18.532 INFO:tasks.workunit.client.0.smithi110.stdout:6/95: dread d1/db/f19 [0,4194304] 0 2024-04-25T15:49:18.532 INFO:tasks.workunit.client.0.smithi110.stdout:5/93: unlink d4/f9 0 2024-04-25T15:49:18.532 INFO:tasks.workunit.client.0.smithi110.stdout:1/190: link d0/d14/l35 d0/d14/l3c 0 2024-04-25T15:49:18.535 INFO:tasks.workunit.client.0.smithi110.stdout:4/79: stat c11 0 2024-04-25T15:49:18.535 INFO:tasks.workunit.client.0.smithi110.stdout:8/117: dwrite f3 [0,4194304] 0 2024-04-25T15:49:18.536 INFO:tasks.workunit.client.0.smithi110.stdout:4/80: truncate fa 236148 0 2024-04-25T15:49:18.536 INFO:tasks.workunit.client.0.smithi110.stdout:8/118: read - d14/f1d zero size 2024-04-25T15:49:18.536 INFO:tasks.workunit.client.0.smithi110.stdout:4/81: chown fc 184 1 2024-04-25T15:49:18.537 INFO:tasks.workunit.client.0.smithi110.stdout:4/82: read - f8 zero size 2024-04-25T15:49:18.537 INFO:tasks.workunit.client.0.smithi110.stdout:3/115: write d0/d7/fa [1386844,88918] 0 2024-04-25T15:49:18.538 INFO:tasks.workunit.client.0.smithi110.stdout:1/191: dread d0/d25/f6 [0,4194304] 0 2024-04-25T15:49:18.539 INFO:tasks.workunit.client.0.smithi110.stdout:1/192: fsync d0/fa 0 2024-04-25T15:49:18.540 INFO:tasks.workunit.client.0.smithi110.stdout:6/96: creat d1/dd/f21 x:0 0 0 2024-04-25T15:49:18.540 INFO:tasks.workunit.client.0.smithi110.stdout:5/94: mknod d4/d1d/d1f/c23 0 2024-04-25T15:49:18.541 INFO:tasks.workunit.client.0.smithi110.stdout:8/119: mkdir d14/d21/d26 0 2024-04-25T15:49:18.546 INFO:tasks.workunit.client.0.smithi110.stdout:3/116: rename d0/d7/d1c/c21 to d0/d7/d1c/c23 0 2024-04-25T15:49:18.548 INFO:tasks.workunit.client.0.smithi110.stdout:6/97: dwrite d1/ff [0,4194304] 0 2024-04-25T15:49:18.549 INFO:tasks.workunit.client.0.smithi110.stdout:3/117: creat d0/d16/f24 x:0 0 0 2024-04-25T15:49:18.550 INFO:tasks.workunit.client.0.smithi110.stdout:8/120: dwrite d14/f15 [0,4194304] 0 2024-04-25T15:49:18.552 INFO:tasks.workunit.client.0.smithi110.stdout:8/121: read d14/f16 [3988647,98261] 0 2024-04-25T15:49:18.557 INFO:tasks.workunit.client.0.smithi110.stdout:8/122: creat d14/d21/f27 x:0 0 0 2024-04-25T15:49:18.559 INFO:tasks.workunit.client.0.smithi110.stdout:6/98: dwrite d1/dd/f13 [0,4194304] 0 2024-04-25T15:49:18.560 INFO:tasks.workunit.client.0.smithi110.stdout:8/123: read f5 [7330748,18794] 0 2024-04-25T15:49:18.566 INFO:tasks.workunit.client.0.smithi110.stdout:6/99: dwrite d1/ff [0,4194304] 0 2024-04-25T15:49:18.575 INFO:tasks.workunit.client.0.smithi110.stdout:6/100: dwrite f0 [0,4194304] 0 2024-04-25T15:49:18.593 INFO:tasks.workunit.client.0.smithi110.stdout:3/118: dread d0/d7/fe [0,4194304] 0 2024-04-25T15:49:18.594 INFO:tasks.workunit.client.0.smithi110.stdout:8/124: mknod d14/d21/c28 0 2024-04-25T15:49:18.598 INFO:tasks.workunit.client.0.smithi110.stdout:6/101: mkdir d1/db/d22 0 2024-04-25T15:49:18.601 INFO:tasks.workunit.client.0.smithi110.stdout:3/119: mknod d0/d7/d1c/c25 0 2024-04-25T15:49:18.603 INFO:tasks.workunit.client.0.smithi110.stdout:8/125: fdatasync d14/f1c 0 2024-04-25T15:49:18.605 INFO:tasks.workunit.client.0.smithi110.stdout:6/102: readlink d1/l7 0 2024-04-25T15:49:18.606 INFO:tasks.workunit.client.0.smithi110.stdout:6/103: write d1/f9 [3530806,80247] 0 2024-04-25T15:49:18.609 INFO:tasks.workunit.client.0.smithi110.stdout:3/120: mknod d0/d16/c26 0 2024-04-25T15:49:18.615 INFO:tasks.workunit.client.0.smithi110.stdout:6/104: unlink d1/db/lc 0 2024-04-25T15:49:18.615 INFO:tasks.workunit.client.0.smithi110.stdout:8/126: dread d14/f1b [0,4194304] 0 2024-04-25T15:49:18.616 INFO:tasks.workunit.client.0.smithi110.stdout:6/105: write f0 [2450918,72026] 0 2024-04-25T15:49:18.618 INFO:tasks.workunit.client.0.smithi110.stdout:7/133: truncate f17 388619 0 2024-04-25T15:49:18.622 INFO:tasks.workunit.client.0.smithi110.stdout:8/127: dread f6 [0,4194304] 0 2024-04-25T15:49:18.624 INFO:tasks.workunit.client.0.smithi110.stdout:6/106: dwrite f0 [4194304,4194304] 0 2024-04-25T15:49:18.627 INFO:tasks.workunit.client.0.smithi110.stdout:7/134: rename d1b/d29/l2e to d1b/l2f 0 2024-04-25T15:49:18.630 INFO:tasks.workunit.client.0.smithi110.stdout:1/193: dwrite d0/d14/f29 [0,4194304] 0 2024-04-25T15:49:18.631 INFO:tasks.workunit.client.0.smithi110.stdout:6/107: write d1/dd/f21 [328578,14064] 0 2024-04-25T15:49:18.633 INFO:tasks.workunit.client.0.smithi110.stdout:1/194: write d0/d25/d33/f24 [1736208,10032] 0 2024-04-25T15:49:18.635 INFO:tasks.workunit.client.0.smithi110.stdout:8/128: dwrite d14/f1d [0,4194304] 0 2024-04-25T15:49:18.636 INFO:tasks.workunit.client.0.smithi110.stdout:8/129: stat d14/c1f 0 2024-04-25T15:49:18.641 INFO:tasks.workunit.client.0.smithi110.stdout:7/135: symlink d1b/d29/l30 0 2024-04-25T15:49:18.642 INFO:tasks.workunit.client.0.smithi110.stdout:1/195: dwrite d0/d14/f32 [0,4194304] 0 2024-04-25T15:49:18.643 INFO:tasks.workunit.client.0.smithi110.stdout:3/121: link d0/l5 d0/d16/l27 0 2024-04-25T15:49:18.643 INFO:tasks.workunit.client.0.smithi110.stdout:7/136: chown d1b/d29/l2c 1863 1 2024-04-25T15:49:18.644 INFO:tasks.workunit.client.0.smithi110.stdout:1/196: chown d0/c28 145531 1 2024-04-25T15:49:18.644 INFO:tasks.workunit.client.0.smithi110.stdout:7/137: chown la 47033 1 2024-04-25T15:49:18.645 INFO:tasks.workunit.client.0.smithi110.stdout:3/122: write d0/f1e [177320,53338] 0 2024-04-25T15:49:18.645 INFO:tasks.workunit.client.0.smithi110.stdout:1/197: write d0/d23/f13 [1803046,21696] 0 2024-04-25T15:49:18.648 INFO:tasks.workunit.client.0.smithi110.stdout:6/108: unlink d1/ff 0 2024-04-25T15:49:18.649 INFO:tasks.workunit.client.0.smithi110.stdout:1/198: dread d0/fa [0,4194304] 0 2024-04-25T15:49:18.649 INFO:tasks.workunit.client.0.smithi110.stdout:1/199: chown d0/fa 2007303 1 2024-04-25T15:49:18.650 INFO:tasks.workunit.client.0.smithi110.stdout:1/200: chown d0/c28 9304737 1 2024-04-25T15:49:18.669 INFO:tasks.workunit.client.0.smithi110.stdout:3/123: unlink d0/l1d 0 2024-04-25T15:49:18.671 INFO:tasks.workunit.client.0.smithi110.stdout:7/138: dread f2 [0,4194304] 0 2024-04-25T15:49:18.673 INFO:tasks.workunit.client.0.smithi110.stdout:1/201: creat d0/d25/f3d x:0 0 0 2024-04-25T15:49:18.676 INFO:tasks.workunit.client.0.smithi110.stdout:8/130: creat d14/d21/d26/f29 x:0 0 0 2024-04-25T15:49:18.677 INFO:tasks.workunit.client.0.smithi110.stdout:8/131: write d14/f20 [41901,90254] 0 2024-04-25T15:49:18.677 INFO:tasks.workunit.client.0.smithi110.stdout:1/202: dread d0/f15 [4194304,4194304] 0 2024-04-25T15:49:18.677 INFO:tasks.workunit.client.0.smithi110.stdout:8/132: stat d14/d21/f27 0 2024-04-25T15:49:18.678 INFO:tasks.workunit.client.0.smithi110.stdout:8/133: chown d14/d21 629254202 1 2024-04-25T15:49:18.679 INFO:tasks.workunit.client.0.smithi110.stdout:3/124: chown d0/f12 3972244 1 2024-04-25T15:49:18.680 INFO:tasks.workunit.client.0.smithi110.stdout:6/109: mknod d1/db/d22/c23 0 2024-04-25T15:49:18.686 INFO:tasks.workunit.client.0.smithi110.stdout:8/134: fsync f6 0 2024-04-25T15:49:18.689 INFO:tasks.workunit.client.0.smithi110.stdout:6/110: mkdir d1/db/d24 0 2024-04-25T15:49:18.690 INFO:tasks.workunit.client.0.smithi110.stdout:7/139: chown f17 3008 1 2024-04-25T15:49:18.698 INFO:tasks.workunit.client.0.smithi110.stdout:3/125: rename d0/c1 to d0/d7/d1c/c28 0 2024-04-25T15:49:18.700 INFO:tasks.workunit.client.0.smithi110.stdout:6/111: rmdir d1/db/d22 39 2024-04-25T15:49:18.701 INFO:tasks.workunit.client.0.smithi110.stdout:6/112: dread - d1/db/f1a zero size 2024-04-25T15:49:18.704 INFO:tasks.workunit.client.0.smithi110.stdout:7/140: chown d1b/c22 0 1 2024-04-25T15:49:18.704 INFO:tasks.workunit.client.0.smithi110.stdout:3/126: dwrite d0/d7/f17 [0,4194304] 0 2024-04-25T15:49:18.705 INFO:tasks.workunit.client.0.smithi110.stdout:3/127: dread - d0/d16/f1f zero size 2024-04-25T15:49:18.706 INFO:tasks.workunit.client.0.smithi110.stdout:1/203: rmdir d0/d14/d2e 0 2024-04-25T15:49:18.706 INFO:tasks.workunit.client.0.smithi110.stdout:3/128: stat d0/d7/d1c/l20 0 2024-04-25T15:49:18.710 INFO:tasks.workunit.client.0.smithi110.stdout:8/135: truncate d14/f16 3426 0 2024-04-25T15:49:18.710 INFO:tasks.workunit.client.0.smithi110.stdout:8/136: stat l12 0 2024-04-25T15:49:18.711 INFO:tasks.workunit.client.0.smithi110.stdout:8/137: write d14/f1d [2903128,24237] 0 2024-04-25T15:49:18.715 INFO:tasks.workunit.client.0.smithi110.stdout:6/113: mknod d1/dd/c25 0 2024-04-25T15:49:18.716 INFO:tasks.workunit.client.0.smithi110.stdout:6/114: truncate d1/d12/f1f 418402 0 2024-04-25T15:49:18.717 INFO:tasks.workunit.client.0.smithi110.stdout:7/141: rename d1b/f24 to d1b/f31 0 2024-04-25T15:49:18.718 INFO:tasks.workunit.client.0.smithi110.stdout:7/142: chown l18 113865 1 2024-04-25T15:49:18.720 INFO:tasks.workunit.client.0.smithi110.stdout:6/115: dread d1/db/f19 [0,4194304] 0 2024-04-25T15:49:18.730 INFO:tasks.workunit.client.0.smithi110.stdout:7/143: symlink d1b/l32 0 2024-04-25T15:49:18.738 INFO:tasks.workunit.client.0.smithi110.stdout:6/116: rename d1/dd/f13 to d1/f26 0 2024-04-25T15:49:18.739 INFO:tasks.workunit.client.0.smithi110.stdout:6/117: truncate d1/f1d 46194 0 2024-04-25T15:49:18.739 INFO:tasks.workunit.client.0.smithi110.stdout:6/118: chown d1/dd 33 1 2024-04-25T15:49:18.739 INFO:tasks.workunit.client.0.smithi110.stdout:6/119: stat d1/dd 0 2024-04-25T15:49:18.745 INFO:tasks.workunit.client.0.smithi110.stdout:6/120: dwrite d1/f9 [0,4194304] 0 2024-04-25T15:49:18.752 INFO:tasks.workunit.client.0.smithi110.stdout:6/121: mkdir d1/db/d22/d27 0 2024-04-25T15:49:18.758 INFO:tasks.workunit.client.0.smithi110.stdout:6/122: dwrite d1/d12/f1f [0,4194304] 0 2024-04-25T15:49:18.767 INFO:tasks.workunit.client.0.smithi110.stdout:6/123: rename d1/dd/c25 to d1/c28 0 2024-04-25T15:49:18.768 INFO:tasks.workunit.client.0.smithi110.stdout:6/124: write d1/dd/f21 [577228,109643] 0 2024-04-25T15:49:18.775 INFO:tasks.workunit.client.0.smithi110.stdout:1/204: rmdir d0 39 2024-04-25T15:49:18.777 INFO:tasks.workunit.client.0.smithi110.stdout:1/205: fsync d0/fa 0 2024-04-25T15:49:18.783 INFO:tasks.workunit.client.0.smithi110.stdout:1/206: dwrite d0/d14/f29 [0,4194304] 0 2024-04-25T15:49:18.787 INFO:tasks.workunit.client.0.smithi110.stdout:1/207: rename d0/d23/fc to d0/d23/f3e 0 2024-04-25T15:49:18.796 INFO:tasks.workunit.client.0.smithi110.stdout:8/138: dwrite d14/f1c [0,4194304] 0 2024-04-25T15:49:18.802 INFO:tasks.workunit.client.0.smithi110.stdout:3/129: truncate d0/d7/fc 1445679 0 2024-04-25T15:49:18.806 INFO:tasks.workunit.client.0.smithi110.stdout:3/130: creat d0/d7/f29 x:0 0 0 2024-04-25T15:49:18.816 INFO:tasks.workunit.client.0.smithi110.stdout:8/139: fdatasync d14/f1c 0 2024-04-25T15:49:18.816 INFO:tasks.workunit.client.0.smithi110.stdout:3/131: dwrite d0/fd [0,4194304] 0 2024-04-25T15:49:18.820 INFO:tasks.workunit.client.0.smithi110.stdout:8/140: creat d14/d21/f2a x:0 0 0 2024-04-25T15:49:18.821 INFO:tasks.workunit.client.0.smithi110.stdout:3/132: creat d0/d7/f2a x:0 0 0 2024-04-25T15:49:18.823 INFO:tasks.workunit.client.0.smithi110.stdout:8/141: creat d14/d21/f2b x:0 0 0 2024-04-25T15:49:18.824 INFO:tasks.workunit.client.0.smithi110.stdout:8/142: write d14/f15 [806344,58913] 0 2024-04-25T15:49:18.829 INFO:tasks.workunit.client.0.smithi110.stdout:3/133: symlink d0/l2b 0 2024-04-25T15:49:18.830 INFO:tasks.workunit.client.0.smithi110.stdout:8/143: dwrite d14/d21/f2a [0,4194304] 0 2024-04-25T15:49:18.831 INFO:tasks.workunit.client.0.smithi110.stdout:8/144: dread - d14/d21/d23/f24 zero size 2024-04-25T15:49:18.831 INFO:tasks.workunit.client.0.smithi110.stdout:8/145: stat d14/c1f 0 2024-04-25T15:49:18.842 INFO:tasks.workunit.client.0.smithi110.stdout:3/134: creat d0/f2c x:0 0 0 2024-04-25T15:49:18.845 INFO:tasks.workunit.client.0.smithi110.stdout:3/135: mkdir d0/d7/d2d 0 2024-04-25T15:49:18.846 INFO:tasks.workunit.client.0.smithi110.stdout:8/146: link d14/l19 d14/d21/d26/l2c 0 2024-04-25T15:49:18.847 INFO:tasks.workunit.client.0.smithi110.stdout:8/147: fsync d14/f15 0 2024-04-25T15:49:18.854 INFO:tasks.workunit.client.0.smithi110.stdout:3/136: mkdir d0/d2e 0 2024-04-25T15:49:18.855 INFO:tasks.workunit.client.0.smithi110.stdout:3/137: truncate d0/d16/f1f 531248 0 2024-04-25T15:49:18.856 INFO:tasks.workunit.client.0.smithi110.stdout:8/148: fdatasync f6 0 2024-04-25T15:49:18.861 INFO:tasks.workunit.client.0.smithi110.stdout:1/208: truncate d0/d14/f29 1757761 0 2024-04-25T15:49:18.861 INFO:tasks.workunit.client.0.smithi110.stdout:3/138: dread d0/f15 [0,4194304] 0 2024-04-25T15:49:18.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:18 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:49:18.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:18 smithi110 ceph-mon[26344]: pgmap v26: 65 pgs: 65 active+clean; 431 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 861 KiB/s rd, 39 MiB/s wr, 375 op/s 2024-04-25T15:49:18.863 INFO:tasks.workunit.client.0.smithi110.stdout:8/149: dwrite d14/d21/f27 [0,4194304] 0 2024-04-25T15:49:18.870 INFO:tasks.workunit.client.0.smithi110.stdout:8/150: readlink d14/d21/d26/l2c 0 2024-04-25T15:49:18.872 INFO:tasks.workunit.client.0.smithi110.stdout:8/151: fsync d14/f20 0 2024-04-25T15:49:18.873 INFO:tasks.workunit.client.0.smithi110.stdout:3/139: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:18.873 INFO:tasks.workunit.client.0.smithi110.stdout:1/209: dwrite d0/d23/f13 [0,4194304] 0 2024-04-25T15:49:18.873 INFO:tasks.workunit.client.0.smithi110.stdout:8/152: stat ff 0 2024-04-25T15:49:18.874 INFO:tasks.workunit.client.0.smithi110.stdout:1/210: dread - d0/d23/f3b zero size 2024-04-25T15:49:18.874 INFO:tasks.workunit.client.0.smithi110.stdout:8/153: chown d14/f1b 188 1 2024-04-25T15:49:18.880 INFO:tasks.workunit.client.0.smithi110.stdout:3/140: dwrite d0/d7/fa [4194304,4194304] 0 2024-04-25T15:49:18.881 INFO:tasks.workunit.client.0.smithi110.stdout:8/154: unlink fe 0 2024-04-25T15:49:18.884 INFO:tasks.workunit.client.0.smithi110.stdout:1/211: creat d0/f3f x:0 0 0 2024-04-25T15:49:18.884 INFO:tasks.workunit.client.0.smithi110.stdout:9/119: sync 2024-04-25T15:49:18.885 INFO:tasks.workunit.client.0.smithi110.stdout:1/212: chown d0/d23/f3e 29032 1 2024-04-25T15:49:18.886 INFO:tasks.workunit.client.0.smithi110.stdout:1/213: read d0/f27 [3798760,117281] 0 2024-04-25T15:49:18.889 INFO:tasks.workunit.client.0.smithi110.stdout:3/141: mkdir d0/d7/d1c/d2f 0 2024-04-25T15:49:18.890 INFO:tasks.workunit.client.0.smithi110.stdout:3/142: write d0/d7/fa [6533857,20055] 0 2024-04-25T15:49:18.892 INFO:tasks.workunit.client.0.smithi110.stdout:8/155: creat d14/f2d x:0 0 0 2024-04-25T15:49:18.893 INFO:tasks.workunit.client.0.smithi110.stdout:9/120: creat d9/d13/f1f x:0 0 0 2024-04-25T15:49:18.898 INFO:tasks.workunit.client.0.smithi110.stdout:1/214: unlink d0/l37 0 2024-04-25T15:49:18.902 INFO:tasks.workunit.client.0.smithi110.stdout:8/156: creat d14/d21/d23/f2e x:0 0 0 2024-04-25T15:49:18.908 INFO:tasks.workunit.client.0.smithi110.stdout:1/215: rename d0/d25/l4 to d0/d25/l40 0 2024-04-25T15:49:18.908 INFO:tasks.workunit.client.0.smithi110.stdout:1/216: stat d0/d14/f2c 0 2024-04-25T15:49:18.910 INFO:tasks.workunit.client.0.smithi110.stdout:8/157: unlink d14/f18 0 2024-04-25T15:49:18.910 INFO:tasks.workunit.client.0.smithi110.stdout:1/217: dread d0/fa [0,4194304] 0 2024-04-25T15:49:18.911 INFO:tasks.workunit.client.0.smithi110.stdout:8/158: write d14/f20 [101323,92399] 0 2024-04-25T15:49:18.913 INFO:tasks.workunit.client.0.smithi110.stdout:1/218: chown d0/d14/l2f 19559 1 2024-04-25T15:49:18.915 INFO:tasks.workunit.client.0.smithi110.stdout:8/159: dread d14/d21/f27 [0,4194304] 0 2024-04-25T15:49:18.916 INFO:tasks.workunit.client.0.smithi110.stdout:8/160: chown la 10774506 1 2024-04-25T15:49:18.916 INFO:tasks.workunit.client.0.smithi110.stdout:1/219: creat d0/d25/f41 x:0 0 0 2024-04-25T15:49:18.918 INFO:tasks.workunit.client.0.smithi110.stdout:8/161: symlink d14/d21/d26/l2f 0 2024-04-25T15:49:18.918 INFO:tasks.workunit.client.0.smithi110.stdout:8/162: dread - d14/d21/d26/f29 zero size 2024-04-25T15:49:18.919 INFO:tasks.workunit.client.0.smithi110.stdout:1/220: symlink d0/d25/d33/l42 0 2024-04-25T15:49:18.920 INFO:tasks.workunit.client.0.smithi110.stdout:8/163: creat d14/d21/d26/f30 x:0 0 0 2024-04-25T15:49:18.920 INFO:tasks.workunit.client.0.smithi110.stdout:8/164: fdatasync d14/d21/f2b 0 2024-04-25T15:49:18.924 INFO:tasks.workunit.client.0.smithi110.stdout:8/165: rename d14/l19 to d14/l31 0 2024-04-25T15:49:18.925 INFO:tasks.workunit.client.0.smithi110.stdout:8/166: write d14/f17 [981011,12429] 0 2024-04-25T15:49:18.931 INFO:tasks.workunit.client.0.smithi110.stdout:8/167: dwrite d14/d21/d23/f25 [0,4194304] 0 2024-04-25T15:49:18.932 INFO:tasks.workunit.client.0.smithi110.stdout:8/168: dread f6 [0,4194304] 0 2024-04-25T15:49:18.935 INFO:tasks.workunit.client.0.smithi110.stdout:8/169: mkdir d14/d32 0 2024-04-25T15:49:18.939 INFO:tasks.workunit.client.0.smithi110.stdout:8/170: dread d14/f1d [0,4194304] 0 2024-04-25T15:49:18.940 INFO:tasks.workunit.client.0.smithi110.stdout:8/171: dread - d14/d21/d26/f29 zero size 2024-04-25T15:49:18.946 INFO:tasks.workunit.client.0.smithi110.stdout:9/121: dwrite f8 [4194304,4194304] 0 2024-04-25T15:49:18.947 INFO:tasks.workunit.client.0.smithi110.stdout:1/221: write d0/d14/f1d [4661259,34509] 0 2024-04-25T15:49:18.950 INFO:tasks.workunit.client.0.smithi110.stdout:1/222: symlink d0/l43 0 2024-04-25T15:49:18.951 INFO:tasks.workunit.client.0.smithi110.stdout:3/143: truncate d0/d7/fc 1803456 0 2024-04-25T15:49:18.956 INFO:tasks.workunit.client.0.smithi110.stdout:9/122: link d9/db/df/l16 d9/l20 0 2024-04-25T15:49:18.957 INFO:tasks.workunit.client.0.smithi110.stdout:9/123: fsync d9/d13/f1f 0 2024-04-25T15:49:18.958 INFO:tasks.workunit.client.0.smithi110.stdout:3/144: creat d0/d2e/f30 x:0 0 0 2024-04-25T15:49:18.958 INFO:tasks.workunit.client.0.smithi110.stdout:3/145: chown d0/d2e 0 1 2024-04-25T15:49:18.960 INFO:tasks.workunit.client.0.smithi110.stdout:3/146: creat d0/d2e/f31 x:0 0 0 2024-04-25T15:49:18.960 INFO:tasks.workunit.client.0.smithi110.stdout:3/147: readlink d0/l1b 0 2024-04-25T15:49:18.961 INFO:tasks.workunit.client.0.smithi110.stdout:9/124: link d9/db/df/f17 d9/db/df/f21 0 2024-04-25T15:49:18.962 INFO:tasks.workunit.client.0.smithi110.stdout:3/148: symlink d0/d2e/l32 0 2024-04-25T15:49:18.963 INFO:tasks.workunit.client.0.smithi110.stdout:9/125: unlink d9/fd 0 2024-04-25T15:49:18.966 INFO:tasks.workunit.client.0.smithi110.stdout:1/223: rmdir d0 39 2024-04-25T15:49:18.970 INFO:tasks.workunit.client.0.smithi110.stdout:9/126: dwrite d9/db/f15 [0,4194304] 0 2024-04-25T15:49:18.971 INFO:tasks.workunit.client.0.smithi110.stdout:9/127: chown d9/db/f11 2500 1 2024-04-25T15:49:18.972 INFO:tasks.workunit.client.0.smithi110.stdout:1/224: truncate d0/d23/f3e 4827919 0 2024-04-25T15:49:18.975 INFO:tasks.workunit.client.0.smithi110.stdout:3/149: dread d0/ff [0,4194304] 0 2024-04-25T15:49:18.976 INFO:tasks.workunit.client.0.smithi110.stdout:1/225: write d0/d14/f29 [1623478,104616] 0 2024-04-25T15:49:18.979 INFO:tasks.workunit.client.0.smithi110.stdout:3/150: readlink d0/d16/l27 0 2024-04-25T15:49:18.979 INFO:tasks.workunit.client.0.smithi110.stdout:1/226: mknod d0/d14/c44 0 2024-04-25T15:49:18.982 INFO:tasks.workunit.client.0.smithi110.stdout:9/128: truncate f0 237658 0 2024-04-25T15:49:18.983 INFO:tasks.workunit.client.0.smithi110.stdout:1/227: creat d0/d25/d33/f45 x:0 0 0 2024-04-25T15:49:18.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/151: rmdir d0/d7/d2d 0 2024-04-25T15:49:18.985 INFO:tasks.workunit.client.0.smithi110.stdout:9/129: fsync f8 0 2024-04-25T15:49:18.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:18 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:49:18.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:18 smithi193 ceph-mon[33747]: pgmap v26: 65 pgs: 65 active+clean; 431 MiB data, 3.0 GiB used, 533 GiB / 536 GiB avail; 861 KiB/s rd, 39 MiB/s wr, 375 op/s 2024-04-25T15:49:18.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/130: readlink d9/db/df/l16 0 2024-04-25T15:49:18.990 INFO:tasks.workunit.client.0.smithi110.stdout:1/228: dwrite d0/f3f [0,4194304] 0 2024-04-25T15:49:18.990 INFO:tasks.workunit.client.0.smithi110.stdout:3/152: mkdir d0/d7/d1c/d2f/d33 0 2024-04-25T15:49:18.992 INFO:tasks.workunit.client.0.smithi110.stdout:9/131: creat d9/db/df/f22 x:0 0 0 2024-04-25T15:49:18.992 INFO:tasks.workunit.client.0.smithi110.stdout:1/229: fdatasync d0/f27 0 2024-04-25T15:49:18.999 INFO:tasks.workunit.client.0.smithi110.stdout:3/153: symlink d0/d7/d1c/d2f/d33/l34 0 2024-04-25T15:49:19.002 INFO:tasks.workunit.client.0.smithi110.stdout:1/230: rename d0/d14/l35 to d0/d25/l46 0 2024-04-25T15:49:19.004 INFO:tasks.workunit.client.0.smithi110.stdout:1/231: truncate d0/d14/f31 20290 0 2024-04-25T15:49:19.004 INFO:tasks.workunit.client.0.smithi110.stdout:9/132: link d9/l20 d9/db/df/l23 0 2024-04-25T15:49:19.007 INFO:tasks.workunit.client.0.smithi110.stdout:3/154: dwrite d0/d2e/f30 [0,4194304] 0 2024-04-25T15:49:19.007 INFO:tasks.workunit.client.0.smithi110.stdout:9/133: readlink d9/db/df/l16 0 2024-04-25T15:49:19.008 INFO:tasks.workunit.client.0.smithi110.stdout:1/232: creat d0/d25/d33/f47 x:0 0 0 2024-04-25T15:49:19.009 INFO:tasks.workunit.client.0.smithi110.stdout:1/233: chown d0/d25/f6 18448866 1 2024-04-25T15:49:19.010 INFO:tasks.workunit.client.0.smithi110.stdout:1/234: chown d0/d23/f30 3639015 1 2024-04-25T15:49:19.022 INFO:tasks.workunit.client.0.smithi110.stdout:3/155: getdents d0 0 2024-04-25T15:49:19.023 INFO:tasks.workunit.client.0.smithi110.stdout:9/134: dwrite f6 [0,4194304] 0 2024-04-25T15:49:19.024 INFO:tasks.workunit.client.0.smithi110.stdout:9/135: dread - d9/d13/f1f zero size 2024-04-25T15:49:19.026 INFO:tasks.workunit.client.0.smithi110.stdout:1/235: truncate d0/d14/f1d 2557479 0 2024-04-25T15:49:19.029 INFO:tasks.workunit.client.0.smithi110.stdout:3/156: link d0/d7/d1c/f22 d0/d7/d1c/f35 0 2024-04-25T15:49:19.030 INFO:tasks.workunit.client.0.smithi110.stdout:1/236: creat d0/d25/d33/f48 x:0 0 0 2024-04-25T15:49:19.034 INFO:tasks.workunit.client.0.smithi110.stdout:3/157: dread d0/d7/f17 [0,4194304] 0 2024-04-25T15:49:19.034 INFO:tasks.workunit.client.0.smithi110.stdout:1/237: link d0/d14/f1f d0/d25/d33/f49 0 2024-04-25T15:49:19.038 INFO:tasks.workunit.client.0.smithi110.stdout:1/238: creat d0/d23/f4a x:0 0 0 2024-04-25T15:49:19.038 INFO:tasks.workunit.client.0.smithi110.stdout:3/158: dread d0/d7/fc [0,4194304] 0 2024-04-25T15:49:19.039 INFO:tasks.workunit.client.0.smithi110.stdout:1/239: write d0/d25/f41 [580455,71808] 0 2024-04-25T15:49:19.044 INFO:tasks.workunit.client.0.smithi110.stdout:9/136: truncate d9/db/f11 2482126 0 2024-04-25T15:49:19.047 INFO:tasks.workunit.client.0.smithi110.stdout:3/159: dwrite d0/d7/fe [0,4194304] 0 2024-04-25T15:49:19.048 INFO:tasks.workunit.client.0.smithi110.stdout:3/160: stat d0/d2e/f31 0 2024-04-25T15:49:19.048 INFO:tasks.workunit.client.0.smithi110.stdout:3/161: stat d0/d16/l27 0 2024-04-25T15:49:19.048 INFO:tasks.workunit.client.0.smithi110.stdout:1/240: getdents d0/d14 0 2024-04-25T15:49:19.049 INFO:tasks.workunit.client.0.smithi110.stdout:1/241: chown d0/d23 150670 1 2024-04-25T15:49:19.049 INFO:tasks.workunit.client.0.smithi110.stdout:3/162: chown d0/d16/l27 2472141 1 2024-04-25T15:49:19.053 INFO:tasks.workunit.client.0.smithi110.stdout:3/163: creat d0/d7/d1c/d2f/d33/f36 x:0 0 0 2024-04-25T15:49:19.056 INFO:tasks.workunit.client.0.smithi110.stdout:1/242: dwrite d0/d23/f4a [0,4194304] 0 2024-04-25T15:49:19.057 INFO:tasks.workunit.client.0.smithi110.stdout:1/243: chown d0/c8 7032 1 2024-04-25T15:49:19.057 INFO:tasks.workunit.client.0.smithi110.stdout:3/164: dread d0/d16/f18 [0,4194304] 0 2024-04-25T15:49:19.058 INFO:tasks.workunit.client.0.smithi110.stdout:3/165: stat d0/d7/c1a 0 2024-04-25T15:49:19.061 INFO:tasks.workunit.client.0.smithi110.stdout:1/244: rename d0/d14/f31 to d0/d14/f4b 0 2024-04-25T15:49:19.064 INFO:tasks.workunit.client.0.smithi110.stdout:9/137: dwrite d9/db/df/f17 [4194304,4194304] 0 2024-04-25T15:49:19.068 INFO:tasks.workunit.client.0.smithi110.stdout:3/166: dwrite d0/d7/f17 [0,4194304] 0 2024-04-25T15:49:19.070 INFO:tasks.workunit.client.0.smithi110.stdout:3/167: write d0/d7/fe [3073075,69325] 0 2024-04-25T15:49:19.075 INFO:tasks.workunit.client.0.smithi110.stdout:3/168: mknod d0/d7/d1c/d2f/d33/c37 0 2024-04-25T15:49:19.076 INFO:tasks.workunit.client.0.smithi110.stdout:3/169: dread - d0/f2c zero size 2024-04-25T15:49:19.078 INFO:tasks.workunit.client.0.smithi110.stdout:3/170: creat d0/d7/f38 x:0 0 0 2024-04-25T15:49:19.084 INFO:tasks.workunit.client.0.smithi110.stdout:3/171: dwrite d0/d16/f1f [0,4194304] 0 2024-04-25T15:49:19.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/172: creat d0/d2e/f39 x:0 0 0 2024-04-25T15:49:19.109 INFO:tasks.workunit.client.0.smithi110.stdout:2/107: sync 2024-04-25T15:49:19.109 INFO:tasks.workunit.client.0.smithi110.stdout:3/173: chown d0/f1e 5552027 1 2024-04-25T15:49:19.109 INFO:tasks.workunit.client.0.smithi110.stdout:2/108: fsync f4 0 2024-04-25T15:49:19.114 INFO:tasks.workunit.client.0.smithi110.stdout:3/174: rename d0/fd to d0/d16/f3a 0 2024-04-25T15:49:19.119 INFO:tasks.workunit.client.0.smithi110.stdout:2/109: rename lc to de/l1d 0 2024-04-25T15:49:19.130 INFO:tasks.workunit.client.0.smithi110.stdout:2/110: rename de/l1d to de/d11/l1e 0 2024-04-25T15:49:19.131 INFO:tasks.workunit.client.0.smithi110.stdout:3/175: dwrite d0/d7/fe [0,4194304] 0 2024-04-25T15:49:19.132 INFO:tasks.workunit.client.0.smithi110.stdout:3/176: truncate d0/d7/f38 752532 0 2024-04-25T15:49:19.133 INFO:tasks.workunit.client.0.smithi110.stdout:2/111: unlink f6 0 2024-04-25T15:49:19.133 INFO:tasks.workunit.client.0.smithi110.stdout:2/112: fsync de/f10 0 2024-04-25T15:49:19.134 INFO:tasks.workunit.client.0.smithi110.stdout:2/113: write de/f1c [273201,74822] 0 2024-04-25T15:49:19.138 INFO:tasks.workunit.client.0.smithi110.stdout:3/177: link d0/d7/f29 d0/d2e/f3b 0 2024-04-25T15:49:19.139 INFO:tasks.workunit.client.0.smithi110.stdout:2/114: dwrite fd [0,4194304] 0 2024-04-25T15:49:19.140 INFO:tasks.workunit.client.0.smithi110.stdout:2/115: truncate de/f10 190355 0 2024-04-25T15:49:19.148 INFO:tasks.workunit.client.0.smithi110.stdout:2/116: dwrite de/d11/f19 [4194304,4194304] 0 2024-04-25T15:49:19.152 INFO:tasks.workunit.client.0.smithi110.stdout:2/117: unlink de/f1c 0 2024-04-25T15:49:19.155 INFO:tasks.workunit.client.0.smithi110.stdout:2/118: unlink f4 0 2024-04-25T15:49:19.156 INFO:tasks.workunit.client.0.smithi110.stdout:2/119: dread de/f10 [0,4194304] 0 2024-04-25T15:49:19.157 INFO:tasks.workunit.client.0.smithi110.stdout:2/120: chown c9 191197 1 2024-04-25T15:49:19.161 INFO:tasks.workunit.client.0.smithi110.stdout:2/121: rmdir de/d11 39 2024-04-25T15:49:19.168 INFO:tasks.workunit.client.0.smithi110.stdout:3/178: truncate d0/d16/f3a 3077358 0 2024-04-25T15:49:19.171 INFO:tasks.workunit.client.0.smithi110.stdout:3/179: symlink d0/l3c 0 2024-04-25T15:49:19.175 INFO:tasks.workunit.client.0.smithi110.stdout:3/180: creat d0/d16/f3d x:0 0 0 2024-04-25T15:49:19.177 INFO:tasks.workunit.client.0.smithi110.stdout:3/181: stat d0/d2e/f39 0 2024-04-25T15:49:19.180 INFO:tasks.workunit.client.0.smithi110.stdout:3/182: write d0/d7/f38 [608106,87720] 0 2024-04-25T15:49:19.182 INFO:tasks.workunit.client.0.smithi110.stdout:2/122: truncate f8 1093120 0 2024-04-25T15:49:19.185 INFO:tasks.workunit.client.0.smithi110.stdout:3/183: dwrite d0/d7/f17 [0,4194304] 0 2024-04-25T15:49:19.192 INFO:tasks.workunit.client.0.smithi110.stdout:3/184: getdents d0/d7/d1c 0 2024-04-25T15:49:19.195 INFO:tasks.workunit.client.0.smithi110.stdout:3/185: symlink d0/d2e/l3e 0 2024-04-25T15:49:19.198 INFO:tasks.workunit.client.0.smithi110.stdout:2/123: fsync fb 0 2024-04-25T15:49:19.199 INFO:tasks.workunit.client.0.smithi110.stdout:3/186: mknod d0/c3f 0 2024-04-25T15:49:19.208 INFO:tasks.workunit.client.0.smithi110.stdout:3/187: write d0/d7/d1c/f22 [841030,124475] 0 2024-04-25T15:49:19.211 INFO:tasks.workunit.client.0.smithi110.stdout:3/188: mkdir d0/d7/d40 0 2024-04-25T15:49:19.218 INFO:tasks.workunit.client.0.smithi110.stdout:3/189: dwrite d0/f2c [0,4194304] 0 2024-04-25T15:49:19.228 INFO:tasks.workunit.client.0.smithi110.stdout:3/190: write d0/d16/f18 [509920,61687] 0 2024-04-25T15:49:19.239 INFO:tasks.workunit.client.0.smithi110.stdout:3/191: dread d0/d16/f18 [0,4194304] 0 2024-04-25T15:49:19.249 INFO:tasks.workunit.client.0.smithi110.stdout:3/192: truncate d0/f1e 7438061 0 2024-04-25T15:49:19.254 INFO:tasks.workunit.client.0.smithi110.stdout:3/193: dwrite d0/f3 [0,4194304] 0 2024-04-25T15:49:19.256 INFO:tasks.workunit.client.0.smithi110.stdout:3/194: symlink d0/d7/d1c/d2f/d33/l41 0 2024-04-25T15:49:19.257 INFO:tasks.workunit.client.0.smithi110.stdout:3/195: write d0/d7/f2a [392513,14956] 0 2024-04-25T15:49:19.260 INFO:tasks.workunit.client.0.smithi110.stdout:3/196: fdatasync d0/d7/f2a 0 2024-04-25T15:49:19.261 INFO:tasks.workunit.client.0.smithi110.stdout:3/197: dread - d0/d16/f3d zero size 2024-04-25T15:49:19.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/83: sync 2024-04-25T15:49:19.269 INFO:tasks.workunit.client.0.smithi110.stdout:0/184: sync 2024-04-25T15:49:19.272 INFO:tasks.workunit.client.0.smithi110.stdout:0/185: mkdir d1/d47 0 2024-04-25T15:49:19.274 INFO:tasks.workunit.client.0.smithi110.stdout:4/84: link ff d13/f16 0 2024-04-25T15:49:19.279 INFO:tasks.workunit.client.0.smithi110.stdout:3/198: write d0/d16/f3a [2181466,38339] 0 2024-04-25T15:49:19.283 INFO:tasks.workunit.client.0.smithi110.stdout:4/85: link d13/c15 d13/c17 0 2024-04-25T15:49:19.284 INFO:tasks.workunit.client.0.smithi110.stdout:0/186: link d1/d20/l21 d1/d3e/l48 0 2024-04-25T15:49:19.284 INFO:tasks.workunit.client.0.smithi110.stdout:0/187: chown d1/d3e 221554094 1 2024-04-25T15:49:19.295 INFO:tasks.workunit.client.0.smithi110.stdout:0/188: creat d1/f49 x:0 0 0 2024-04-25T15:49:19.297 INFO:tasks.workunit.client.0.smithi110.stdout:4/86: dwrite d13/f16 [0,4194304] 0 2024-04-25T15:49:19.298 INFO:tasks.workunit.client.0.smithi110.stdout:4/87: dread - f12 zero size 2024-04-25T15:49:19.299 INFO:tasks.workunit.client.0.smithi110.stdout:0/189: symlink d1/d23/d30/d33/l4a 0 2024-04-25T15:49:19.299 INFO:tasks.workunit.client.0.smithi110.stdout:0/190: chown d1/d23/d30 1 1 2024-04-25T15:49:19.306 INFO:tasks.workunit.client.0.smithi110.stdout:4/88: dwrite fc [0,4194304] 0 2024-04-25T15:49:19.308 INFO:tasks.workunit.client.0.smithi110.stdout:4/89: fdatasync f8 0 2024-04-25T15:49:19.308 INFO:tasks.workunit.client.0.smithi110.stdout:4/90: write f12 [205985,111435] 0 2024-04-25T15:49:19.314 INFO:tasks.workunit.client.0.smithi110.stdout:4/91: dwrite f8 [0,4194304] 0 2024-04-25T15:49:19.316 INFO:tasks.workunit.client.0.smithi110.stdout:4/92: chown fc 241 1 2024-04-25T15:49:19.316 INFO:tasks.workunit.client.0.smithi110.stdout:3/199: dread d0/d7/fa [0,4194304] 0 2024-04-25T15:49:19.316 INFO:tasks.workunit.client.0.smithi110.stdout:4/93: chown d13 369168 1 2024-04-25T15:49:19.319 INFO:tasks.workunit.client.0.smithi110.stdout:3/200: dread - d0/f12 zero size 2024-04-25T15:49:19.324 INFO:tasks.workunit.client.0.smithi110.stdout:4/94: dwrite fa [0,4194304] 0 2024-04-25T15:49:19.325 INFO:tasks.workunit.client.0.smithi110.stdout:4/95: chown ff 1831640 1 2024-04-25T15:49:19.329 INFO:tasks.workunit.client.0.smithi110.stdout:4/96: fdatasync fd 0 2024-04-25T15:49:19.333 INFO:tasks.workunit.client.0.smithi110.stdout:4/97: dread d13/f16 [0,4194304] 0 2024-04-25T15:49:19.336 INFO:tasks.workunit.client.0.smithi110.stdout:3/201: truncate d0/d7/d1c/f22 263 0 2024-04-25T15:49:19.340 INFO:tasks.workunit.client.0.smithi110.stdout:4/98: dwrite f8 [4194304,4194304] 0 2024-04-25T15:49:19.341 INFO:tasks.workunit.client.0.smithi110.stdout:4/99: dread - d13/f14 zero size 2024-04-25T15:49:19.341 INFO:tasks.workunit.client.0.smithi110.stdout:4/100: read ff [1593501,106809] 0 2024-04-25T15:49:19.345 INFO:tasks.workunit.client.0.smithi110.stdout:3/202: link d0/l1b d0/d7/d40/l42 0 2024-04-25T15:49:19.349 INFO:tasks.workunit.client.0.smithi110.stdout:4/101: dwrite f12 [0,4194304] 0 2024-04-25T15:49:19.353 INFO:tasks.workunit.client.0.smithi110.stdout:4/102: symlink d13/l18 0 2024-04-25T15:49:19.357 INFO:tasks.workunit.client.0.smithi110.stdout:4/103: link d13/l18 d13/l19 0 2024-04-25T15:49:19.357 INFO:tasks.workunit.client.0.smithi110.stdout:3/203: write d0/d2e/f3b [754549,129584] 0 2024-04-25T15:49:19.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/204: creat d0/d7/d40/f43 x:0 0 0 2024-04-25T15:49:19.363 INFO:tasks.workunit.client.0.smithi110.stdout:4/104: dwrite f12 [0,4194304] 0 2024-04-25T15:49:19.365 INFO:tasks.workunit.client.0.smithi110.stdout:4/105: readlink d13/l18 0 2024-04-25T15:49:19.366 INFO:tasks.workunit.client.0.smithi110.stdout:3/205: creat d0/d7/d1c/d2f/d33/f44 x:0 0 0 2024-04-25T15:49:19.370 INFO:tasks.workunit.client.0.smithi110.stdout:4/106: dread fc [0,4194304] 0 2024-04-25T15:49:19.371 INFO:tasks.workunit.client.0.smithi110.stdout:3/206: mkdir d0/d2e/d45 0 2024-04-25T15:49:19.374 INFO:tasks.workunit.client.0.smithi110.stdout:4/107: dread f8 [4194304,4194304] 0 2024-04-25T15:49:19.377 INFO:tasks.workunit.client.0.smithi110.stdout:4/108: rename d13/f14 to d13/f1a 0 2024-04-25T15:49:19.378 INFO:tasks.workunit.client.0.smithi110.stdout:3/207: getdents d0/d7/d1c/d2f 0 2024-04-25T15:49:19.378 INFO:tasks.workunit.client.0.smithi110.stdout:4/109: rename d13 to d13/d1b 22 2024-04-25T15:49:19.378 INFO:tasks.workunit.client.0.smithi110.stdout:4/110: chown fc 230548482 1 2024-04-25T15:49:19.379 INFO:tasks.workunit.client.0.smithi110.stdout:3/208: rmdir d0/d7/d1c/d2f 39 2024-04-25T15:49:19.380 INFO:tasks.workunit.client.0.smithi110.stdout:4/111: mkdir d13/d1c 0 2024-04-25T15:49:19.382 INFO:tasks.workunit.client.0.smithi110.stdout:3/209: mknod d0/c46 0 2024-04-25T15:49:19.383 INFO:tasks.workunit.client.0.smithi110.stdout:4/112: mknod d13/d1c/c1d 0 2024-04-25T15:49:19.389 INFO:tasks.workunit.client.0.smithi110.stdout:4/113: link l3 d13/d1c/l1e 0 2024-04-25T15:49:19.390 INFO:tasks.workunit.client.0.smithi110.stdout:4/114: mkdir d13/d1f 0 2024-04-25T15:49:19.392 INFO:tasks.workunit.client.0.smithi110.stdout:3/210: getdents d0/d7/d1c/d2f/d33 0 2024-04-25T15:49:19.393 INFO:tasks.workunit.client.0.smithi110.stdout:4/115: creat d13/d1c/f20 x:0 0 0 2024-04-25T15:49:19.396 INFO:tasks.workunit.client.0.smithi110.stdout:4/116: unlink l5 0 2024-04-25T15:49:19.399 INFO:tasks.workunit.client.0.smithi110.stdout:4/117: getdents d13/d1f 0 2024-04-25T15:49:19.399 INFO:tasks.workunit.client.0.smithi110.stdout:4/118: chown d13/f16 109647124 1 2024-04-25T15:49:19.401 INFO:tasks.workunit.client.0.smithi110.stdout:4/119: unlink d13/f16 0 2024-04-25T15:49:19.404 INFO:tasks.workunit.client.0.smithi110.stdout:4/120: symlink d13/l21 0 2024-04-25T15:49:19.405 INFO:tasks.workunit.client.0.smithi110.stdout:3/211: write d0/f1e [8324513,44304] 0 2024-04-25T15:49:19.406 INFO:tasks.workunit.client.0.smithi110.stdout:3/212: fdatasync d0/d7/fe 0 2024-04-25T15:49:19.406 INFO:tasks.workunit.client.0.smithi110.stdout:4/121: creat d13/f22 x:0 0 0 2024-04-25T15:49:19.409 INFO:tasks.workunit.client.0.smithi110.stdout:3/213: creat d0/d7/d40/f47 x:0 0 0 2024-04-25T15:49:19.411 INFO:tasks.workunit.client.0.smithi110.stdout:3/214: write d0/f15 [2755103,17790] 0 2024-04-25T15:49:19.413 INFO:tasks.workunit.client.0.smithi110.stdout:3/215: symlink d0/d7/l48 0 2024-04-25T15:49:19.413 INFO:tasks.workunit.client.0.smithi110.stdout:4/122: dwrite fc [0,4194304] 0 2024-04-25T15:49:19.416 INFO:tasks.workunit.client.0.smithi110.stdout:3/216: mknod d0/d7/d40/c49 0 2024-04-25T15:49:19.421 INFO:tasks.workunit.client.0.smithi110.stdout:4/123: creat d13/d1f/f23 x:0 0 0 2024-04-25T15:49:19.422 INFO:tasks.workunit.client.0.smithi110.stdout:4/124: readlink d13/l21 0 2024-04-25T15:49:19.423 INFO:tasks.workunit.client.0.smithi110.stdout:4/125: truncate d13/d1f/f23 183079 0 2024-04-25T15:49:19.424 INFO:tasks.workunit.client.0.smithi110.stdout:3/217: dwrite d0/d16/f24 [0,4194304] 0 2024-04-25T15:49:19.425 INFO:tasks.workunit.client.0.smithi110.stdout:4/126: write fa [5087854,52280] 0 2024-04-25T15:49:19.428 INFO:tasks.workunit.client.0.smithi110.stdout:3/218: symlink d0/d7/l4a 0 2024-04-25T15:49:19.434 INFO:tasks.workunit.client.0.smithi110.stdout:4/127: rename d13/d1f to d13/d24 0 2024-04-25T15:49:19.435 INFO:tasks.workunit.client.0.smithi110.stdout:4/128: stat d13/d24 0 2024-04-25T15:49:19.436 INFO:tasks.workunit.client.0.smithi110.stdout:3/219: dwrite d0/d7/d1c/d2f/d33/f36 [0,4194304] 0 2024-04-25T15:49:19.437 INFO:tasks.workunit.client.0.smithi110.stdout:4/129: write f6 [32182,52609] 0 2024-04-25T15:49:19.440 INFO:tasks.workunit.client.0.smithi110.stdout:4/130: readlink d13/l18 0 2024-04-25T15:49:19.441 INFO:tasks.workunit.client.0.smithi110.stdout:4/131: read - d13/f22 zero size 2024-04-25T15:49:19.444 INFO:tasks.workunit.client.0.smithi110.stdout:3/220: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:19.446 INFO:tasks.workunit.client.0.smithi110.stdout:4/132: getdents d13 0 2024-04-25T15:49:19.447 INFO:tasks.workunit.client.0.smithi110.stdout:4/133: write fc [1998432,97098] 0 2024-04-25T15:49:19.448 INFO:tasks.workunit.client.0.smithi110.stdout:4/134: read f12 [2682741,124996] 0 2024-04-25T15:49:19.449 INFO:tasks.workunit.client.0.smithi110.stdout:4/135: truncate d13/f22 400074 0 2024-04-25T15:49:19.450 INFO:tasks.workunit.client.0.smithi110.stdout:4/136: dread - d13/f1a zero size 2024-04-25T15:49:19.452 INFO:tasks.workunit.client.0.smithi110.stdout:4/137: creat d13/d1c/f25 x:0 0 0 2024-04-25T15:49:19.455 INFO:tasks.workunit.client.0.smithi110.stdout:4/138: mkdir d13/d24/d26 0 2024-04-25T15:49:19.455 INFO:tasks.workunit.client.0.smithi110.stdout:4/139: readlink d13/l21 0 2024-04-25T15:49:19.456 INFO:tasks.workunit.client.0.smithi110.stdout:4/140: read f6 [1135501,63119] 0 2024-04-25T15:49:19.458 INFO:tasks.workunit.client.0.smithi110.stdout:5/95: sync 2024-04-25T15:49:19.459 INFO:tasks.workunit.client.0.smithi110.stdout:4/141: mknod d13/c27 0 2024-04-25T15:49:19.459 INFO:tasks.workunit.client.0.smithi110.stdout:5/96: truncate d4/f8 1139136 0 2024-04-25T15:49:19.460 INFO:tasks.workunit.client.0.smithi110.stdout:4/142: fsync fa 0 2024-04-25T15:49:19.460 INFO:tasks.workunit.client.0.smithi110.stdout:5/97: dread - d4/d1d/d1f/f20 zero size 2024-04-25T15:49:19.466 INFO:tasks.workunit.client.0.smithi110.stdout:4/143: link d13/l19 d13/l28 0 2024-04-25T15:49:19.468 INFO:tasks.workunit.client.0.smithi110.stdout:3/221: dwrite d0/d7/d1c/f35 [0,4194304] 0 2024-04-25T15:49:19.469 INFO:tasks.workunit.client.0.smithi110.stdout:4/144: symlink d13/l29 0 2024-04-25T15:49:19.470 INFO:tasks.workunit.client.0.smithi110.stdout:4/145: write d13/f22 [828355,124977] 0 2024-04-25T15:49:19.474 INFO:tasks.workunit.client.0.smithi110.stdout:3/222: symlink d0/d16/l4b 0 2024-04-25T15:49:19.478 INFO:tasks.workunit.client.0.smithi110.stdout:3/223: mkdir d0/d7/d1c/d2f/d4c 0 2024-04-25T15:49:19.479 INFO:tasks.workunit.client.0.smithi110.stdout:5/98: dread d4/f6 [0,4194304] 0 2024-04-25T15:49:19.480 INFO:tasks.workunit.client.0.smithi110.stdout:5/99: chown d4/f16 525491868 1 2024-04-25T15:49:19.484 INFO:tasks.workunit.client.0.smithi110.stdout:4/146: dwrite ff [0,4194304] 0 2024-04-25T15:49:19.488 INFO:tasks.workunit.client.0.smithi110.stdout:5/100: creat d4/f24 x:0 0 0 2024-04-25T15:49:19.489 INFO:tasks.workunit.client.0.smithi110.stdout:5/101: dread - d4/d1d/d1f/f20 zero size 2024-04-25T15:49:19.490 INFO:tasks.workunit.client.0.smithi110.stdout:3/224: link d0/d7/d1c/d2f/d33/c37 d0/c4d 0 2024-04-25T15:49:19.493 INFO:tasks.workunit.client.0.smithi110.stdout:3/225: symlink d0/d7/l4e 0 2024-04-25T15:49:19.495 INFO:tasks.workunit.client.0.smithi110.stdout:4/147: getdents d13/d1c 0 2024-04-25T15:49:19.496 INFO:tasks.workunit.client.0.smithi110.stdout:5/102: dwrite d4/f24 [0,4194304] 0 2024-04-25T15:49:19.496 INFO:tasks.workunit.client.0.smithi110.stdout:4/148: dread - d13/d1c/f25 zero size 2024-04-25T15:49:19.497 INFO:tasks.workunit.client.0.smithi110.stdout:3/226: fdatasync d0/d7/fc 0 2024-04-25T15:49:19.501 INFO:tasks.workunit.client.0.smithi110.stdout:3/227: write d0/f1e [3987212,20559] 0 2024-04-25T15:49:19.503 INFO:tasks.workunit.client.0.smithi110.stdout:5/103: unlink d4/f24 0 2024-04-25T15:49:19.507 INFO:tasks.workunit.client.0.smithi110.stdout:4/149: rename d13/d24/f23 to d13/d24/f2a 0 2024-04-25T15:49:19.508 INFO:tasks.workunit.client.0.smithi110.stdout:3/228: creat d0/d2e/f4f x:0 0 0 2024-04-25T15:49:19.509 INFO:tasks.workunit.client.0.smithi110.stdout:3/229: fdatasync d0/d7/f17 0 2024-04-25T15:49:19.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/150: dwrite f8 [0,4194304] 0 2024-04-25T15:49:19.518 INFO:tasks.workunit.client.0.smithi110.stdout:3/230: mkdir d0/d7/d1c/d50 0 2024-04-25T15:49:19.521 INFO:tasks.workunit.client.0.smithi110.stdout:5/104: rename d4/d1d/d1f/f20 to d4/f25 0 2024-04-25T15:49:19.524 INFO:tasks.workunit.client.0.smithi110.stdout:3/231: symlink d0/d16/l51 0 2024-04-25T15:49:19.525 INFO:tasks.workunit.client.0.smithi110.stdout:5/105: dread d4/f10 [0,4194304] 0 2024-04-25T15:49:19.526 INFO:tasks.workunit.client.0.smithi110.stdout:5/106: fsync d4/d15/f1a 0 2024-04-25T15:49:19.527 INFO:tasks.workunit.client.0.smithi110.stdout:5/107: write d4/d15/f1a [319954,116227] 0 2024-04-25T15:49:19.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/151: rename d13/d1c/c1d to d13/d24/c2b 0 2024-04-25T15:49:19.529 INFO:tasks.workunit.client.0.smithi110.stdout:4/152: truncate d13/d1c/f20 420671 0 2024-04-25T15:49:19.532 INFO:tasks.workunit.client.0.smithi110.stdout:4/153: creat d13/d1c/f2c x:0 0 0 2024-04-25T15:49:19.533 INFO:tasks.workunit.client.0.smithi110.stdout:4/154: dread - d13/f1a zero size 2024-04-25T15:49:19.534 INFO:tasks.workunit.client.0.smithi110.stdout:3/232: getdents d0/d2e 0 2024-04-25T15:49:19.535 INFO:tasks.workunit.client.0.smithi110.stdout:4/155: rmdir d13/d24 39 2024-04-25T15:49:19.539 INFO:tasks.workunit.client.0.smithi110.stdout:5/108: dread d4/de/f17 [0,4194304] 0 2024-04-25T15:49:19.540 INFO:tasks.workunit.client.0.smithi110.stdout:4/156: mkdir d13/d24/d2d 0 2024-04-25T15:49:19.540 INFO:tasks.workunit.client.0.smithi110.stdout:3/233: dread d0/ff [0,4194304] 0 2024-04-25T15:49:19.540 INFO:tasks.workunit.client.0.smithi110.stdout:4/157: write fa [5267073,114338] 0 2024-04-25T15:49:19.542 INFO:tasks.workunit.client.0.smithi110.stdout:5/109: rmdir d4/d15 39 2024-04-25T15:49:19.546 INFO:tasks.workunit.client.0.smithi110.stdout:3/234: dwrite d0/f12 [0,4194304] 0 2024-04-25T15:49:19.546 INFO:tasks.workunit.client.0.smithi110.stdout:5/110: mknod d4/d1d/c26 0 2024-04-25T15:49:19.547 INFO:tasks.workunit.client.0.smithi110.stdout:4/158: getdents d13/d24/d2d 0 2024-04-25T15:49:19.548 INFO:tasks.workunit.client.0.smithi110.stdout:5/111: write d4/f8 [450087,23523] 0 2024-04-25T15:49:19.549 INFO:tasks.workunit.client.0.smithi110.stdout:3/235: write d0/d7/d40/f47 [11103,16413] 0 2024-04-25T15:49:19.551 INFO:tasks.workunit.client.0.smithi110.stdout:5/112: write d4/f8 [498573,123297] 0 2024-04-25T15:49:19.555 INFO:tasks.workunit.client.0.smithi110.stdout:7/144: sync 2024-04-25T15:49:19.555 INFO:tasks.workunit.client.0.smithi110.stdout:5/113: dread d4/f6 [0,4194304] 0 2024-04-25T15:49:19.556 INFO:tasks.workunit.client.0.smithi110.stdout:7/145: truncate f19 970090 0 2024-04-25T15:49:19.558 INFO:tasks.workunit.client.0.smithi110.stdout:4/159: symlink d13/d24/d26/l2e 0 2024-04-25T15:49:19.560 INFO:tasks.workunit.client.0.smithi110.stdout:7/146: rename d1b/c22 to d1b/c33 0 2024-04-25T15:49:19.562 INFO:tasks.workunit.client.0.smithi110.stdout:5/114: dwrite d4/d1d/f1e [0,4194304] 0 2024-04-25T15:49:19.564 INFO:tasks.workunit.client.0.smithi110.stdout:4/160: unlink c11 0 2024-04-25T15:49:19.566 INFO:tasks.workunit.client.0.smithi110.stdout:4/161: chown ff 24 1 2024-04-25T15:49:19.567 INFO:tasks.workunit.client.0.smithi110.stdout:7/147: dwrite f19 [0,4194304] 0 2024-04-25T15:49:19.569 INFO:tasks.workunit.client.0.smithi110.stdout:7/148: truncate d1b/f26 450904 0 2024-04-25T15:49:19.572 INFO:tasks.workunit.client.0.smithi110.stdout:5/115: creat d4/de/f27 x:0 0 0 2024-04-25T15:49:19.574 INFO:tasks.workunit.client.0.smithi110.stdout:4/162: dwrite d13/d1c/f20 [0,4194304] 0 2024-04-25T15:49:19.576 INFO:tasks.workunit.client.0.smithi110.stdout:3/236: link d0/c9 d0/d7/d1c/d2f/c52 0 2024-04-25T15:49:19.583 INFO:tasks.workunit.client.0.smithi110.stdout:5/116: rename d4/de/l1b to d4/de/l28 0 2024-04-25T15:49:19.583 INFO:tasks.workunit.client.0.smithi110.stdout:5/117: chown d4 952 1 2024-04-25T15:49:19.585 INFO:tasks.workunit.client.0.smithi110.stdout:4/163: mknod d13/d24/d26/c2f 0 2024-04-25T15:49:19.588 INFO:tasks.workunit.client.0.smithi110.stdout:4/164: rmdir d13 39 2024-04-25T15:49:19.590 INFO:tasks.workunit.client.0.smithi110.stdout:7/149: link d1b/l2d d1b/d29/l34 0 2024-04-25T15:49:19.593 INFO:tasks.workunit.client.0.smithi110.stdout:3/237: link d0/l19 d0/d16/l53 0 2024-04-25T15:49:19.594 INFO:tasks.workunit.client.0.smithi110.stdout:7/150: truncate fd 676297 0 2024-04-25T15:49:19.596 INFO:tasks.workunit.client.0.smithi110.stdout:4/165: rename d13/l29 to d13/d1c/l30 0 2024-04-25T15:49:19.597 INFO:tasks.workunit.client.0.smithi110.stdout:4/166: write fa [3800413,82802] 0 2024-04-25T15:49:19.597 INFO:tasks.workunit.client.0.smithi110.stdout:7/151: mknod d1b/c35 0 2024-04-25T15:49:19.599 INFO:tasks.workunit.client.0.smithi110.stdout:3/238: dwrite d0/d7/f17 [0,4194304] 0 2024-04-25T15:49:19.602 INFO:tasks.workunit.client.0.smithi110.stdout:3/239: write d0/d16/f1f [3996112,4329] 0 2024-04-25T15:49:19.602 INFO:tasks.workunit.client.0.smithi110.stdout:7/152: creat d1b/f36 x:0 0 0 2024-04-25T15:49:19.604 INFO:tasks.workunit.client.0.smithi110.stdout:3/240: read d0/d7/f17 [2111070,23171] 0 2024-04-25T15:49:19.607 INFO:tasks.workunit.client.0.smithi110.stdout:4/167: dwrite d13/d1c/f2c [0,4194304] 0 2024-04-25T15:49:19.611 INFO:tasks.workunit.client.0.smithi110.stdout:7/153: creat d1b/d29/f37 x:0 0 0 2024-04-25T15:49:19.612 INFO:tasks.workunit.client.0.smithi110.stdout:3/241: dread d0/d7/f29 [0,4194304] 0 2024-04-25T15:49:19.625 INFO:tasks.workunit.client.0.smithi110.stdout:4/168: link f8 d13/d24/d26/f31 0 2024-04-25T15:49:19.625 INFO:tasks.workunit.client.0.smithi110.stdout:4/169: chown ff 371944 1 2024-04-25T15:49:19.628 INFO:tasks.workunit.client.0.smithi110.stdout:3/242: symlink d0/d7/d1c/d2f/d4c/l54 0 2024-04-25T15:49:19.629 INFO:tasks.workunit.client.0.smithi110.stdout:4/170: mknod d13/d24/c32 0 2024-04-25T15:49:19.630 INFO:tasks.workunit.client.0.smithi110.stdout:4/171: stat d13/f22 0 2024-04-25T15:49:19.632 INFO:tasks.workunit.client.0.smithi110.stdout:3/243: mknod d0/d7/d1c/d2f/d4c/c55 0 2024-04-25T15:49:19.634 INFO:tasks.workunit.client.0.smithi110.stdout:4/172: readlink d13/d1c/l30 0 2024-04-25T15:49:19.636 INFO:tasks.workunit.client.0.smithi110.stdout:3/244: dread d0/d16/f1f [0,4194304] 0 2024-04-25T15:49:19.637 INFO:tasks.workunit.client.0.smithi110.stdout:4/173: write d13/f1a [512694,103649] 0 2024-04-25T15:49:19.647 INFO:tasks.workunit.client.0.smithi110.stdout:4/174: rename d13/d1c/l1e to d13/d24/l33 0 2024-04-25T15:49:19.650 INFO:tasks.workunit.client.0.smithi110.stdout:3/245: link d0/ff d0/d7/d1c/f56 0 2024-04-25T15:49:19.660 INFO:tasks.workunit.client.0.smithi110.stdout:3/246: rename d0/c4d to d0/d7/d1c/d2f/d4c/c57 0 2024-04-25T15:49:19.662 INFO:tasks.workunit.client.0.smithi110.stdout:3/247: mknod d0/d7/d40/c58 0 2024-04-25T15:49:19.665 INFO:tasks.workunit.client.0.smithi110.stdout:3/248: write d0/d2e/f3b [1420048,86752] 0 2024-04-25T15:49:19.675 INFO:tasks.workunit.client.0.smithi110.stdout:4/175: dwrite fd [0,4194304] 0 2024-04-25T15:49:19.676 INFO:tasks.workunit.client.0.smithi110.stdout:4/176: write d13/d1c/f2c [3837353,105096] 0 2024-04-25T15:49:19.677 INFO:tasks.workunit.client.0.smithi110.stdout:7/154: truncate fd 843610 0 2024-04-25T15:49:19.678 INFO:tasks.workunit.client.0.smithi110.stdout:3/249: getdents d0/d7/d1c/d2f/d4c 0 2024-04-25T15:49:19.690 INFO:tasks.workunit.client.0.smithi110.stdout:2/124: sync 2024-04-25T15:49:19.690 INFO:tasks.workunit.client.0.smithi110.stdout:1/245: sync 2024-04-25T15:49:19.690 INFO:tasks.workunit.client.0.smithi110.stdout:8/172: sync 2024-04-25T15:49:19.690 INFO:tasks.workunit.client.0.smithi110.stdout:0/191: sync 2024-04-25T15:49:19.690 INFO:tasks.workunit.client.0.smithi110.stdout:9/138: sync 2024-04-25T15:49:19.690 INFO:tasks.workunit.client.0.smithi110.stdout:6/125: sync 2024-04-25T15:49:19.691 INFO:tasks.workunit.client.0.smithi110.stdout:1/246: chown d0/d25 331 1 2024-04-25T15:49:19.692 INFO:tasks.workunit.client.0.smithi110.stdout:8/173: write ff [3227213,125178] 0 2024-04-25T15:49:19.692 INFO:tasks.workunit.client.0.smithi110.stdout:8/174: fdatasync f11 0 2024-04-25T15:49:19.692 INFO:tasks.workunit.client.0.smithi110.stdout:1/247: chown d0/d14/f1f 1 1 2024-04-25T15:49:19.693 INFO:tasks.workunit.client.0.smithi110.stdout:8/175: fdatasync f11 0 2024-04-25T15:49:19.694 INFO:tasks.workunit.client.0.smithi110.stdout:6/126: read d1/f26 [1472624,117452] 0 2024-04-25T15:49:19.699 INFO:tasks.workunit.client.0.smithi110.stdout:4/177: link d13/l21 d13/d24/d26/l34 0 2024-04-25T15:49:19.704 INFO:tasks.workunit.client.0.smithi110.stdout:0/192: dread d1/d23/d30/d33/d35/f36 [0,4194304] 0 2024-04-25T15:49:19.704 INFO:tasks.workunit.client.0.smithi110.stdout:3/250: truncate d0/d16/f18 6635 0 2024-04-25T15:49:19.704 INFO:tasks.workunit.client.0.smithi110.stdout:0/193: readlink d1/d23/l25 0 2024-04-25T15:49:19.709 INFO:tasks.workunit.client.0.smithi110.stdout:8/176: symlink d14/l33 0 2024-04-25T15:49:19.709 INFO:tasks.workunit.client.0.smithi110.stdout:8/177: fsync d14/f17 0 2024-04-25T15:49:19.710 INFO:tasks.workunit.client.0.smithi110.stdout:8/178: chown c10 64998 1 2024-04-25T15:49:19.712 INFO:tasks.workunit.client.0.smithi110.stdout:3/251: dwrite d0/f12 [0,4194304] 0 2024-04-25T15:49:19.713 INFO:tasks.workunit.client.0.smithi110.stdout:7/155: dwrite fd [0,4194304] 0 2024-04-25T15:49:19.713 INFO:tasks.workunit.client.0.smithi110.stdout:2/125: unlink de/d11/c1a 0 2024-04-25T15:49:19.713 INFO:tasks.workunit.client.0.smithi110.stdout:3/252: chown d0/d7/f29 8 1 2024-04-25T15:49:19.720 INFO:tasks.workunit.client.0.smithi110.stdout:1/248: write d0/d25/d33/f49 [3726577,23771] 0 2024-04-25T15:49:19.726 INFO:tasks.workunit.client.0.smithi110.stdout:0/194: creat d1/d23/d30/d33/f4b x:0 0 0 2024-04-25T15:49:19.729 INFO:tasks.workunit.client.0.smithi110.stdout:6/127: creat d1/db/d24/f29 x:0 0 0 2024-04-25T15:49:19.733 INFO:tasks.workunit.client.0.smithi110.stdout:2/126: creat de/d11/f1f x:0 0 0 2024-04-25T15:49:19.734 INFO:tasks.workunit.client.0.smithi110.stdout:7/156: mkdir d1b/d29/d38 0 2024-04-25T15:49:19.735 INFO:tasks.workunit.client.0.smithi110.stdout:7/157: write f14 [695167,129692] 0 2024-04-25T15:49:19.738 INFO:tasks.workunit.client.0.smithi110.stdout:9/139: creat d9/f24 x:0 0 0 2024-04-25T15:49:19.739 INFO:tasks.workunit.client.0.smithi110.stdout:9/140: stat d9/db/df/f22 0 2024-04-25T15:49:19.740 INFO:tasks.workunit.client.0.smithi110.stdout:4/178: mknod d13/d24/d2d/c35 0 2024-04-25T15:49:19.741 INFO:tasks.workunit.client.0.smithi110.stdout:4/179: write ff [5235587,57171] 0 2024-04-25T15:49:19.742 INFO:tasks.workunit.client.0.smithi110.stdout:4/180: read d13/d24/d26/f31 [479296,76066] 0 2024-04-25T15:49:19.748 INFO:tasks.workunit.client.0.smithi110.stdout:4/181: dwrite d13/f22 [0,4194304] 0 2024-04-25T15:49:19.749 INFO:tasks.workunit.client.0.smithi110.stdout:4/182: read fd [2848279,89944] 0 2024-04-25T15:49:19.750 INFO:tasks.workunit.client.0.smithi110.stdout:2/127: unlink de/l1b 0 2024-04-25T15:49:19.753 INFO:tasks.workunit.client.0.smithi110.stdout:3/253: creat d0/d7/d1c/d50/f59 x:0 0 0 2024-04-25T15:49:19.754 INFO:tasks.workunit.client.0.smithi110.stdout:9/141: mkdir d9/db/df/d25 0 2024-04-25T15:49:19.758 INFO:tasks.workunit.client.0.smithi110.stdout:7/158: dwrite f4 [4194304,4194304] 0 2024-04-25T15:49:19.763 INFO:tasks.workunit.client.0.smithi110.stdout:0/195: dread d1/f19 [0,4194304] 0 2024-04-25T15:49:19.764 INFO:tasks.workunit.client.0.smithi110.stdout:6/128: link f0 d1/d12/f2a 0 2024-04-25T15:49:19.765 INFO:tasks.workunit.client.0.smithi110.stdout:6/129: fsync d1/dd/f17 0 2024-04-25T15:49:19.768 INFO:tasks.workunit.client.0.smithi110.stdout:4/183: mkdir d13/d24/d2d/d36 0 2024-04-25T15:49:19.770 INFO:tasks.workunit.client.0.smithi110.stdout:2/128: fdatasync de/d11/f19 0 2024-04-25T15:49:19.772 INFO:tasks.workunit.client.0.smithi110.stdout:3/254: mknod d0/d7/d40/c5a 0 2024-04-25T15:49:19.775 INFO:tasks.workunit.client.0.smithi110.stdout:7/159: symlink d1b/l39 0 2024-04-25T15:49:19.778 INFO:tasks.workunit.client.0.smithi110.stdout:3/255: dwrite d0/d2e/f3b [0,4194304] 0 2024-04-25T15:49:19.781 INFO:tasks.workunit.client.0.smithi110.stdout:0/196: unlink d1/d23/d30/f3c 0 2024-04-25T15:49:19.784 INFO:tasks.workunit.client.0.smithi110.stdout:7/160: dwrite d1b/d29/f37 [0,4194304] 0 2024-04-25T15:49:19.784 INFO:tasks.workunit.client.0.smithi110.stdout:7/161: chown d1b/d29 6769 1 2024-04-25T15:49:19.790 INFO:tasks.workunit.client.0.smithi110.stdout:4/184: creat d13/d24/d2d/f37 x:0 0 0 2024-04-25T15:49:19.794 INFO:tasks.workunit.client.0.smithi110.stdout:4/185: dread f8 [0,4194304] 0 2024-04-25T15:49:19.797 INFO:tasks.workunit.client.0.smithi110.stdout:9/142: mknod d9/db/df/d25/c26 0 2024-04-25T15:49:19.797 INFO:tasks.workunit.client.0.smithi110.stdout:4/186: dread f8 [4194304,4194304] 0 2024-04-25T15:49:19.799 INFO:tasks.workunit.client.0.smithi110.stdout:1/249: link d0/d14/l3c d0/d23/l4c 0 2024-04-25T15:49:19.804 INFO:tasks.workunit.client.0.smithi110.stdout:3/256: mknod d0/d2e/c5b 0 2024-04-25T15:49:19.807 INFO:tasks.workunit.client.0.smithi110.stdout:7/162: symlink d1b/d29/l3a 0 2024-04-25T15:49:19.809 INFO:tasks.workunit.client.0.smithi110.stdout:6/130: link d1/db/d24/f29 d1/db/d22/d27/f2b 0 2024-04-25T15:49:19.811 INFO:tasks.workunit.client.0.smithi110.stdout:7/163: dread f11 [4194304,4194304] 0 2024-04-25T15:49:19.811 INFO:tasks.workunit.client.0.smithi110.stdout:7/164: stat d1b 0 2024-04-25T15:49:19.813 INFO:tasks.workunit.client.0.smithi110.stdout:6/131: dread d1/dd/f21 [0,4194304] 0 2024-04-25T15:49:19.814 INFO:tasks.workunit.client.0.smithi110.stdout:6/132: readlink d1/db/l16 0 2024-04-25T15:49:19.815 INFO:tasks.workunit.client.0.smithi110.stdout:9/143: rename d9/d13/f1f to d9/d13/f27 0 2024-04-25T15:49:19.820 INFO:tasks.workunit.client.0.smithi110.stdout:3/257: symlink d0/d7/d1c/d50/l5c 0 2024-04-25T15:49:19.823 INFO:tasks.workunit.client.0.smithi110.stdout:0/197: mkdir d1/d4c 0 2024-04-25T15:49:19.823 INFO:tasks.workunit.client.0.smithi110.stdout:1/250: dwrite d0/d14/f2c [0,4194304] 0 2024-04-25T15:49:19.825 INFO:tasks.workunit.client.0.smithi110.stdout:1/251: dread d0/d14/f4b [0,4194304] 0 2024-04-25T15:49:19.827 INFO:tasks.workunit.client.0.smithi110.stdout:1/252: truncate d0/d25/f3d 760907 0 2024-04-25T15:49:19.829 INFO:tasks.workunit.client.0.smithi110.stdout:6/133: dread d1/d12/f1f [0,4194304] 0 2024-04-25T15:49:19.836 INFO:tasks.workunit.client.0.smithi110.stdout:4/187: rename d13/l19 to d13/d24/d26/l38 0 2024-04-25T15:49:19.841 INFO:tasks.workunit.client.0.smithi110.stdout:4/188: dwrite ff [4194304,4194304] 0 2024-04-25T15:49:19.852 INFO:tasks.workunit.client.0.smithi110.stdout:1/253: symlink d0/d14/l4d 0 2024-04-25T15:49:19.853 INFO:tasks.workunit.client.0.smithi110.stdout:1/254: write d0/f3f [1639641,69273] 0 2024-04-25T15:49:19.856 INFO:tasks.workunit.client.0.smithi110.stdout:1/255: read d0/d25/d33/f24 [4176450,105496] 0 2024-04-25T15:49:19.857 INFO:tasks.workunit.client.0.smithi110.stdout:6/134: unlink d1/db/d22/c23 0 2024-04-25T15:49:19.858 INFO:tasks.workunit.client.0.smithi110.stdout:6/135: truncate d1/f1d 721859 0 2024-04-25T15:49:19.858 INFO:tasks.workunit.client.0.smithi110.stdout:2/129: getdents de 0 2024-04-25T15:49:19.859 INFO:tasks.workunit.client.0.smithi110.stdout:2/130: write de/d11/f1f [395100,53133] 0 2024-04-25T15:49:19.863 INFO:tasks.workunit.client.0.smithi110.stdout:2/131: dread de/d11/f1f [0,4194304] 0 2024-04-25T15:49:19.866 INFO:tasks.workunit.client.0.smithi110.stdout:3/258: symlink d0/l5d 0 2024-04-25T15:49:19.870 INFO:tasks.workunit.client.0.smithi110.stdout:3/259: dread d0/d7/f17 [0,4194304] 0 2024-04-25T15:49:19.872 INFO:tasks.workunit.client.0.smithi110.stdout:1/256: creat d0/d23/f4e x:0 0 0 2024-04-25T15:49:19.874 INFO:tasks.workunit.client.0.smithi110.stdout:6/136: write d1/d12/f1f [1332530,87930] 0 2024-04-25T15:49:19.877 INFO:tasks.workunit.client.0.smithi110.stdout:2/132: symlink de/l20 0 2024-04-25T15:49:19.880 INFO:tasks.workunit.client.0.smithi110.stdout:0/198: creat d1/f4d x:0 0 0 2024-04-25T15:49:19.883 INFO:tasks.workunit.client.0.smithi110.stdout:7/165: truncate fb 71391 0 2024-04-25T15:49:19.883 INFO:tasks.workunit.client.0.smithi110.stdout:2/133: dwrite fd [0,4194304] 0 2024-04-25T15:49:19.885 INFO:tasks.workunit.client.0.smithi110.stdout:7/166: write d1b/d29/f2b [285077,25491] 0 2024-04-25T15:49:19.887 INFO:tasks.workunit.client.0.smithi110.stdout:6/137: rmdir d1/d12 39 2024-04-25T15:49:19.889 INFO:tasks.workunit.client.0.smithi110.stdout:9/144: write f0 [803378,75293] 0 2024-04-25T15:49:19.890 INFO:tasks.workunit.client.0.smithi110.stdout:6/138: chown d1/l7 0 1 2024-04-25T15:49:19.891 INFO:tasks.workunit.client.0.smithi110.stdout:0/199: dwrite d1/f49 [0,4194304] 0 2024-04-25T15:49:19.892 INFO:tasks.workunit.client.0.smithi110.stdout:6/139: chown d1/db/d22/d27 619 1 2024-04-25T15:49:19.897 INFO:tasks.workunit.client.0.smithi110.stdout:7/167: dwrite d1b/d29/f37 [0,4194304] 0 2024-04-25T15:49:19.962 INFO:tasks.workunit.client.0.smithi110.stdout:9/145: rmdir d9/d13 39 2024-04-25T15:49:19.963 INFO:tasks.workunit.client.0.smithi110.stdout:9/146: write f8 [3666735,6048] 0 2024-04-25T15:49:19.964 INFO:tasks.workunit.client.0.smithi110.stdout:4/189: getdents d13/d24 0 2024-04-25T15:49:19.967 INFO:tasks.workunit.client.0.smithi110.stdout:6/140: creat d1/db/d22/f2c x:0 0 0 2024-04-25T15:49:19.968 INFO:tasks.workunit.client.0.smithi110.stdout:6/141: chown d1/db/l16 5 1 2024-04-25T15:49:19.971 INFO:tasks.workunit.client.0.smithi110.stdout:3/260: truncate d0/d7/fa 400846 0 2024-04-25T15:49:19.977 INFO:tasks.workunit.client.0.smithi110.stdout:1/257: getdents d0/d14 0 2024-04-25T15:49:19.978 INFO:tasks.workunit.client.0.smithi110.stdout:6/142: mkdir d1/db/d24/d2d 0 2024-04-25T15:49:19.979 INFO:tasks.workunit.client.0.smithi110.stdout:7/168: rename d1b/l1f to d1b/d29/d38/l3b 0 2024-04-25T15:49:19.981 INFO:tasks.workunit.client.0.smithi110.stdout:2/134: truncate de/f10 267216 0 2024-04-25T15:49:19.982 INFO:tasks.workunit.client.0.smithi110.stdout:2/135: chown fb 45418771 1 2024-04-25T15:49:19.986 INFO:tasks.workunit.client.0.smithi110.stdout:1/258: mknod d0/d25/d33/c4f 0 2024-04-25T15:49:19.986 INFO:tasks.workunit.client.0.smithi110.stdout:3/261: dwrite d0/d16/f1f [0,4194304] 0 2024-04-25T15:49:19.987 INFO:tasks.workunit.client.0.smithi110.stdout:3/262: truncate d0/d2e/f3b 4955776 0 2024-04-25T15:49:19.991 INFO:tasks.workunit.client.0.smithi110.stdout:6/143: symlink d1/db/d22/l2e 0 2024-04-25T15:49:19.995 INFO:tasks.workunit.client.0.smithi110.stdout:2/136: unlink fd 0 2024-04-25T15:49:19.996 INFO:tasks.workunit.client.0.smithi110.stdout:0/200: getdents d1/d23/d30/d33 0 2024-04-25T15:49:19.997 INFO:tasks.workunit.client.0.smithi110.stdout:7/169: rmdir d1b/d29 39 2024-04-25T15:49:19.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/170: fsync fd 0 2024-04-25T15:49:19.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/171: fsync fd 0 2024-04-25T15:49:20.000 INFO:tasks.workunit.client.0.smithi110.stdout:1/259: creat d0/d25/d33/f50 x:0 0 0 2024-04-25T15:49:20.001 INFO:tasks.workunit.client.0.smithi110.stdout:3/263: symlink d0/d7/l5e 0 2024-04-25T15:49:20.001 INFO:tasks.workunit.client.0.smithi110.stdout:4/190: truncate d13/f22 3080296 0 2024-04-25T15:49:20.002 INFO:tasks.workunit.client.0.smithi110.stdout:6/144: creat d1/dd/f2f x:0 0 0 2024-04-25T15:49:20.003 INFO:tasks.workunit.client.0.smithi110.stdout:0/201: dwrite d1/f49 [0,4194304] 0 2024-04-25T15:49:20.014 INFO:tasks.workunit.client.0.smithi110.stdout:1/260: mkdir d0/d51 0 2024-04-25T15:49:20.016 INFO:tasks.workunit.client.0.smithi110.stdout:4/191: mknod d13/d24/d2d/c39 0 2024-04-25T15:49:20.016 INFO:tasks.workunit.client.0.smithi110.stdout:4/192: chown c10 7365 1 2024-04-25T15:49:20.020 INFO:tasks.workunit.client.0.smithi110.stdout:9/147: link d9/l20 d9/l28 0 2024-04-25T15:49:20.021 INFO:tasks.workunit.client.0.smithi110.stdout:1/261: dwrite d0/d25/f6 [4194304,4194304] 0 2024-04-25T15:49:20.022 INFO:tasks.workunit.client.0.smithi110.stdout:1/262: chown d0/d25/d33/f47 228170072 1 2024-04-25T15:49:20.029 INFO:tasks.workunit.client.0.smithi110.stdout:4/193: mknod d13/d24/c3a 0 2024-04-25T15:49:20.030 INFO:tasks.workunit.client.0.smithi110.stdout:4/194: chown d13/c27 46 1 2024-04-25T15:49:20.038 INFO:tasks.workunit.client.0.smithi110.stdout:7/172: dwrite f1a [0,4194304] 0 2024-04-25T15:49:20.040 INFO:tasks.workunit.client.0.smithi110.stdout:6/145: creat d1/f30 x:0 0 0 2024-04-25T15:49:20.040 INFO:tasks.workunit.client.0.smithi110.stdout:5/118: sync 2024-04-25T15:49:20.041 INFO:tasks.workunit.client.0.smithi110.stdout:3/264: stat d0/f1e 0 2024-04-25T15:49:20.041 INFO:tasks.workunit.client.0.smithi110.stdout:0/202: creat d1/f4e x:0 0 0 2024-04-25T15:49:20.042 INFO:tasks.workunit.client.0.smithi110.stdout:2/137: write de/d11/f19 [185472,33925] 0 2024-04-25T15:49:20.043 INFO:tasks.workunit.client.0.smithi110.stdout:1/263: dread d0/d25/d33/f49 [0,4194304] 0 2024-04-25T15:49:20.045 INFO:tasks.workunit.client.0.smithi110.stdout:4/195: creat d13/d24/d26/f3b x:0 0 0 2024-04-25T15:49:20.045 INFO:tasks.workunit.client.0.smithi110.stdout:4/196: chown d13/f1a 0 1 2024-04-25T15:49:20.047 INFO:tasks.workunit.client.0.smithi110.stdout:7/173: dread f2 [4194304,4194304] 0 2024-04-25T15:49:20.051 INFO:tasks.workunit.client.0.smithi110.stdout:6/146: read d1/dd/f21 [357087,49943] 0 2024-04-25T15:49:20.052 INFO:tasks.workunit.client.0.smithi110.stdout:6/147: write d1/f1d [556261,56861] 0 2024-04-25T15:49:20.053 INFO:tasks.workunit.client.0.smithi110.stdout:2/138: creat de/d11/f21 x:0 0 0 2024-04-25T15:49:20.053 INFO:tasks.workunit.client.0.smithi110.stdout:6/148: chown d1/db/d24/f29 0 1 2024-04-25T15:49:20.054 INFO:tasks.workunit.client.0.smithi110.stdout:3/265: chown d0/d7/d1c/d2f/d33/c37 0 1 2024-04-25T15:49:20.054 INFO:tasks.workunit.client.0.smithi110.stdout:6/149: chown d1/f9 3 1 2024-04-25T15:49:20.055 INFO:tasks.workunit.client.0.smithi110.stdout:3/266: truncate d0/d7/d40/f43 711430 0 2024-04-25T15:49:20.056 INFO:tasks.workunit.client.0.smithi110.stdout:1/264: truncate d0/d25/d33/f24 3298240 0 2024-04-25T15:49:20.057 INFO:tasks.workunit.client.0.smithi110.stdout:4/197: fdatasync d13/d24/f2a 0 2024-04-25T15:49:20.060 INFO:tasks.workunit.client.0.smithi110.stdout:1/265: dread d0/d14/f2c [0,4194304] 0 2024-04-25T15:49:20.061 INFO:tasks.workunit.client.0.smithi110.stdout:7/174: dread fb [0,4194304] 0 2024-04-25T15:49:20.061 INFO:tasks.workunit.client.0.smithi110.stdout:5/119: rename d4/d1d/d1f/l21 to d4/d15/l29 0 2024-04-25T15:49:20.061 INFO:tasks.workunit.client.0.smithi110.stdout:7/175: fdatasync fd 0 2024-04-25T15:49:20.062 INFO:tasks.workunit.client.0.smithi110.stdout:0/203: symlink d1/l4f 0 2024-04-25T15:49:20.065 INFO:tasks.workunit.client.0.smithi110.stdout:9/148: getdents d9 0 2024-04-25T15:49:20.066 INFO:tasks.workunit.client.0.smithi110.stdout:6/150: dread - d1/d12/f1b zero size 2024-04-25T15:49:20.068 INFO:tasks.workunit.client.0.smithi110.stdout:0/204: dread d1/d23/d30/d33/d35/f12 [0,4194304] 0 2024-04-25T15:49:20.068 INFO:tasks.workunit.client.0.smithi110.stdout:6/151: read d1/f9 [818300,60689] 0 2024-04-25T15:49:20.069 INFO:tasks.workunit.client.0.smithi110.stdout:4/198: stat d13/d24/d26/l38 0 2024-04-25T15:49:20.071 INFO:tasks.workunit.client.0.smithi110.stdout:7/176: dwrite f14 [0,4194304] 0 2024-04-25T15:49:20.073 INFO:tasks.workunit.client.0.smithi110.stdout:5/120: mkdir d4/de/d2a 0 2024-04-25T15:49:20.075 INFO:tasks.workunit.client.0.smithi110.stdout:5/121: chown d4/d1d/c26 0 1 2024-04-25T15:49:20.079 INFO:tasks.workunit.client.0.smithi110.stdout:9/149: creat d9/d13/f29 x:0 0 0 2024-04-25T15:49:20.081 INFO:tasks.workunit.client.0.smithi110.stdout:4/199: mkdir d13/d24/d3c 0 2024-04-25T15:49:20.083 INFO:tasks.workunit.client.0.smithi110.stdout:8/179: sync 2024-04-25T15:49:20.083 INFO:tasks.workunit.client.0.smithi110.stdout:5/122: symlink d4/d15/l2b 0 2024-04-25T15:49:20.084 INFO:tasks.workunit.client.0.smithi110.stdout:3/267: creat d0/d7/d1c/f5f x:0 0 0 2024-04-25T15:49:20.086 INFO:tasks.workunit.client.0.smithi110.stdout:9/150: symlink d9/l2a 0 2024-04-25T15:49:20.088 INFO:tasks.workunit.client.0.smithi110.stdout:2/139: rename de/d11/f1f to de/f22 0 2024-04-25T15:49:20.090 INFO:tasks.workunit.client.0.smithi110.stdout:7/177: mknod d1b/d29/c3c 0 2024-04-25T15:49:20.092 INFO:tasks.workunit.client.0.smithi110.stdout:0/205: link d1/d23/d30/d33/d35/f12 d1/d20/f50 0 2024-04-25T15:49:20.092 INFO:tasks.workunit.client.0.smithi110.stdout:0/206: chown d1/d3e 19537062 1 2024-04-25T15:49:20.096 INFO:tasks.workunit.client.0.smithi110.stdout:3/268: unlink d0/c46 0 2024-04-25T15:49:20.101 INFO:tasks.workunit.client.0.smithi110.stdout:1/266: rename d0/d14/f32 to d0/d51/f52 0 2024-04-25T15:49:20.103 INFO:tasks.workunit.client.0.smithi110.stdout:6/152: link d1/db/d22/l2e d1/db/d24/l31 0 2024-04-25T15:49:20.104 INFO:tasks.workunit.client.0.smithi110.stdout:1/267: dread d0/f3f [0,4194304] 0 2024-04-25T15:49:20.109 INFO:tasks.workunit.client.0.smithi110.stdout:2/140: symlink de/l23 0 2024-04-25T15:49:20.110 INFO:tasks.workunit.client.0.smithi110.stdout:1/268: dwrite d0/f3f [0,4194304] 0 2024-04-25T15:49:20.111 INFO:tasks.workunit.client.0.smithi110.stdout:1/269: dread - d0/d23/f3b zero size 2024-04-25T15:49:20.111 INFO:tasks.workunit.client.0.smithi110.stdout:4/200: write f8 [8875130,25291] 0 2024-04-25T15:49:20.117 INFO:tasks.workunit.client.0.smithi110.stdout:4/201: dread fc [0,4194304] 0 2024-04-25T15:49:20.117 INFO:tasks.workunit.client.0.smithi110.stdout:4/202: readlink d13/l18 0 2024-04-25T15:49:20.120 INFO:tasks.workunit.client.0.smithi110.stdout:8/180: rename d14/d21/d26/l2f to d14/l34 0 2024-04-25T15:49:20.123 INFO:tasks.workunit.client.0.smithi110.stdout:0/207: creat d1/d3e/f51 x:0 0 0 2024-04-25T15:49:20.125 INFO:tasks.workunit.client.0.smithi110.stdout:5/123: rename d4/de to d4/d2c 0 2024-04-25T15:49:20.125 INFO:tasks.workunit.client.0.smithi110.stdout:5/124: chown d4/d1d/d1f 4754949 1 2024-04-25T15:49:20.126 INFO:tasks.workunit.client.0.smithi110.stdout:5/125: write d4/f8 [344096,55952] 0 2024-04-25T15:49:20.129 INFO:tasks.workunit.client.0.smithi110.stdout:3/269: write d0/f1e [568072,8384] 0 2024-04-25T15:49:20.133 INFO:tasks.workunit.client.0.smithi110.stdout:8/181: mkdir d14/d21/d35 0 2024-04-25T15:49:20.135 INFO:tasks.workunit.client.0.smithi110.stdout:2/141: dread de/d11/f19 [0,4194304] 0 2024-04-25T15:49:20.136 INFO:tasks.workunit.client.0.smithi110.stdout:5/126: dwrite d4/f8 [0,4194304] 0 2024-04-25T15:49:20.139 INFO:tasks.workunit.client.0.smithi110.stdout:5/127: chown d4/f6 14209505 1 2024-04-25T15:49:20.143 INFO:tasks.workunit.client.0.smithi110.stdout:4/203: creat d13/f3d x:0 0 0 2024-04-25T15:49:20.143 INFO:tasks.workunit.client.0.smithi110.stdout:3/270: dwrite d0/d2e/f30 [4194304,4194304] 0 2024-04-25T15:49:20.144 INFO:tasks.workunit.client.0.smithi110.stdout:9/151: link c7 d9/c2b 0 2024-04-25T15:49:20.145 INFO:tasks.workunit.client.0.smithi110.stdout:8/182: mkdir d14/d32/d36 0 2024-04-25T15:49:20.147 INFO:tasks.workunit.client.0.smithi110.stdout:5/128: write d4/f25 [845295,61436] 0 2024-04-25T15:49:20.148 INFO:tasks.workunit.client.0.smithi110.stdout:0/208: dwrite d1/f2a [0,4194304] 0 2024-04-25T15:49:20.149 INFO:tasks.workunit.client.0.smithi110.stdout:8/183: fsync d14/d21/d26/f29 0 2024-04-25T15:49:20.151 INFO:tasks.workunit.client.0.smithi110.stdout:4/204: symlink d13/d24/d26/l3e 0 2024-04-25T15:49:20.152 INFO:tasks.workunit.client.0.smithi110.stdout:9/152: fdatasync d9/db/df/f22 0 2024-04-25T15:49:20.153 INFO:tasks.workunit.client.0.smithi110.stdout:2/142: creat de/f24 x:0 0 0 2024-04-25T15:49:20.157 INFO:tasks.workunit.client.0.smithi110.stdout:2/143: chown fb 110848 1 2024-04-25T15:49:20.159 INFO:tasks.workunit.client.0.smithi110.stdout:5/129: creat d4/d1d/f2d x:0 0 0 2024-04-25T15:49:20.159 INFO:tasks.workunit.client.0.smithi110.stdout:3/271: dwrite d0/d7/fa [0,4194304] 0 2024-04-25T15:49:20.162 INFO:tasks.workunit.client.0.smithi110.stdout:9/153: mkdir d9/db/df/d2c 0 2024-04-25T15:49:20.162 INFO:tasks.workunit.client.0.smithi110.stdout:9/154: write f6 [7338385,88265] 0 2024-04-25T15:49:20.166 INFO:tasks.workunit.client.0.smithi110.stdout:0/209: dread d1/f19 [0,4194304] 0 2024-04-25T15:49:20.167 INFO:tasks.workunit.client.0.smithi110.stdout:4/205: symlink d13/l3f 0 2024-04-25T15:49:20.170 INFO:tasks.workunit.client.0.smithi110.stdout:6/153: dwrite d1/db/f19 [0,4194304] 0 2024-04-25T15:49:20.172 INFO:tasks.workunit.client.0.smithi110.stdout:6/154: stat d1/db/d24/f29 0 2024-04-25T15:49:20.173 INFO:tasks.workunit.client.0.smithi110.stdout:3/272: symlink d0/d7/l60 0 2024-04-25T15:49:20.180 INFO:tasks.workunit.client.0.smithi110.stdout:1/270: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:20.181 INFO:tasks.workunit.client.0.smithi110.stdout:8/184: rmdir d14/d21/d35 0 2024-04-25T15:49:20.183 INFO:tasks.workunit.client.0.smithi110.stdout:1/271: dread - d0/d25/d33/f45 zero size 2024-04-25T15:49:20.184 INFO:tasks.workunit.client.0.smithi110.stdout:8/185: write d14/f15 [1429497,34196] 0 2024-04-25T15:49:20.184 INFO:tasks.workunit.client.0.smithi110.stdout:4/206: dwrite d13/d1c/f25 [0,4194304] 0 2024-04-25T15:49:20.187 INFO:tasks.workunit.client.0.smithi110.stdout:8/186: chown d14/d21/d26/f30 323188227 1 2024-04-25T15:49:20.187 INFO:tasks.workunit.client.0.smithi110.stdout:1/272: chown d0/d25/d33/f50 3857486 1 2024-04-25T15:49:20.189 INFO:tasks.workunit.client.0.smithi110.stdout:5/130: mkdir d4/d2e 0 2024-04-25T15:49:20.189 INFO:tasks.workunit.client.0.smithi110.stdout:3/273: creat d0/d7/d40/f61 x:0 0 0 2024-04-25T15:49:20.190 INFO:tasks.workunit.client.0.smithi110.stdout:9/155: mkdir d9/d2d 0 2024-04-25T15:49:20.191 INFO:tasks.workunit.client.0.smithi110.stdout:6/155: mkdir d1/db/d24/d32 0 2024-04-25T15:49:20.192 INFO:tasks.workunit.client.0.smithi110.stdout:0/210: dread d1/f32 [0,4194304] 0 2024-04-25T15:49:20.197 INFO:tasks.workunit.client.0.smithi110.stdout:1/273: dwrite d0/d25/d33/f50 [0,4194304] 0 2024-04-25T15:49:20.200 INFO:tasks.workunit.client.0.smithi110.stdout:6/156: mkdir d1/db/d22/d33 0 2024-04-25T15:49:20.204 INFO:tasks.workunit.client.0.smithi110.stdout:3/274: rename d0/d16/c26 to d0/d7/d1c/d50/c62 0 2024-04-25T15:49:20.207 INFO:tasks.workunit.client.0.smithi110.stdout:4/207: creat d13/d24/d2d/d36/f40 x:0 0 0 2024-04-25T15:49:20.208 INFO:tasks.workunit.client.0.smithi110.stdout:4/208: stat c10 0 2024-04-25T15:49:20.209 INFO:tasks.workunit.client.0.smithi110.stdout:9/156: dwrite d9/f24 [0,4194304] 0 2024-04-25T15:49:20.213 INFO:tasks.workunit.client.0.smithi110.stdout:0/211: rename d1/d20/f50 to d1/d47/f52 0 2024-04-25T15:49:20.216 INFO:tasks.workunit.client.0.smithi110.stdout:3/275: dread d0/d16/f18 [0,4194304] 0 2024-04-25T15:49:20.217 INFO:tasks.workunit.client.0.smithi110.stdout:0/212: chown d1/d23/d30/d33/d35/lc 416512 1 2024-04-25T15:49:20.217 INFO:tasks.workunit.client.0.smithi110.stdout:2/144: dwrite de/f10 [0,4194304] 0 2024-04-25T15:49:20.220 INFO:tasks.workunit.client.0.smithi110.stdout:0/213: truncate d1/d23/d30/d33/f4b 497441 0 2024-04-25T15:49:20.224 INFO:tasks.workunit.client.0.smithi110.stdout:6/157: rename d1/f30 to d1/db/d24/d2d/f34 0 2024-04-25T15:49:20.226 INFO:tasks.workunit.client.0.smithi110.stdout:6/158: write d1/d12/f1b [1034498,80832] 0 2024-04-25T15:49:20.226 INFO:tasks.workunit.client.0.smithi110.stdout:8/187: getdents d14 0 2024-04-25T15:49:20.227 INFO:tasks.workunit.client.0.smithi110.stdout:5/131: link d4/d2c/f1c d4/f2f 0 2024-04-25T15:49:20.229 INFO:tasks.workunit.client.0.smithi110.stdout:3/276: mkdir d0/d7/d1c/d2f/d4c/d63 0 2024-04-25T15:49:20.235 INFO:tasks.workunit.client.0.smithi110.stdout:6/159: dwrite d1/f3 [0,4194304] 0 2024-04-25T15:49:20.236 INFO:tasks.workunit.client.0.smithi110.stdout:3/277: dread d0/f12 [0,4194304] 0 2024-04-25T15:49:20.236 INFO:tasks.workunit.client.0.smithi110.stdout:6/160: chown d1/d12/f1f 302109 1 2024-04-25T15:49:20.237 INFO:tasks.workunit.client.0.smithi110.stdout:3/278: chown d0/d7/l4e 7 1 2024-04-25T15:49:20.239 INFO:tasks.workunit.client.0.smithi110.stdout:5/132: symlink d4/d1d/l30 0 2024-04-25T15:49:20.242 INFO:tasks.workunit.client.0.smithi110.stdout:1/274: dread d0/d23/f3e [0,4194304] 0 2024-04-25T15:49:20.242 INFO:tasks.workunit.client.0.smithi110.stdout:0/214: creat d1/d4c/f53 x:0 0 0 2024-04-25T15:49:20.243 INFO:tasks.workunit.client.0.smithi110.stdout:1/275: chown d0/d14/f1d 334 1 2024-04-25T15:49:20.243 INFO:tasks.workunit.client.0.smithi110.stdout:0/215: write d1/d23/d30/d33/f2d [1375940,87311] 0 2024-04-25T15:49:20.244 INFO:tasks.workunit.client.0.smithi110.stdout:2/145: creat de/f25 x:0 0 0 2024-04-25T15:49:20.246 INFO:tasks.workunit.client.0.smithi110.stdout:4/209: rmdir d13/d24/d2d 39 2024-04-25T15:49:20.249 INFO:tasks.workunit.client.0.smithi110.stdout:3/279: mknod d0/d7/d1c/d50/c64 0 2024-04-25T15:49:20.252 INFO:tasks.workunit.client.0.smithi110.stdout:9/157: dread d9/db/f11 [0,4194304] 0 2024-04-25T15:49:20.258 INFO:tasks.workunit.client.0.smithi110.stdout:9/158: dwrite d9/f24 [0,4194304] 0 2024-04-25T15:49:20.258 INFO:tasks.workunit.client.0.smithi110.stdout:2/146: unlink de/l20 0 2024-04-25T15:49:20.259 INFO:tasks.workunit.client.0.smithi110.stdout:2/147: readlink de/l23 0 2024-04-25T15:49:20.260 INFO:tasks.workunit.client.0.smithi110.stdout:8/188: link c7 d14/d32/d36/c37 0 2024-04-25T15:49:20.263 INFO:tasks.workunit.client.0.smithi110.stdout:6/161: symlink d1/db/l35 0 2024-04-25T15:49:20.264 INFO:tasks.workunit.client.0.smithi110.stdout:0/216: dwrite d1/f16 [4194304,4194304] 0 2024-04-25T15:49:20.264 INFO:tasks.workunit.client.0.smithi110.stdout:6/162: chown d1/f1d 15 1 2024-04-25T15:49:20.266 INFO:tasks.workunit.client.0.smithi110.stdout:3/280: symlink d0/d7/d1c/d2f/d4c/l65 0 2024-04-25T15:49:20.271 INFO:tasks.workunit.client.0.smithi110.stdout:0/217: write d1/d23/d30/d33/f2d [1233801,27388] 0 2024-04-25T15:49:20.273 INFO:tasks.workunit.client.0.smithi110.stdout:5/133: truncate d4/fd 1116832 0 2024-04-25T15:49:20.273 INFO:tasks.workunit.client.0.smithi110.stdout:8/189: dread d14/f17 [0,4194304] 0 2024-04-25T15:49:20.276 INFO:tasks.workunit.client.0.smithi110.stdout:3/281: dread d0/d7/d1c/d2f/d33/f36 [0,4194304] 0 2024-04-25T15:49:20.278 INFO:tasks.workunit.client.0.smithi110.stdout:2/148: fdatasync de/d11/f19 0 2024-04-25T15:49:20.279 INFO:tasks.workunit.client.0.smithi110.stdout:5/134: dread d4/d2c/f17 [0,4194304] 0 2024-04-25T15:49:20.283 INFO:tasks.workunit.client.0.smithi110.stdout:9/159: dread f6 [4194304,4194304] 0 2024-04-25T15:49:20.285 INFO:tasks.workunit.client.0.smithi110.stdout:0/218: mkdir d1/d47/d54 0 2024-04-25T15:49:20.285 INFO:tasks.workunit.client.0.smithi110.stdout:1/276: write d0/f27 [3951610,27726] 0 2024-04-25T15:49:20.286 INFO:tasks.workunit.client.0.smithi110.stdout:3/282: dwrite d0/d7/d1c/d2f/d33/f36 [4194304,4194304] 0 2024-04-25T15:49:20.288 INFO:tasks.workunit.client.0.smithi110.stdout:8/190: rmdir d14/d21 39 2024-04-25T15:49:20.293 INFO:tasks.workunit.client.0.smithi110.stdout:6/163: mkdir d1/db/d22/d33/d36 0 2024-04-25T15:49:20.295 INFO:tasks.workunit.client.0.smithi110.stdout:6/164: readlink d1/db/l16 0 2024-04-25T15:49:20.295 INFO:tasks.workunit.client.0.smithi110.stdout:9/160: dwrite d9/db/df/f21 [4194304,4194304] 0 2024-04-25T15:49:20.303 INFO:tasks.workunit.client.0.smithi110.stdout:0/219: rename d1/d23/d30/d33/d35/l22 to d1/d23/d30/d33/d35/l55 0 2024-04-25T15:49:20.304 INFO:tasks.workunit.client.0.smithi110.stdout:8/191: chown d14/d32/d36/c37 925434078 1 2024-04-25T15:49:20.311 INFO:tasks.workunit.client.0.smithi110.stdout:6/165: symlink d1/db/d24/d2d/l37 0 2024-04-25T15:49:20.311 INFO:tasks.workunit.client.0.smithi110.stdout:8/192: dwrite d14/f15 [0,4194304] 0 2024-04-25T15:49:20.315 INFO:tasks.workunit.client.0.smithi110.stdout:9/161: dwrite f8 [4194304,4194304] 0 2024-04-25T15:49:20.318 INFO:tasks.workunit.client.0.smithi110.stdout:2/149: write de/f22 [1436883,6787] 0 2024-04-25T15:49:20.322 INFO:tasks.workunit.client.0.smithi110.stdout:2/150: write de/f25 [211091,110421] 0 2024-04-25T15:49:20.322 INFO:tasks.workunit.client.0.smithi110.stdout:5/135: link d4/d15/l18 d4/d2c/d2a/l31 0 2024-04-25T15:49:20.325 INFO:tasks.workunit.client.0.smithi110.stdout:5/136: dread d4/f8 [0,4194304] 0 2024-04-25T15:49:20.326 INFO:tasks.workunit.client.0.smithi110.stdout:6/166: creat d1/db/d22/f38 x:0 0 0 2024-04-25T15:49:20.327 INFO:tasks.workunit.client.0.smithi110.stdout:1/277: truncate d0/d51/f52 167853 0 2024-04-25T15:49:20.328 INFO:tasks.workunit.client.0.smithi110.stdout:3/283: truncate d0/d7/fa 2663826 0 2024-04-25T15:49:20.329 INFO:tasks.workunit.client.0.smithi110.stdout:0/220: symlink d1/d47/d54/l56 0 2024-04-25T15:49:20.330 INFO:tasks.workunit.client.0.smithi110.stdout:5/137: read d4/d2c/f19 [2789135,100834] 0 2024-04-25T15:49:20.332 INFO:tasks.workunit.client.0.smithi110.stdout:8/193: creat d14/d21/d23/f38 x:0 0 0 2024-04-25T15:49:20.333 INFO:tasks.workunit.client.0.smithi110.stdout:8/194: chown d14/d21/f2b 5069754 1 2024-04-25T15:49:20.335 INFO:tasks.workunit.client.0.smithi110.stdout:8/195: truncate d14/d21/d26/f29 825189 0 2024-04-25T15:49:20.335 INFO:tasks.workunit.client.0.smithi110.stdout:9/162: mknod d9/d13/c2e 0 2024-04-25T15:49:20.336 INFO:tasks.workunit.client.0.smithi110.stdout:1/278: unlink d0/d23/f4a 0 2024-04-25T15:49:20.337 INFO:tasks.workunit.client.0.smithi110.stdout:1/279: chown d0/d14/f1d 20387 1 2024-04-25T15:49:20.339 INFO:tasks.workunit.client.0.smithi110.stdout:0/221: rename d1/d20/c31 to d1/d3e/c57 0 2024-04-25T15:49:20.340 INFO:tasks.workunit.client.0.smithi110.stdout:2/151: mknod de/c26 0 2024-04-25T15:49:20.341 INFO:tasks.workunit.client.0.smithi110.stdout:2/152: stat de/d11/f21 0 2024-04-25T15:49:20.342 INFO:tasks.workunit.client.0.smithi110.stdout:2/153: readlink de/l23 0 2024-04-25T15:49:20.343 INFO:tasks.workunit.client.0.smithi110.stdout:2/154: chown de/f22 43 1 2024-04-25T15:49:20.343 INFO:tasks.workunit.client.0.smithi110.stdout:3/284: dwrite d0/d7/f17 [4194304,4194304] 0 2024-04-25T15:49:20.351 INFO:tasks.workunit.client.0.smithi110.stdout:1/280: creat d0/d14/f53 x:0 0 0 2024-04-25T15:49:20.352 INFO:tasks.workunit.client.0.smithi110.stdout:0/222: mknod d1/d47/d54/c58 0 2024-04-25T15:49:20.352 INFO:tasks.workunit.client.0.smithi110.stdout:0/223: chown d1/d20 101 1 2024-04-25T15:49:20.353 INFO:tasks.workunit.client.0.smithi110.stdout:0/224: fdatasync d1/d23/f42 0 2024-04-25T15:49:20.357 INFO:tasks.workunit.client.0.smithi110.stdout:3/285: mkdir d0/d7/d1c/d2f/d66 0 2024-04-25T15:49:20.358 INFO:tasks.workunit.client.0.smithi110.stdout:3/286: write d0/d7/f2a [1156507,80104] 0 2024-04-25T15:49:20.358 INFO:tasks.workunit.client.0.smithi110.stdout:9/163: creat d9/d2d/f2f x:0 0 0 2024-04-25T15:49:20.364 INFO:tasks.workunit.client.0.smithi110.stdout:1/281: mkdir d0/d23/d54 0 2024-04-25T15:49:20.364 INFO:tasks.workunit.client.0.smithi110.stdout:3/287: dwrite d0/d7/f2a [0,4194304] 0 2024-04-25T15:49:20.366 INFO:tasks.workunit.client.0.smithi110.stdout:0/225: rename d1/l40 to d1/d47/l59 0 2024-04-25T15:49:20.368 INFO:tasks.workunit.client.0.smithi110.stdout:9/164: creat d9/d2d/f30 x:0 0 0 2024-04-25T15:49:20.370 INFO:tasks.workunit.client.0.smithi110.stdout:8/196: link d14/c1e d14/d32/d36/c39 0 2024-04-25T15:49:20.371 INFO:tasks.workunit.client.0.smithi110.stdout:8/197: chown d14/f2d 3219995 1 2024-04-25T15:49:20.372 INFO:tasks.workunit.client.0.smithi110.stdout:6/167: getdents d1/db/d22 0 2024-04-25T15:49:20.374 INFO:tasks.workunit.client.0.smithi110.stdout:1/282: rename d0/d23/f3e to d0/d23/f55 0 2024-04-25T15:49:20.379 INFO:tasks.workunit.client.0.smithi110.stdout:0/226: unlink d1/d23/d30/d33/d35/f1b 0 2024-04-25T15:49:20.384 INFO:tasks.workunit.client.0.smithi110.stdout:9/165: mknod d9/db/df/c31 0 2024-04-25T15:49:20.385 INFO:tasks.workunit.client.0.smithi110.stdout:6/168: dread d1/d12/f2a [4194304,4194304] 0 2024-04-25T15:49:20.385 INFO:tasks.workunit.client.0.smithi110.stdout:6/169: stat d1/ca 0 2024-04-25T15:49:20.391 INFO:tasks.workunit.client.0.smithi110.stdout:9/166: dwrite d9/d13/f29 [0,4194304] 0 2024-04-25T15:49:20.392 INFO:tasks.workunit.client.0.smithi110.stdout:9/167: truncate d9/d13/f27 70953 0 2024-04-25T15:49:20.393 INFO:tasks.workunit.client.0.smithi110.stdout:2/155: getdents de/d11 0 2024-04-25T15:49:20.395 INFO:tasks.workunit.client.0.smithi110.stdout:2/156: dread de/d11/f19 [0,4194304] 0 2024-04-25T15:49:20.397 INFO:tasks.workunit.client.0.smithi110.stdout:0/227: symlink d1/d47/d54/l5a 0 2024-04-25T15:49:20.399 INFO:tasks.workunit.client.0.smithi110.stdout:9/168: dwrite f8 [4194304,4194304] 0 2024-04-25T15:49:20.401 INFO:tasks.workunit.client.0.smithi110.stdout:6/170: mknod d1/db/d24/c39 0 2024-04-25T15:49:20.402 INFO:tasks.workunit.client.0.smithi110.stdout:2/157: dread de/f25 [0,4194304] 0 2024-04-25T15:49:20.408 INFO:tasks.workunit.client.0.smithi110.stdout:8/198: dwrite f5 [0,4194304] 0 2024-04-25T15:49:20.410 INFO:tasks.workunit.client.0.smithi110.stdout:1/283: truncate d0/d51/f52 486863 0 2024-04-25T15:49:20.412 INFO:tasks.workunit.client.0.smithi110.stdout:0/228: mkdir d1/d23/d30/d33/d5b 0 2024-04-25T15:49:20.414 INFO:tasks.workunit.client.0.smithi110.stdout:9/169: symlink d9/db/df/d25/l32 0 2024-04-25T15:49:20.415 INFO:tasks.workunit.client.0.smithi110.stdout:9/170: write d9/f24 [3009113,121985] 0 2024-04-25T15:49:20.417 INFO:tasks.workunit.client.0.smithi110.stdout:3/288: link d0/d7/d1c/d50/c62 d0/d7/d1c/d2f/c67 0 2024-04-25T15:49:20.418 INFO:tasks.workunit.client.0.smithi110.stdout:3/289: fdatasync d0/d7/d40/f61 0 2024-04-25T15:49:20.420 INFO:tasks.workunit.client.0.smithi110.stdout:8/199: rename c4 to d14/d21/d26/c3a 0 2024-04-25T15:49:20.420 INFO:tasks.workunit.client.0.smithi110.stdout:8/200: truncate f11 4765111 0 2024-04-25T15:49:20.423 INFO:tasks.workunit.client.0.smithi110.stdout:1/284: unlink d0/d23/f4e 0 2024-04-25T15:49:20.429 INFO:tasks.workunit.client.0.smithi110.stdout:9/171: creat d9/db/df/f33 x:0 0 0 2024-04-25T15:49:20.430 INFO:tasks.workunit.client.0.smithi110.stdout:6/171: write f0 [3599880,98589] 0 2024-04-25T15:49:20.431 INFO:tasks.workunit.client.0.smithi110.stdout:0/229: dwrite d1/f19 [0,4194304] 0 2024-04-25T15:49:20.432 INFO:tasks.workunit.client.0.smithi110.stdout:2/158: mknod de/c27 0 2024-04-25T15:49:20.433 INFO:tasks.workunit.client.0.smithi110.stdout:3/290: creat d0/d7/f68 x:0 0 0 2024-04-25T15:49:20.434 INFO:tasks.workunit.client.0.smithi110.stdout:2/159: dread - de/f24 zero size 2024-04-25T15:49:20.436 INFO:tasks.workunit.client.0.smithi110.stdout:8/201: dread d14/d21/f2a [0,4194304] 0 2024-04-25T15:49:20.438 INFO:tasks.workunit.client.0.smithi110.stdout:6/172: dread d1/fe [0,4194304] 0 2024-04-25T15:49:20.438 INFO:tasks.workunit.client.0.smithi110.stdout:0/230: mknod d1/d23/d30/d33/d35/c5c 0 2024-04-25T15:49:20.440 INFO:tasks.workunit.client.0.smithi110.stdout:6/173: dread - d1/db/d22/f38 zero size 2024-04-25T15:49:20.440 INFO:tasks.workunit.client.0.smithi110.stdout:6/174: chown d1/c15 8 1 2024-04-25T15:49:20.444 INFO:tasks.workunit.client.0.smithi110.stdout:3/291: rename d0/d7/fa to d0/d7/f69 0 2024-04-25T15:49:20.447 INFO:tasks.workunit.client.0.smithi110.stdout:9/172: creat d9/f34 x:0 0 0 2024-04-25T15:49:20.448 INFO:tasks.workunit.client.0.smithi110.stdout:9/173: dread - d9/d13/f1c zero size 2024-04-25T15:49:20.452 INFO:tasks.workunit.client.0.smithi110.stdout:2/160: creat de/f28 x:0 0 0 2024-04-25T15:49:20.457 INFO:tasks.workunit.client.0.smithi110.stdout:3/292: dread d0/d7/d1c/f35 [0,4194304] 0 2024-04-25T15:49:20.457 INFO:tasks.workunit.client.0.smithi110.stdout:6/175: link d1/dd/f2f d1/db/d24/d2d/f3a 0 2024-04-25T15:49:20.458 INFO:tasks.workunit.client.0.smithi110.stdout:6/176: dread - d1/db/d22/d27/f2b zero size 2024-04-25T15:49:20.463 INFO:tasks.workunit.client.0.smithi110.stdout:1/285: truncate d0/d25/d33/f49 3489936 0 2024-04-25T15:49:20.464 INFO:tasks.workunit.client.0.smithi110.stdout:1/286: fsync d0/d25/d33/f50 0 2024-04-25T15:49:20.465 INFO:tasks.workunit.client.0.smithi110.stdout:6/177: creat d1/db/d22/d33/f3b x:0 0 0 2024-04-25T15:49:20.469 INFO:tasks.workunit.client.0.smithi110.stdout:3/293: link d0/d16/f24 d0/d7/d1c/d2f/d4c/f6a 0 2024-04-25T15:49:20.469 INFO:tasks.workunit.client.0.smithi110.stdout:8/202: truncate d14/f22 2081924 0 2024-04-25T15:49:20.469 INFO:tasks.workunit.client.0.smithi110.stdout:3/294: stat d0/d2e/l3e 0 2024-04-25T15:49:20.470 INFO:tasks.workunit.client.0.smithi110.stdout:8/203: write d14/d21/d23/f25 [4110988,9650] 0 2024-04-25T15:49:20.471 INFO:tasks.workunit.client.0.smithi110.stdout:3/295: write d0/d16/f1f [429450,71452] 0 2024-04-25T15:49:20.474 INFO:tasks.workunit.client.0.smithi110.stdout:1/287: mkdir d0/d23/d56 0 2024-04-25T15:49:20.476 INFO:tasks.workunit.client.0.smithi110.stdout:9/174: getdents d9/db/df 0 2024-04-25T15:49:20.477 INFO:tasks.workunit.client.0.smithi110.stdout:9/175: fdatasync d9/db/df/f17 0 2024-04-25T15:49:20.480 INFO:tasks.workunit.client.0.smithi110.stdout:6/178: dread d1/d12/f2a [0,4194304] 0 2024-04-25T15:49:20.480 INFO:tasks.workunit.client.0.smithi110.stdout:8/204: mknod d14/d21/d26/c3b 0 2024-04-25T15:49:20.488 INFO:tasks.workunit.client.0.smithi110.stdout:9/176: mknod d9/d13/c35 0 2024-04-25T15:49:20.489 INFO:tasks.workunit.client.0.smithi110.stdout:6/179: mkdir d1/dd/d3c 0 2024-04-25T15:49:20.490 INFO:tasks.workunit.client.0.smithi110.stdout:6/180: chown d1/dd/d3c 7614 1 2024-04-25T15:49:20.495 INFO:tasks.workunit.client.0.smithi110.stdout:8/205: creat d14/d32/d36/f3c x:0 0 0 2024-04-25T15:49:20.496 INFO:tasks.workunit.client.0.smithi110.stdout:8/206: truncate d14/d21/f2a 4362767 0 2024-04-25T15:49:20.496 INFO:tasks.workunit.client.0.smithi110.stdout:6/181: mkdir d1/d12/d3d 0 2024-04-25T15:49:20.498 INFO:tasks.workunit.client.0.smithi110.stdout:8/207: dread d14/f17 [0,4194304] 0 2024-04-25T15:49:20.499 INFO:tasks.workunit.client.0.smithi110.stdout:8/208: fsync d14/f2d 0 2024-04-25T15:49:20.499 INFO:tasks.workunit.client.0.smithi110.stdout:3/296: rmdir d0/d7/d1c/d2f/d66 0 2024-04-25T15:49:20.500 INFO:tasks.workunit.client.0.smithi110.stdout:9/177: unlink d9/c2b 0 2024-04-25T15:49:20.504 INFO:tasks.workunit.client.0.smithi110.stdout:6/182: mkdir d1/db/d24/d3e 0 2024-04-25T15:49:20.505 INFO:tasks.workunit.client.0.smithi110.stdout:8/209: unlink d14/d32/d36/c37 0 2024-04-25T15:49:20.505 INFO:tasks.workunit.client.0.smithi110.stdout:8/210: read - d14/d21/d23/f24 zero size 2024-04-25T15:49:20.511 INFO:tasks.workunit.client.0.smithi110.stdout:3/297: creat d0/d7/d1c/d2f/d4c/d63/f6b x:0 0 0 2024-04-25T15:49:20.512 INFO:tasks.workunit.client.0.smithi110.stdout:3/298: fsync d0/d16/f3a 0 2024-04-25T15:49:20.514 INFO:tasks.workunit.client.0.smithi110.stdout:8/211: rename c10 to d14/d32/c3d 0 2024-04-25T15:49:20.515 INFO:tasks.workunit.client.0.smithi110.stdout:8/212: dread - d14/d21/f2b zero size 2024-04-25T15:49:20.516 INFO:tasks.workunit.client.0.smithi110.stdout:9/178: link d9/db/df/f17 d9/db/df/d2c/f36 0 2024-04-25T15:49:20.517 INFO:tasks.workunit.client.0.smithi110.stdout:6/183: symlink d1/db/l3f 0 2024-04-25T15:49:20.520 INFO:tasks.workunit.client.0.smithi110.stdout:9/179: dread d9/f24 [0,4194304] 0 2024-04-25T15:49:20.521 INFO:tasks.workunit.client.0.smithi110.stdout:3/299: creat d0/d7/f6c x:0 0 0 2024-04-25T15:49:20.521 INFO:tasks.workunit.client.0.smithi110.stdout:8/213: rmdir d14/d32 39 2024-04-25T15:49:20.525 INFO:tasks.workunit.client.0.smithi110.stdout:6/184: symlink d1/dd/l40 0 2024-04-25T15:49:20.526 INFO:tasks.workunit.client.0.smithi110.stdout:3/300: unlink d0/d7/d1c/d2f/d4c/d63/f6b 0 2024-04-25T15:49:20.528 INFO:tasks.workunit.client.0.smithi110.stdout:8/214: fsync f3 0 2024-04-25T15:49:20.530 INFO:tasks.workunit.client.0.smithi110.stdout:9/180: dwrite d9/d13/f1c [0,4194304] 0 2024-04-25T15:49:20.532 INFO:tasks.workunit.client.0.smithi110.stdout:9/181: dread - d9/d2d/f2f zero size 2024-04-25T15:49:20.532 INFO:tasks.workunit.client.0.smithi110.stdout:1/288: dwrite d0/d14/f1f [0,4194304] 0 2024-04-25T15:49:20.537 INFO:tasks.workunit.client.0.smithi110.stdout:1/289: dread d0/d14/f4b [0,4194304] 0 2024-04-25T15:49:20.538 INFO:tasks.workunit.client.0.smithi110.stdout:3/301: creat d0/d7/d1c/d50/f6d x:0 0 0 2024-04-25T15:49:20.539 INFO:tasks.workunit.client.0.smithi110.stdout:1/290: write d0/d25/d33/f49 [1332511,40729] 0 2024-04-25T15:49:20.540 INFO:tasks.workunit.client.0.smithi110.stdout:6/185: dwrite d1/f26 [0,4194304] 0 2024-04-25T15:49:20.542 INFO:tasks.workunit.client.0.smithi110.stdout:9/182: creat d9/db/df/d25/f37 x:0 0 0 2024-04-25T15:49:20.547 INFO:tasks.workunit.client.0.smithi110.stdout:1/291: creat d0/d14/f57 x:0 0 0 2024-04-25T15:49:20.548 INFO:tasks.workunit.client.0.smithi110.stdout:1/292: write d0/d25/d33/f45 [483450,85000] 0 2024-04-25T15:49:20.551 INFO:tasks.workunit.client.0.smithi110.stdout:6/186: symlink d1/db/d24/d3e/l41 0 2024-04-25T15:49:20.552 INFO:tasks.workunit.client.0.smithi110.stdout:1/293: mkdir d0/d25/d58 0 2024-04-25T15:49:20.553 INFO:tasks.workunit.client.0.smithi110.stdout:1/294: stat d0/d25/d33/f50 0 2024-04-25T15:49:20.554 INFO:tasks.workunit.client.0.smithi110.stdout:8/215: getdents d14/d21/d26 0 2024-04-25T15:49:20.554 INFO:tasks.workunit.client.0.smithi110.stdout:1/295: dread - d0/d25/d33/f47 zero size 2024-04-25T15:49:20.555 INFO:tasks.workunit.client.0.smithi110.stdout:1/296: write d0/f27 [1655595,45057] 0 2024-04-25T15:49:20.557 INFO:tasks.workunit.client.0.smithi110.stdout:9/183: getdents d9/d13 0 2024-04-25T15:49:20.558 INFO:tasks.workunit.client.0.smithi110.stdout:6/187: rename d1/c11 to d1/c42 0 2024-04-25T15:49:20.560 INFO:tasks.workunit.client.0.smithi110.stdout:8/216: creat d14/d32/f3e x:0 0 0 2024-04-25T15:49:20.561 INFO:tasks.workunit.client.0.smithi110.stdout:9/184: creat d9/d2d/f38 x:0 0 0 2024-04-25T15:49:20.561 INFO:tasks.workunit.client.0.smithi110.stdout:8/217: dread - d14/d21/d23/f38 zero size 2024-04-25T15:49:20.562 INFO:tasks.workunit.client.0.smithi110.stdout:9/185: write d9/d13/f1a [4886105,101439] 0 2024-04-25T15:49:20.563 INFO:tasks.workunit.client.0.smithi110.stdout:7/178: sync 2024-04-25T15:49:20.566 INFO:tasks.workunit.client.0.smithi110.stdout:1/297: dread d0/fa [0,4194304] 0 2024-04-25T15:49:20.568 INFO:tasks.workunit.client.0.smithi110.stdout:1/298: read d0/d14/f2c [3181192,37787] 0 2024-04-25T15:49:20.570 INFO:tasks.workunit.client.0.smithi110.stdout:8/218: link d14/d21/d26/f29 d14/d32/f3f 0 2024-04-25T15:49:20.573 INFO:tasks.workunit.client.0.smithi110.stdout:1/299: mknod d0/d14/c59 0 2024-04-25T15:49:20.574 INFO:tasks.workunit.client.0.smithi110.stdout:9/186: dwrite d9/db/df/f33 [0,4194304] 0 2024-04-25T15:49:20.574 INFO:tasks.workunit.client.0.smithi110.stdout:9/187: stat d9/db 0 2024-04-25T15:49:20.579 INFO:tasks.workunit.client.0.smithi110.stdout:8/219: creat d14/f40 x:0 0 0 2024-04-25T15:49:20.580 INFO:tasks.workunit.client.0.smithi110.stdout:1/300: dread d0/d25/d33/f50 [0,4194304] 0 2024-04-25T15:49:20.581 INFO:tasks.workunit.client.0.smithi110.stdout:1/301: write d0/f27 [2065208,98607] 0 2024-04-25T15:49:20.582 INFO:tasks.workunit.client.0.smithi110.stdout:9/188: getdents d9/d13 0 2024-04-25T15:49:20.584 INFO:tasks.workunit.client.0.smithi110.stdout:8/220: chown d14/d32/d36/c39 5 1 2024-04-25T15:49:20.585 INFO:tasks.workunit.client.0.smithi110.stdout:3/302: write d0/d7/f69 [3638411,6618] 0 2024-04-25T15:49:20.587 INFO:tasks.workunit.client.0.smithi110.stdout:6/188: rename d1/db/d24/d3e to d1/db/d22/d33/d43 0 2024-04-25T15:49:20.587 INFO:tasks.workunit.client.0.smithi110.stdout:1/302: creat d0/d23/f5a x:0 0 0 2024-04-25T15:49:20.591 INFO:tasks.workunit.client.0.smithi110.stdout:1/303: dread d0/d25/d33/f45 [0,4194304] 0 2024-04-25T15:49:20.592 INFO:tasks.workunit.client.0.smithi110.stdout:1/304: read d0/d14/f4b [14,81505] 0 2024-04-25T15:49:20.594 INFO:tasks.workunit.client.0.smithi110.stdout:7/179: rename f11 to d1b/d29/d38/f3d 0 2024-04-25T15:49:20.594 INFO:tasks.workunit.client.0.smithi110.stdout:7/180: chown d1b/c1c 12690591 1 2024-04-25T15:49:20.609 INFO:tasks.workunit.client.0.smithi110.stdout:7/181: dread f19 [0,4194304] 0 2024-04-25T15:49:20.613 INFO:tasks.workunit.client.0.smithi110.stdout:7/182: dread d1b/d29/f37 [0,4194304] 0 2024-04-25T15:49:20.616 INFO:tasks.workunit.client.0.smithi110.stdout:9/189: creat d9/f39 x:0 0 0 2024-04-25T15:49:20.617 INFO:tasks.workunit.client.0.smithi110.stdout:9/190: write d9/d13/f1a [357482,92429] 0 2024-04-25T15:49:20.621 INFO:tasks.workunit.client.0.smithi110.stdout:3/303: rmdir d0/d2e/d45 0 2024-04-25T15:49:20.622 INFO:tasks.workunit.client.0.smithi110.stdout:8/221: dwrite d14/f22 [0,4194304] 0 2024-04-25T15:49:20.622 INFO:tasks.workunit.client.0.smithi110.stdout:3/304: chown d0/d7/d1c/c25 2 1 2024-04-25T15:49:20.624 INFO:tasks.workunit.client.0.smithi110.stdout:1/305: symlink d0/d23/d54/l5b 0 2024-04-25T15:49:20.628 INFO:tasks.workunit.client.0.smithi110.stdout:6/189: creat d1/db/f44 x:0 0 0 2024-04-25T15:49:20.633 INFO:tasks.workunit.client.0.smithi110.stdout:8/222: symlink d14/d32/l41 0 2024-04-25T15:49:20.634 INFO:tasks.workunit.client.0.smithi110.stdout:9/191: dread d9/f24 [0,4194304] 0 2024-04-25T15:49:20.635 INFO:tasks.workunit.client.0.smithi110.stdout:9/192: chown d9/d13/l1e 4917 1 2024-04-25T15:49:20.636 INFO:tasks.workunit.client.0.smithi110.stdout:1/306: creat d0/d23/f5c x:0 0 0 2024-04-25T15:49:20.639 INFO:tasks.workunit.client.0.smithi110.stdout:6/190: readlink d1/db/d24/l31 0 2024-04-25T15:49:20.641 INFO:tasks.workunit.client.0.smithi110.stdout:7/183: mknod d1b/c3e 0 2024-04-25T15:49:20.642 INFO:tasks.workunit.client.0.smithi110.stdout:1/307: dwrite d0/d25/f6 [4194304,4194304] 0 2024-04-25T15:49:20.642 INFO:tasks.workunit.client.0.smithi110.stdout:7/184: write f1a [2781097,62600] 0 2024-04-25T15:49:20.645 INFO:tasks.workunit.client.0.smithi110.stdout:7/185: chown d1b/l21 4063 1 2024-04-25T15:49:20.650 INFO:tasks.workunit.client.0.smithi110.stdout:1/308: dwrite d0/d25/f6 [0,4194304] 0 2024-04-25T15:49:20.651 INFO:tasks.workunit.client.0.smithi110.stdout:1/309: fdatasync d0/f15 0 2024-04-25T15:49:20.652 INFO:tasks.workunit.client.0.smithi110.stdout:6/191: creat d1/db/d24/f45 x:0 0 0 2024-04-25T15:49:20.658 INFO:tasks.workunit.client.0.smithi110.stdout:8/223: mkdir d14/d42 0 2024-04-25T15:49:20.664 INFO:tasks.workunit.client.0.smithi110.stdout:4/210: sync 2024-04-25T15:49:20.665 INFO:tasks.workunit.client.0.smithi110.stdout:3/305: write d0/d16/f18 [211050,18994] 0 2024-04-25T15:49:20.665 INFO:tasks.workunit.client.0.smithi110.stdout:4/211: stat d13/d24/f2a 0 2024-04-25T15:49:20.666 INFO:tasks.workunit.client.0.smithi110.stdout:8/224: mknod d14/d21/d26/c43 0 2024-04-25T15:49:20.673 INFO:tasks.workunit.client.0.smithi110.stdout:3/306: symlink d0/d7/d1c/d50/l6e 0 2024-04-25T15:49:20.675 INFO:tasks.workunit.client.0.smithi110.stdout:8/225: unlink d14/f20 0 2024-04-25T15:49:20.678 INFO:tasks.workunit.client.0.smithi110.stdout:6/192: link d1/db/d24/d2d/l37 d1/d12/d3d/l46 0 2024-04-25T15:49:20.681 INFO:tasks.workunit.client.0.smithi110.stdout:3/307: creat d0/d7/d1c/d50/f6f x:0 0 0 2024-04-25T15:49:20.684 INFO:tasks.workunit.client.0.smithi110.stdout:8/226: rename d14/c1f to d14/d32/c44 0 2024-04-25T15:49:20.686 INFO:tasks.workunit.client.0.smithi110.stdout:7/186: rename d1b/l2d to d1b/l3f 0 2024-04-25T15:49:20.691 INFO:tasks.workunit.client.0.smithi110.stdout:3/308: rmdir d0/d16 39 2024-04-25T15:49:20.693 INFO:tasks.workunit.client.0.smithi110.stdout:7/187: dwrite d1b/d29/f2b [0,4194304] 0 2024-04-25T15:49:20.693 INFO:tasks.workunit.client.0.smithi110.stdout:3/309: fsync d0/d2e/f3b 0 2024-04-25T15:49:20.698 INFO:tasks.workunit.client.0.smithi110.stdout:4/212: dwrite fc [0,4194304] 0 2024-04-25T15:49:20.705 INFO:tasks.workunit.client.0.smithi110.stdout:4/213: dwrite d13/d1c/f20 [0,4194304] 0 2024-04-25T15:49:20.705 INFO:tasks.workunit.client.0.smithi110.stdout:7/188: symlink d1b/d29/l40 0 2024-04-25T15:49:20.706 INFO:tasks.workunit.client.0.smithi110.stdout:7/189: fdatasync f14 0 2024-04-25T15:49:20.706 INFO:tasks.workunit.client.0.smithi110.stdout:4/214: write d13/f1a [912181,27909] 0 2024-04-25T15:49:20.707 INFO:tasks.workunit.client.0.smithi110.stdout:4/215: readlink d13/l3f 0 2024-04-25T15:49:20.708 INFO:tasks.workunit.client.0.smithi110.stdout:4/216: chown d13/d1c/f20 19 1 2024-04-25T15:49:20.710 INFO:tasks.workunit.client.0.smithi110.stdout:4/217: write d13/f1a [265712,65422] 0 2024-04-25T15:49:20.712 INFO:tasks.workunit.client.0.smithi110.stdout:6/193: write d1/dd/f2f [142020,113345] 0 2024-04-25T15:49:20.716 INFO:tasks.workunit.client.0.smithi110.stdout:8/227: write d14/d21/d26/f29 [1851954,64046] 0 2024-04-25T15:49:20.719 INFO:tasks.workunit.client.0.smithi110.stdout:8/228: dread d14/f16 [0,4194304] 0 2024-04-25T15:49:20.721 INFO:tasks.workunit.client.0.smithi110.stdout:3/310: creat d0/f70 x:0 0 0 2024-04-25T15:49:20.722 INFO:tasks.workunit.client.0.smithi110.stdout:3/311: chown d0/f15 11861078 1 2024-04-25T15:49:20.722 INFO:tasks.workunit.client.0.smithi110.stdout:6/194: mknod d1/dd/c47 0 2024-04-25T15:49:20.725 INFO:tasks.workunit.client.0.smithi110.stdout:3/312: dread d0/d7/d1c/f35 [0,4194304] 0 2024-04-25T15:49:20.726 INFO:tasks.workunit.client.0.smithi110.stdout:3/313: chown d0/d2e/c5b 5 1 2024-04-25T15:49:20.730 INFO:tasks.workunit.client.0.smithi110.stdout:3/314: creat d0/d2e/f71 x:0 0 0 2024-04-25T15:49:20.731 INFO:tasks.workunit.client.0.smithi110.stdout:6/195: mkdir d1/d48 0 2024-04-25T15:49:20.735 INFO:tasks.workunit.client.0.smithi110.stdout:7/190: link d1b/c33 d1b/c41 0 2024-04-25T15:49:20.737 INFO:tasks.workunit.client.0.smithi110.stdout:3/315: dwrite d0/d7/d1c/f5f [0,4194304] 0 2024-04-25T15:49:20.738 INFO:tasks.workunit.client.0.smithi110.stdout:6/196: dread d1/fe [0,4194304] 0 2024-04-25T15:49:20.739 INFO:tasks.workunit.client.0.smithi110.stdout:7/191: mkdir d1b/d29/d42 0 2024-04-25T15:49:20.743 INFO:tasks.workunit.client.0.smithi110.stdout:8/229: dread ff [4194304,4194304] 0 2024-04-25T15:49:20.744 INFO:tasks.workunit.client.0.smithi110.stdout:8/230: rename d14 to d14/d21/d45 22 2024-04-25T15:49:20.745 INFO:tasks.workunit.client.0.smithi110.stdout:8/231: fsync d14/d32/d36/f3c 0 2024-04-25T15:49:20.745 INFO:tasks.workunit.client.0.smithi110.stdout:6/197: unlink d1/dd/f2f 0 2024-04-25T15:49:20.746 INFO:tasks.workunit.client.0.smithi110.stdout:6/198: stat d1/dd/l18 0 2024-04-25T15:49:20.746 INFO:tasks.workunit.client.0.smithi110.stdout:7/192: fdatasync d1b/f31 0 2024-04-25T15:49:20.747 INFO:tasks.workunit.client.0.smithi110.stdout:3/316: getdents d0/d7/d1c/d2f/d33 0 2024-04-25T15:49:20.748 INFO:tasks.workunit.client.0.smithi110.stdout:3/317: write d0/f2c [4273226,100160] 0 2024-04-25T15:49:20.751 INFO:tasks.workunit.client.0.smithi110.stdout:8/232: symlink d14/d21/d26/l46 0 2024-04-25T15:49:20.751 INFO:tasks.workunit.client.0.smithi110.stdout:8/233: write d14/f17 [549660,129587] 0 2024-04-25T15:49:20.758 INFO:tasks.workunit.client.0.smithi110.stdout:3/318: write d0/d16/f18 [727309,12169] 0 2024-04-25T15:49:20.765 INFO:tasks.workunit.client.0.smithi110.stdout:3/319: dwrite d0/f2c [0,4194304] 0 2024-04-25T15:49:20.766 INFO:tasks.workunit.client.0.smithi110.stdout:3/320: write d0/d7/d40/f43 [338130,67063] 0 2024-04-25T15:49:20.771 INFO:tasks.workunit.client.0.smithi110.stdout:8/234: symlink d14/l47 0 2024-04-25T15:49:20.772 INFO:tasks.workunit.client.0.smithi110.stdout:8/235: fsync f11 0 2024-04-25T15:49:20.773 INFO:tasks.workunit.client.0.smithi110.stdout:3/321: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:20.774 INFO:tasks.workunit.client.0.smithi110.stdout:6/199: rmdir d1/db/d22/d33/d36 0 2024-04-25T15:49:20.783 INFO:tasks.workunit.client.0.smithi110.stdout:8/236: dwrite d14/d32/d36/f3c [0,4194304] 0 2024-04-25T15:49:20.784 INFO:tasks.workunit.client.0.smithi110.stdout:6/200: dread d1/fe [0,4194304] 0 2024-04-25T15:49:20.785 INFO:tasks.workunit.client.0.smithi110.stdout:8/237: dread - d14/d21/f2b zero size 2024-04-25T15:49:20.786 INFO:tasks.workunit.client.0.smithi110.stdout:3/322: dwrite d0/d7/d40/f61 [0,4194304] 0 2024-04-25T15:49:20.788 INFO:tasks.workunit.client.0.smithi110.stdout:3/323: stat d0/d2e/l3e 0 2024-04-25T15:49:20.789 INFO:tasks.workunit.client.0.smithi110.stdout:4/218: truncate fa 1939863 0 2024-04-25T15:49:20.791 INFO:tasks.workunit.client.0.smithi110.stdout:6/201: creat d1/d12/f49 x:0 0 0 2024-04-25T15:49:20.791 INFO:tasks.workunit.client.0.smithi110.stdout:3/324: rename d0/d7 to d0/d7/d1c/d2f/d4c/d63/d72 22 2024-04-25T15:49:20.791 INFO:tasks.workunit.client.0.smithi110.stdout:4/219: write d13/d24/d26/f3b [842970,54587] 0 2024-04-25T15:49:20.792 INFO:tasks.workunit.client.0.smithi110.stdout:3/325: stat d0/d7/d1c/d50/f59 0 2024-04-25T15:49:20.793 INFO:tasks.workunit.client.0.smithi110.stdout:3/326: chown d0/d7/d40/c5a 906509 1 2024-04-25T15:49:20.793 INFO:tasks.workunit.client.0.smithi110.stdout:8/238: mkdir d14/d32/d36/d48 0 2024-04-25T15:49:20.796 INFO:tasks.workunit.client.0.smithi110.stdout:4/220: creat d13/d1c/f41 x:0 0 0 2024-04-25T15:49:20.801 INFO:tasks.workunit.client.0.smithi110.stdout:3/327: dread d0/d7/f38 [0,4194304] 0 2024-04-25T15:49:20.802 INFO:tasks.workunit.client.0.smithi110.stdout:8/239: rename d14/d32/l41 to d14/d21/d23/l49 0 2024-04-25T15:49:20.806 INFO:tasks.workunit.client.0.smithi110.stdout:8/240: dread d14/d21/f2a [0,4194304] 0 2024-04-25T15:49:20.813 INFO:tasks.workunit.client.0.smithi110.stdout:8/241: creat d14/d21/d26/f4a x:0 0 0 2024-04-25T15:49:20.820 INFO:tasks.workunit.client.0.smithi110.stdout:4/221: link d13/d24/d2d/f37 d13/d24/d2d/d36/f42 0 2024-04-25T15:49:20.820 INFO:tasks.workunit.client.0.smithi110.stdout:4/222: read - d13/f3d zero size 2024-04-25T15:49:20.822 INFO:tasks.workunit.client.0.smithi110.stdout:7/193: write fb [462385,123181] 0 2024-04-25T15:49:20.826 INFO:tasks.workunit.client.0.smithi110.stdout:7/194: dread d1b/d29/f2b [0,4194304] 0 2024-04-25T15:49:20.826 INFO:tasks.workunit.client.0.smithi110.stdout:7/195: truncate d1b/f26 704016 0 2024-04-25T15:49:20.828 INFO:tasks.workunit.client.0.smithi110.stdout:4/223: readlink d13/d24/d26/l34 0 2024-04-25T15:49:20.838 INFO:tasks.workunit.client.0.smithi110.stdout:7/196: mknod d1b/c43 0 2024-04-25T15:49:20.839 INFO:tasks.workunit.client.0.smithi110.stdout:8/242: truncate f11 4003745 0 2024-04-25T15:49:20.839 INFO:tasks.workunit.client.0.smithi110.stdout:8/243: chown d14/d21 20 1 2024-04-25T15:49:20.843 INFO:tasks.workunit.client.0.smithi110.stdout:4/224: link d13/d1c/f41 d13/d24/f43 0 2024-04-25T15:49:20.844 INFO:tasks.workunit.client.0.smithi110.stdout:8/244: mknod d14/d21/d23/c4b 0 2024-04-25T15:49:20.846 INFO:tasks.workunit.client.0.smithi110.stdout:8/245: write d14/d21/d23/f24 [231513,29499] 0 2024-04-25T15:49:20.846 INFO:tasks.workunit.client.0.smithi110.stdout:8/246: fsync d14/d21/d23/f25 0 2024-04-25T15:49:20.847 INFO:tasks.workunit.client.0.smithi110.stdout:8/247: readlink d14/l34 0 2024-04-25T15:49:20.850 INFO:tasks.workunit.client.0.smithi110.stdout:4/225: write d13/d24/f43 [191194,80706] 0 2024-04-25T15:49:20.853 INFO:tasks.workunit.client.0.smithi110.stdout:7/197: creat d1b/f44 x:0 0 0 2024-04-25T15:49:20.855 INFO:tasks.workunit.client.0.smithi110.stdout:4/226: read f6 [1192577,69570] 0 2024-04-25T15:49:20.859 INFO:tasks.workunit.client.0.smithi110.stdout:7/198: dwrite f1a [0,4194304] 0 2024-04-25T15:49:20.861 INFO:tasks.workunit.client.0.smithi110.stdout:4/227: mkdir d13/d24/d2d/d44 0 2024-04-25T15:49:20.867 INFO:tasks.workunit.client.0.smithi110.stdout:3/328: truncate d0/d7/d1c/d2f/d33/f36 5264979 0 2024-04-25T15:49:20.872 INFO:tasks.workunit.client.0.smithi110.stdout:7/199: creat d1b/f45 x:0 0 0 2024-04-25T15:49:20.874 INFO:tasks.workunit.client.0.smithi110.stdout:3/329: unlink d0/d2e/l3e 0 2024-04-25T15:49:20.879 INFO:tasks.workunit.client.0.smithi110.stdout:3/330: dwrite d0/d7/d40/f61 [0,4194304] 0 2024-04-25T15:49:20.882 INFO:tasks.workunit.client.0.smithi110.stdout:3/331: read d0/d16/f3a [1674375,101756] 0 2024-04-25T15:49:20.883 INFO:tasks.workunit.client.0.smithi110.stdout:7/200: truncate d1b/d29/f2b 2862900 0 2024-04-25T15:49:20.884 INFO:tasks.workunit.client.0.smithi110.stdout:7/201: write d1b/f36 [377721,93402] 0 2024-04-25T15:49:20.886 INFO:tasks.workunit.client.0.smithi110.stdout:3/332: mknod d0/d7/d1c/d50/c73 0 2024-04-25T15:49:20.890 INFO:tasks.workunit.client.0.smithi110.stdout:3/333: rmdir d0/d16 39 2024-04-25T15:49:20.894 INFO:tasks.workunit.client.0.smithi110.stdout:7/202: fsync f17 0 2024-04-25T15:49:20.900 INFO:tasks.workunit.client.0.smithi110.stdout:7/203: mkdir d1b/d29/d46 0 2024-04-25T15:49:20.902 INFO:tasks.workunit.client.0.smithi110.stdout:3/334: symlink d0/d16/l74 0 2024-04-25T15:49:20.902 INFO:tasks.workunit.client.0.smithi110.stdout:3/335: fsync d0/f2c 0 2024-04-25T15:49:20.903 INFO:tasks.workunit.client.0.smithi110.stdout:7/204: fdatasync f17 0 2024-04-25T15:49:20.914 INFO:tasks.workunit.client.0.smithi110.stdout:4/228: dwrite d13/f22 [0,4194304] 0 2024-04-25T15:49:20.916 INFO:tasks.workunit.client.0.smithi110.stdout:4/229: write d13/d24/d26/f31 [3398445,5573] 0 2024-04-25T15:49:20.917 INFO:tasks.workunit.client.0.smithi110.stdout:7/205: truncate f2 654862 0 2024-04-25T15:49:20.917 INFO:tasks.workunit.client.0.smithi110.stdout:4/230: read d13/f22 [904290,127668] 0 2024-04-25T15:49:20.917 INFO:tasks.workunit.client.0.smithi110.stdout:7/206: chown lf 14 1 2024-04-25T15:49:20.927 INFO:tasks.workunit.client.0.smithi110.stdout:7/207: rename d1b/f44 to d1b/d29/d46/f47 0 2024-04-25T15:49:20.937 INFO:tasks.workunit.client.0.smithi110.stdout:4/231: creat d13/d24/f45 x:0 0 0 2024-04-25T15:49:20.940 INFO:tasks.workunit.client.0.smithi110.stdout:4/232: symlink d13/d1c/l46 0 2024-04-25T15:49:20.941 INFO:tasks.workunit.client.0.smithi110.stdout:4/233: read d13/f1a [316994,105348] 0 2024-04-25T15:49:20.942 INFO:tasks.workunit.client.0.smithi110.stdout:7/208: creat d1b/f48 x:0 0 0 2024-04-25T15:49:20.947 INFO:tasks.workunit.client.0.smithi110.stdout:3/336: dwrite d0/d16/f24 [0,4194304] 0 2024-04-25T15:49:20.952 INFO:tasks.workunit.client.0.smithi110.stdout:3/337: dread d0/d16/f24 [0,4194304] 0 2024-04-25T15:49:20.971 INFO:tasks.workunit.client.0.smithi110.stdout:7/209: dread d1b/d29/f2b [0,4194304] 0 2024-04-25T15:49:20.972 INFO:tasks.workunit.client.0.smithi110.stdout:3/338: link d0/d2e/f30 d0/f75 0 2024-04-25T15:49:20.973 INFO:tasks.workunit.client.0.smithi110.stdout:3/339: chown d0/f75 6668 1 2024-04-25T15:49:20.973 INFO:tasks.workunit.client.0.smithi110.stdout:7/210: dread f19 [0,4194304] 0 2024-04-25T15:49:20.978 INFO:tasks.workunit.client.0.smithi110.stdout:4/234: write d13/d24/d2d/f37 [689497,47212] 0 2024-04-25T15:49:20.979 INFO:tasks.workunit.client.0.smithi110.stdout:4/235: stat d13/f3d 0 2024-04-25T15:49:21.003 INFO:tasks.workunit.client.0.smithi110.stdout:5/138: sync 2024-04-25T15:49:21.004 INFO:tasks.workunit.client.0.smithi110.stdout:7/211: link d1b/c3e d1b/c49 0 2024-04-25T15:49:21.004 INFO:tasks.workunit.client.0.smithi110.stdout:4/236: creat d13/d1c/f47 x:0 0 0 2024-04-25T15:49:21.005 INFO:tasks.workunit.client.0.smithi110.stdout:4/237: write d13/d1c/f41 [1150769,73047] 0 2024-04-25T15:49:21.007 INFO:tasks.workunit.client.0.smithi110.stdout:4/238: write d13/f3d [87356,73014] 0 2024-04-25T15:49:21.011 INFO:tasks.workunit.client.0.smithi110.stdout:7/212: dwrite d1b/f36 [0,4194304] 0 2024-04-25T15:49:21.012 INFO:tasks.workunit.client.0.smithi110.stdout:7/213: stat l3 0 2024-04-25T15:49:21.012 INFO:tasks.workunit.client.0.smithi110.stdout:5/139: dread d4/f8 [0,4194304] 0 2024-04-25T15:49:21.017 INFO:tasks.workunit.client.0.smithi110.stdout:5/140: rename d4/d1d/f2d to d4/d1d/d1f/f32 0 2024-04-25T15:49:21.019 INFO:tasks.workunit.client.0.smithi110.stdout:5/141: symlink d4/d1d/d1f/l33 0 2024-04-25T15:49:21.066 INFO:tasks.workunit.client.0.smithi110.stdout:7/214: write d1b/d29/f37 [2929525,76719] 0 2024-04-25T15:49:21.066 INFO:tasks.workunit.client.0.smithi110.stdout:7/215: chown c5 295982106 1 2024-04-25T15:49:21.070 INFO:tasks.workunit.client.0.smithi110.stdout:7/216: truncate fd 3620371 0 2024-04-25T15:49:21.072 INFO:tasks.workunit.client.0.smithi110.stdout:7/217: symlink d1b/d29/l4a 0 2024-04-25T15:49:21.073 INFO:tasks.workunit.client.0.smithi110.stdout:7/218: stat d1b/d29/d46/f47 0 2024-04-25T15:49:21.074 INFO:tasks.workunit.client.0.smithi110.stdout:5/142: write d4/fd [1163021,4907] 0 2024-04-25T15:49:21.077 INFO:tasks.workunit.client.0.smithi110.stdout:7/219: dread d1b/f31 [0,4194304] 0 2024-04-25T15:49:21.079 INFO:tasks.workunit.client.0.smithi110.stdout:5/143: rename d4/d2c/f1c to d4/d1d/f34 0 2024-04-25T15:49:21.079 INFO:tasks.workunit.client.0.smithi110.stdout:5/144: readlink d4/lf 0 2024-04-25T15:49:21.080 INFO:tasks.workunit.client.0.smithi110.stdout:5/145: write d4/f25 [1774963,76625] 0 2024-04-25T15:49:21.090 INFO:tasks.workunit.client.0.smithi110.stdout:5/146: getdents d4/d1d/d1f 0 2024-04-25T15:49:21.094 INFO:tasks.workunit.client.0.smithi110.stdout:5/147: rmdir d4/d2e 0 2024-04-25T15:49:21.095 INFO:tasks.workunit.client.0.smithi110.stdout:5/148: mkdir d4/d1d/d1f/d35 0 2024-04-25T15:49:21.096 INFO:tasks.workunit.client.0.smithi110.stdout:5/149: write d4/d2c/f27 [959768,71873] 0 2024-04-25T15:49:21.110 INFO:tasks.workunit.client.0.smithi110.stdout:7/220: truncate fb 499501 0 2024-04-25T15:49:21.125 INFO:tasks.workunit.client.0.smithi110.stdout:7/221: dread d1b/d29/d38/f3d [0,4194304] 0 2024-04-25T15:49:21.128 INFO:tasks.workunit.client.0.smithi110.stdout:7/222: creat d1b/f4b x:0 0 0 2024-04-25T15:49:21.128 INFO:tasks.workunit.client.0.smithi110.stdout:7/223: readlink d1b/d29/l30 0 2024-04-25T15:49:21.413 INFO:tasks.workunit.client.0.smithi110.stdout:0/231: sync 2024-04-25T15:49:21.415 INFO:tasks.workunit.client.0.smithi110.stdout:0/232: unlink d1/f28 0 2024-04-25T15:49:21.417 INFO:tasks.workunit.client.0.smithi110.stdout:0/233: creat d1/d4c/f5d x:0 0 0 2024-04-25T15:49:21.417 INFO:tasks.workunit.client.0.smithi110.stdout:0/234: stat d1/d4c/f5d 0 2024-04-25T15:49:21.418 INFO:tasks.workunit.client.0.smithi110.stdout:0/235: fdatasync d1/f49 0 2024-04-25T15:49:21.422 INFO:tasks.workunit.client.0.smithi110.stdout:0/236: link d1/d23/d30/d33/d35/lc d1/d20/l5e 0 2024-04-25T15:49:21.422 INFO:tasks.workunit.client.0.smithi110.stdout:0/237: chown d1/l4f 33466042 1 2024-04-25T15:49:21.487 INFO:tasks.workunit.client.0.smithi110.stdout:0/238: dwrite d1/d23/d30/d33/d35/f11 [0,4194304] 0 2024-04-25T15:49:21.489 INFO:tasks.workunit.client.0.smithi110.stdout:0/239: creat d1/d4c/f5f x:0 0 0 2024-04-25T15:49:21.492 INFO:tasks.workunit.client.0.smithi110.stdout:0/240: mknod d1/d20/c60 0 2024-04-25T15:49:21.492 INFO:tasks.workunit.client.0.smithi110.stdout:0/241: stat d1/f2a 0 2024-04-25T15:49:21.494 INFO:tasks.workunit.client.0.smithi110.stdout:0/242: creat d1/d47/f61 x:0 0 0 2024-04-25T15:49:21.523 INFO:tasks.workunit.client.0.smithi110.stdout:1/310: sync 2024-04-25T15:49:21.523 INFO:tasks.workunit.client.0.smithi110.stdout:9/193: sync 2024-04-25T15:49:21.523 INFO:tasks.workunit.client.0.smithi110.stdout:2/161: sync 2024-04-25T15:49:21.523 INFO:tasks.workunit.client.0.smithi110.stdout:1/311: fdatasync d0/d25/f41 0 2024-04-25T15:49:21.525 INFO:tasks.workunit.client.0.smithi110.stdout:1/312: truncate d0/d23/f5a 608159 0 2024-04-25T15:49:21.526 INFO:tasks.workunit.client.0.smithi110.stdout:2/162: rename de/d11/l15 to de/d11/l29 0 2024-04-25T15:49:21.527 INFO:tasks.workunit.client.0.smithi110.stdout:2/163: fdatasync de/f28 0 2024-04-25T15:49:21.527 INFO:tasks.workunit.client.0.smithi110.stdout:1/313: mknod d0/d14/c5d 0 2024-04-25T15:49:21.528 INFO:tasks.workunit.client.0.smithi110.stdout:1/314: readlink d0/d25/l40 0 2024-04-25T15:49:21.529 INFO:tasks.workunit.client.0.smithi110.stdout:1/315: creat d0/d23/d54/f5e x:0 0 0 2024-04-25T15:49:21.530 INFO:tasks.workunit.client.0.smithi110.stdout:1/316: stat d0/d23/f5c 0 2024-04-25T15:49:21.624 INFO:tasks.workunit.client.0.smithi110.stdout:2/164: chown de/d11/l29 5877558 1 2024-04-25T15:49:21.624 INFO:tasks.workunit.client.0.smithi110.stdout:1/317: dwrite d0/d23/f5c [0,4194304] 0 2024-04-25T15:49:21.626 INFO:tasks.workunit.client.0.smithi110.stdout:2/165: write de/d11/f19 [1681517,115002] 0 2024-04-25T15:49:21.627 INFO:tasks.workunit.client.0.smithi110.stdout:2/166: write de/f22 [933071,116352] 0 2024-04-25T15:49:21.627 INFO:tasks.workunit.client.0.smithi110.stdout:2/167: fdatasync de/d11/f21 0 2024-04-25T15:49:21.627 INFO:tasks.workunit.client.0.smithi110.stdout:2/168: stat de/d11 0 2024-04-25T15:49:21.629 INFO:tasks.workunit.client.0.smithi110.stdout:1/318: mkdir d0/d14/d5f 0 2024-04-25T15:49:21.629 INFO:tasks.workunit.client.0.smithi110.stdout:1/319: fdatasync d0/d25/f3d 0 2024-04-25T15:49:21.632 INFO:tasks.workunit.client.0.smithi110.stdout:1/320: dread d0/d25/f3d [0,4194304] 0 2024-04-25T15:49:21.632 INFO:tasks.workunit.client.0.smithi110.stdout:1/321: readlink d0/d25/l36 0 2024-04-25T15:49:21.633 INFO:tasks.workunit.client.0.smithi110.stdout:1/322: chown d0/f15 12 1 2024-04-25T15:49:21.634 INFO:tasks.workunit.client.0.smithi110.stdout:2/169: rmdir de 39 2024-04-25T15:49:21.641 INFO:tasks.workunit.client.0.smithi110.stdout:2/170: unlink fb 0 2024-04-25T15:49:21.643 INFO:tasks.workunit.client.0.smithi110.stdout:1/323: mknod d0/d23/d56/c60 0 2024-04-25T15:49:21.662 INFO:tasks.workunit.client.0.smithi110.stdout:6/202: sync 2024-04-25T15:49:21.671 INFO:tasks.workunit.client.0.smithi110.stdout:6/203: creat d1/d12/d3d/f4a x:0 0 0 2024-04-25T15:49:21.695 INFO:tasks.workunit.client.0.smithi110.stdout:8/248: sync 2024-04-25T15:49:21.695 INFO:tasks.workunit.client.0.smithi110.stdout:3/340: sync 2024-04-25T15:49:21.696 INFO:tasks.workunit.client.0.smithi110.stdout:3/341: fdatasync d0/d16/f1f 0 2024-04-25T15:49:21.697 INFO:tasks.workunit.client.0.smithi110.stdout:3/342: write d0/d7/f29 [3133621,64349] 0 2024-04-25T15:49:21.702 INFO:tasks.workunit.client.0.smithi110.stdout:8/249: mkdir d14/d4c 0 2024-04-25T15:49:21.702 INFO:tasks.workunit.client.0.smithi110.stdout:8/250: fdatasync d14/f15 0 2024-04-25T15:49:21.703 INFO:tasks.workunit.client.0.smithi110.stdout:2/171: write de/f25 [343376,12008] 0 2024-04-25T15:49:21.704 INFO:tasks.workunit.client.0.smithi110.stdout:3/343: fdatasync d0/d7/d1c/d2f/d33/f36 0 2024-04-25T15:49:21.706 INFO:tasks.workunit.client.0.smithi110.stdout:1/324: truncate d0/d25/f6 1069140 0 2024-04-25T15:49:21.707 INFO:tasks.workunit.client.0.smithi110.stdout:8/251: readlink d14/l31 0 2024-04-25T15:49:21.712 INFO:tasks.workunit.client.0.smithi110.stdout:1/325: mkdir d0/d23/d54/d61 0 2024-04-25T15:49:21.720 INFO:tasks.workunit.client.0.smithi110.stdout:1/326: dwrite d0/f3f [4194304,4194304] 0 2024-04-25T15:49:21.721 INFO:tasks.workunit.client.0.smithi110.stdout:6/204: write d1/db/d24/f29 [708159,76422] 0 2024-04-25T15:49:21.721 INFO:tasks.workunit.client.0.smithi110.stdout:1/327: chown d0/fa 18496156 1 2024-04-25T15:49:21.722 INFO:tasks.workunit.client.0.smithi110.stdout:6/205: chown d1/dd/c47 8051631 1 2024-04-25T15:49:21.726 INFO:tasks.workunit.client.0.smithi110.stdout:3/344: getdents d0 0 2024-04-25T15:49:21.727 INFO:tasks.workunit.client.0.smithi110.stdout:3/345: chown d0/d7/d1c/d50/f6f 2044269141 1 2024-04-25T15:49:21.729 INFO:tasks.workunit.client.0.smithi110.stdout:3/346: chown d0/d7/d1c/d50/l6e 630298120 1 2024-04-25T15:49:21.729 INFO:tasks.workunit.client.0.smithi110.stdout:1/328: read d0/d51/f52 [154200,27080] 0 2024-04-25T15:49:21.730 INFO:tasks.workunit.client.0.smithi110.stdout:3/347: readlink d0/d7/d1c/d2f/d33/l41 0 2024-04-25T15:49:21.730 INFO:tasks.workunit.client.0.smithi110.stdout:6/206: mknod d1/db/d22/d33/c4b 0 2024-04-25T15:49:21.731 INFO:tasks.workunit.client.0.smithi110.stdout:3/348: read d0/d16/f1f [2706998,46754] 0 2024-04-25T15:49:21.738 INFO:tasks.workunit.client.0.smithi110.stdout:3/349: rename d0/d7/d40/f47 to d0/d2e/f76 0 2024-04-25T15:49:21.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:21 smithi193 ceph-mon[33747]: pgmap v27: 65 pgs: 65 active+clean; 489 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.4 MiB/s rd, 46 MiB/s wr, 320 op/s 2024-04-25T15:49:21.745 INFO:tasks.workunit.client.0.smithi110.stdout:6/207: mkdir d1/db/d24/d2d/d4c 0 2024-04-25T15:49:21.747 INFO:tasks.workunit.client.0.smithi110.stdout:1/329: getdents d0/d14 0 2024-04-25T15:49:21.750 INFO:tasks.workunit.client.0.smithi110.stdout:3/350: link d0/f2c d0/d2e/f77 0 2024-04-25T15:49:21.753 INFO:tasks.workunit.client.0.smithi110.stdout:6/208: creat d1/db/d22/f4d x:0 0 0 2024-04-25T15:49:21.753 INFO:tasks.workunit.client.0.smithi110.stdout:1/330: dwrite d0/d23/d54/f5e [0,4194304] 0 2024-04-25T15:49:21.758 INFO:tasks.workunit.client.0.smithi110.stdout:3/351: rename d0/d2e/f77 to d0/d7/d1c/d2f/d4c/f78 0 2024-04-25T15:49:21.763 INFO:tasks.workunit.client.0.smithi110.stdout:1/331: creat d0/d14/f62 x:0 0 0 2024-04-25T15:49:21.765 INFO:tasks.workunit.client.0.smithi110.stdout:3/352: dwrite d0/d7/f68 [0,4194304] 0 2024-04-25T15:49:21.766 INFO:tasks.workunit.client.0.smithi110.stdout:3/353: chown d0/d7/d1c/d50/l5c 11639410 1 2024-04-25T15:49:21.771 INFO:tasks.workunit.client.0.smithi110.stdout:3/354: rmdir d0/d7/d1c/d2f/d33 39 2024-04-25T15:49:21.772 INFO:tasks.workunit.client.0.smithi110.stdout:3/355: dread - d0/d7/f6c zero size 2024-04-25T15:49:21.776 INFO:tasks.workunit.client.0.smithi110.stdout:3/356: fdatasync d0/d7/d1c/d2f/d4c/f6a 0 2024-04-25T15:49:21.778 INFO:tasks.workunit.client.0.smithi110.stdout:8/252: truncate f5 4325108 0 2024-04-25T15:49:21.779 INFO:tasks.workunit.client.0.smithi110.stdout:8/253: dread f3 [0,4194304] 0 2024-04-25T15:49:21.782 INFO:tasks.workunit.client.0.smithi110.stdout:3/357: dwrite d0/f1e [0,4194304] 0 2024-04-25T15:49:21.783 INFO:tasks.workunit.client.0.smithi110.stdout:8/254: dread d14/d21/d23/f24 [0,4194304] 0 2024-04-25T15:49:21.787 INFO:tasks.workunit.client.0.smithi110.stdout:3/358: chown d0/d7/d40/l42 1 1 2024-04-25T15:49:21.791 INFO:tasks.workunit.client.0.smithi110.stdout:3/359: creat d0/d16/f79 x:0 0 0 2024-04-25T15:49:21.794 INFO:tasks.workunit.client.0.smithi110.stdout:3/360: dread d0/d7/f2a [0,4194304] 0 2024-04-25T15:49:21.795 INFO:tasks.workunit.client.0.smithi110.stdout:3/361: readlink d0/d16/l74 0 2024-04-25T15:49:21.798 INFO:tasks.workunit.client.0.smithi110.stdout:3/362: mknod d0/d7/d1c/d2f/d4c/d63/c7a 0 2024-04-25T15:49:21.806 INFO:tasks.workunit.client.0.smithi110.stdout:3/363: rmdir d0/d2e 39 2024-04-25T15:49:21.809 INFO:tasks.workunit.client.0.smithi110.stdout:6/209: write d1/f9 [1170451,125229] 0 2024-04-25T15:49:21.812 INFO:tasks.workunit.client.0.smithi110.stdout:6/210: creat d1/db/d22/d33/d43/f4e x:0 0 0 2024-04-25T15:49:21.815 INFO:tasks.workunit.client.0.smithi110.stdout:6/211: write d1/d12/f2a [2650023,98210] 0 2024-04-25T15:49:21.817 INFO:tasks.workunit.client.0.smithi110.stdout:6/212: symlink d1/d12/l4f 0 2024-04-25T15:49:21.818 INFO:tasks.workunit.client.0.smithi110.stdout:1/332: truncate d0/f27 2616211 0 2024-04-25T15:49:21.825 INFO:tasks.workunit.client.0.smithi110.stdout:6/213: dwrite d1/db/d22/f2c [0,4194304] 0 2024-04-25T15:49:21.827 INFO:tasks.workunit.client.0.smithi110.stdout:1/333: dwrite d0/d14/f62 [0,4194304] 0 2024-04-25T15:49:21.830 INFO:tasks.workunit.client.0.smithi110.stdout:1/334: dread d0/d25/f3d [0,4194304] 0 2024-04-25T15:49:21.831 INFO:tasks.workunit.client.0.smithi110.stdout:6/214: unlink d1/dd/c47 0 2024-04-25T15:49:21.832 INFO:tasks.workunit.client.0.smithi110.stdout:1/335: mknod d0/d25/c63 0 2024-04-25T15:49:21.834 INFO:tasks.workunit.client.0.smithi110.stdout:1/336: read - d0/d14/f57 zero size 2024-04-25T15:49:21.835 INFO:tasks.workunit.client.0.smithi110.stdout:1/337: write d0/d23/f5a [296806,61703] 0 2024-04-25T15:49:21.838 INFO:tasks.workunit.client.0.smithi110.stdout:1/338: creat d0/d23/d56/f64 x:0 0 0 2024-04-25T15:49:21.839 INFO:tasks.workunit.client.0.smithi110.stdout:1/339: write d0/d25/d33/f48 [269184,63302] 0 2024-04-25T15:49:21.840 INFO:tasks.workunit.client.0.smithi110.stdout:1/340: chown d0/d14/d5f 13751 1 2024-04-25T15:49:21.856 INFO:tasks.workunit.client.0.smithi110.stdout:5/150: sync 2024-04-25T15:49:21.856 INFO:tasks.workunit.client.0.smithi110.stdout:7/224: sync 2024-04-25T15:49:21.856 INFO:tasks.workunit.client.0.smithi110.stdout:9/194: sync 2024-04-25T15:49:21.856 INFO:tasks.workunit.client.0.smithi110.stdout:0/243: sync 2024-04-25T15:49:21.856 INFO:tasks.workunit.client.0.smithi110.stdout:4/239: sync 2024-04-25T15:49:21.858 INFO:tasks.workunit.client.0.smithi110.stdout:1/341: rmdir d0/d23/d56 39 2024-04-25T15:49:21.860 INFO:tasks.workunit.client.0.smithi110.stdout:4/240: rmdir d13/d24/d2d/d36 39 2024-04-25T15:49:21.860 INFO:tasks.workunit.client.0.smithi110.stdout:6/215: dwrite d1/db/d22/f2c [4194304,4194304] 0 2024-04-25T15:49:21.861 INFO:tasks.workunit.client.0.smithi110.stdout:4/241: chown d13/d24 318556558 1 2024-04-25T15:49:21.862 INFO:tasks.workunit.client.0.smithi110.stdout:6/216: fdatasync d1/d12/f1b 0 2024-04-25T15:49:21.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:21 smithi110 ceph-mon[26344]: pgmap v27: 65 pgs: 65 active+clean; 489 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.4 MiB/s rd, 46 MiB/s wr, 320 op/s 2024-04-25T15:49:21.864 INFO:tasks.workunit.client.0.smithi110.stdout:5/151: fdatasync d4/d1d/d1f/f32 0 2024-04-25T15:49:21.865 INFO:tasks.workunit.client.0.smithi110.stdout:0/244: dread d1/f19 [0,4194304] 0 2024-04-25T15:49:21.868 INFO:tasks.workunit.client.0.smithi110.stdout:9/195: dread d9/db/f11 [0,4194304] 0 2024-04-25T15:49:21.871 INFO:tasks.workunit.client.0.smithi110.stdout:6/217: write d1/db/f19 [3359548,126975] 0 2024-04-25T15:49:21.873 INFO:tasks.workunit.client.0.smithi110.stdout:5/152: dwrite d4/f25 [0,4194304] 0 2024-04-25T15:49:21.874 INFO:tasks.workunit.client.0.smithi110.stdout:7/225: symlink d1b/d29/d42/l4c 0 2024-04-25T15:49:21.875 INFO:tasks.workunit.client.0.smithi110.stdout:7/226: read d1b/f31 [675703,88789] 0 2024-04-25T15:49:21.875 INFO:tasks.workunit.client.0.smithi110.stdout:1/342: fsync d0/d25/f6 0 2024-04-25T15:49:21.876 INFO:tasks.workunit.client.0.smithi110.stdout:0/245: truncate d1/d23/f2f 581415 0 2024-04-25T15:49:21.881 INFO:tasks.workunit.client.0.smithi110.stdout:9/196: rename f0 to d9/db/df/d2c/f3a 0 2024-04-25T15:49:21.882 INFO:tasks.workunit.client.0.smithi110.stdout:9/197: dread - d9/db/df/d25/f37 zero size 2024-04-25T15:49:21.885 INFO:tasks.workunit.client.0.smithi110.stdout:9/198: dread d9/d13/f29 [0,4194304] 0 2024-04-25T15:49:21.888 INFO:tasks.workunit.client.0.smithi110.stdout:4/242: creat d13/d24/f48 x:0 0 0 2024-04-25T15:49:21.890 INFO:tasks.workunit.client.0.smithi110.stdout:6/218: mknod d1/db/d22/d27/c50 0 2024-04-25T15:49:21.894 INFO:tasks.workunit.client.0.smithi110.stdout:6/219: dread d1/db/f19 [0,4194304] 0 2024-04-25T15:49:21.894 INFO:tasks.workunit.client.0.smithi110.stdout:1/343: creat d0/d14/d5f/f65 x:0 0 0 2024-04-25T15:49:21.899 INFO:tasks.workunit.client.0.smithi110.stdout:4/243: creat d13/d24/d26/f49 x:0 0 0 2024-04-25T15:49:21.901 INFO:tasks.workunit.client.0.smithi110.stdout:5/153: link d4/d1d/c26 d4/d1d/d1f/d35/c36 0 2024-04-25T15:49:21.902 INFO:tasks.workunit.client.0.smithi110.stdout:7/227: link d1b/l32 d1b/d29/l4d 0 2024-04-25T15:49:21.905 INFO:tasks.workunit.client.0.smithi110.stdout:1/344: mknod d0/d14/d5f/c66 0 2024-04-25T15:49:21.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/244: fdatasync d13/f1a 0 2024-04-25T15:49:21.909 INFO:tasks.workunit.client.0.smithi110.stdout:9/199: write d9/db/df/f21 [8880620,86644] 0 2024-04-25T15:49:21.910 INFO:tasks.workunit.client.0.smithi110.stdout:7/228: rmdir d1b/d29 39 2024-04-25T15:49:21.911 INFO:tasks.workunit.client.0.smithi110.stdout:6/220: creat d1/dd/d3c/f51 x:0 0 0 2024-04-25T15:49:21.913 INFO:tasks.workunit.client.0.smithi110.stdout:0/246: write d1/d23/f2f [1309923,2902] 0 2024-04-25T15:49:21.914 INFO:tasks.workunit.client.0.smithi110.stdout:0/247: stat d1/d23/d30/d33/d35/f41 0 2024-04-25T15:49:21.915 INFO:tasks.workunit.client.0.smithi110.stdout:0/248: fdatasync d1/d4c/f53 0 2024-04-25T15:49:21.917 INFO:tasks.workunit.client.0.smithi110.stdout:9/200: symlink d9/db/df/l3b 0 2024-04-25T15:49:21.918 INFO:tasks.workunit.client.0.smithi110.stdout:7/229: chown d1b/c3e 84085648 1 2024-04-25T15:49:21.919 INFO:tasks.workunit.client.0.smithi110.stdout:1/345: creat d0/d23/d56/f67 x:0 0 0 2024-04-25T15:49:21.920 INFO:tasks.workunit.client.0.smithi110.stdout:6/221: rename d1/db/d22/d27/c50 to d1/d12/c52 0 2024-04-25T15:49:21.921 INFO:tasks.workunit.client.0.smithi110.stdout:6/222: chown d1/dd/l18 14737162 1 2024-04-25T15:49:21.924 INFO:tasks.workunit.client.0.smithi110.stdout:1/346: symlink d0/l68 0 2024-04-25T15:49:21.925 INFO:tasks.workunit.client.0.smithi110.stdout:1/347: chown d0/d25/c1b 701496 1 2024-04-25T15:49:21.927 INFO:tasks.workunit.client.0.smithi110.stdout:5/154: getdents d4/d1d 0 2024-04-25T15:49:21.929 INFO:tasks.workunit.client.0.smithi110.stdout:6/223: rename d1/d12/f2a to d1/db/d22/d33/d43/f53 0 2024-04-25T15:49:21.929 INFO:tasks.workunit.client.0.smithi110.stdout:6/224: write d1/db/f44 [877915,98384] 0 2024-04-25T15:49:21.932 INFO:tasks.workunit.client.0.smithi110.stdout:7/230: mknod d1b/d29/d46/c4e 0 2024-04-25T15:49:21.932 INFO:tasks.workunit.client.0.smithi110.stdout:7/231: readlink d1b/l28 0 2024-04-25T15:49:21.933 INFO:tasks.workunit.client.0.smithi110.stdout:7/232: dread - d1b/f48 zero size 2024-04-25T15:49:21.934 INFO:tasks.workunit.client.0.smithi110.stdout:4/245: getdents d13/d24/d26 0 2024-04-25T15:49:21.936 INFO:tasks.workunit.client.0.smithi110.stdout:4/246: write d13/d24/d26/f3b [1399135,31987] 0 2024-04-25T15:49:21.936 INFO:tasks.workunit.client.0.smithi110.stdout:1/348: mknod d0/d25/c69 0 2024-04-25T15:49:21.939 INFO:tasks.workunit.client.0.smithi110.stdout:0/249: dwrite d1/d47/f52 [0,4194304] 0 2024-04-25T15:49:21.941 INFO:tasks.workunit.client.0.smithi110.stdout:7/233: chown d1b/d29/l4d 1 1 2024-04-25T15:49:21.951 INFO:tasks.workunit.client.0.smithi110.stdout:4/247: dread d13/d1c/f2c [0,4194304] 0 2024-04-25T15:49:21.952 INFO:tasks.workunit.client.0.smithi110.stdout:0/250: fdatasync d1/f19 0 2024-04-25T15:49:21.957 INFO:tasks.workunit.client.0.smithi110.stdout:1/349: mknod d0/d25/d58/c6a 0 2024-04-25T15:49:21.957 INFO:tasks.workunit.client.0.smithi110.stdout:4/248: truncate fd 2589806 0 2024-04-25T15:49:21.958 INFO:tasks.workunit.client.0.smithi110.stdout:6/225: dwrite d1/fe [0,4194304] 0 2024-04-25T15:49:21.959 INFO:tasks.workunit.client.0.smithi110.stdout:0/251: creat d1/d4c/f62 x:0 0 0 2024-04-25T15:49:21.960 INFO:tasks.workunit.client.0.smithi110.stdout:4/249: chown ff 52318 1 2024-04-25T15:49:21.960 INFO:tasks.workunit.client.0.smithi110.stdout:1/350: fsync d0/d23/f13 0 2024-04-25T15:49:21.961 INFO:tasks.workunit.client.0.smithi110.stdout:6/226: fsync d1/d12/f1b 0 2024-04-25T15:49:21.962 INFO:tasks.workunit.client.0.smithi110.stdout:4/250: readlink d13/d1c/l30 0 2024-04-25T15:49:21.965 INFO:tasks.workunit.client.0.smithi110.stdout:5/155: dwrite d4/d1d/f34 [4194304,4194304] 0 2024-04-25T15:49:21.966 INFO:tasks.workunit.client.0.smithi110.stdout:1/351: mkdir d0/d51/d6b 0 2024-04-25T15:49:21.966 INFO:tasks.workunit.client.0.smithi110.stdout:6/227: read d1/d12/f1f [4070432,33889] 0 2024-04-25T15:49:21.966 INFO:tasks.workunit.client.0.smithi110.stdout:7/234: dread fd [0,4194304] 0 2024-04-25T15:49:21.971 INFO:tasks.workunit.client.0.smithi110.stdout:0/252: unlink d1/d4c/f62 0 2024-04-25T15:49:21.971 INFO:tasks.workunit.client.0.smithi110.stdout:6/228: write d1/db/d22/d33/f3b [550658,16535] 0 2024-04-25T15:49:21.974 INFO:tasks.workunit.client.0.smithi110.stdout:4/251: mknod d13/d1c/c4a 0 2024-04-25T15:49:21.977 INFO:tasks.workunit.client.0.smithi110.stdout:5/156: dwrite d4/fd [0,4194304] 0 2024-04-25T15:49:21.979 INFO:tasks.workunit.client.0.smithi110.stdout:7/235: dwrite d1b/f36 [0,4194304] 0 2024-04-25T15:49:21.987 INFO:tasks.workunit.client.0.smithi110.stdout:6/229: fsync d1/db/d22/d33/d43/f53 0 2024-04-25T15:49:21.988 INFO:tasks.workunit.client.0.smithi110.stdout:5/157: mknod d4/d1d/d1f/c37 0 2024-04-25T15:49:21.991 INFO:tasks.workunit.client.0.smithi110.stdout:7/236: creat d1b/d29/f4f x:0 0 0 2024-04-25T15:49:21.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/253: symlink d1/d23/d30/d33/d5b/l63 0 2024-04-25T15:49:21.994 INFO:tasks.workunit.client.0.smithi110.stdout:4/252: symlink d13/l4b 0 2024-04-25T15:49:21.995 INFO:tasks.workunit.client.0.smithi110.stdout:4/253: write d13/d24/d26/f3b [34802,84445] 0 2024-04-25T15:49:21.996 INFO:tasks.workunit.client.0.smithi110.stdout:4/254: readlink d13/d24/d26/l3e 0 2024-04-25T15:49:21.996 INFO:tasks.workunit.client.0.smithi110.stdout:1/352: rename d0/d23/d56/c60 to d0/d23/c6c 0 2024-04-25T15:49:21.998 INFO:tasks.workunit.client.0.smithi110.stdout:0/254: creat d1/d20/f64 x:0 0 0 2024-04-25T15:49:22.001 INFO:tasks.workunit.client.0.smithi110.stdout:4/255: rename f12 to d13/d1c/f4c 0 2024-04-25T15:49:22.004 INFO:tasks.workunit.client.0.smithi110.stdout:6/230: symlink d1/db/d24/d2d/l54 0 2024-04-25T15:49:22.004 INFO:tasks.workunit.client.0.smithi110.stdout:6/231: read d1/db/d22/f2c [925086,8987] 0 2024-04-25T15:49:22.005 INFO:tasks.workunit.client.0.smithi110.stdout:4/256: unlink d13/l4b 0 2024-04-25T15:49:22.007 INFO:tasks.workunit.client.0.smithi110.stdout:1/353: rename d0/d23/c22 to d0/d14/c6d 0 2024-04-25T15:49:22.007 INFO:tasks.workunit.client.0.smithi110.stdout:0/255: rename d1/d23/d30/d33 to d1/d23/d30/d33/d35/d65 22 2024-04-25T15:49:22.007 INFO:tasks.workunit.client.0.smithi110.stdout:1/354: chown d0/d23/d56/f64 3 1 2024-04-25T15:49:22.008 INFO:tasks.workunit.client.0.smithi110.stdout:0/256: truncate d1/f4e 98570 0 2024-04-25T15:49:22.008 INFO:tasks.workunit.client.0.smithi110.stdout:7/237: creat d1b/f50 x:0 0 0 2024-04-25T15:49:22.009 INFO:tasks.workunit.client.0.smithi110.stdout:7/238: dread - d1b/d29/f4f zero size 2024-04-25T15:49:22.013 INFO:tasks.workunit.client.0.smithi110.stdout:7/239: dread d1b/f36 [0,4194304] 0 2024-04-25T15:49:22.014 INFO:tasks.workunit.client.0.smithi110.stdout:6/232: rename d1/db/d24/f45 to d1/db/d22/f55 0 2024-04-25T15:49:22.015 INFO:tasks.workunit.client.0.smithi110.stdout:1/355: creat d0/d25/d58/f6e x:0 0 0 2024-04-25T15:49:22.020 INFO:tasks.workunit.client.0.smithi110.stdout:4/257: creat d13/d24/d2d/d36/f4d x:0 0 0 2024-04-25T15:49:22.022 INFO:tasks.workunit.client.0.smithi110.stdout:5/158: truncate d4/f6 5685669 0 2024-04-25T15:49:22.023 INFO:tasks.workunit.client.0.smithi110.stdout:5/159: fsync d4/fd 0 2024-04-25T15:49:22.024 INFO:tasks.workunit.client.0.smithi110.stdout:7/240: rename f1a to d1b/d29/d46/f51 0 2024-04-25T15:49:22.026 INFO:tasks.workunit.client.0.smithi110.stdout:0/257: mknod d1/d23/d30/c66 0 2024-04-25T15:49:22.028 INFO:tasks.workunit.client.0.smithi110.stdout:4/258: symlink d13/d1c/l4e 0 2024-04-25T15:49:22.028 INFO:tasks.workunit.client.0.smithi110.stdout:4/259: dread - d13/d24/d26/f49 zero size 2024-04-25T15:49:22.029 INFO:tasks.workunit.client.0.smithi110.stdout:4/260: fdatasync d13/d24/f48 0 2024-04-25T15:49:22.033 INFO:tasks.workunit.client.0.smithi110.stdout:5/160: creat d4/d1d/f38 x:0 0 0 2024-04-25T15:49:22.035 INFO:tasks.workunit.client.0.smithi110.stdout:7/241: creat d1b/d29/d46/f52 x:0 0 0 2024-04-25T15:49:22.038 INFO:tasks.workunit.client.0.smithi110.stdout:1/356: rename d0/d25/d58 to d0/d14/d5f/d6f 0 2024-04-25T15:49:22.039 INFO:tasks.workunit.client.0.smithi110.stdout:1/357: read d0/d23/f30 [8952,128058] 0 2024-04-25T15:49:22.039 INFO:tasks.workunit.client.0.smithi110.stdout:1/358: chown d0/d14/d5f/d6f 5 1 2024-04-25T15:49:22.040 INFO:tasks.workunit.client.0.smithi110.stdout:7/242: dwrite f4 [4194304,4194304] 0 2024-04-25T15:49:22.041 INFO:tasks.workunit.client.0.smithi110.stdout:7/243: dread - d1b/f45 zero size 2024-04-25T15:49:22.063 INFO:tasks.workunit.client.0.smithi110.stdout:2/172: sync 2024-04-25T15:49:22.064 INFO:tasks.workunit.client.0.smithi110.stdout:5/161: rmdir d4/d2c 39 2024-04-25T15:49:22.072 INFO:tasks.workunit.client.0.smithi110.stdout:7/244: creat d1b/d29/d38/f53 x:0 0 0 2024-04-25T15:49:22.073 INFO:tasks.workunit.client.0.smithi110.stdout:7/245: chown d1b/c23 1300 1 2024-04-25T15:49:22.075 INFO:tasks.workunit.client.0.smithi110.stdout:6/233: dwrite d1/db/d24/d2d/f3a [0,4194304] 0 2024-04-25T15:49:22.075 INFO:tasks.workunit.client.0.smithi110.stdout:7/246: write d1b/d29/d38/f53 [825771,118156] 0 2024-04-25T15:49:22.078 INFO:tasks.workunit.client.0.smithi110.stdout:0/258: link d1/d23/d30/l3b d1/d23/d30/d33/l67 0 2024-04-25T15:49:22.079 INFO:tasks.workunit.client.0.smithi110.stdout:1/359: mknod d0/d51/d6b/c70 0 2024-04-25T15:49:22.084 INFO:tasks.workunit.client.0.smithi110.stdout:4/261: creat d13/d24/d2d/f4f x:0 0 0 2024-04-25T15:49:22.084 INFO:tasks.workunit.client.0.smithi110.stdout:4/262: stat d13/f1a 0 2024-04-25T15:49:22.085 INFO:tasks.workunit.client.0.smithi110.stdout:6/234: unlink d1/ca 0 2024-04-25T15:49:22.090 INFO:tasks.workunit.client.0.smithi110.stdout:5/162: unlink d4/d2c/f17 0 2024-04-25T15:49:22.091 INFO:tasks.workunit.client.0.smithi110.stdout:5/163: stat d4/d2c/l12 0 2024-04-25T15:49:22.092 INFO:tasks.workunit.client.0.smithi110.stdout:0/259: creat d1/d4c/f68 x:0 0 0 2024-04-25T15:49:22.098 INFO:tasks.workunit.client.0.smithi110.stdout:0/260: mkdir d1/d20/d69 0 2024-04-25T15:49:22.101 INFO:tasks.workunit.client.0.smithi110.stdout:2/173: getdents de 0 2024-04-25T15:49:22.102 INFO:tasks.workunit.client.0.smithi110.stdout:4/263: mknod d13/d24/c50 0 2024-04-25T15:49:22.103 INFO:tasks.workunit.client.0.smithi110.stdout:4/264: write d13/d1c/f47 [409808,109948] 0 2024-04-25T15:49:22.106 INFO:tasks.workunit.client.0.smithi110.stdout:7/247: write fb [1544338,108916] 0 2024-04-25T15:49:22.107 INFO:tasks.workunit.client.0.smithi110.stdout:0/261: fsync d1/fd 0 2024-04-25T15:49:22.107 INFO:tasks.workunit.client.0.smithi110.stdout:1/360: link d0/d51/d6b/c70 d0/d25/c71 0 2024-04-25T15:49:22.107 INFO:tasks.workunit.client.0.smithi110.stdout:7/248: write d1b/f45 [140749,33207] 0 2024-04-25T15:49:22.109 INFO:tasks.workunit.client.0.smithi110.stdout:0/262: write d1/d23/f42 [470346,16806] 0 2024-04-25T15:49:22.110 INFO:tasks.workunit.client.0.smithi110.stdout:1/361: write d0/d25/d33/f49 [4171956,2440] 0 2024-04-25T15:49:22.114 INFO:tasks.workunit.client.0.smithi110.stdout:1/362: dread d0/d14/f62 [0,4194304] 0 2024-04-25T15:49:22.115 INFO:tasks.workunit.client.0.smithi110.stdout:1/363: fsync d0/d23/f13 0 2024-04-25T15:49:22.116 INFO:tasks.workunit.client.0.smithi110.stdout:7/249: rename fd to d1b/d29/d38/f54 0 2024-04-25T15:49:22.120 INFO:tasks.workunit.client.0.smithi110.stdout:6/235: truncate d1/db/d24/d2d/f3a 2141395 0 2024-04-25T15:49:22.122 INFO:tasks.workunit.client.0.smithi110.stdout:0/263: creat d1/d20/f6a x:0 0 0 2024-04-25T15:49:22.122 INFO:tasks.workunit.client.0.smithi110.stdout:4/265: write f6 [3461747,97230] 0 2024-04-25T15:49:22.122 INFO:tasks.workunit.client.0.smithi110.stdout:4/266: fdatasync d13/d24/d26/f49 0 2024-04-25T15:49:22.126 INFO:tasks.workunit.client.0.smithi110.stdout:4/267: dread d13/d1c/f2c [0,4194304] 0 2024-04-25T15:49:22.127 INFO:tasks.workunit.client.0.smithi110.stdout:7/250: rmdir d1b/d29 39 2024-04-25T15:49:22.133 INFO:tasks.workunit.client.0.smithi110.stdout:6/236: rmdir d1/db/d22 39 2024-04-25T15:49:22.133 INFO:tasks.workunit.client.0.smithi110.stdout:2/174: dwrite de/d11/f19 [0,4194304] 0 2024-04-25T15:49:22.134 INFO:tasks.workunit.client.0.smithi110.stdout:2/175: chown de/d11 2907420 1 2024-04-25T15:49:22.138 INFO:tasks.workunit.client.0.smithi110.stdout:4/268: mkdir d13/d24/d2d/d36/d51 0 2024-04-25T15:49:22.144 INFO:tasks.workunit.client.0.smithi110.stdout:2/176: mknod de/d11/c2a 0 2024-04-25T15:49:22.147 INFO:tasks.workunit.client.0.smithi110.stdout:4/269: dwrite ff [4194304,4194304] 0 2024-04-25T15:49:22.149 INFO:tasks.workunit.client.0.smithi110.stdout:7/251: dwrite d1b/d29/f37 [0,4194304] 0 2024-04-25T15:49:22.150 INFO:tasks.workunit.client.0.smithi110.stdout:4/270: fsync d13/d1c/f25 0 2024-04-25T15:49:22.150 INFO:tasks.workunit.client.0.smithi110.stdout:2/177: mknod de/c2b 0 2024-04-25T15:49:22.152 INFO:tasks.workunit.client.0.smithi110.stdout:7/252: write d1b/f31 [53553,60080] 0 2024-04-25T15:49:22.153 INFO:tasks.workunit.client.0.smithi110.stdout:4/271: write d13/d1c/f41 [2237146,50289] 0 2024-04-25T15:49:22.155 INFO:tasks.workunit.client.0.smithi110.stdout:1/364: truncate d0/d23/f5a 426071 0 2024-04-25T15:49:22.159 INFO:tasks.workunit.client.0.smithi110.stdout:6/237: link d1/db/d22/f4d d1/db/d22/d33/f56 0 2024-04-25T15:49:22.163 INFO:tasks.workunit.client.0.smithi110.stdout:7/253: truncate d1b/f36 5091800 0 2024-04-25T15:49:22.164 INFO:tasks.workunit.client.0.smithi110.stdout:1/365: chown d0/d23/c6c 3598389 1 2024-04-25T15:49:22.168 INFO:tasks.workunit.client.0.smithi110.stdout:7/254: mkdir d1b/d29/d38/d55 0 2024-04-25T15:49:22.173 INFO:tasks.workunit.client.0.smithi110.stdout:6/238: dread d1/f9 [0,4194304] 0 2024-04-25T15:49:22.173 INFO:tasks.workunit.client.0.smithi110.stdout:1/366: rename d0/d23/d54 to d0/d23/d56/d72 0 2024-04-25T15:49:22.174 INFO:tasks.workunit.client.0.smithi110.stdout:1/367: read - d0/d23/f3b zero size 2024-04-25T15:49:22.175 INFO:tasks.workunit.client.0.smithi110.stdout:1/368: fdatasync d0/d25/d33/f47 0 2024-04-25T15:49:22.175 INFO:tasks.workunit.client.0.smithi110.stdout:7/255: chown d1b/d29/d38/l3b 1440 1 2024-04-25T15:49:22.178 INFO:tasks.workunit.client.0.smithi110.stdout:6/239: creat d1/db/d22/f57 x:0 0 0 2024-04-25T15:49:22.179 INFO:tasks.workunit.client.0.smithi110.stdout:1/369: rmdir d0/d14/d5f/d6f 39 2024-04-25T15:49:22.180 INFO:tasks.workunit.client.0.smithi110.stdout:1/370: dread d0/d23/f30 [0,4194304] 0 2024-04-25T15:49:22.182 INFO:tasks.workunit.client.0.smithi110.stdout:7/256: mknod d1b/d29/d46/c56 0 2024-04-25T15:49:22.182 INFO:tasks.workunit.client.0.smithi110.stdout:4/272: dread d13/d1c/f25 [0,4194304] 0 2024-04-25T15:49:22.183 INFO:tasks.workunit.client.0.smithi110.stdout:4/273: fsync d13/d24/f43 0 2024-04-25T15:49:22.184 INFO:tasks.workunit.client.0.smithi110.stdout:4/274: chown d13/d1c/f25 5 1 2024-04-25T15:49:22.185 INFO:tasks.workunit.client.0.smithi110.stdout:6/240: unlink d1/db/d24/c39 0 2024-04-25T15:49:22.186 INFO:tasks.workunit.client.0.smithi110.stdout:1/371: mknod d0/d14/c73 0 2024-04-25T15:49:22.189 INFO:tasks.workunit.client.0.smithi110.stdout:7/257: creat d1b/d29/f57 x:0 0 0 2024-04-25T15:49:22.190 INFO:tasks.workunit.client.0.smithi110.stdout:4/275: rename d13/d24 to d13/d24/d52 22 2024-04-25T15:49:22.191 INFO:tasks.workunit.client.0.smithi110.stdout:6/241: symlink d1/d12/l58 0 2024-04-25T15:49:22.192 INFO:tasks.workunit.client.0.smithi110.stdout:1/372: mknod d0/d23/d56/c74 0 2024-04-25T15:49:22.195 INFO:tasks.workunit.client.0.smithi110.stdout:2/178: rename de/f25 to de/f2c 0 2024-04-25T15:49:22.196 INFO:tasks.workunit.client.0.smithi110.stdout:4/276: dwrite d13/d24/f2a [0,4194304] 0 2024-04-25T15:49:22.197 INFO:tasks.workunit.client.0.smithi110.stdout:4/277: fsync fc 0 2024-04-25T15:49:22.204 INFO:tasks.workunit.client.0.smithi110.stdout:4/278: dwrite d13/d24/d2d/d36/f40 [0,4194304] 0 2024-04-25T15:49:22.205 INFO:tasks.workunit.client.0.smithi110.stdout:6/242: creat d1/db/d24/d2d/f59 x:0 0 0 2024-04-25T15:49:22.208 INFO:tasks.workunit.client.0.smithi110.stdout:4/279: unlink d13/d24/f43 0 2024-04-25T15:49:22.215 INFO:tasks.workunit.client.0.smithi110.stdout:4/280: creat d13/d24/d2d/d36/f53 x:0 0 0 2024-04-25T15:49:22.216 INFO:tasks.workunit.client.0.smithi110.stdout:2/179: fdatasync de/f10 0 2024-04-25T15:49:22.219 INFO:tasks.workunit.client.0.smithi110.stdout:2/180: chown de/d11/f21 115 1 2024-04-25T15:49:22.220 INFO:tasks.workunit.client.0.smithi110.stdout:2/181: write de/d11/f19 [3385391,17746] 0 2024-04-25T15:49:22.221 INFO:tasks.workunit.client.0.smithi110.stdout:4/281: getdents d13/d24/d2d 0 2024-04-25T15:49:22.223 INFO:tasks.workunit.client.0.smithi110.stdout:6/243: rmdir d1/db/d24/d2d 39 2024-04-25T15:49:22.224 INFO:tasks.workunit.client.0.smithi110.stdout:6/244: fsync d1/db/d22/d27/f2b 0 2024-04-25T15:49:22.224 INFO:tasks.workunit.client.0.smithi110.stdout:2/182: dread de/d11/f19 [0,4194304] 0 2024-04-25T15:49:22.225 INFO:tasks.workunit.client.0.smithi110.stdout:6/245: truncate d1/d12/f49 357653 0 2024-04-25T15:49:22.227 INFO:tasks.workunit.client.0.smithi110.stdout:4/282: fsync d13/f1a 0 2024-04-25T15:49:22.228 INFO:tasks.workunit.client.0.smithi110.stdout:4/283: stat d13/d24/d2d/d36/d51 0 2024-04-25T15:49:22.228 INFO:tasks.workunit.client.0.smithi110.stdout:4/284: chown d13/d24/d26 20484373 1 2024-04-25T15:49:22.235 INFO:tasks.workunit.client.0.smithi110.stdout:2/183: dread de/f22 [0,4194304] 0 2024-04-25T15:49:22.235 INFO:tasks.workunit.client.0.smithi110.stdout:2/184: write de/f10 [4951747,69887] 0 2024-04-25T15:49:22.236 INFO:tasks.workunit.client.0.smithi110.stdout:6/246: creat d1/db/d24/d32/f5a x:0 0 0 2024-04-25T15:49:22.238 INFO:tasks.workunit.client.0.smithi110.stdout:4/285: creat d13/d24/d2d/d36/d51/f54 x:0 0 0 2024-04-25T15:49:22.241 INFO:tasks.workunit.client.0.smithi110.stdout:6/247: creat d1/d12/f5b x:0 0 0 2024-04-25T15:49:22.246 INFO:tasks.workunit.client.0.smithi110.stdout:2/185: creat de/f2d x:0 0 0 2024-04-25T15:49:22.249 INFO:tasks.workunit.client.0.smithi110.stdout:4/286: dread fd [0,4194304] 0 2024-04-25T15:49:22.251 INFO:tasks.workunit.client.0.smithi110.stdout:2/186: symlink de/l2e 0 2024-04-25T15:49:22.251 INFO:tasks.workunit.client.0.smithi110.stdout:4/287: mkdir d13/d24/d2d/d55 0 2024-04-25T15:49:22.253 INFO:tasks.workunit.client.0.smithi110.stdout:6/248: rmdir d1/db/d24/d2d/d4c 0 2024-04-25T15:49:22.256 INFO:tasks.workunit.client.0.smithi110.stdout:4/288: write d13/d1c/f2c [1618306,43847] 0 2024-04-25T15:49:22.257 INFO:tasks.workunit.client.0.smithi110.stdout:6/249: mknod d1/db/d22/d33/c5c 0 2024-04-25T15:49:22.258 INFO:tasks.workunit.client.0.smithi110.stdout:2/187: creat de/d11/f2f x:0 0 0 2024-04-25T15:49:22.260 INFO:tasks.workunit.client.0.smithi110.stdout:4/289: mkdir d13/d24/d3c/d56 0 2024-04-25T15:49:22.263 INFO:tasks.workunit.client.0.smithi110.stdout:4/290: fdatasync fd 0 2024-04-25T15:49:22.264 INFO:tasks.workunit.client.0.smithi110.stdout:4/291: chown d13/d24/c32 228634 1 2024-04-25T15:49:22.264 INFO:tasks.workunit.client.0.smithi110.stdout:2/188: mkdir de/d30 0 2024-04-25T15:49:22.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/292: rename d13/f3d to d13/d24/d3c/d56/f57 0 2024-04-25T15:49:22.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/293: fdatasync f6 0 2024-04-25T15:49:22.273 INFO:tasks.workunit.client.0.smithi110.stdout:4/294: creat d13/d24/f58 x:0 0 0 2024-04-25T15:49:22.274 INFO:tasks.workunit.client.0.smithi110.stdout:4/295: readlink l3 0 2024-04-25T15:49:22.279 INFO:tasks.workunit.client.0.smithi110.stdout:4/296: rmdir d13/d24/d2d 39 2024-04-25T15:49:22.284 INFO:tasks.workunit.client.0.smithi110.stdout:2/189: rmdir de/d11 39 2024-04-25T15:49:22.286 INFO:tasks.workunit.client.0.smithi110.stdout:4/297: rename d13/d24/c2b to d13/d24/d2d/d36/d51/c59 0 2024-04-25T15:49:22.287 INFO:tasks.workunit.client.0.smithi110.stdout:4/298: write d13/d24/d2d/d36/f42 [1585182,89284] 0 2024-04-25T15:49:22.288 INFO:tasks.workunit.client.0.smithi110.stdout:6/250: truncate d1/f1d 582883 0 2024-04-25T15:49:22.298 INFO:tasks.workunit.client.0.smithi110.stdout:2/190: unlink de/f10 0 2024-04-25T15:49:22.299 INFO:tasks.workunit.client.0.smithi110.stdout:4/299: dread ff [0,4194304] 0 2024-04-25T15:49:22.301 INFO:tasks.workunit.client.0.smithi110.stdout:6/251: link d1/d12/f1b d1/dd/d3c/f5d 0 2024-04-25T15:49:22.304 INFO:tasks.workunit.client.0.smithi110.stdout:2/191: write de/d11/f21 [391525,122955] 0 2024-04-25T15:49:22.307 INFO:tasks.workunit.client.0.smithi110.stdout:2/192: chown de/l23 996055800 1 2024-04-25T15:49:22.309 INFO:tasks.workunit.client.0.smithi110.stdout:4/300: rmdir d13 39 2024-04-25T15:49:22.311 INFO:tasks.workunit.client.0.smithi110.stdout:6/252: creat d1/f5e x:0 0 0 2024-04-25T15:49:22.315 INFO:tasks.workunit.client.0.smithi110.stdout:4/301: fsync d13/d24/d26/f31 0 2024-04-25T15:49:22.317 INFO:tasks.workunit.client.0.smithi110.stdout:4/302: dread - d13/d24/d2d/d36/f4d zero size 2024-04-25T15:49:22.319 INFO:tasks.workunit.client.0.smithi110.stdout:4/303: write fc [615482,107285] 0 2024-04-25T15:49:22.319 INFO:tasks.workunit.client.0.smithi110.stdout:6/253: dwrite d1/fe [0,4194304] 0 2024-04-25T15:49:22.322 INFO:tasks.workunit.client.0.smithi110.stdout:4/304: chown d13/d24/d2d/d36/d51 65651 1 2024-04-25T15:49:22.323 INFO:tasks.workunit.client.0.smithi110.stdout:2/193: dwrite de/f2c [0,4194304] 0 2024-04-25T15:49:22.330 INFO:tasks.workunit.client.0.smithi110.stdout:6/254: rename d1/db/d24/d2d/l37 to d1/d12/l5f 0 2024-04-25T15:49:22.332 INFO:tasks.workunit.client.0.smithi110.stdout:2/194: symlink de/l31 0 2024-04-25T15:49:22.336 INFO:tasks.workunit.client.0.smithi110.stdout:6/255: creat d1/db/d22/d33/d43/f60 x:0 0 0 2024-04-25T15:49:22.339 INFO:tasks.workunit.client.0.smithi110.stdout:2/195: unlink de/l12 0 2024-04-25T15:49:22.343 INFO:tasks.workunit.client.0.smithi110.stdout:2/196: rename de/l14 to de/d30/l32 0 2024-04-25T15:49:22.344 INFO:tasks.workunit.client.0.smithi110.stdout:2/197: fsync f8 0 2024-04-25T15:49:22.352 INFO:tasks.workunit.client.0.smithi110.stdout:2/198: dwrite de/f24 [0,4194304] 0 2024-04-25T15:49:22.354 INFO:tasks.workunit.client.0.smithi110.stdout:4/305: dread d13/d1c/f4c [0,4194304] 0 2024-04-25T15:49:22.357 INFO:tasks.workunit.client.0.smithi110.stdout:4/306: read d13/d24/d3c/d56/f57 [30490,44158] 0 2024-04-25T15:49:22.360 INFO:tasks.workunit.client.0.smithi110.stdout:2/199: dwrite de/d11/f2f [0,4194304] 0 2024-04-25T15:49:22.362 INFO:tasks.workunit.client.0.smithi110.stdout:4/307: rename d13/d24/d2d/d36/f42 to d13/d24/d2d/d36/f5a 0 2024-04-25T15:49:22.366 INFO:tasks.workunit.client.0.smithi110.stdout:2/200: write de/d11/f19 [4889525,4488] 0 2024-04-25T15:49:22.371 INFO:tasks.workunit.client.0.smithi110.stdout:2/201: getdents de/d11 0 2024-04-25T15:49:22.377 INFO:tasks.workunit.client.0.smithi110.stdout:2/202: dwrite de/f2c [0,4194304] 0 2024-04-25T15:49:22.379 INFO:tasks.workunit.client.0.smithi110.stdout:2/203: write de/d11/f21 [143670,21573] 0 2024-04-25T15:49:22.379 INFO:tasks.workunit.client.0.smithi110.stdout:2/204: write de/f2d [183214,91682] 0 2024-04-25T15:49:22.383 INFO:tasks.workunit.client.0.smithi110.stdout:2/205: chown de/f28 16522383 1 2024-04-25T15:49:22.385 INFO:tasks.workunit.client.0.smithi110.stdout:4/308: dwrite d13/d1c/f4c [0,4194304] 0 2024-04-25T15:49:22.390 INFO:tasks.workunit.client.0.smithi110.stdout:2/206: rename de/f22 to de/d30/f33 0 2024-04-25T15:49:22.392 INFO:tasks.workunit.client.0.smithi110.stdout:2/207: creat de/d30/f34 x:0 0 0 2024-04-25T15:49:22.396 INFO:tasks.workunit.client.0.smithi110.stdout:2/208: rename de/d11/f2f to de/d30/f35 0 2024-04-25T15:49:22.396 INFO:tasks.workunit.client.0.smithi110.stdout:2/209: dread - de/d30/f34 zero size 2024-04-25T15:49:22.398 INFO:tasks.workunit.client.0.smithi110.stdout:2/210: write de/f28 [1010416,56550] 0 2024-04-25T15:49:22.399 INFO:tasks.workunit.client.0.smithi110.stdout:4/309: dread f8 [0,4194304] 0 2024-04-25T15:49:22.402 INFO:tasks.workunit.client.0.smithi110.stdout:4/310: read d13/d24/d2d/d36/f5a [1191393,59473] 0 2024-04-25T15:49:22.402 INFO:tasks.workunit.client.0.smithi110.stdout:2/211: dread f8 [0,4194304] 0 2024-04-25T15:49:22.403 INFO:tasks.workunit.client.0.smithi110.stdout:2/212: chown de/l23 193 1 2024-04-25T15:49:22.407 INFO:tasks.workunit.client.0.smithi110.stdout:4/311: rename d13/d24/d26/l38 to d13/d24/d2d/l5b 0 2024-04-25T15:49:22.407 INFO:tasks.workunit.client.0.smithi110.stdout:2/213: dread f8 [0,4194304] 0 2024-04-25T15:49:22.407 INFO:tasks.workunit.client.0.smithi110.stdout:2/214: stat de/d11/l29 0 2024-04-25T15:49:22.408 INFO:tasks.workunit.client.0.smithi110.stdout:2/215: write de/d30/f34 [576489,38638] 0 2024-04-25T15:49:22.411 INFO:tasks.workunit.client.0.smithi110.stdout:2/216: dread de/f24 [0,4194304] 0 2024-04-25T15:49:22.436 INFO:tasks.workunit.client.0.smithi110.stdout:3/364: sync 2024-04-25T15:49:22.436 INFO:tasks.workunit.client.0.smithi110.stdout:8/255: sync 2024-04-25T15:49:22.436 INFO:tasks.workunit.client.0.smithi110.stdout:4/312: mknod d13/d24/d2d/d36/c5c 0 2024-04-25T15:49:22.442 INFO:tasks.workunit.client.0.smithi110.stdout:8/256: dread d14/d21/d26/f29 [0,4194304] 0 2024-04-25T15:49:22.443 INFO:tasks.workunit.client.0.smithi110.stdout:8/257: chown d14/d42 26511 1 2024-04-25T15:49:22.444 INFO:tasks.workunit.client.0.smithi110.stdout:4/313: dwrite d13/d1c/f47 [0,4194304] 0 2024-04-25T15:49:22.445 INFO:tasks.workunit.client.0.smithi110.stdout:2/217: dread de/d30/f33 [0,4194304] 0 2024-04-25T15:49:22.451 INFO:tasks.workunit.client.0.smithi110.stdout:4/314: dwrite d13/d1c/f4c [0,4194304] 0 2024-04-25T15:49:22.454 INFO:tasks.workunit.client.0.smithi110.stdout:2/218: creat de/d30/f36 x:0 0 0 2024-04-25T15:49:22.458 INFO:tasks.workunit.client.0.smithi110.stdout:2/219: symlink de/d30/l37 0 2024-04-25T15:49:22.459 INFO:tasks.workunit.client.0.smithi110.stdout:8/258: dwrite d14/d21/d26/f29 [0,4194304] 0 2024-04-25T15:49:22.461 INFO:tasks.workunit.client.0.smithi110.stdout:2/220: truncate de/d11/f21 611536 0 2024-04-25T15:49:22.463 INFO:tasks.workunit.client.0.smithi110.stdout:2/221: write de/f28 [1593598,46914] 0 2024-04-25T15:49:22.471 INFO:tasks.workunit.client.0.smithi110.stdout:8/259: mknod d14/d4c/c4d 0 2024-04-25T15:49:22.472 INFO:tasks.workunit.client.0.smithi110.stdout:3/365: link d0/d7/d1c/d50/c62 d0/d7/d1c/d2f/d4c/c7b 0 2024-04-25T15:49:22.477 INFO:tasks.workunit.client.0.smithi110.stdout:2/222: write de/f24 [5004116,33597] 0 2024-04-25T15:49:22.480 INFO:tasks.workunit.client.0.smithi110.stdout:3/366: rename d0/d7/d1c/d2f/d4c/d63/c7a to d0/d7/d1c/c7c 0 2024-04-25T15:49:22.480 INFO:tasks.workunit.client.0.smithi110.stdout:8/260: dwrite d14/f40 [0,4194304] 0 2024-04-25T15:49:22.481 INFO:tasks.workunit.client.0.smithi110.stdout:4/315: dwrite fa [0,4194304] 0 2024-04-25T15:49:22.482 INFO:tasks.workunit.client.0.smithi110.stdout:3/367: stat d0/d7/d1c/d2f/d4c/f78 0 2024-04-25T15:49:22.483 INFO:tasks.workunit.client.0.smithi110.stdout:3/368: fsync d0/d7/d40/f43 0 2024-04-25T15:49:22.490 INFO:tasks.workunit.client.0.smithi110.stdout:3/369: mkdir d0/d7/d7d 0 2024-04-25T15:49:22.492 INFO:tasks.workunit.client.0.smithi110.stdout:3/370: truncate d0/d7/f6c 1008573 0 2024-04-25T15:49:22.493 INFO:tasks.workunit.client.0.smithi110.stdout:2/223: creat de/f38 x:0 0 0 2024-04-25T15:49:22.493 INFO:tasks.workunit.client.0.smithi110.stdout:3/371: fdatasync d0/d7/f69 0 2024-04-25T15:49:22.495 INFO:tasks.workunit.client.0.smithi110.stdout:8/261: rmdir d14/d42 0 2024-04-25T15:49:22.498 INFO:tasks.workunit.client.0.smithi110.stdout:8/262: mknod d14/d4c/c4e 0 2024-04-25T15:49:22.498 INFO:tasks.workunit.client.0.smithi110.stdout:3/372: fdatasync d0/f2c 0 2024-04-25T15:49:22.499 INFO:tasks.workunit.client.0.smithi110.stdout:8/263: write d14/d32/f3e [699127,116828] 0 2024-04-25T15:49:22.501 INFO:tasks.workunit.client.0.smithi110.stdout:8/264: dread f6 [0,4194304] 0 2024-04-25T15:49:22.501 INFO:tasks.workunit.client.0.smithi110.stdout:4/316: dread d13/d1c/f20 [0,4194304] 0 2024-04-25T15:49:22.503 INFO:tasks.workunit.client.0.smithi110.stdout:3/373: creat d0/d7/d40/f7e x:0 0 0 2024-04-25T15:49:22.505 INFO:tasks.workunit.client.0.smithi110.stdout:2/224: getdents de/d30 0 2024-04-25T15:49:22.506 INFO:tasks.workunit.client.0.smithi110.stdout:8/265: symlink d14/d21/d23/l4f 0 2024-04-25T15:49:22.508 INFO:tasks.workunit.client.0.smithi110.stdout:4/317: rename d13/d24/c50 to d13/d24/d2d/d55/c5d 0 2024-04-25T15:49:22.514 INFO:tasks.workunit.client.0.smithi110.stdout:4/318: mkdir d13/d24/d2d/d5e 0 2024-04-25T15:49:22.515 INFO:tasks.workunit.client.0.smithi110.stdout:3/374: link d0/d7/d1c/d50/l6e d0/d7/d40/l7f 0 2024-04-25T15:49:22.515 INFO:tasks.workunit.client.0.smithi110.stdout:8/266: dwrite d14/d21/f27 [0,4194304] 0 2024-04-25T15:49:22.516 INFO:tasks.workunit.client.0.smithi110.stdout:3/375: stat d0/d7/d40/f7e 0 2024-04-25T15:49:22.518 INFO:tasks.workunit.client.0.smithi110.stdout:8/267: creat d14/d4c/f50 x:0 0 0 2024-04-25T15:49:22.526 INFO:tasks.workunit.client.0.smithi110.stdout:3/376: dwrite d0/d16/f1f [0,4194304] 0 2024-04-25T15:49:22.528 INFO:tasks.workunit.client.0.smithi110.stdout:3/377: truncate d0/d7/d1c/d50/f6d 572341 0 2024-04-25T15:49:22.531 INFO:tasks.workunit.client.0.smithi110.stdout:3/378: symlink d0/d7/d1c/d50/l80 0 2024-04-25T15:49:22.535 INFO:tasks.workunit.client.0.smithi110.stdout:3/379: creat d0/d7/d7d/f81 x:0 0 0 2024-04-25T15:49:22.536 INFO:tasks.workunit.client.0.smithi110.stdout:3/380: chown d0/d7/d1c/d50/f6d 17675 1 2024-04-25T15:49:22.541 INFO:tasks.workunit.client.0.smithi110.stdout:4/319: truncate d13/d1c/f20 2445094 0 2024-04-25T15:49:22.541 INFO:tasks.workunit.client.0.smithi110.stdout:8/268: write f5 [5014531,26092] 0 2024-04-25T15:49:22.544 INFO:tasks.workunit.client.0.smithi110.stdout:8/269: symlink d14/d4c/l51 0 2024-04-25T15:49:22.544 INFO:tasks.workunit.client.0.smithi110.stdout:4/320: getdents d13/d24/d2d/d5e 0 2024-04-25T15:49:22.546 INFO:tasks.workunit.client.0.smithi110.stdout:8/270: symlink d14/d32/d36/l52 0 2024-04-25T15:49:22.548 INFO:tasks.workunit.client.0.smithi110.stdout:4/321: symlink d13/d24/l5f 0 2024-04-25T15:49:22.549 INFO:tasks.workunit.client.0.smithi110.stdout:8/271: symlink d14/d4c/l53 0 2024-04-25T15:49:22.551 INFO:tasks.workunit.client.0.smithi110.stdout:8/272: truncate d14/f2d 437459 0 2024-04-25T15:49:22.552 INFO:tasks.workunit.client.0.smithi110.stdout:8/273: read d14/f1c [3844639,105944] 0 2024-04-25T15:49:22.552 INFO:tasks.workunit.client.0.smithi110.stdout:4/322: creat d13/d24/d2d/d44/f60 x:0 0 0 2024-04-25T15:49:22.553 INFO:tasks.workunit.client.0.smithi110.stdout:8/274: fdatasync d14/f2d 0 2024-04-25T15:49:22.553 INFO:tasks.workunit.client.0.smithi110.stdout:3/381: dread d0/d2e/f3b [0,4194304] 0 2024-04-25T15:49:22.556 INFO:tasks.workunit.client.0.smithi110.stdout:4/323: mknod d13/d24/c61 0 2024-04-25T15:49:22.562 INFO:tasks.workunit.client.0.smithi110.stdout:8/275: getdents d14 0 2024-04-25T15:49:22.563 INFO:tasks.workunit.client.0.smithi110.stdout:8/276: chown d14/d21/d26 0 1 2024-04-25T15:49:22.563 INFO:tasks.workunit.client.0.smithi110.stdout:4/324: creat d13/d24/d2d/f62 x:0 0 0 2024-04-25T15:49:22.565 INFO:tasks.workunit.client.0.smithi110.stdout:3/382: dread d0/d7/d1c/f5f [0,4194304] 0 2024-04-25T15:49:22.566 INFO:tasks.workunit.client.0.smithi110.stdout:8/277: truncate f6 704199 0 2024-04-25T15:49:22.570 INFO:tasks.workunit.client.0.smithi110.stdout:4/325: dread d13/d24/d26/f3b [0,4194304] 0 2024-04-25T15:49:22.570 INFO:tasks.workunit.client.0.smithi110.stdout:8/278: mknod d14/d4c/c54 0 2024-04-25T15:49:22.570 INFO:tasks.workunit.client.0.smithi110.stdout:4/326: write fa [5224900,93616] 0 2024-04-25T15:49:22.574 INFO:tasks.workunit.client.0.smithi110.stdout:4/327: dread d13/d24/f2a [0,4194304] 0 2024-04-25T15:49:22.575 INFO:tasks.workunit.client.0.smithi110.stdout:8/279: mkdir d14/d4c/d55 0 2024-04-25T15:49:22.582 INFO:tasks.workunit.client.0.smithi110.stdout:8/280: dwrite d14/d21/d26/f30 [0,4194304] 0 2024-04-25T15:49:22.583 INFO:tasks.workunit.client.0.smithi110.stdout:4/328: rename d13/d1c/c4a to d13/c63 0 2024-04-25T15:49:22.584 INFO:tasks.workunit.client.0.smithi110.stdout:8/281: symlink d14/l56 0 2024-04-25T15:49:22.587 INFO:tasks.workunit.client.0.smithi110.stdout:3/383: write d0/f2c [4394881,1871] 0 2024-04-25T15:49:22.596 INFO:tasks.workunit.client.0.smithi110.stdout:8/282: dwrite ff [8388608,4194304] 0 2024-04-25T15:49:22.597 INFO:tasks.workunit.client.0.smithi110.stdout:3/384: dwrite d0/d16/f1f [0,4194304] 0 2024-04-25T15:49:22.602 INFO:tasks.workunit.client.0.smithi110.stdout:3/385: symlink d0/d16/l82 0 2024-04-25T15:49:22.606 INFO:tasks.workunit.client.0.smithi110.stdout:3/386: symlink d0/d7/d40/l83 0 2024-04-25T15:49:22.609 INFO:tasks.workunit.client.0.smithi110.stdout:3/387: truncate d0/f70 768873 0 2024-04-25T15:49:22.612 INFO:tasks.workunit.client.0.smithi110.stdout:3/388: unlink d0/d7/d1c/d2f/d4c/f6a 0 2024-04-25T15:49:22.618 INFO:tasks.workunit.client.0.smithi110.stdout:3/389: unlink d0/d7/d1c/d50/l6e 0 2024-04-25T15:49:22.628 INFO:tasks.workunit.client.0.smithi110.stdout:8/283: truncate d14/d21/d26/f30 3373903 0 2024-04-25T15:49:22.628 INFO:tasks.workunit.client.0.smithi110.stdout:8/284: readlink l8 0 2024-04-25T15:49:22.633 INFO:tasks.workunit.client.0.smithi110.stdout:8/285: rmdir d14/d4c 39 2024-04-25T15:49:22.635 INFO:tasks.workunit.client.0.smithi110.stdout:3/390: dwrite d0/d2e/f30 [4194304,4194304] 0 2024-04-25T15:49:22.640 INFO:tasks.workunit.client.0.smithi110.stdout:3/391: dread - d0/d7/d1c/d2f/d33/f44 zero size 2024-04-25T15:49:22.658 INFO:tasks.workunit.client.0.smithi110.stdout:9/201: sync 2024-04-25T15:49:22.658 INFO:tasks.workunit.client.0.smithi110.stdout:5/164: sync 2024-04-25T15:49:22.663 INFO:tasks.workunit.client.0.smithi110.stdout:8/286: dwrite d14/d4c/f50 [0,4194304] 0 2024-04-25T15:49:22.665 INFO:tasks.workunit.client.0.smithi110.stdout:9/202: mkdir d9/db/df/d2c/d3c 0 2024-04-25T15:49:22.669 INFO:tasks.workunit.client.0.smithi110.stdout:8/287: creat d14/d4c/f57 x:0 0 0 2024-04-25T15:49:22.671 INFO:tasks.workunit.client.0.smithi110.stdout:9/203: dwrite d9/d2d/f30 [0,4194304] 0 2024-04-25T15:49:22.672 INFO:tasks.workunit.client.0.smithi110.stdout:9/204: readlink d9/d13/l18 0 2024-04-25T15:49:22.678 INFO:tasks.workunit.client.0.smithi110.stdout:8/288: creat d14/d21/f58 x:0 0 0 2024-04-25T15:49:22.680 INFO:tasks.workunit.client.0.smithi110.stdout:9/205: symlink d9/l3d 0 2024-04-25T15:49:22.682 INFO:tasks.workunit.client.0.smithi110.stdout:8/289: unlink d14/d21/f58 0 2024-04-25T15:49:22.685 INFO:tasks.workunit.client.0.smithi110.stdout:9/206: rmdir d9/d13 39 2024-04-25T15:49:22.688 INFO:tasks.workunit.client.0.smithi110.stdout:5/165: dwrite d4/d2c/f19 [0,4194304] 0 2024-04-25T15:49:22.690 INFO:tasks.workunit.client.0.smithi110.stdout:8/290: rename d14/d21/d23/c4b to d14/d21/c59 0 2024-04-25T15:49:22.696 INFO:tasks.workunit.client.0.smithi110.stdout:9/207: symlink d9/db/df/d2c/d3c/l3e 0 2024-04-25T15:49:22.701 INFO:tasks.workunit.client.0.smithi110.stdout:9/208: truncate d9/d13/f27 842797 0 2024-04-25T15:49:22.702 INFO:tasks.workunit.client.0.smithi110.stdout:9/209: chown d9/db/df/d25/l32 6069 1 2024-04-25T15:49:22.702 INFO:tasks.workunit.client.0.smithi110.stdout:9/210: write d9/db/df/d2c/f36 [5691777,120062] 0 2024-04-25T15:49:22.703 INFO:tasks.workunit.client.0.smithi110.stdout:9/211: chown d9/db/df/d25 0 1 2024-04-25T15:49:22.706 INFO:tasks.workunit.client.0.smithi110.stdout:9/212: creat d9/d13/f3f x:0 0 0 2024-04-25T15:49:22.709 INFO:tasks.workunit.client.0.smithi110.stdout:9/213: symlink d9/db/l40 0 2024-04-25T15:49:22.724 INFO:tasks.workunit.client.0.smithi110.stdout:5/166: write d4/d1d/f34 [1496664,85622] 0 2024-04-25T15:49:22.725 INFO:tasks.workunit.client.0.smithi110.stdout:5/167: readlink d4/lc 0 2024-04-25T15:49:22.725 INFO:tasks.workunit.client.0.smithi110.stdout:5/168: stat d4/d2c/f27 0 2024-04-25T15:49:22.726 INFO:tasks.workunit.client.0.smithi110.stdout:5/169: read - d4/d1d/f38 zero size 2024-04-25T15:49:22.726 INFO:tasks.workunit.client.0.smithi110.stdout:8/291: write f3 [1075146,85204] 0 2024-04-25T15:49:22.727 INFO:tasks.workunit.client.0.smithi110.stdout:8/292: chown f3 40 1 2024-04-25T15:49:22.729 INFO:tasks.workunit.client.0.smithi110.stdout:5/170: rmdir d4/d2c/d2a 39 2024-04-25T15:49:22.730 INFO:tasks.workunit.client.0.smithi110.stdout:8/293: fsync d14/d21/d23/f24 0 2024-04-25T15:49:22.735 INFO:tasks.workunit.client.0.smithi110.stdout:8/294: creat d14/d21/f5a x:0 0 0 2024-04-25T15:49:22.740 INFO:tasks.workunit.client.0.smithi110.stdout:8/295: rename d14/d21/d26/l2c to d14/d21/l5b 0 2024-04-25T15:49:22.740 INFO:tasks.workunit.client.0.smithi110.stdout:9/214: truncate d9/db/df/f33 1656938 0 2024-04-25T15:49:22.742 INFO:tasks.workunit.client.0.smithi110.stdout:5/171: dread d4/d1d/f34 [0,4194304] 0 2024-04-25T15:49:22.745 INFO:tasks.workunit.client.0.smithi110.stdout:5/172: dread d4/f6 [0,4194304] 0 2024-04-25T15:49:22.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/296: creat d14/f5c x:0 0 0 2024-04-25T15:49:22.753 INFO:tasks.workunit.client.0.smithi110.stdout:5/173: rename d4/f16 to d4/d1d/d1f/d35/f39 0 2024-04-25T15:49:22.755 INFO:tasks.workunit.client.0.smithi110.stdout:8/297: symlink d14/d4c/d55/l5d 0 2024-04-25T15:49:22.756 INFO:tasks.workunit.client.0.smithi110.stdout:8/298: dread - d14/d21/d26/f4a zero size 2024-04-25T15:49:22.758 INFO:tasks.workunit.client.0.smithi110.stdout:5/174: symlink d4/d15/l3a 0 2024-04-25T15:49:22.759 INFO:tasks.workunit.client.0.smithi110.stdout:5/175: dread - d4/f22 zero size 2024-04-25T15:49:22.762 INFO:tasks.workunit.client.0.smithi110.stdout:5/176: mknod d4/d15/c3b 0 2024-04-25T15:49:22.765 INFO:tasks.workunit.client.0.smithi110.stdout:5/177: dread d4/d15/f1a [0,4194304] 0 2024-04-25T15:49:22.766 INFO:tasks.workunit.client.0.smithi110.stdout:5/178: write d4/fd [2619775,33382] 0 2024-04-25T15:49:22.767 INFO:tasks.workunit.client.0.smithi110.stdout:8/299: dread d14/d21/d23/f25 [0,4194304] 0 2024-04-25T15:49:22.771 INFO:tasks.workunit.client.0.smithi110.stdout:5/179: fsync d4/f2f 0 2024-04-25T15:49:22.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/180: stat d4/d1d/f38 0 2024-04-25T15:49:22.776 INFO:tasks.workunit.client.0.smithi110.stdout:5/181: write d4/d15/f1a [1171249,6560] 0 2024-04-25T15:49:22.778 INFO:tasks.workunit.client.0.smithi110.stdout:8/300: chown d14/d32/c3d 9 1 2024-04-25T15:49:22.785 INFO:tasks.workunit.client.0.smithi110.stdout:8/301: rename d14/f17 to d14/d32/d36/d48/f5e 0 2024-04-25T15:49:22.789 INFO:tasks.workunit.client.0.smithi110.stdout:8/302: dread d14/d32/f3f [0,4194304] 0 2024-04-25T15:49:22.791 INFO:tasks.workunit.client.0.smithi110.stdout:8/303: symlink d14/d32/d36/l5f 0 2024-04-25T15:49:22.796 INFO:tasks.workunit.client.0.smithi110.stdout:9/215: dwrite d9/db/df/f33 [0,4194304] 0 2024-04-25T15:49:22.800 INFO:tasks.workunit.client.0.smithi110.stdout:9/216: rename d9/l2a to d9/db/df/l41 0 2024-04-25T15:49:22.802 INFO:tasks.workunit.client.0.smithi110.stdout:5/182: truncate d4/d15/f1a 455550 0 2024-04-25T15:49:22.804 INFO:tasks.workunit.client.0.smithi110.stdout:5/183: creat d4/d15/f3c x:0 0 0 2024-04-25T15:49:22.808 INFO:tasks.workunit.client.0.smithi110.stdout:5/184: mkdir d4/d1d/d1f/d35/d3d 0 2024-04-25T15:49:22.808 INFO:tasks.workunit.client.0.smithi110.stdout:9/217: link d9/db/df/c31 d9/db/df/d25/c42 0 2024-04-25T15:49:22.808 INFO:tasks.workunit.client.0.smithi110.stdout:5/185: read d4/f6 [300809,3779] 0 2024-04-25T15:49:22.810 INFO:tasks.workunit.client.0.smithi110.stdout:9/218: write d9/d2d/f38 [397275,38982] 0 2024-04-25T15:49:22.810 INFO:tasks.workunit.client.0.smithi110.stdout:5/186: truncate d4/d2c/f27 1238308 0 2024-04-25T15:49:22.812 INFO:tasks.workunit.client.0.smithi110.stdout:8/304: truncate fb 4513021 0 2024-04-25T15:49:22.815 INFO:tasks.workunit.client.0.smithi110.stdout:8/305: creat d14/d21/d23/f60 x:0 0 0 2024-04-25T15:49:22.816 INFO:tasks.workunit.client.0.smithi110.stdout:9/219: rename d9/db/df/c31 to d9/db/c43 0 2024-04-25T15:49:22.817 INFO:tasks.workunit.client.0.smithi110.stdout:8/306: creat d14/d32/f61 x:0 0 0 2024-04-25T15:49:22.818 INFO:tasks.workunit.client.0.smithi110.stdout:9/220: dread f6 [8388608,4194304] 0 2024-04-25T15:49:22.885 INFO:tasks.workunit.client.0.smithi110.stdout:0/264: sync 2024-04-25T15:49:22.889 INFO:tasks.workunit.client.0.smithi110.stdout:8/307: chown d14/d21/d23/l49 154 1 2024-04-25T15:49:22.892 INFO:tasks.workunit.client.0.smithi110.stdout:0/265: creat d1/d23/d30/d33/d35/f6b x:0 0 0 2024-04-25T15:49:22.897 INFO:tasks.workunit.client.0.smithi110.stdout:8/308: unlink d14/d21/f5a 0 2024-04-25T15:49:22.899 INFO:tasks.workunit.client.0.smithi110.stdout:9/221: mkdir d9/d44 0 2024-04-25T15:49:22.904 INFO:tasks.workunit.client.0.smithi110.stdout:8/309: creat d14/d21/f62 x:0 0 0 2024-04-25T15:49:22.905 INFO:tasks.workunit.client.0.smithi110.stdout:9/222: symlink d9/db/df/d25/l45 0 2024-04-25T15:49:22.905 INFO:tasks.workunit.client.0.smithi110.stdout:5/187: write d4/d15/f1a [242494,23147] 0 2024-04-25T15:49:22.907 INFO:tasks.workunit.client.0.smithi110.stdout:0/266: rmdir d1/d23/d30 39 2024-04-25T15:49:22.909 INFO:tasks.workunit.client.0.smithi110.stdout:8/310: symlink d14/l63 0 2024-04-25T15:49:22.914 INFO:tasks.workunit.client.0.smithi110.stdout:5/188: mknod d4/d1d/d1f/d35/d3d/c3e 0 2024-04-25T15:49:22.915 INFO:tasks.workunit.client.0.smithi110.stdout:0/267: mkdir d1/d6c 0 2024-04-25T15:49:22.915 INFO:tasks.workunit.client.0.smithi110.stdout:5/189: write d4/d15/f1a [1001556,104080] 0 2024-04-25T15:49:22.917 INFO:tasks.workunit.client.0.smithi110.stdout:0/268: write d1/d4c/f5d [361616,109701] 0 2024-04-25T15:49:22.919 INFO:tasks.workunit.client.0.smithi110.stdout:5/190: mknod d4/d2c/d2a/c3f 0 2024-04-25T15:49:22.919 INFO:tasks.workunit.client.0.smithi110.stdout:9/223: link d9/d13/c35 d9/c46 0 2024-04-25T15:49:22.923 INFO:tasks.workunit.client.0.smithi110.stdout:0/269: creat d1/d47/d54/f6d x:0 0 0 2024-04-25T15:49:22.927 INFO:tasks.workunit.client.0.smithi110.stdout:8/311: getdents d14 0 2024-04-25T15:49:22.931 INFO:tasks.workunit.client.0.smithi110.stdout:0/270: write d1/d23/d30/d33/d35/f36 [648752,130684] 0 2024-04-25T15:49:22.931 INFO:tasks.workunit.client.0.smithi110.stdout:8/312: write d14/d21/f27 [1797279,96903] 0 2024-04-25T15:49:22.933 INFO:tasks.workunit.client.0.smithi110.stdout:5/191: dwrite d4/f10 [0,4194304] 0 2024-04-25T15:49:22.936 INFO:tasks.workunit.client.0.smithi110.stdout:0/271: write d1/d23/d30/d33/d35/f6b [884445,73590] 0 2024-04-25T15:49:22.938 INFO:tasks.workunit.client.0.smithi110.stdout:9/224: dread d9/db/df/d2c/f36 [4194304,4194304] 0 2024-04-25T15:49:22.940 INFO:tasks.workunit.client.0.smithi110.stdout:0/272: truncate d1/d4c/f5d 787040 0 2024-04-25T15:49:22.941 INFO:tasks.workunit.client.0.smithi110.stdout:8/313: dread d14/f40 [0,4194304] 0 2024-04-25T15:49:22.942 INFO:tasks.workunit.client.0.smithi110.stdout:0/273: dread - d1/d20/f64 zero size 2024-04-25T15:49:22.943 INFO:tasks.workunit.client.0.smithi110.stdout:9/225: fsync f8 0 2024-04-25T15:49:22.944 INFO:tasks.workunit.client.0.smithi110.stdout:5/192: dwrite d4/f10 [0,4194304] 0 2024-04-25T15:49:22.944 INFO:tasks.workunit.client.0.smithi110.stdout:0/274: chown d1/d23/d30 1 1 2024-04-25T15:49:22.947 INFO:tasks.workunit.client.0.smithi110.stdout:8/314: creat d14/d32/d36/f64 x:0 0 0 2024-04-25T15:49:22.950 INFO:tasks.workunit.client.0.smithi110.stdout:5/193: symlink d4/d1d/d1f/d35/d3d/l40 0 2024-04-25T15:49:22.951 INFO:tasks.workunit.client.0.smithi110.stdout:9/226: mknod d9/c47 0 2024-04-25T15:49:22.952 INFO:tasks.workunit.client.0.smithi110.stdout:8/315: symlink d14/d21/d23/l65 0 2024-04-25T15:49:22.953 INFO:tasks.workunit.client.0.smithi110.stdout:0/275: dread d1/d23/f42 [0,4194304] 0 2024-04-25T15:49:22.954 INFO:tasks.workunit.client.0.smithi110.stdout:9/227: fsync d9/f24 0 2024-04-25T15:49:22.958 INFO:tasks.workunit.client.0.smithi110.stdout:5/194: dwrite d4/f22 [0,4194304] 0 2024-04-25T15:49:22.958 INFO:tasks.workunit.client.0.smithi110.stdout:9/228: fsync d9/db/df/d2c/f3a 0 2024-04-25T15:49:22.959 INFO:tasks.workunit.client.0.smithi110.stdout:9/229: dread - d9/f34 zero size 2024-04-25T15:49:22.963 INFO:tasks.workunit.client.0.smithi110.stdout:0/276: getdents d1 0 2024-04-25T15:49:22.964 INFO:tasks.workunit.client.0.smithi110.stdout:5/195: dread d4/f10 [0,4194304] 0 2024-04-25T15:49:22.965 INFO:tasks.workunit.client.0.smithi110.stdout:9/230: chown d9/c46 2 1 2024-04-25T15:49:22.966 INFO:tasks.workunit.client.0.smithi110.stdout:7/258: sync 2024-04-25T15:49:22.967 INFO:tasks.workunit.client.0.smithi110.stdout:0/277: rename d1/l18 to d1/d3e/l6e 0 2024-04-25T15:49:22.967 INFO:tasks.workunit.client.0.smithi110.stdout:7/259: stat d1b/d29/f57 0 2024-04-25T15:49:22.970 INFO:tasks.workunit.client.0.smithi110.stdout:9/231: mknod d9/db/df/c48 0 2024-04-25T15:49:22.971 INFO:tasks.workunit.client.0.smithi110.stdout:5/196: dwrite d4/d1d/f1e [4194304,4194304] 0 2024-04-25T15:49:22.972 INFO:tasks.workunit.client.0.smithi110.stdout:9/232: chown d9/f34 388 1 2024-04-25T15:49:22.973 INFO:tasks.workunit.client.0.smithi110.stdout:7/260: fdatasync d1b/d29/d46/f47 0 2024-04-25T15:49:22.974 INFO:tasks.workunit.client.0.smithi110.stdout:7/261: fdatasync d1b/d29/d46/f47 0 2024-04-25T15:49:22.975 INFO:tasks.workunit.client.0.smithi110.stdout:7/262: read d1b/d29/f37 [4084720,128416] 0 2024-04-25T15:49:22.976 INFO:tasks.workunit.client.0.smithi110.stdout:5/197: symlink d4/d1d/d1f/l41 0 2024-04-25T15:49:22.980 INFO:tasks.workunit.client.0.smithi110.stdout:8/316: truncate d14/d21/f2a 4103515 0 2024-04-25T15:49:22.982 INFO:tasks.workunit.client.0.smithi110.stdout:5/198: creat d4/d1d/d1f/f42 x:0 0 0 2024-04-25T15:49:22.987 INFO:tasks.workunit.client.0.smithi110.stdout:5/199: mknod d4/d2c/d2a/c43 0 2024-04-25T15:49:22.988 INFO:tasks.workunit.client.0.smithi110.stdout:7/263: link l7 d1b/d29/l58 0 2024-04-25T15:49:22.990 INFO:tasks.workunit.client.0.smithi110.stdout:8/317: link d14/d32/d36/d48/f5e d14/d32/d36/d48/f66 0 2024-04-25T15:49:22.996 INFO:tasks.workunit.client.0.smithi110.stdout:7/264: creat d1b/d29/d38/d55/f59 x:0 0 0 2024-04-25T15:49:22.999 INFO:tasks.workunit.client.0.smithi110.stdout:5/200: creat d4/f44 x:0 0 0 2024-04-25T15:49:23.001 INFO:tasks.workunit.client.0.smithi110.stdout:8/318: dwrite d14/d32/d36/d48/f5e [0,4194304] 0 2024-04-25T15:49:23.001 INFO:tasks.workunit.client.0.smithi110.stdout:0/278: dread d1/d23/d30/d33/d35/f11 [0,4194304] 0 2024-04-25T15:49:23.003 INFO:tasks.workunit.client.0.smithi110.stdout:8/319: fdatasync d14/f22 0 2024-04-25T15:49:23.011 INFO:tasks.workunit.client.0.smithi110.stdout:8/320: dwrite d14/f5c [0,4194304] 0 2024-04-25T15:49:23.014 INFO:tasks.workunit.client.0.smithi110.stdout:7/265: getdents d1b/d29/d42 0 2024-04-25T15:49:23.017 INFO:tasks.workunit.client.0.smithi110.stdout:5/201: dread d4/f8 [0,4194304] 0 2024-04-25T15:49:23.017 INFO:tasks.workunit.client.0.smithi110.stdout:8/321: creat d14/d21/f67 x:0 0 0 2024-04-25T15:49:23.022 INFO:tasks.workunit.client.0.smithi110.stdout:8/322: symlink d14/d32/d36/l68 0 2024-04-25T15:49:23.027 INFO:tasks.workunit.client.0.smithi110.stdout:5/202: rename d4/d1d/d1f/d35/d3d/c3e to d4/d15/c45 0 2024-04-25T15:49:23.028 INFO:tasks.workunit.client.0.smithi110.stdout:5/203: chown d4/d15/f3c 10 1 2024-04-25T15:49:23.029 INFO:tasks.workunit.client.0.smithi110.stdout:8/323: dwrite d14/f16 [0,4194304] 0 2024-04-25T15:49:23.031 INFO:tasks.workunit.client.0.smithi110.stdout:8/324: stat d14/d21/d26/c3a 0 2024-04-25T15:49:23.032 INFO:tasks.workunit.client.0.smithi110.stdout:0/279: link d1/d20/l21 d1/d23/d30/d33/l6f 0 2024-04-25T15:49:23.033 INFO:tasks.workunit.client.0.smithi110.stdout:7/266: link d1b/d29/l58 d1b/d29/l5a 0 2024-04-25T15:49:23.035 INFO:tasks.workunit.client.0.smithi110.stdout:0/280: symlink d1/d20/l70 0 2024-04-25T15:49:23.037 INFO:tasks.workunit.client.0.smithi110.stdout:5/204: link d4/f10 d4/d2c/d2a/f46 0 2024-04-25T15:49:23.042 INFO:tasks.workunit.client.0.smithi110.stdout:0/281: rename d1/d3e/f51 to d1/d47/d54/f71 0 2024-04-25T15:49:23.043 INFO:tasks.workunit.client.0.smithi110.stdout:5/205: dwrite d4/f25 [0,4194304] 0 2024-04-25T15:49:23.046 INFO:tasks.workunit.client.0.smithi110.stdout:7/267: dread d1b/d29/d46/f51 [0,4194304] 0 2024-04-25T15:49:23.046 INFO:tasks.workunit.client.0.smithi110.stdout:0/282: unlink d1/d20/f64 0 2024-04-25T15:49:23.047 INFO:tasks.workunit.client.0.smithi110.stdout:7/268: chown la 1037756 1 2024-04-25T15:49:23.047 INFO:tasks.workunit.client.0.smithi110.stdout:5/206: creat d4/d1d/d1f/f47 x:0 0 0 2024-04-25T15:49:23.048 INFO:tasks.workunit.client.0.smithi110.stdout:5/207: stat d4/d1d/d1f/d35/d3d 0 2024-04-25T15:49:23.049 INFO:tasks.workunit.client.0.smithi110.stdout:0/283: mkdir d1/d23/d30/d33/d5b/d72 0 2024-04-25T15:49:23.051 INFO:tasks.workunit.client.0.smithi110.stdout:5/208: rename d4/lf to d4/d15/l48 0 2024-04-25T15:49:23.055 INFO:tasks.workunit.client.0.smithi110.stdout:0/284: link d1/d47/d54/f71 d1/d20/f73 0 2024-04-25T15:49:23.055 INFO:tasks.workunit.client.0.smithi110.stdout:5/209: dread d4/f25 [0,4194304] 0 2024-04-25T15:49:23.059 INFO:tasks.workunit.client.0.smithi110.stdout:5/210: creat d4/d1d/d1f/f49 x:0 0 0 2024-04-25T15:49:23.062 INFO:tasks.workunit.client.0.smithi110.stdout:5/211: dread d4/f2f [0,4194304] 0 2024-04-25T15:49:23.064 INFO:tasks.workunit.client.0.smithi110.stdout:5/212: mknod d4/d1d/d1f/c4a 0 2024-04-25T15:49:23.064 INFO:tasks.workunit.client.0.smithi110.stdout:5/213: fdatasync d4/f22 0 2024-04-25T15:49:23.067 INFO:tasks.workunit.client.0.smithi110.stdout:0/285: rmdir d1/d3e 39 2024-04-25T15:49:23.072 INFO:tasks.workunit.client.0.smithi110.stdout:0/286: dread - d1/d47/d54/f71 zero size 2024-04-25T15:49:23.073 INFO:tasks.workunit.client.0.smithi110.stdout:5/214: link d4/d15/c3b d4/d1d/d1f/d35/c4b 0 2024-04-25T15:49:23.074 INFO:tasks.workunit.client.0.smithi110.stdout:5/215: truncate d4/d1d/d1f/f42 3554 0 2024-04-25T15:49:23.078 INFO:tasks.workunit.client.0.smithi110.stdout:0/287: creat d1/d47/d54/f74 x:0 0 0 2024-04-25T15:49:23.081 INFO:tasks.workunit.client.0.smithi110.stdout:5/216: rename d4/f2f to d4/d1d/d1f/d35/f4c 0 2024-04-25T15:49:23.083 INFO:tasks.workunit.client.0.smithi110.stdout:0/288: dread - d1/f4d zero size 2024-04-25T15:49:23.085 INFO:tasks.workunit.client.0.smithi110.stdout:5/217: creat d4/d1d/d1f/d35/d3d/f4d x:0 0 0 2024-04-25T15:49:23.085 INFO:tasks.workunit.client.0.smithi110.stdout:5/218: fdatasync d4/f44 0 2024-04-25T15:49:23.092 INFO:tasks.workunit.client.0.smithi110.stdout:0/289: dwrite d1/f4e [0,4194304] 0 2024-04-25T15:49:23.093 INFO:tasks.workunit.client.0.smithi110.stdout:5/219: link d4/d2c/d2a/l31 d4/d2c/l4e 0 2024-04-25T15:49:23.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/290: mknod d1/d20/c75 0 2024-04-25T15:49:23.106 INFO:tasks.workunit.client.0.smithi110.stdout:0/291: link d1/d23/d30/d33/d35/c1f d1/d23/d30/c76 0 2024-04-25T15:49:23.108 INFO:tasks.workunit.client.0.smithi110.stdout:5/220: dwrite d4/f8 [0,4194304] 0 2024-04-25T15:49:23.108 INFO:tasks.workunit.client.0.smithi110.stdout:0/292: dread - d1/d47/f61 zero size 2024-04-25T15:49:23.110 INFO:tasks.workunit.client.0.smithi110.stdout:5/221: readlink d4/d2c/l4e 0 2024-04-25T15:49:23.116 INFO:tasks.workunit.client.0.smithi110.stdout:5/222: unlink d4/d15/c3b 0 2024-04-25T15:49:23.116 INFO:tasks.workunit.client.0.smithi110.stdout:0/293: getdents d1/d23/d30/d33/d5b 0 2024-04-25T15:49:23.117 INFO:tasks.workunit.client.0.smithi110.stdout:0/294: fdatasync d1/d47/d54/f74 0 2024-04-25T15:49:23.121 INFO:tasks.workunit.client.0.smithi110.stdout:5/223: creat d4/d2c/f4f x:0 0 0 2024-04-25T15:49:23.122 INFO:tasks.workunit.client.0.smithi110.stdout:0/295: mkdir d1/d23/d30/d33/d5b/d72/d77 0 2024-04-25T15:49:23.122 INFO:tasks.workunit.client.0.smithi110.stdout:5/224: readlink d4/d1d/d1f/l33 0 2024-04-25T15:49:23.123 INFO:tasks.workunit.client.0.smithi110.stdout:0/296: dread - d1/d20/f73 zero size 2024-04-25T15:49:23.127 INFO:tasks.workunit.client.0.smithi110.stdout:0/297: mkdir d1/d23/d30/d33/d5b/d78 0 2024-04-25T15:49:23.130 INFO:tasks.workunit.client.0.smithi110.stdout:5/225: dwrite d4/d1d/d1f/d35/d3d/f4d [0,4194304] 0 2024-04-25T15:49:23.130 INFO:tasks.workunit.client.0.smithi110.stdout:5/226: stat d4/f6 0 2024-04-25T15:49:23.131 INFO:tasks.workunit.client.0.smithi110.stdout:5/227: readlink d4/d15/l18 0 2024-04-25T15:49:23.132 INFO:tasks.workunit.client.0.smithi110.stdout:5/228: write d4/fd [3238338,60518] 0 2024-04-25T15:49:23.132 INFO:tasks.workunit.client.0.smithi110.stdout:0/298: dread d1/f4e [0,4194304] 0 2024-04-25T15:49:23.136 INFO:tasks.workunit.client.0.smithi110.stdout:2/225: sync 2024-04-25T15:49:23.136 INFO:tasks.workunit.client.0.smithi110.stdout:1/373: sync 2024-04-25T15:49:23.136 INFO:tasks.workunit.client.0.smithi110.stdout:6/256: sync 2024-04-25T15:49:23.137 INFO:tasks.workunit.client.0.smithi110.stdout:0/299: write d1/f19 [2106768,16999] 0 2024-04-25T15:49:23.137 INFO:tasks.workunit.client.0.smithi110.stdout:5/229: symlink d4/d1d/l50 0 2024-04-25T15:49:23.138 INFO:tasks.workunit.client.0.smithi110.stdout:6/257: dread - d1/db/d22/d33/d43/f60 zero size 2024-04-25T15:49:23.139 INFO:tasks.workunit.client.0.smithi110.stdout:5/230: write d4/f8 [2750936,76123] 0 2024-04-25T15:49:23.139 INFO:tasks.workunit.client.0.smithi110.stdout:6/258: dread - d1/db/d22/f57 zero size 2024-04-25T15:49:23.140 INFO:tasks.workunit.client.0.smithi110.stdout:1/374: mkdir d0/d75 0 2024-04-25T15:49:23.142 INFO:tasks.workunit.client.0.smithi110.stdout:2/226: write de/d30/f35 [3586242,52793] 0 2024-04-25T15:49:23.142 INFO:tasks.workunit.client.0.smithi110.stdout:1/375: write d0/d25/f41 [1026091,29482] 0 2024-04-25T15:49:23.146 INFO:tasks.workunit.client.0.smithi110.stdout:5/231: dwrite d4/d1d/d1f/f32 [0,4194304] 0 2024-04-25T15:49:23.178 INFO:tasks.workunit.client.0.smithi110.stdout:0/300: chown d1/f38 45763596 1 2024-04-25T15:49:23.183 INFO:tasks.workunit.client.0.smithi110.stdout:6/259: mknod d1/db/d22/d33/d43/c61 0 2024-04-25T15:49:23.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/376: readlink d0/d25/l46 0 2024-04-25T15:49:23.186 INFO:tasks.workunit.client.0.smithi110.stdout:2/227: unlink de/d30/f34 0 2024-04-25T15:49:23.187 INFO:tasks.workunit.client.0.smithi110.stdout:5/232: mkdir d4/d15/d51 0 2024-04-25T15:49:23.193 INFO:tasks.workunit.client.0.smithi110.stdout:2/228: creat de/d11/f39 x:0 0 0 2024-04-25T15:49:23.194 INFO:tasks.workunit.client.0.smithi110.stdout:2/229: write de/d30/f35 [5007954,6365] 0 2024-04-25T15:49:23.196 INFO:tasks.workunit.client.0.smithi110.stdout:0/301: mkdir d1/d3e/d79 0 2024-04-25T15:49:23.197 INFO:tasks.workunit.client.0.smithi110.stdout:6/260: dread d1/f1d [0,4194304] 0 2024-04-25T15:49:23.197 INFO:tasks.workunit.client.0.smithi110.stdout:1/377: mknod d0/d14/d5f/c76 0 2024-04-25T15:49:23.201 INFO:tasks.workunit.client.0.smithi110.stdout:2/230: creat de/d11/f3a x:0 0 0 2024-04-25T15:49:23.201 INFO:tasks.workunit.client.0.smithi110.stdout:0/302: mknod d1/d3e/c7a 0 2024-04-25T15:49:23.205 INFO:tasks.workunit.client.0.smithi110.stdout:2/231: dread de/f28 [0,4194304] 0 2024-04-25T15:49:23.205 INFO:tasks.workunit.client.0.smithi110.stdout:1/378: mknod d0/d25/d33/c77 0 2024-04-25T15:49:23.206 INFO:tasks.workunit.client.0.smithi110.stdout:1/379: write d0/d25/f3d [1232336,694] 0 2024-04-25T15:49:23.207 INFO:tasks.workunit.client.0.smithi110.stdout:1/380: fsync d0/d23/f3b 0 2024-04-25T15:49:23.211 INFO:tasks.workunit.client.0.smithi110.stdout:1/381: write d0/d14/d5f/f65 [642535,60796] 0 2024-04-25T15:49:23.212 INFO:tasks.workunit.client.0.smithi110.stdout:0/303: creat d1/d47/f7b x:0 0 0 2024-04-25T15:49:23.214 INFO:tasks.workunit.client.0.smithi110.stdout:0/304: write d1/d20/f6a [649823,65994] 0 2024-04-25T15:49:23.214 INFO:tasks.workunit.client.0.smithi110.stdout:5/233: truncate d4/f8 2399142 0 2024-04-25T15:49:23.215 INFO:tasks.workunit.client.0.smithi110.stdout:5/234: stat d4/d15 0 2024-04-25T15:49:23.220 INFO:tasks.workunit.client.0.smithi110.stdout:6/261: creat d1/db/f62 x:0 0 0 2024-04-25T15:49:23.220 INFO:tasks.workunit.client.0.smithi110.stdout:1/382: rename d0/d14 to d0/d14/d5f/d78 22 2024-04-25T15:49:23.221 INFO:tasks.workunit.client.0.smithi110.stdout:0/305: readlink d1/d3e/l6e 0 2024-04-25T15:49:23.222 INFO:tasks.workunit.client.0.smithi110.stdout:5/235: dwrite d4/d1d/f38 [0,4194304] 0 2024-04-25T15:49:23.224 INFO:tasks.workunit.client.0.smithi110.stdout:5/236: write d4/d2c/f19 [55004,60149] 0 2024-04-25T15:49:23.224 INFO:tasks.workunit.client.0.smithi110.stdout:6/262: readlink d1/d12/l5f 0 2024-04-25T15:49:23.226 INFO:tasks.workunit.client.0.smithi110.stdout:1/383: creat d0/d51/f79 x:0 0 0 2024-04-25T15:49:23.238 INFO:tasks.workunit.client.0.smithi110.stdout:2/232: link de/c26 de/c3b 0 2024-04-25T15:49:23.242 INFO:tasks.workunit.client.0.smithi110.stdout:1/384: link d0/d23/f13 d0/d14/f7a 0 2024-04-25T15:49:23.244 INFO:tasks.workunit.client.0.smithi110.stdout:2/233: rmdir de/d30 39 2024-04-25T15:49:23.245 INFO:tasks.workunit.client.0.smithi110.stdout:5/237: mkdir d4/d1d/d1f/d35/d3d/d52 0 2024-04-25T15:49:23.247 INFO:tasks.workunit.client.0.smithi110.stdout:0/306: dread d1/d23/d30/d33/d35/f3a [0,4194304] 0 2024-04-25T15:49:23.248 INFO:tasks.workunit.client.0.smithi110.stdout:1/385: dread d0/d25/f41 [0,4194304] 0 2024-04-25T15:49:23.252 INFO:tasks.workunit.client.0.smithi110.stdout:6/263: getdents d1/db/d22/d27 0 2024-04-25T15:49:23.254 INFO:tasks.workunit.client.0.smithi110.stdout:0/307: mkdir d1/d3e/d7c 0 2024-04-25T15:49:23.256 INFO:tasks.workunit.client.0.smithi110.stdout:2/234: rmdir de 39 2024-04-25T15:49:23.256 INFO:tasks.workunit.client.0.smithi110.stdout:1/386: dread d0/d14/f1d [0,4194304] 0 2024-04-25T15:49:23.256 INFO:tasks.workunit.client.0.smithi110.stdout:4/329: sync 2024-04-25T15:49:23.257 INFO:tasks.workunit.client.0.smithi110.stdout:6/264: mkdir d1/db/d22/d33/d63 0 2024-04-25T15:49:23.260 INFO:tasks.workunit.client.0.smithi110.stdout:0/308: write d1/d23/d30/d33/d35/f11 [432115,38186] 0 2024-04-25T15:49:23.263 INFO:tasks.workunit.client.0.smithi110.stdout:6/265: creat d1/dd/f64 x:0 0 0 2024-04-25T15:49:23.265 INFO:tasks.workunit.client.0.smithi110.stdout:4/330: dwrite d13/d24/d2d/d36/d51/f54 [0,4194304] 0 2024-04-25T15:49:23.265 INFO:tasks.workunit.client.0.smithi110.stdout:6/266: dread d1/f1d [0,4194304] 0 2024-04-25T15:49:23.267 INFO:tasks.workunit.client.0.smithi110.stdout:2/235: read de/d30/f33 [1427840,55675] 0 2024-04-25T15:49:23.267 INFO:tasks.workunit.client.0.smithi110.stdout:4/331: write d13/d24/d26/f49 [645877,43191] 0 2024-04-25T15:49:23.271 INFO:tasks.workunit.client.0.smithi110.stdout:0/309: rename d1/d23/d30/d33/d35/f11 to d1/d23/d30/d33/d5b/d78/f7d 0 2024-04-25T15:49:23.271 INFO:tasks.workunit.client.0.smithi110.stdout:0/310: readlink d1/d20/l39 0 2024-04-25T15:49:23.275 INFO:tasks.workunit.client.0.smithi110.stdout:6/267: mknod d1/db/d24/d32/c65 0 2024-04-25T15:49:23.276 INFO:tasks.workunit.client.0.smithi110.stdout:6/268: dread - d1/dd/d3c/f51 zero size 2024-04-25T15:49:23.276 INFO:tasks.workunit.client.0.smithi110.stdout:5/238: chown d4/f8 1014202 1 2024-04-25T15:49:23.278 INFO:tasks.workunit.client.0.smithi110.stdout:2/236: symlink de/d30/l3c 0 2024-04-25T15:49:23.284 INFO:tasks.workunit.client.0.smithi110.stdout:5/239: creat d4/d1d/d1f/d35/d3d/f53 x:0 0 0 2024-04-25T15:49:23.284 INFO:tasks.workunit.client.0.smithi110.stdout:5/240: chown d4/d2c/l12 14 1 2024-04-25T15:49:23.285 INFO:tasks.workunit.client.0.smithi110.stdout:5/241: write d4/fd [3624215,72010] 0 2024-04-25T15:49:23.286 INFO:tasks.workunit.client.0.smithi110.stdout:2/237: rename de/f38 to de/d30/f3d 0 2024-04-25T15:49:23.287 INFO:tasks.workunit.client.0.smithi110.stdout:2/238: chown de/d11/f19 0 1 2024-04-25T15:49:23.289 INFO:tasks.workunit.client.0.smithi110.stdout:5/242: mknod d4/d15/c54 0 2024-04-25T15:49:23.291 INFO:tasks.workunit.client.0.smithi110.stdout:0/311: symlink d1/d6c/l7e 0 2024-04-25T15:49:23.292 INFO:tasks.workunit.client.0.smithi110.stdout:5/243: write d4/d1d/d1f/f47 [754217,4419] 0 2024-04-25T15:49:23.292 INFO:tasks.workunit.client.0.smithi110.stdout:6/269: creat d1/db/d24/f66 x:0 0 0 2024-04-25T15:49:23.293 INFO:tasks.workunit.client.0.smithi110.stdout:4/332: getdents d13/d24/d2d/d36 0 2024-04-25T15:49:23.295 INFO:tasks.workunit.client.0.smithi110.stdout:2/239: dwrite de/d11/f3a [0,4194304] 0 2024-04-25T15:49:23.296 INFO:tasks.workunit.client.0.smithi110.stdout:6/270: dread - d1/db/d24/d2d/f59 zero size 2024-04-25T15:49:23.298 INFO:tasks.workunit.client.0.smithi110.stdout:6/271: write d1/db/d22/d27/f2b [184751,45807] 0 2024-04-25T15:49:23.302 INFO:tasks.workunit.client.0.smithi110.stdout:2/240: creat de/d30/f3e x:0 0 0 2024-04-25T15:49:23.304 INFO:tasks.workunit.client.0.smithi110.stdout:4/333: mkdir d13/d24/d2d/d44/d64 0 2024-04-25T15:49:23.306 INFO:tasks.workunit.client.0.smithi110.stdout:0/312: dwrite d1/d23/d30/f37 [4194304,4194304] 0 2024-04-25T15:49:23.313 INFO:tasks.workunit.client.0.smithi110.stdout:4/334: mkdir d13/d24/d2d/d44/d65 0 2024-04-25T15:49:23.317 INFO:tasks.workunit.client.0.smithi110.stdout:0/313: rename d1/d3e to d1/d6c/d7f 0 2024-04-25T15:49:23.319 INFO:tasks.workunit.client.0.smithi110.stdout:2/241: mknod de/c3f 0 2024-04-25T15:49:23.321 INFO:tasks.workunit.client.0.smithi110.stdout:9/233: sync 2024-04-25T15:49:23.321 INFO:tasks.workunit.client.0.smithi110.stdout:8/325: sync 2024-04-25T15:49:23.321 INFO:tasks.workunit.client.0.smithi110.stdout:7/269: sync 2024-04-25T15:49:23.322 INFO:tasks.workunit.client.0.smithi110.stdout:3/392: sync 2024-04-25T15:49:23.323 INFO:tasks.workunit.client.0.smithi110.stdout:5/244: truncate d4/fd 3277153 0 2024-04-25T15:49:23.323 INFO:tasks.workunit.client.0.smithi110.stdout:9/234: readlink d9/l3d 0 2024-04-25T15:49:23.325 INFO:tasks.workunit.client.0.smithi110.stdout:0/314: rmdir d1/d4c 39 2024-04-25T15:49:23.326 INFO:tasks.workunit.client.0.smithi110.stdout:8/326: dread d14/f2d [0,4194304] 0 2024-04-25T15:49:23.328 INFO:tasks.workunit.client.0.smithi110.stdout:7/270: truncate d1b/d29/d46/f52 629089 0 2024-04-25T15:49:23.330 INFO:tasks.workunit.client.0.smithi110.stdout:6/272: dwrite d1/dd/f21 [0,4194304] 0 2024-04-25T15:49:23.331 INFO:tasks.workunit.client.0.smithi110.stdout:7/271: write d1b/f4b [241414,117634] 0 2024-04-25T15:49:23.332 INFO:tasks.workunit.client.0.smithi110.stdout:4/335: dwrite d13/d24/d2d/f4f [0,4194304] 0 2024-04-25T15:49:23.340 INFO:tasks.workunit.client.0.smithi110.stdout:4/336: dwrite d13/d24/d26/f49 [0,4194304] 0 2024-04-25T15:49:23.364 INFO:tasks.workunit.client.0.smithi110.stdout:5/245: chown d4/d15/c45 85 1 2024-04-25T15:49:23.371 INFO:tasks.workunit.client.0.smithi110.stdout:2/242: rename de/d30 to de/d40 0 2024-04-25T15:49:23.372 INFO:tasks.workunit.client.0.smithi110.stdout:9/235: creat d9/db/df/d2c/d3c/f49 x:0 0 0 2024-04-25T15:49:23.374 INFO:tasks.workunit.client.0.smithi110.stdout:0/315: mkdir d1/d23/d30/d33/d80 0 2024-04-25T15:49:23.374 INFO:tasks.workunit.client.0.smithi110.stdout:0/316: chown d1/d20/d69 12 1 2024-04-25T15:49:23.375 INFO:tasks.workunit.client.0.smithi110.stdout:6/273: fsync d1/dd/d3c/f5d 0 2024-04-25T15:49:23.378 INFO:tasks.workunit.client.0.smithi110.stdout:7/272: creat d1b/d29/d38/f5b x:0 0 0 2024-04-25T15:49:23.379 INFO:tasks.workunit.client.0.smithi110.stdout:4/337: chown d13/l18 92 1 2024-04-25T15:49:23.381 INFO:tasks.workunit.client.0.smithi110.stdout:7/273: dread d1b/f45 [0,4194304] 0 2024-04-25T15:49:23.382 INFO:tasks.workunit.client.0.smithi110.stdout:3/393: rename d0/d16/l53 to d0/d7/d1c/d2f/d33/l84 0 2024-04-25T15:49:23.384 INFO:tasks.workunit.client.0.smithi110.stdout:7/274: truncate d1b/d29/f57 679866 0 2024-04-25T15:49:23.384 INFO:tasks.workunit.client.0.smithi110.stdout:2/243: fdatasync de/d11/f19 0 2024-04-25T15:49:23.384 INFO:tasks.workunit.client.0.smithi110.stdout:3/394: chown d0/d7/d1c/d2f/d4c/d63 1015 1 2024-04-25T15:49:23.435 INFO:tasks.workunit.client.0.smithi110.stdout:9/236: mknod d9/d2d/c4a 0 2024-04-25T15:49:23.435 INFO:tasks.workunit.client.0.smithi110.stdout:2/244: dread - de/d40/f3e zero size 2024-04-25T15:49:23.436 INFO:tasks.workunit.client.0.smithi110.stdout:4/338: dwrite d13/d24/f45 [0,4194304] 0 2024-04-25T15:49:23.436 INFO:tasks.workunit.client.0.smithi110.stdout:8/327: rename d14/d21/d23/f38 to d14/d32/d36/d48/f69 0 2024-04-25T15:49:23.437 INFO:tasks.workunit.client.0.smithi110.stdout:7/275: rmdir d1b 39 2024-04-25T15:49:23.439 INFO:tasks.workunit.client.0.smithi110.stdout:9/237: write d9/d13/f27 [1214421,69358] 0 2024-04-25T15:49:23.441 INFO:tasks.workunit.client.0.smithi110.stdout:5/246: rename d4/d15/f3c to d4/d1d/d1f/d35/d3d/d52/f55 0 2024-04-25T15:49:23.442 INFO:tasks.workunit.client.0.smithi110.stdout:5/247: fdatasync d4/d1d/d1f/f42 0 2024-04-25T15:49:23.442 INFO:tasks.workunit.client.0.smithi110.stdout:8/328: mkdir d14/d21/d26/d6a 0 2024-04-25T15:49:23.445 INFO:tasks.workunit.client.0.smithi110.stdout:3/395: mkdir d0/d7/d1c/d2f/d85 0 2024-04-25T15:49:23.447 INFO:tasks.workunit.client.0.smithi110.stdout:9/238: mknod d9/db/c4b 0 2024-04-25T15:49:23.448 INFO:tasks.workunit.client.0.smithi110.stdout:5/248: mkdir d4/d1d/d1f/d56 0 2024-04-25T15:49:23.450 INFO:tasks.workunit.client.0.smithi110.stdout:7/276: creat d1b/d29/d46/f5c x:0 0 0 2024-04-25T15:49:23.451 INFO:tasks.workunit.client.0.smithi110.stdout:9/239: creat d9/db/df/d25/f4c x:0 0 0 2024-04-25T15:49:23.452 INFO:tasks.workunit.client.0.smithi110.stdout:5/249: truncate d4/f8 604780 0 2024-04-25T15:49:23.453 INFO:tasks.workunit.client.0.smithi110.stdout:3/396: mknod d0/c86 0 2024-04-25T15:49:23.453 INFO:tasks.workunit.client.0.smithi110.stdout:5/250: dread - d4/d1d/d1f/f49 zero size 2024-04-25T15:49:23.455 INFO:tasks.workunit.client.0.smithi110.stdout:0/317: rename d1/d6c/d7f/d7c to d1/d23/d81 0 2024-04-25T15:49:23.455 INFO:tasks.workunit.client.0.smithi110.stdout:7/277: fdatasync f19 0 2024-04-25T15:49:23.456 INFO:tasks.workunit.client.0.smithi110.stdout:0/318: write d1/d47/d54/f6d [149310,83034] 0 2024-04-25T15:49:23.456 INFO:tasks.workunit.client.0.smithi110.stdout:9/240: mkdir d9/db/df/d4d 0 2024-04-25T15:49:23.459 INFO:tasks.workunit.client.0.smithi110.stdout:3/397: chown d0/d7/d1c/d2f/c67 1 1 2024-04-25T15:49:23.461 INFO:tasks.workunit.client.0.smithi110.stdout:2/245: rename de/d11/c2a to de/d40/c41 0 2024-04-25T15:49:23.462 INFO:tasks.workunit.client.0.smithi110.stdout:2/246: write de/d11/f21 [1438690,33276] 0 2024-04-25T15:49:23.462 INFO:tasks.workunit.client.0.smithi110.stdout:7/278: creat d1b/d29/f5d x:0 0 0 2024-04-25T15:49:23.465 INFO:tasks.workunit.client.0.smithi110.stdout:8/329: getdents d14/d21/d23 0 2024-04-25T15:49:23.466 INFO:tasks.workunit.client.0.smithi110.stdout:9/241: chown d9/db/df/l41 430 1 2024-04-25T15:49:23.468 INFO:tasks.workunit.client.0.smithi110.stdout:0/319: dread d1/d47/f52 [0,4194304] 0 2024-04-25T15:49:23.469 INFO:tasks.workunit.client.0.smithi110.stdout:3/398: creat d0/d2e/f87 x:0 0 0 2024-04-25T15:49:23.471 INFO:tasks.workunit.client.0.smithi110.stdout:4/339: rename d13/d24/d2d/d36/f53 to d13/d24/d3c/d56/f66 0 2024-04-25T15:49:23.471 INFO:tasks.workunit.client.0.smithi110.stdout:4/340: dread - d13/d24/d2d/f62 zero size 2024-04-25T15:49:23.472 INFO:tasks.workunit.client.0.smithi110.stdout:4/341: chown d13/d24/d2d/f4f 6997630 1 2024-04-25T15:49:23.473 INFO:tasks.workunit.client.0.smithi110.stdout:4/342: stat d13/d24/d3c/d56/f57 0 2024-04-25T15:49:23.474 INFO:tasks.workunit.client.0.smithi110.stdout:8/330: symlink d14/d32/l6b 0 2024-04-25T15:49:23.475 INFO:tasks.workunit.client.0.smithi110.stdout:8/331: write d14/f2d [713930,101097] 0 2024-04-25T15:49:23.479 INFO:tasks.workunit.client.0.smithi110.stdout:0/320: creat d1/d23/d30/d33/d5b/d72/f82 x:0 0 0 2024-04-25T15:49:23.483 INFO:tasks.workunit.client.0.smithi110.stdout:5/251: rename d4/d15/c54 to d4/d1d/d1f/d35/d3d/c57 0 2024-04-25T15:49:23.485 INFO:tasks.workunit.client.0.smithi110.stdout:4/343: symlink d13/d24/d2d/d55/l67 0 2024-04-25T15:49:23.488 INFO:tasks.workunit.client.0.smithi110.stdout:5/252: dread d4/f22 [0,4194304] 0 2024-04-25T15:49:23.491 INFO:tasks.workunit.client.0.smithi110.stdout:8/332: rename d14/d32/c3d to d14/d4c/d55/c6c 0 2024-04-25T15:49:23.495 INFO:tasks.workunit.client.0.smithi110.stdout:5/253: dwrite d4/f44 [0,4194304] 0 2024-04-25T15:49:23.499 INFO:tasks.workunit.client.0.smithi110.stdout:7/279: getdents d1b/d29/d38 0 2024-04-25T15:49:23.500 INFO:tasks.workunit.client.0.smithi110.stdout:7/280: dread f17 [0,4194304] 0 2024-04-25T15:49:23.504 INFO:tasks.workunit.client.0.smithi110.stdout:8/333: creat d14/d4c/d55/f6d x:0 0 0 2024-04-25T15:49:23.504 INFO:tasks.workunit.client.0.smithi110.stdout:9/242: getdents d9/db/df/d25 0 2024-04-25T15:49:23.506 INFO:tasks.workunit.client.0.smithi110.stdout:4/344: truncate d13/d1c/f41 3076137 0 2024-04-25T15:49:23.510 INFO:tasks.workunit.client.0.smithi110.stdout:5/254: dread d4/d1d/d1f/d35/f4c [4194304,4194304] 0 2024-04-25T15:49:23.511 INFO:tasks.workunit.client.0.smithi110.stdout:8/334: creat d14/f6e x:0 0 0 2024-04-25T15:49:23.513 INFO:tasks.workunit.client.0.smithi110.stdout:4/345: creat d13/d24/d2d/d36/d51/f68 x:0 0 0 2024-04-25T15:49:23.514 INFO:tasks.workunit.client.0.smithi110.stdout:4/346: dread - d13/d24/d2d/d36/f4d zero size 2024-04-25T15:49:23.515 INFO:tasks.workunit.client.0.smithi110.stdout:4/347: stat d13/d24/d2d/d36/d51 0 2024-04-25T15:49:23.516 INFO:tasks.workunit.client.0.smithi110.stdout:4/348: write fc [4729108,58399] 0 2024-04-25T15:49:23.517 INFO:tasks.workunit.client.0.smithi110.stdout:7/281: getdents d1b/d29/d38 0 2024-04-25T15:49:23.521 INFO:tasks.workunit.client.0.smithi110.stdout:4/349: dread d13/d24/d2d/f4f [0,4194304] 0 2024-04-25T15:49:23.522 INFO:tasks.workunit.client.0.smithi110.stdout:3/399: write d0/d7/d1c/f56 [2506019,19297] 0 2024-04-25T15:49:23.523 INFO:tasks.workunit.client.0.smithi110.stdout:3/400: readlink d0/d2e/l32 0 2024-04-25T15:49:23.524 INFO:tasks.workunit.client.0.smithi110.stdout:7/282: unlink cc 0 2024-04-25T15:49:23.526 INFO:tasks.workunit.client.0.smithi110.stdout:8/335: getdents d14/d21/d23 0 2024-04-25T15:49:23.526 INFO:tasks.workunit.client.0.smithi110.stdout:9/243: dread d9/db/df/d2c/f36 [0,4194304] 0 2024-04-25T15:49:23.529 INFO:tasks.workunit.client.0.smithi110.stdout:4/350: mknod d13/d24/d2d/d36/d51/c69 0 2024-04-25T15:49:23.530 INFO:tasks.workunit.client.0.smithi110.stdout:0/321: truncate d1/d23/d30/d33/f4b 477829 0 2024-04-25T15:49:23.532 INFO:tasks.workunit.client.0.smithi110.stdout:8/336: write d14/f2d [279253,24798] 0 2024-04-25T15:49:23.534 INFO:tasks.workunit.client.0.smithi110.stdout:9/244: symlink d9/db/df/d2c/d3c/l4e 0 2024-04-25T15:49:23.534 INFO:tasks.workunit.client.0.smithi110.stdout:7/283: getdents d1b/d29/d42 0 2024-04-25T15:49:23.534 INFO:tasks.workunit.client.0.smithi110.stdout:0/322: dread d1/d20/f6a [0,4194304] 0 2024-04-25T15:49:23.537 INFO:tasks.workunit.client.0.smithi110.stdout:4/351: creat d13/d24/d2d/d36/d51/f6a x:0 0 0 2024-04-25T15:49:23.542 INFO:tasks.workunit.client.0.smithi110.stdout:7/284: dwrite d1b/d29/f5d [0,4194304] 0 2024-04-25T15:49:23.544 INFO:tasks.workunit.client.0.smithi110.stdout:8/337: dread - d14/d32/d36/d48/f69 zero size 2024-04-25T15:49:23.544 INFO:tasks.workunit.client.0.smithi110.stdout:4/352: dwrite d13/d24/d26/f49 [0,4194304] 0 2024-04-25T15:49:23.546 INFO:tasks.workunit.client.0.smithi110.stdout:9/245: creat d9/db/df/f4f x:0 0 0 2024-04-25T15:49:23.552 INFO:tasks.workunit.client.0.smithi110.stdout:8/338: dread d14/d32/d36/d48/f5e [0,4194304] 0 2024-04-25T15:49:23.554 INFO:tasks.workunit.client.0.smithi110.stdout:7/285: unlink d1b/d29/f4f 0 2024-04-25T15:49:23.558 INFO:tasks.workunit.client.0.smithi110.stdout:0/323: rename d1/d4c to d1/d23/d30/d33/d5b/d83 0 2024-04-25T15:49:23.562 INFO:tasks.workunit.client.0.smithi110.stdout:9/246: dwrite d9/db/df/d2c/f36 [4194304,4194304] 0 2024-04-25T15:49:23.564 INFO:tasks.workunit.client.0.smithi110.stdout:4/353: creat d13/d24/d2d/d5e/f6b x:0 0 0 2024-04-25T15:49:23.594 INFO:tasks.workunit.client.0.smithi110.stdout:0/324: rmdir d1/d47/d54 39 2024-04-25T15:49:23.600 INFO:tasks.workunit.client.0.smithi110.stdout:0/325: creat d1/d23/d30/d33/d5b/d83/f84 x:0 0 0 2024-04-25T15:49:23.603 INFO:tasks.workunit.client.0.smithi110.stdout:8/339: getdents d14/d21 0 2024-04-25T15:49:23.607 INFO:tasks.workunit.client.0.smithi110.stdout:8/340: symlink d14/d21/d23/l6f 0 2024-04-25T15:49:23.608 INFO:tasks.workunit.client.0.smithi110.stdout:0/326: mknod d1/d23/d30/d33/d5b/d72/d77/c85 0 2024-04-25T15:49:23.609 INFO:tasks.workunit.client.0.smithi110.stdout:0/327: write d1/d23/d30/d33/d35/f41 [69777,45308] 0 2024-04-25T15:49:23.612 INFO:tasks.workunit.client.0.smithi110.stdout:8/341: stat d14/d4c/d55/f6d 0 2024-04-25T15:49:23.612 INFO:tasks.workunit.client.0.smithi110.stdout:4/354: getdents d13/d24/d2d/d5e 0 2024-04-25T15:49:23.612 INFO:tasks.workunit.client.0.smithi110.stdout:8/342: dread - d14/d21/f2b zero size 2024-04-25T15:49:23.613 INFO:tasks.workunit.client.0.smithi110.stdout:8/343: stat c13 0 2024-04-25T15:49:23.613 INFO:tasks.workunit.client.0.smithi110.stdout:8/344: write f5 [3099394,64731] 0 2024-04-25T15:49:23.620 INFO:tasks.workunit.client.0.smithi110.stdout:4/355: rename d13/d24/d2d/d44/d64 to d13/d24/d3c/d56/d6c 0 2024-04-25T15:49:23.621 INFO:tasks.workunit.client.0.smithi110.stdout:8/345: mknod d14/d21/d26/c70 0 2024-04-25T15:49:23.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/346: mkdir d14/d4c/d71 0 2024-04-25T15:49:23.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/347: readlink ld 0 2024-04-25T15:49:23.632 INFO:tasks.workunit.client.0.smithi110.stdout:8/348: dread ff [8388608,4194304] 0 2024-04-25T15:49:23.633 INFO:tasks.workunit.client.0.smithi110.stdout:8/349: read d14/f2d [342978,96374] 0 2024-04-25T15:49:23.648 INFO:tasks.workunit.client.0.smithi110.stdout:9/247: write d9/db/f11 [1813454,113008] 0 2024-04-25T15:49:23.654 INFO:tasks.workunit.client.0.smithi110.stdout:9/248: dwrite d9/db/f11 [0,4194304] 0 2024-04-25T15:49:23.656 INFO:tasks.workunit.client.0.smithi110.stdout:1/387: sync 2024-04-25T15:49:23.661 INFO:tasks.workunit.client.0.smithi110.stdout:9/249: rename d9/db/df/d25/c42 to d9/db/df/d4d/c50 0 2024-04-25T15:49:23.662 INFO:tasks.workunit.client.0.smithi110.stdout:1/388: dwrite d0/d25/d33/f47 [0,4194304] 0 2024-04-25T15:49:23.663 INFO:tasks.workunit.client.0.smithi110.stdout:1/389: fsync d0/d14/f57 0 2024-04-25T15:49:23.670 INFO:tasks.workunit.client.0.smithi110.stdout:1/390: symlink d0/d23/d56/l7b 0 2024-04-25T15:49:23.671 INFO:tasks.workunit.client.0.smithi110.stdout:9/250: dwrite d9/db/df/d2c/d3c/f49 [0,4194304] 0 2024-04-25T15:49:23.677 INFO:tasks.workunit.client.0.smithi110.stdout:1/391: creat d0/d51/d6b/f7c x:0 0 0 2024-04-25T15:49:23.678 INFO:tasks.workunit.client.0.smithi110.stdout:1/392: stat d0/d23/f5c 0 2024-04-25T15:49:23.679 INFO:tasks.workunit.client.0.smithi110.stdout:9/251: creat d9/db/df/d25/f51 x:0 0 0 2024-04-25T15:49:23.680 INFO:tasks.workunit.client.0.smithi110.stdout:0/328: truncate d1/d23/d30/d33/d35/f6b 759326 0 2024-04-25T15:49:23.684 INFO:tasks.workunit.client.0.smithi110.stdout:9/252: mknod d9/db/df/d25/c52 0 2024-04-25T15:49:23.692 INFO:tasks.workunit.client.0.smithi110.stdout:4/356: dwrite d13/d1c/f20 [0,4194304] 0 2024-04-25T15:49:23.697 INFO:tasks.workunit.client.0.smithi110.stdout:9/253: creat d9/d44/f53 x:0 0 0 2024-04-25T15:49:23.698 INFO:tasks.workunit.client.0.smithi110.stdout:9/254: write d9/db/df/f4f [276676,128517] 0 2024-04-25T15:49:23.702 INFO:tasks.workunit.client.0.smithi110.stdout:4/357: dread d13/d24/f2a [0,4194304] 0 2024-04-25T15:49:23.703 INFO:tasks.workunit.client.0.smithi110.stdout:9/255: unlink d9/db/df/f17 0 2024-04-25T15:49:23.707 INFO:tasks.workunit.client.0.smithi110.stdout:4/358: mknod d13/d24/d3c/d56/c6d 0 2024-04-25T15:49:23.711 INFO:tasks.workunit.client.0.smithi110.stdout:8/350: creat d14/d21/f72 x:0 0 0 2024-04-25T15:49:23.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:23 smithi193 ceph-mon[33747]: pgmap v28: 65 pgs: 65 active+clean; 489 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.4 MiB/s rd, 46 MiB/s wr, 268 op/s 2024-04-25T15:49:23.751 INFO:tasks.workunit.client.0.smithi110.stdout:1/393: rename d0/d23/d56 to d0/d14/d5f/d7d 0 2024-04-25T15:49:23.756 INFO:tasks.workunit.client.0.smithi110.stdout:9/256: rename d9/db/df/d2c/f3a to d9/d13/f54 0 2024-04-25T15:49:23.757 INFO:tasks.workunit.client.0.smithi110.stdout:9/257: chown d9/f34 4044340 1 2024-04-25T15:49:23.758 INFO:tasks.workunit.client.0.smithi110.stdout:1/394: write d0/d25/f41 [460443,52901] 0 2024-04-25T15:49:23.768 INFO:tasks.workunit.client.0.smithi110.stdout:1/395: creat d0/d14/d5f/d7d/d72/f7e x:0 0 0 2024-04-25T15:49:23.770 INFO:tasks.workunit.client.0.smithi110.stdout:4/359: rename d13/c17 to d13/d24/d26/c6e 0 2024-04-25T15:49:23.772 INFO:tasks.workunit.client.0.smithi110.stdout:1/396: mknod d0/c7f 0 2024-04-25T15:49:23.773 INFO:tasks.workunit.client.0.smithi110.stdout:0/329: truncate d1/d23/d30/d33/d35/f36 19584 0 2024-04-25T15:49:23.782 INFO:tasks.workunit.client.0.smithi110.stdout:1/397: dwrite d0/d14/f4b [0,4194304] 0 2024-04-25T15:49:23.786 INFO:tasks.workunit.client.0.smithi110.stdout:4/360: dread d13/d1c/f25 [0,4194304] 0 2024-04-25T15:49:23.787 INFO:tasks.workunit.client.0.smithi110.stdout:4/361: stat d13/d24/d2d/d36/d51 0 2024-04-25T15:49:23.790 INFO:tasks.workunit.client.0.smithi110.stdout:1/398: truncate d0/d25/f6 1072245 0 2024-04-25T15:49:23.793 INFO:tasks.workunit.client.0.smithi110.stdout:0/330: rename d1/f49 to d1/d47/f86 0 2024-04-25T15:49:23.794 INFO:tasks.workunit.client.0.smithi110.stdout:0/331: chown d1/d23/d30/d33/d5b/d72 2 1 2024-04-25T15:49:23.796 INFO:tasks.workunit.client.0.smithi110.stdout:4/362: dread fd [0,4194304] 0 2024-04-25T15:49:23.798 INFO:tasks.workunit.client.0.smithi110.stdout:0/332: mkdir d1/d23/d30/d33/d5b/d83/d87 0 2024-04-25T15:49:23.798 INFO:tasks.workunit.client.0.smithi110.stdout:8/351: truncate d14/f16 2411296 0 2024-04-25T15:49:23.798 INFO:tasks.workunit.client.0.smithi110.stdout:0/333: chown d1/f4e 229238 1 2024-04-25T15:49:23.802 INFO:tasks.workunit.client.0.smithi110.stdout:8/352: mknod d14/d4c/c73 0 2024-04-25T15:49:23.805 INFO:tasks.workunit.client.0.smithi110.stdout:4/363: mknod d13/d24/d2d/c6f 0 2024-04-25T15:49:23.808 INFO:tasks.workunit.client.0.smithi110.stdout:0/334: link d1/f4e d1/d23/d30/d33/f88 0 2024-04-25T15:49:23.809 INFO:tasks.workunit.client.0.smithi110.stdout:0/335: chown d1/d23/d30/d33/d35/f3a 3 1 2024-04-25T15:49:23.811 INFO:tasks.workunit.client.0.smithi110.stdout:8/353: write d14/f1d [399702,82402] 0 2024-04-25T15:49:23.815 INFO:tasks.workunit.client.0.smithi110.stdout:8/354: mknod d14/d32/c74 0 2024-04-25T15:49:23.817 INFO:tasks.workunit.client.0.smithi110.stdout:4/364: creat d13/d24/f70 x:0 0 0 2024-04-25T15:49:23.819 INFO:tasks.workunit.client.0.smithi110.stdout:0/336: creat d1/d23/f89 x:0 0 0 2024-04-25T15:49:23.822 INFO:tasks.workunit.client.0.smithi110.stdout:8/355: unlink d14/d21/f2b 0 2024-04-25T15:49:23.825 INFO:tasks.workunit.client.0.smithi110.stdout:0/337: creat d1/d23/d30/d33/d5b/d83/f8a x:0 0 0 2024-04-25T15:49:23.827 INFO:tasks.workunit.client.0.smithi110.stdout:0/338: fsync d1/d23/d30/d33/d5b/d83/f53 0 2024-04-25T15:49:23.828 INFO:tasks.workunit.client.0.smithi110.stdout:0/339: stat d1/d23/f42 0 2024-04-25T15:49:23.829 INFO:tasks.workunit.client.0.smithi110.stdout:8/356: dwrite f5 [0,4194304] 0 2024-04-25T15:49:23.832 INFO:tasks.workunit.client.0.smithi110.stdout:4/365: dread d13/d24/d26/f31 [0,4194304] 0 2024-04-25T15:49:23.833 INFO:tasks.workunit.client.0.smithi110.stdout:9/258: rmdir d9 39 2024-04-25T15:49:23.833 INFO:tasks.workunit.client.0.smithi110.stdout:4/366: chown d13/d24/f45 12033492 1 2024-04-25T15:49:23.835 INFO:tasks.workunit.client.0.smithi110.stdout:8/357: creat d14/d4c/f75 x:0 0 0 2024-04-25T15:49:23.840 INFO:tasks.workunit.client.0.smithi110.stdout:9/259: chown d9/d2d/f38 3 1 2024-04-25T15:49:23.841 INFO:tasks.workunit.client.0.smithi110.stdout:4/367: creat d13/d24/d2d/d5e/f71 x:0 0 0 2024-04-25T15:49:23.845 INFO:tasks.workunit.client.0.smithi110.stdout:6/274: sync 2024-04-25T15:49:23.847 INFO:tasks.workunit.client.0.smithi110.stdout:9/260: fdatasync d9/d13/f29 0 2024-04-25T15:49:23.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:23 smithi110 ceph-mon[26344]: pgmap v28: 65 pgs: 65 active+clean; 489 MiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 1.4 MiB/s rd, 46 MiB/s wr, 268 op/s 2024-04-25T15:49:23.879 INFO:tasks.workunit.client.0.smithi110.stdout:0/340: truncate d1/f8 2213443 0 2024-04-25T15:49:23.883 INFO:tasks.workunit.client.0.smithi110.stdout:8/358: truncate d14/f1d 2074991 0 2024-04-25T15:49:23.885 INFO:tasks.workunit.client.0.smithi110.stdout:8/359: chown d14/f40 5 1 2024-04-25T15:49:23.887 INFO:tasks.workunit.client.0.smithi110.stdout:8/360: rmdir d14/d21/d23 39 2024-04-25T15:49:23.891 INFO:tasks.workunit.client.0.smithi110.stdout:6/275: dwrite d1/db/d22/f4d [0,4194304] 0 2024-04-25T15:49:23.891 INFO:tasks.workunit.client.0.smithi110.stdout:8/361: mknod d14/d4c/c76 0 2024-04-25T15:49:23.893 INFO:tasks.workunit.client.0.smithi110.stdout:4/368: dwrite d13/d1c/f41 [0,4194304] 0 2024-04-25T15:49:23.896 INFO:tasks.workunit.client.0.smithi110.stdout:6/276: rename d1/f3 to d1/dd/f67 0 2024-04-25T15:49:23.898 INFO:tasks.workunit.client.0.smithi110.stdout:8/362: creat d14/d32/f77 x:0 0 0 2024-04-25T15:49:23.900 INFO:tasks.workunit.client.0.smithi110.stdout:6/277: truncate d1/dd/d3c/f51 263128 0 2024-04-25T15:49:23.905 INFO:tasks.workunit.client.0.smithi110.stdout:4/369: dwrite d13/d24/d2d/d44/f60 [0,4194304] 0 2024-04-25T15:49:23.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/370: read d13/d24/f2a [3882741,25695] 0 2024-04-25T15:49:23.909 INFO:tasks.workunit.client.0.smithi110.stdout:6/278: creat d1/d12/f68 x:0 0 0 2024-04-25T15:49:23.912 INFO:tasks.workunit.client.0.smithi110.stdout:0/341: dwrite d1/f38 [0,4194304] 0 2024-04-25T15:49:23.914 INFO:tasks.workunit.client.0.smithi110.stdout:8/363: creat d14/d21/d26/d6a/f78 x:0 0 0 2024-04-25T15:49:23.921 INFO:tasks.workunit.client.0.smithi110.stdout:4/371: dwrite d13/d24/d3c/d56/f57 [0,4194304] 0 2024-04-25T15:49:23.923 INFO:tasks.workunit.client.0.smithi110.stdout:6/279: mkdir d1/db/d24/d69 0 2024-04-25T15:49:23.923 INFO:tasks.workunit.client.0.smithi110.stdout:4/372: dread - d13/d24/d2d/d36/d51/f68 zero size 2024-04-25T15:49:23.932 INFO:tasks.workunit.client.0.smithi110.stdout:6/280: creat d1/db/d22/d33/d43/f6a x:0 0 0 2024-04-25T15:49:23.933 INFO:tasks.workunit.client.0.smithi110.stdout:6/281: readlink d1/d12/d3d/l46 0 2024-04-25T15:49:23.934 INFO:tasks.workunit.client.0.smithi110.stdout:6/282: chown d1/db/d22/d27/f2b 44866170 1 2024-04-25T15:49:23.936 INFO:tasks.workunit.client.0.smithi110.stdout:0/342: rename d1/d23/d30/d33/d5b/d83/f5f to d1/f8b 0 2024-04-25T15:49:23.938 INFO:tasks.workunit.client.0.smithi110.stdout:4/373: unlink d13/d24/d26/f3b 0 2024-04-25T15:49:23.943 INFO:tasks.workunit.client.0.smithi110.stdout:8/364: write f11 [1067401,113616] 0 2024-04-25T15:49:23.944 INFO:tasks.workunit.client.0.smithi110.stdout:0/343: fdatasync d1/d23/f42 0 2024-04-25T15:49:23.946 INFO:tasks.workunit.client.0.smithi110.stdout:4/374: rmdir d13/d24/d2d/d44 39 2024-04-25T15:49:23.948 INFO:tasks.workunit.client.0.smithi110.stdout:4/375: dread - d13/d24/d2d/d36/f4d zero size 2024-04-25T15:49:23.948 INFO:tasks.workunit.client.0.smithi110.stdout:8/365: mknod d14/d4c/c79 0 2024-04-25T15:49:23.955 INFO:tasks.workunit.client.0.smithi110.stdout:0/344: dwrite d1/d23/d30/d33/d35/f12 [0,4194304] 0 2024-04-25T15:49:23.957 INFO:tasks.workunit.client.0.smithi110.stdout:6/283: getdents d1/db/d24/d32 0 2024-04-25T15:49:23.960 INFO:tasks.workunit.client.0.smithi110.stdout:0/345: mknod d1/d23/d81/c8c 0 2024-04-25T15:49:23.962 INFO:tasks.workunit.client.0.smithi110.stdout:8/366: dwrite d14/f5c [0,4194304] 0 2024-04-25T15:49:23.963 INFO:tasks.workunit.client.0.smithi110.stdout:8/367: dread - d14/d21/f72 zero size 2024-04-25T15:49:23.965 INFO:tasks.workunit.client.0.smithi110.stdout:6/284: rename d1/db/l16 to d1/db/d22/d33/l6b 0 2024-04-25T15:49:23.971 INFO:tasks.workunit.client.0.smithi110.stdout:8/368: dwrite d14/d32/d36/f64 [0,4194304] 0 2024-04-25T15:49:23.974 INFO:tasks.workunit.client.0.smithi110.stdout:6/285: getdents d1/db/d22 0 2024-04-25T15:49:23.977 INFO:tasks.workunit.client.0.smithi110.stdout:8/369: stat d14/d21/d23/l49 0 2024-04-25T15:49:23.978 INFO:tasks.workunit.client.0.smithi110.stdout:8/370: read - d14/d32/d36/d48/f69 zero size 2024-04-25T15:49:23.980 INFO:tasks.workunit.client.0.smithi110.stdout:8/371: stat d14/l63 0 2024-04-25T15:49:23.983 INFO:tasks.workunit.client.0.smithi110.stdout:8/372: creat d14/d21/d26/f7a x:0 0 0 2024-04-25T15:49:23.983 INFO:tasks.workunit.client.0.smithi110.stdout:8/373: chown l12 123935 1 2024-04-25T15:49:23.988 INFO:tasks.workunit.client.0.smithi110.stdout:8/374: write d14/f2d [599017,119712] 0 2024-04-25T15:49:23.991 INFO:tasks.workunit.client.0.smithi110.stdout:8/375: symlink d14/d21/d23/l7b 0 2024-04-25T15:49:23.992 INFO:tasks.workunit.client.0.smithi110.stdout:8/376: creat d14/d21/d23/f7c x:0 0 0 2024-04-25T15:49:23.999 INFO:tasks.workunit.client.0.smithi110.stdout:8/377: write f3 [1065912,14141] 0 2024-04-25T15:49:24.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/378: creat d14/d4c/d55/f7d x:0 0 0 2024-04-25T15:49:24.001 INFO:tasks.workunit.client.0.smithi110.stdout:8/379: stat d14/d21/f62 0 2024-04-25T15:49:24.001 INFO:tasks.workunit.client.0.smithi110.stdout:8/380: chown d14/d21/f27 162 1 2024-04-25T15:49:24.003 INFO:tasks.workunit.client.0.smithi110.stdout:8/381: unlink l12 0 2024-04-25T15:49:24.005 INFO:tasks.workunit.client.0.smithi110.stdout:8/382: mkdir d14/d21/d26/d6a/d7e 0 2024-04-25T15:49:24.012 INFO:tasks.workunit.client.0.smithi110.stdout:8/383: write d14/f1d [2731973,84895] 0 2024-04-25T15:49:24.013 INFO:tasks.workunit.client.0.smithi110.stdout:8/384: dread - d14/d21/d26/f4a zero size 2024-04-25T15:49:24.020 INFO:tasks.workunit.client.0.smithi110.stdout:8/385: dwrite d14/d21/f67 [0,4194304] 0 2024-04-25T15:49:24.025 INFO:tasks.workunit.client.0.smithi110.stdout:8/386: dread d14/f40 [0,4194304] 0 2024-04-25T15:49:24.028 INFO:tasks.workunit.client.0.smithi110.stdout:8/387: creat d14/d4c/d71/f7f x:0 0 0 2024-04-25T15:49:24.034 INFO:tasks.workunit.client.0.smithi110.stdout:8/388: dwrite d14/d21/f72 [0,4194304] 0 2024-04-25T15:49:24.038 INFO:tasks.workunit.client.0.smithi110.stdout:8/389: truncate d14/d32/d36/d48/f5e 4870305 0 2024-04-25T15:49:24.066 INFO:tasks.workunit.client.0.smithi110.stdout:2/247: sync 2024-04-25T15:49:24.069 INFO:tasks.workunit.client.0.smithi110.stdout:8/390: rename d14/f22 to d14/f80 0 2024-04-25T15:49:24.072 INFO:tasks.workunit.client.0.smithi110.stdout:8/391: fdatasync d14/d21/f2a 0 2024-04-25T15:49:24.073 INFO:tasks.workunit.client.0.smithi110.stdout:8/392: dread - d14/d32/d36/d48/f69 zero size 2024-04-25T15:49:24.076 INFO:tasks.workunit.client.0.smithi110.stdout:8/393: rmdir d14/d21 39 2024-04-25T15:49:24.081 INFO:tasks.workunit.client.0.smithi110.stdout:2/248: truncate de/d40/f35 2738469 0 2024-04-25T15:49:24.083 INFO:tasks.workunit.client.0.smithi110.stdout:2/249: creat de/d11/f42 x:0 0 0 2024-04-25T15:49:24.085 INFO:tasks.workunit.client.0.smithi110.stdout:2/250: write de/d40/f3d [630522,75970] 0 2024-04-25T15:49:24.086 INFO:tasks.workunit.client.0.smithi110.stdout:2/251: write f8 [3556235,67070] 0 2024-04-25T15:49:24.088 INFO:tasks.workunit.client.0.smithi110.stdout:2/252: fsync de/d11/f21 0 2024-04-25T15:49:24.089 INFO:tasks.workunit.client.0.smithi110.stdout:2/253: stat de/d11/f39 0 2024-04-25T15:49:24.091 INFO:tasks.workunit.client.0.smithi110.stdout:8/394: dwrite d14/d21/d26/f30 [0,4194304] 0 2024-04-25T15:49:24.093 INFO:tasks.workunit.client.0.smithi110.stdout:8/395: dread - d14/d4c/f75 zero size 2024-04-25T15:49:24.097 INFO:tasks.workunit.client.0.smithi110.stdout:2/254: dwrite de/d11/f3a [0,4194304] 0 2024-04-25T15:49:24.099 INFO:tasks.workunit.client.0.smithi110.stdout:8/396: dread d14/d21/f67 [0,4194304] 0 2024-04-25T15:49:24.100 INFO:tasks.workunit.client.0.smithi110.stdout:8/397: chown d14/f80 205421986 1 2024-04-25T15:49:24.100 INFO:tasks.workunit.client.0.smithi110.stdout:2/255: mknod de/d40/c43 0 2024-04-25T15:49:24.101 INFO:tasks.workunit.client.0.smithi110.stdout:8/398: chown d14/l56 3 1 2024-04-25T15:49:24.102 INFO:tasks.workunit.client.0.smithi110.stdout:2/256: mkdir de/d11/d44 0 2024-04-25T15:49:24.106 INFO:tasks.workunit.client.0.smithi110.stdout:8/399: creat d14/f81 x:0 0 0 2024-04-25T15:49:24.110 INFO:tasks.workunit.client.0.smithi110.stdout:2/257: rename de/c3b to de/c45 0 2024-04-25T15:49:24.110 INFO:tasks.workunit.client.0.smithi110.stdout:8/400: dread f5 [0,4194304] 0 2024-04-25T15:49:24.119 INFO:tasks.workunit.client.0.smithi110.stdout:2/258: write de/d40/f35 [2470486,67534] 0 2024-04-25T15:49:24.121 INFO:tasks.workunit.client.0.smithi110.stdout:8/401: write d14/d32/d36/d48/f5e [4293210,85921] 0 2024-04-25T15:49:24.122 INFO:tasks.workunit.client.0.smithi110.stdout:2/259: symlink de/d40/l46 0 2024-04-25T15:49:24.123 INFO:tasks.workunit.client.0.smithi110.stdout:8/402: symlink d14/d21/d26/d6a/l82 0 2024-04-25T15:49:24.124 INFO:tasks.workunit.client.0.smithi110.stdout:8/403: write d14/f2d [370051,117898] 0 2024-04-25T15:49:24.126 INFO:tasks.workunit.client.0.smithi110.stdout:2/260: creat de/d11/d44/f47 x:0 0 0 2024-04-25T15:49:24.127 INFO:tasks.workunit.client.0.smithi110.stdout:2/261: fsync de/d11/f21 0 2024-04-25T15:49:24.128 INFO:tasks.workunit.client.0.smithi110.stdout:2/262: fdatasync de/f2c 0 2024-04-25T15:49:24.128 INFO:tasks.workunit.client.0.smithi110.stdout:2/263: readlink de/l23 0 2024-04-25T15:49:24.132 INFO:tasks.workunit.client.0.smithi110.stdout:8/404: creat d14/f83 x:0 0 0 2024-04-25T15:49:24.132 INFO:tasks.workunit.client.0.smithi110.stdout:2/264: mknod de/c48 0 2024-04-25T15:49:24.137 INFO:tasks.workunit.client.0.smithi110.stdout:2/265: unlink de/d40/l32 0 2024-04-25T15:49:24.140 INFO:tasks.workunit.client.0.smithi110.stdout:2/266: creat de/d11/d44/f49 x:0 0 0 2024-04-25T15:49:24.152 INFO:tasks.workunit.client.0.smithi110.stdout:8/405: dread d14/d4c/f50 [0,4194304] 0 2024-04-25T15:49:24.155 INFO:tasks.workunit.client.0.smithi110.stdout:8/406: unlink d14/d32/l6b 0 2024-04-25T15:49:24.156 INFO:tasks.workunit.client.0.smithi110.stdout:8/407: write d14/d32/d36/f3c [3890313,113587] 0 2024-04-25T15:49:24.160 INFO:tasks.workunit.client.0.smithi110.stdout:8/408: rename d14/d32/f77 to d14/d32/d36/f84 0 2024-04-25T15:49:24.162 INFO:tasks.workunit.client.0.smithi110.stdout:2/267: dread de/f2c [0,4194304] 0 2024-04-25T15:49:24.164 INFO:tasks.workunit.client.0.smithi110.stdout:2/268: read de/f24 [1325741,123286] 0 2024-04-25T15:49:24.164 INFO:tasks.workunit.client.0.smithi110.stdout:8/409: rename d14/d21/d23/l4f to d14/d21/d26/d6a/d7e/l85 0 2024-04-25T15:49:24.165 INFO:tasks.workunit.client.0.smithi110.stdout:2/269: fdatasync de/d40/f35 0 2024-04-25T15:49:24.174 INFO:tasks.workunit.client.0.smithi110.stdout:8/410: dwrite d14/d21/f67 [4194304,4194304] 0 2024-04-25T15:49:24.177 INFO:tasks.workunit.client.0.smithi110.stdout:2/270: getdents de/d40 0 2024-04-25T15:49:24.177 INFO:tasks.workunit.client.0.smithi110.stdout:8/411: symlink d14/d21/d26/d6a/d7e/l86 0 2024-04-25T15:49:24.179 INFO:tasks.workunit.client.0.smithi110.stdout:2/271: write de/d11/d44/f49 [978428,15518] 0 2024-04-25T15:49:24.181 INFO:tasks.workunit.client.0.smithi110.stdout:8/412: mknod d14/d21/d26/d6a/c87 0 2024-04-25T15:49:24.183 INFO:tasks.workunit.client.0.smithi110.stdout:8/413: dread - d14/d32/d36/f84 zero size 2024-04-25T15:49:24.187 INFO:tasks.workunit.client.0.smithi110.stdout:2/272: dwrite de/d40/f3e [0,4194304] 0 2024-04-25T15:49:24.188 INFO:tasks.workunit.client.0.smithi110.stdout:8/414: unlink d14/d21/d23/l49 0 2024-04-25T15:49:24.197 INFO:tasks.workunit.client.0.smithi110.stdout:8/415: dwrite f6 [0,4194304] 0 2024-04-25T15:49:24.197 INFO:tasks.workunit.client.0.smithi110.stdout:2/273: mknod de/c4a 0 2024-04-25T15:49:24.198 INFO:tasks.workunit.client.0.smithi110.stdout:2/274: read - de/d40/f36 zero size 2024-04-25T15:49:24.199 INFO:tasks.workunit.client.0.smithi110.stdout:2/275: write de/d11/f21 [1263510,127182] 0 2024-04-25T15:49:24.201 INFO:tasks.workunit.client.0.smithi110.stdout:8/416: write d14/f40 [2940434,129048] 0 2024-04-25T15:49:24.202 INFO:tasks.workunit.client.0.smithi110.stdout:8/417: stat d14/d4c/d71 0 2024-04-25T15:49:24.203 INFO:tasks.workunit.client.0.smithi110.stdout:8/418: truncate d14/f6e 994203 0 2024-04-25T15:49:24.204 INFO:tasks.workunit.client.0.smithi110.stdout:8/419: dread - d14/d21/d26/f4a zero size 2024-04-25T15:49:24.206 INFO:tasks.workunit.client.0.smithi110.stdout:2/276: creat de/f4b x:0 0 0 2024-04-25T15:49:24.206 INFO:tasks.workunit.client.0.smithi110.stdout:8/420: creat d14/d4c/d55/f88 x:0 0 0 2024-04-25T15:49:24.208 INFO:tasks.workunit.client.0.smithi110.stdout:2/277: creat de/d11/d44/f4c x:0 0 0 2024-04-25T15:49:24.209 INFO:tasks.workunit.client.0.smithi110.stdout:8/421: mknod d14/d32/d36/d48/c89 0 2024-04-25T15:49:24.209 INFO:tasks.workunit.client.0.smithi110.stdout:2/278: symlink de/d40/l4d 0 2024-04-25T15:49:24.214 INFO:tasks.workunit.client.0.smithi110.stdout:8/422: truncate d14/d32/f3f 3949153 0 2024-04-25T15:49:24.214 INFO:tasks.workunit.client.0.smithi110.stdout:2/279: creat de/d11/d44/f4e x:0 0 0 2024-04-25T15:49:24.215 INFO:tasks.workunit.client.0.smithi110.stdout:2/280: write de/f2c [2981019,110225] 0 2024-04-25T15:49:24.217 INFO:tasks.workunit.client.0.smithi110.stdout:2/281: write de/f2c [2121735,30041] 0 2024-04-25T15:49:24.234 INFO:tasks.workunit.client.0.smithi110.stdout:2/282: rename de/d11/d44 to de/d40/d4f 0 2024-04-25T15:49:24.288 INFO:tasks.workunit.client.0.smithi110.stdout:5/255: sync 2024-04-25T15:49:24.291 INFO:tasks.workunit.client.0.smithi110.stdout:2/283: write de/f24 [3197659,10092] 0 2024-04-25T15:49:24.293 INFO:tasks.workunit.client.0.smithi110.stdout:2/284: dread - de/d40/d4f/f47 zero size 2024-04-25T15:49:24.295 INFO:tasks.workunit.client.0.smithi110.stdout:5/256: dwrite d4/d2c/f27 [0,4194304] 0 2024-04-25T15:49:24.299 INFO:tasks.workunit.client.0.smithi110.stdout:5/257: chown d4/d1d/d1f/d35/d3d/c57 52883 1 2024-04-25T15:49:24.301 INFO:tasks.workunit.client.0.smithi110.stdout:2/285: dwrite de/d11/f42 [0,4194304] 0 2024-04-25T15:49:24.302 INFO:tasks.workunit.client.0.smithi110.stdout:5/258: creat d4/d1d/d1f/d35/d3d/f58 x:0 0 0 2024-04-25T15:49:24.302 INFO:tasks.workunit.client.0.smithi110.stdout:5/259: readlink d4/d2c/d2a/l31 0 2024-04-25T15:49:24.303 INFO:tasks.workunit.client.0.smithi110.stdout:2/286: mkdir de/d11/d50 0 2024-04-25T15:49:24.303 INFO:tasks.workunit.client.0.smithi110.stdout:5/260: write d4/d1d/f38 [4273753,60802] 0 2024-04-25T15:49:24.305 INFO:tasks.workunit.client.0.smithi110.stdout:5/261: write d4/d1d/d1f/d35/d3d/f4d [3412071,36969] 0 2024-04-25T15:49:24.305 INFO:tasks.workunit.client.0.smithi110.stdout:2/287: mknod de/d11/c51 0 2024-04-25T15:49:24.306 INFO:tasks.workunit.client.0.smithi110.stdout:2/288: read de/d11/f3a [3608796,4979] 0 2024-04-25T15:49:24.310 INFO:tasks.workunit.client.0.smithi110.stdout:5/262: creat d4/d1d/d1f/d35/d3d/d52/f59 x:0 0 0 2024-04-25T15:49:24.311 INFO:tasks.workunit.client.0.smithi110.stdout:5/263: chown d4/d1d/d1f/d35/d3d 171 1 2024-04-25T15:49:24.311 INFO:tasks.workunit.client.0.smithi110.stdout:2/289: mkdir de/d11/d52 0 2024-04-25T15:49:24.315 INFO:tasks.workunit.client.0.smithi110.stdout:5/264: mknod d4/d15/c5a 0 2024-04-25T15:49:24.317 INFO:tasks.workunit.client.0.smithi110.stdout:5/265: fsync d4/d2c/d2a/f46 0 2024-04-25T15:49:24.319 INFO:tasks.workunit.client.0.smithi110.stdout:5/266: write d4/f6 [1549701,44407] 0 2024-04-25T15:49:24.319 INFO:tasks.workunit.client.0.smithi110.stdout:5/267: dread - d4/d1d/d1f/d35/d3d/f53 zero size 2024-04-25T15:49:24.331 INFO:tasks.workunit.client.0.smithi110.stdout:5/268: truncate d4/d1d/f34 77786 0 2024-04-25T15:49:24.338 INFO:tasks.workunit.client.0.smithi110.stdout:5/269: truncate d4/d2c/f19 755131 0 2024-04-25T15:49:24.345 INFO:tasks.workunit.client.0.smithi110.stdout:5/270: dwrite d4/d1d/d1f/d35/d3d/f4d [4194304,4194304] 0 2024-04-25T15:49:24.346 INFO:tasks.workunit.client.0.smithi110.stdout:5/271: stat d4/d1d/d1f/f42 0 2024-04-25T15:49:24.348 INFO:tasks.workunit.client.0.smithi110.stdout:5/272: symlink d4/d1d/d1f/l5b 0 2024-04-25T15:49:24.351 INFO:tasks.workunit.client.0.smithi110.stdout:5/273: creat d4/d2c/d2a/f5c x:0 0 0 2024-04-25T15:49:24.356 INFO:tasks.workunit.client.0.smithi110.stdout:5/274: rename d4/d1d/d1f/d35/d3d/f4d to d4/d1d/d1f/d35/d3d/f5d 0 2024-04-25T15:49:24.361 INFO:tasks.workunit.client.0.smithi110.stdout:5/275: truncate d4/f22 111503 0 2024-04-25T15:49:24.368 INFO:tasks.workunit.client.0.smithi110.stdout:5/276: rename d4/d2c/l4e to d4/l5e 0 2024-04-25T15:49:24.368 INFO:tasks.workunit.client.0.smithi110.stdout:5/277: chown d4/d2c/l12 22 1 2024-04-25T15:49:24.374 INFO:tasks.workunit.client.0.smithi110.stdout:5/278: getdents d4/d1d/d1f/d35 0 2024-04-25T15:49:24.388 INFO:tasks.workunit.client.0.smithi110.stdout:5/279: dwrite d4/f22 [0,4194304] 0 2024-04-25T15:49:24.393 INFO:tasks.workunit.client.0.smithi110.stdout:5/280: rename d4/d1d/d1f/d35/d3d/d52/f59 to d4/d1d/d1f/d35/d3d/d52/f5f 0 2024-04-25T15:49:24.396 INFO:tasks.workunit.client.0.smithi110.stdout:5/281: creat d4/d1d/d1f/d35/d3d/f60 x:0 0 0 2024-04-25T15:49:24.397 INFO:tasks.workunit.client.0.smithi110.stdout:5/282: dread - d4/d1d/d1f/d35/d3d/d52/f5f zero size 2024-04-25T15:49:24.405 INFO:tasks.workunit.client.0.smithi110.stdout:5/283: dwrite d4/d1d/f38 [0,4194304] 0 2024-04-25T15:49:24.406 INFO:tasks.workunit.client.0.smithi110.stdout:5/284: mknod d4/d1d/c61 0 2024-04-25T15:49:24.407 INFO:tasks.workunit.client.0.smithi110.stdout:5/285: mknod d4/d1d/d1f/c62 0 2024-04-25T15:49:24.414 INFO:tasks.workunit.client.0.smithi110.stdout:5/286: dwrite d4/d1d/d1f/d35/d3d/f58 [0,4194304] 0 2024-04-25T15:49:24.417 INFO:tasks.workunit.client.0.smithi110.stdout:5/287: readlink d4/d15/l18 0 2024-04-25T15:49:24.419 INFO:tasks.workunit.client.0.smithi110.stdout:5/288: write d4/d2c/d2a/f5c [627455,8172] 0 2024-04-25T15:49:24.428 INFO:tasks.workunit.client.0.smithi110.stdout:5/289: dwrite d4/d1d/d1f/f47 [0,4194304] 0 2024-04-25T15:49:24.430 INFO:tasks.workunit.client.0.smithi110.stdout:5/290: truncate d4/d2c/f4f 128988 0 2024-04-25T15:49:24.433 INFO:tasks.workunit.client.1.smithi193.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-25T15:49:24.438 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-25T15:49:24.438 INFO:tasks.workunit.client.1.smithi193.stderr:+ make 2024-04-25T15:49:24.440 INFO:tasks.workunit.client.0.smithi110.stdout:5/291: dread d4/d1d/d1f/d35/f39 [0,4194304] 0 2024-04-25T15:49:24.443 INFO:tasks.workunit.client.0.smithi110.stdout:5/292: fsync d4/f10 0 2024-04-25T15:49:24.445 INFO:tasks.workunit.client.0.smithi110.stdout:5/293: write d4/d1d/d1f/d35/f39 [480965,49060] 0 2024-04-25T15:49:24.449 INFO:tasks.workunit.client.0.smithi110.stdout:5/294: mknod d4/d1d/c63 0 2024-04-25T15:49:24.452 INFO:tasks.workunit.client.0.smithi110.stdout:5/295: creat d4/d1d/d1f/f64 x:0 0 0 2024-04-25T15:49:24.452 INFO:tasks.workunit.client.0.smithi110.stdout:5/296: stat d4/d1d/l30 0 2024-04-25T15:49:24.454 INFO:tasks.workunit.client.0.smithi110.stdout:5/297: write d4/f10 [3370794,16070] 0 2024-04-25T15:49:24.455 INFO:tasks.workunit.client.0.smithi110.stdout:5/298: stat d4/d2c/d2a/c3f 0 2024-04-25T15:49:24.456 INFO:tasks.workunit.client.0.smithi110.stdout:5/299: write d4/d2c/d2a/f5c [1278345,123866] 0 2024-04-25T15:49:24.460 INFO:tasks.workunit.client.0.smithi110.stdout:5/300: creat d4/d1d/f65 x:0 0 0 2024-04-25T15:49:24.462 INFO:tasks.workunit.client.0.smithi110.stdout:5/301: mknod d4/d15/c66 0 2024-04-25T15:49:24.474 INFO:tasks.workunit.client.1.smithi193.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-25T15:49:24.504 INFO:tasks.workunit.client.0.smithi110.stdout:7/286: sync 2024-04-25T15:49:24.504 INFO:tasks.workunit.client.0.smithi110.stdout:3/401: sync 2024-04-25T15:49:24.506 INFO:tasks.workunit.client.0.smithi110.stdout:7/287: fsync d1b/d29/d46/f51 0 2024-04-25T15:49:24.508 INFO:tasks.workunit.client.0.smithi110.stdout:7/288: creat d1b/d29/d38/f5e x:0 0 0 2024-04-25T15:49:24.509 INFO:tasks.workunit.client.0.smithi110.stdout:3/402: dread d0/d7/f38 [0,4194304] 0 2024-04-25T15:49:24.510 INFO:tasks.workunit.client.0.smithi110.stdout:7/289: mknod d1b/c5f 0 2024-04-25T15:49:24.510 INFO:tasks.workunit.client.0.smithi110.stdout:3/403: mkdir d0/d2e/d88 0 2024-04-25T15:49:24.511 INFO:tasks.workunit.client.0.smithi110.stdout:7/290: creat d1b/d29/f60 x:0 0 0 2024-04-25T15:49:24.513 INFO:tasks.workunit.client.0.smithi110.stdout:7/291: creat d1b/d29/d38/f61 x:0 0 0 2024-04-25T15:49:24.513 INFO:tasks.workunit.client.0.smithi110.stdout:3/404: truncate d0/f12 3533897 0 2024-04-25T15:49:24.516 INFO:tasks.workunit.client.0.smithi110.stdout:3/405: creat d0/d7/d1c/d2f/d85/f89 x:0 0 0 2024-04-25T15:49:24.518 INFO:tasks.workunit.client.0.smithi110.stdout:3/406: mknod d0/d2e/d88/c8a 0 2024-04-25T15:49:24.519 INFO:tasks.workunit.client.0.smithi110.stdout:3/407: write d0/d7/d1c/d50/f59 [619255,47522] 0 2024-04-25T15:49:24.520 INFO:tasks.workunit.client.0.smithi110.stdout:7/292: write d1b/f45 [1211104,16383] 0 2024-04-25T15:49:24.521 INFO:tasks.workunit.client.0.smithi110.stdout:7/293: dread - d1b/d29/d38/f5e zero size 2024-04-25T15:49:24.524 INFO:tasks.workunit.client.0.smithi110.stdout:7/294: mkdir d1b/d29/d38/d62 0 2024-04-25T15:49:24.525 INFO:tasks.workunit.client.0.smithi110.stdout:7/295: fsync f14 0 2024-04-25T15:49:24.526 INFO:tasks.workunit.client.0.smithi110.stdout:3/408: dwrite d0/d16/f18 [0,4194304] 0 2024-04-25T15:49:24.529 INFO:tasks.workunit.client.0.smithi110.stdout:3/409: creat d0/d7/f8b x:0 0 0 2024-04-25T15:49:24.535 INFO:tasks.workunit.client.0.smithi110.stdout:3/410: dwrite d0/d7/d7d/f81 [0,4194304] 0 2024-04-25T15:49:24.538 INFO:tasks.workunit.client.0.smithi110.stdout:3/411: mkdir d0/d16/d8c 0 2024-04-25T15:49:24.540 INFO:tasks.workunit.client.0.smithi110.stdout:3/412: symlink d0/d7/d1c/d2f/d85/l8d 0 2024-04-25T15:49:24.542 INFO:tasks.workunit.client.0.smithi110.stdout:3/413: dread d0/d7/f38 [0,4194304] 0 2024-04-25T15:49:24.543 INFO:tasks.workunit.client.0.smithi110.stdout:3/414: fdatasync d0/d16/f3d 0 2024-04-25T15:49:24.543 INFO:tasks.workunit.client.0.smithi110.stdout:3/415: write d0/d7/d1c/d2f/d33/f44 [218984,31738] 0 2024-04-25T15:49:24.547 INFO:tasks.workunit.client.0.smithi110.stdout:3/416: link d0/d7/d1c/d2f/d4c/l65 d0/d16/d8c/l8e 0 2024-04-25T15:49:24.548 INFO:tasks.workunit.client.0.smithi110.stdout:3/417: write d0/f70 [203060,50173] 0 2024-04-25T15:49:24.551 INFO:tasks.workunit.client.0.smithi110.stdout:3/418: rename d0/d2e/d88/c8a to d0/d7/c8f 0 2024-04-25T15:49:24.559 INFO:tasks.workunit.client.0.smithi110.stdout:3/419: dread d0/d7/d1c/f56 [0,4194304] 0 2024-04-25T15:49:24.561 INFO:tasks.workunit.client.0.smithi110.stdout:3/420: mkdir d0/d16/d8c/d90 0 2024-04-25T15:49:24.563 INFO:tasks.workunit.client.0.smithi110.stdout:3/421: unlink d0/d7/c1a 0 2024-04-25T15:49:24.564 INFO:tasks.workunit.client.0.smithi110.stdout:3/422: chown d0/d16/d8c 10221724 1 2024-04-25T15:49:24.573 INFO:tasks.workunit.client.0.smithi110.stdout:3/423: dread d0/d7/d1c/f35 [0,4194304] 0 2024-04-25T15:49:24.573 INFO:tasks.workunit.client.0.smithi110.stdout:3/424: stat d0/d2e/f71 0 2024-04-25T15:49:24.576 INFO:tasks.workunit.client.0.smithi110.stdout:3/425: mknod d0/d7/d1c/c91 0 2024-04-25T15:49:24.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:24 smithi193 ceph-mon[33747]: pgmap v29: 65 pgs: 65 active+clean; 859 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 17 MiB/s rd, 112 MiB/s wr, 419 op/s 2024-04-25T15:49:24.806 INFO:tasks.workunit.client.0.smithi110.stdout:1/399: sync 2024-04-25T15:49:24.806 INFO:tasks.workunit.client.0.smithi110.stdout:9/261: sync 2024-04-25T15:49:24.811 INFO:tasks.workunit.client.0.smithi110.stdout:9/262: write d9/db/df/d2c/d3c/f49 [1750482,57695] 0 2024-04-25T15:49:24.813 INFO:tasks.workunit.client.0.smithi110.stdout:9/263: dread - d9/db/df/d25/f37 zero size 2024-04-25T15:49:24.813 INFO:tasks.workunit.client.0.smithi110.stdout:9/264: mkdir d9/db/df/d55 0 2024-04-25T15:49:24.814 INFO:tasks.workunit.client.0.smithi110.stdout:9/265: read d9/db/df/f33 [450030,120385] 0 2024-04-25T15:49:24.815 INFO:tasks.workunit.client.0.smithi110.stdout:9/266: dread - d9/db/df/d25/f37 zero size 2024-04-25T15:49:24.816 INFO:tasks.workunit.client.0.smithi110.stdout:9/267: chown d9/db/df 883896 1 2024-04-25T15:49:24.831 INFO:tasks.workunit.client.0.smithi110.stdout:9/268: dread d9/db/f1b [4194304,4194304] 0 2024-04-25T15:49:24.832 INFO:tasks.workunit.client.0.smithi110.stdout:9/269: readlink d9/db/df/d2c/d3c/l4e 0 2024-04-25T15:49:24.860 INFO:tasks.workunit.client.0.smithi110.stdout:1/400: dwrite d0/d51/f52 [0,4194304] 0 2024-04-25T15:49:24.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:24 smithi110 ceph-mon[26344]: pgmap v29: 65 pgs: 65 active+clean; 859 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 17 MiB/s rd, 112 MiB/s wr, 419 op/s 2024-04-25T15:49:24.863 INFO:tasks.workunit.client.0.smithi110.stdout:4/376: sync 2024-04-25T15:49:24.863 INFO:tasks.workunit.client.0.smithi110.stdout:0/346: sync 2024-04-25T15:49:24.865 INFO:tasks.workunit.client.0.smithi110.stdout:0/347: write d1/d23/d30/d33/f2d [520289,34224] 0 2024-04-25T15:49:24.865 INFO:tasks.workunit.client.0.smithi110.stdout:4/377: rmdir d13/d24/d2d/d5e 39 2024-04-25T15:49:24.868 INFO:tasks.workunit.client.0.smithi110.stdout:0/348: chown d1/d23/d30/d33/f88 14334 1 2024-04-25T15:49:24.868 INFO:tasks.workunit.client.0.smithi110.stdout:1/401: dwrite d0/d14/f2c [4194304,4194304] 0 2024-04-25T15:49:24.875 INFO:tasks.workunit.client.0.smithi110.stdout:1/402: fdatasync d0/d25/f6 0 2024-04-25T15:49:24.877 INFO:tasks.workunit.client.0.smithi110.stdout:1/403: readlink d0/d25/d33/l42 0 2024-04-25T15:49:24.877 INFO:tasks.workunit.client.0.smithi110.stdout:4/378: creat d13/d24/d2d/d44/d65/f72 x:0 0 0 2024-04-25T15:49:24.878 INFO:tasks.workunit.client.0.smithi110.stdout:0/349: dwrite d1/d47/f7b [0,4194304] 0 2024-04-25T15:49:24.880 INFO:tasks.workunit.client.0.smithi110.stdout:0/350: chown d1/d6c/d7f/c7a 1680758 1 2024-04-25T15:49:24.881 INFO:tasks.workunit.client.0.smithi110.stdout:1/404: stat d0/d14/d5f/d6f 0 2024-04-25T15:49:24.884 INFO:tasks.workunit.client.0.smithi110.stdout:0/351: truncate d1/d23/f24 433100 0 2024-04-25T15:49:24.887 INFO:tasks.workunit.client.0.smithi110.stdout:0/352: readlink d1/l1e 0 2024-04-25T15:49:24.888 INFO:tasks.workunit.client.0.smithi110.stdout:4/379: dwrite d13/d24/d2d/d36/f4d [0,4194304] 0 2024-04-25T15:49:24.889 INFO:tasks.workunit.client.0.smithi110.stdout:4/380: readlink d13/l21 0 2024-04-25T15:49:24.892 INFO:tasks.workunit.client.0.smithi110.stdout:1/405: dwrite d0/d51/f79 [0,4194304] 0 2024-04-25T15:49:24.894 INFO:tasks.workunit.client.0.smithi110.stdout:1/406: write d0/d51/d6b/f7c [484090,49652] 0 2024-04-25T15:49:24.900 INFO:tasks.workunit.client.0.smithi110.stdout:0/353: link d1/f32 d1/f8d 0 2024-04-25T15:49:24.901 INFO:tasks.workunit.client.0.smithi110.stdout:4/381: getdents d13/d24/d26 0 2024-04-25T15:49:24.902 INFO:tasks.workunit.client.0.smithi110.stdout:0/354: stat d1/d23/f24 0 2024-04-25T15:49:24.903 INFO:tasks.workunit.client.0.smithi110.stdout:4/382: stat d13/d24/d2d/d36/c5c 0 2024-04-25T15:49:24.906 INFO:tasks.workunit.client.0.smithi110.stdout:4/383: creat d13/d24/d2d/d44/f73 x:0 0 0 2024-04-25T15:49:24.909 INFO:tasks.workunit.client.0.smithi110.stdout:0/355: rename d1/d23/d30/d33/d35/f41 to d1/d47/f8e 0 2024-04-25T15:49:24.911 INFO:tasks.workunit.client.0.smithi110.stdout:4/384: link d13/d24/f48 d13/d24/d2d/d36/f74 0 2024-04-25T15:49:24.912 INFO:tasks.workunit.client.0.smithi110.stdout:0/356: mknod d1/c8f 0 2024-04-25T15:49:24.913 INFO:tasks.workunit.client.0.smithi110.stdout:0/357: readlink d1/d23/d30/d33/d5b/l63 0 2024-04-25T15:49:24.913 INFO:tasks.workunit.client.0.smithi110.stdout:4/385: mknod d13/d24/d3c/d56/c75 0 2024-04-25T15:49:24.916 INFO:tasks.workunit.client.0.smithi110.stdout:0/358: read d1/fd [329607,68378] 0 2024-04-25T15:49:24.918 INFO:tasks.workunit.client.0.smithi110.stdout:4/386: rename d13/d24/d26/f31 to d13/d24/d3c/d56/f76 0 2024-04-25T15:49:24.921 INFO:tasks.workunit.client.0.smithi110.stdout:4/387: creat d13/f77 x:0 0 0 2024-04-25T15:49:24.922 INFO:tasks.workunit.client.0.smithi110.stdout:1/407: dread d0/f3f [4194304,4194304] 0 2024-04-25T15:49:24.923 INFO:tasks.workunit.client.0.smithi110.stdout:4/388: creat d13/d24/d2d/d36/d51/f78 x:0 0 0 2024-04-25T15:49:24.924 INFO:tasks.workunit.client.0.smithi110.stdout:1/408: write d0/d14/f57 [150030,3071] 0 2024-04-25T15:49:24.925 INFO:tasks.workunit.client.0.smithi110.stdout:1/409: dread - d0/d23/f3b zero size 2024-04-25T15:49:24.928 INFO:tasks.workunit.client.0.smithi110.stdout:1/410: mknod d0/d14/d5f/d7d/d72/c80 0 2024-04-25T15:49:24.930 INFO:tasks.workunit.client.0.smithi110.stdout:4/389: dwrite d13/d24/f58 [0,4194304] 0 2024-04-25T15:49:24.931 INFO:tasks.workunit.client.0.smithi110.stdout:1/411: truncate d0/f18 2618935 0 2024-04-25T15:49:24.935 INFO:tasks.workunit.client.0.smithi110.stdout:1/412: write d0/d14/d5f/d7d/f64 [669920,29783] 0 2024-04-25T15:49:24.937 INFO:tasks.workunit.client.0.smithi110.stdout:0/359: dwrite d1/d47/d54/f71 [0,4194304] 0 2024-04-25T15:49:24.939 INFO:tasks.workunit.client.0.smithi110.stdout:0/360: fdatasync d1/d23/d30/d33/d5b/d83/f5d 0 2024-04-25T15:49:24.940 INFO:tasks.workunit.client.0.smithi110.stdout:0/361: stat d1/d23/d30/d33/c2e 0 2024-04-25T15:49:24.941 INFO:tasks.workunit.client.0.smithi110.stdout:0/362: fdatasync d1/d23/d30/d33/d35/f12 0 2024-04-25T15:49:24.942 INFO:tasks.workunit.client.0.smithi110.stdout:4/390: creat d13/d24/d2d/f79 x:0 0 0 2024-04-25T15:49:24.946 INFO:tasks.workunit.client.0.smithi110.stdout:4/391: creat d13/d24/d3c/f7a x:0 0 0 2024-04-25T15:49:24.947 INFO:tasks.workunit.client.0.smithi110.stdout:0/363: truncate d1/d23/f44 428352 0 2024-04-25T15:49:24.948 INFO:tasks.workunit.client.0.smithi110.stdout:4/392: write d13/d24/d26/f49 [4363413,879] 0 2024-04-25T15:49:24.953 INFO:tasks.workunit.client.0.smithi110.stdout:0/364: rename d1/d23/d81/c8c to d1/d23/c90 0 2024-04-25T15:49:24.954 INFO:tasks.workunit.client.0.smithi110.stdout:1/413: dwrite d0/f27 [0,4194304] 0 2024-04-25T15:49:24.955 INFO:tasks.workunit.client.0.smithi110.stdout:4/393: creat d13/d24/d2d/d5e/f7b x:0 0 0 2024-04-25T15:49:24.955 INFO:tasks.workunit.client.0.smithi110.stdout:1/414: chown d0/d23/f3b 130441561 1 2024-04-25T15:49:24.956 INFO:tasks.workunit.client.0.smithi110.stdout:4/394: write d13/d1c/f4c [2329683,21088] 0 2024-04-25T15:49:24.956 INFO:tasks.workunit.client.0.smithi110.stdout:4/395: stat d13/l3f 0 2024-04-25T15:49:24.957 INFO:tasks.workunit.client.0.smithi110.stdout:0/365: creat d1/d20/d69/f91 x:0 0 0 2024-04-25T15:49:24.958 INFO:tasks.workunit.client.0.smithi110.stdout:4/396: write fa [5869075,62944] 0 2024-04-25T15:49:24.958 INFO:tasks.workunit.client.0.smithi110.stdout:0/366: chown d1/d20/l39 6280 1 2024-04-25T15:49:24.959 INFO:tasks.workunit.client.0.smithi110.stdout:1/415: fdatasync d0/d14/f1d 0 2024-04-25T15:49:24.960 INFO:tasks.workunit.client.0.smithi110.stdout:1/416: chown d0/d14/f1f 0 1 2024-04-25T15:49:24.962 INFO:tasks.workunit.client.0.smithi110.stdout:4/397: rename d13/d1c/f25 to d13/d24/d2d/d5e/f7c 0 2024-04-25T15:49:24.963 INFO:tasks.workunit.client.0.smithi110.stdout:0/367: mknod d1/d23/d30/d33/d35/c92 0 2024-04-25T15:49:24.964 INFO:tasks.workunit.client.0.smithi110.stdout:4/398: write d13/d24/d2d/f79 [444704,47528] 0 2024-04-25T15:49:24.966 INFO:tasks.workunit.client.0.smithi110.stdout:1/417: mkdir d0/d14/d81 0 2024-04-25T15:49:24.972 INFO:tasks.workunit.client.0.smithi110.stdout:4/399: mkdir d13/d24/d2d/d7d 0 2024-04-25T15:49:24.973 INFO:tasks.workunit.client.0.smithi110.stdout:1/418: getdents d0/d14/d81 0 2024-04-25T15:49:24.973 INFO:tasks.workunit.client.0.smithi110.stdout:4/400: write d13/d24/d2d/d36/d51/f78 [370930,40584] 0 2024-04-25T15:49:24.974 INFO:tasks.workunit.client.0.smithi110.stdout:1/419: readlink d0/d25/l40 0 2024-04-25T15:49:24.977 INFO:tasks.workunit.client.0.smithi110.stdout:4/401: mknod d13/d24/d2d/d5e/c7e 0 2024-04-25T15:49:24.979 INFO:tasks.workunit.client.0.smithi110.stdout:1/420: getdents d0/d75 0 2024-04-25T15:49:24.980 INFO:tasks.workunit.client.0.smithi110.stdout:0/368: dread d1/f16 [0,4194304] 0 2024-04-25T15:49:24.981 INFO:tasks.workunit.client.0.smithi110.stdout:4/402: dread f8 [0,4194304] 0 2024-04-25T15:49:24.982 INFO:tasks.workunit.client.0.smithi110.stdout:1/421: fdatasync d0/d25/d33/f45 0 2024-04-25T15:49:24.983 INFO:tasks.workunit.client.0.smithi110.stdout:0/369: symlink d1/d23/d30/d33/d35/l93 0 2024-04-25T15:49:24.986 INFO:tasks.workunit.client.0.smithi110.stdout:4/403: rename d13/d24/d3c/d56/d6c to d13/d24/d26/d7f 0 2024-04-25T15:49:24.989 INFO:tasks.workunit.client.0.smithi110.stdout:1/422: dwrite d0/d23/f3b [0,4194304] 0 2024-04-25T15:49:24.989 INFO:tasks.workunit.client.0.smithi110.stdout:4/404: fdatasync d13/d24/d3c/d56/f76 0 2024-04-25T15:49:24.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/370: rename d1/d20/f43 to d1/d6c/f94 0 2024-04-25T15:49:24.993 INFO:tasks.workunit.client.0.smithi110.stdout:4/405: mknod d13/d1c/c80 0 2024-04-25T15:49:24.995 INFO:tasks.workunit.client.0.smithi110.stdout:1/423: creat d0/d14/d5f/d6f/f82 x:0 0 0 2024-04-25T15:49:24.997 INFO:tasks.workunit.client.0.smithi110.stdout:4/406: rename d13/d24/d2d/d5e/f7c to d13/d24/d2d/d7d/f81 0 2024-04-25T15:49:25.000 INFO:tasks.workunit.client.0.smithi110.stdout:1/424: creat d0/d23/f83 x:0 0 0 2024-04-25T15:49:25.003 INFO:tasks.workunit.client.0.smithi110.stdout:4/407: dwrite d13/d1c/f47 [0,4194304] 0 2024-04-25T15:49:25.005 INFO:tasks.workunit.client.0.smithi110.stdout:2/290: sync 2024-04-25T15:49:25.005 INFO:tasks.workunit.client.0.smithi110.stdout:7/296: sync 2024-04-25T15:49:25.005 INFO:tasks.workunit.client.0.smithi110.stdout:5/302: sync 2024-04-25T15:49:25.005 INFO:tasks.workunit.client.0.smithi110.stdout:6/286: sync 2024-04-25T15:49:25.005 INFO:tasks.workunit.client.0.smithi110.stdout:8/423: sync 2024-04-25T15:49:25.005 INFO:tasks.workunit.client.0.smithi110.stdout:3/426: sync 2024-04-25T15:49:25.007 INFO:tasks.workunit.client.0.smithi110.stdout:4/408: fdatasync ff 0 2024-04-25T15:49:25.007 INFO:tasks.workunit.client.0.smithi110.stdout:7/297: chown d1b/d29/d42 161371310 1 2024-04-25T15:49:25.011 INFO:tasks.workunit.client.0.smithi110.stdout:1/425: dwrite d0/d25/d33/f47 [0,4194304] 0 2024-04-25T15:49:25.012 INFO:tasks.workunit.client.0.smithi110.stdout:8/424: mknod d14/d21/d26/d6a/d7e/c8a 0 2024-04-25T15:49:25.014 INFO:tasks.workunit.client.0.smithi110.stdout:5/303: creat d4/d1d/d1f/d56/f67 x:0 0 0 2024-04-25T15:49:25.015 INFO:tasks.workunit.client.0.smithi110.stdout:3/427: read - d0/d7/d40/f7e zero size 2024-04-25T15:49:25.016 INFO:tasks.workunit.client.0.smithi110.stdout:6/287: creat d1/d48/f6c x:0 0 0 2024-04-25T15:49:25.017 INFO:tasks.workunit.client.0.smithi110.stdout:1/426: mknod d0/d51/c84 0 2024-04-25T15:49:25.019 INFO:tasks.workunit.client.0.smithi110.stdout:8/425: symlink d14/l8b 0 2024-04-25T15:49:25.020 INFO:tasks.workunit.client.0.smithi110.stdout:3/428: write d0/d2e/f4f [665879,24083] 0 2024-04-25T15:49:25.021 INFO:tasks.workunit.client.0.smithi110.stdout:6/288: fsync d1/d12/d3d/f4a 0 2024-04-25T15:49:25.023 INFO:tasks.workunit.client.0.smithi110.stdout:8/426: readlink d14/l63 0 2024-04-25T15:49:25.024 INFO:tasks.workunit.client.0.smithi110.stdout:1/427: chown d0/d25/c34 31804884 1 2024-04-25T15:49:25.027 INFO:tasks.workunit.client.0.smithi110.stdout:2/291: dwrite de/d11/f21 [0,4194304] 0 2024-04-25T15:49:25.031 INFO:tasks.workunit.client.0.smithi110.stdout:6/289: mkdir d1/d12/d6d 0 2024-04-25T15:49:25.034 INFO:tasks.workunit.client.0.smithi110.stdout:8/427: fsync fb 0 2024-04-25T15:49:25.037 INFO:tasks.workunit.client.0.smithi110.stdout:7/298: dwrite d1b/d29/d46/f47 [0,4194304] 0 2024-04-25T15:49:25.039 INFO:tasks.workunit.client.0.smithi110.stdout:4/409: dwrite d13/d24/d2d/d5e/f7b [0,4194304] 0 2024-04-25T15:49:25.042 INFO:tasks.workunit.client.0.smithi110.stdout:5/304: dwrite d4/d2c/d2a/f46 [0,4194304] 0 2024-04-25T15:49:25.043 INFO:tasks.workunit.client.0.smithi110.stdout:7/299: fdatasync d1b/d29/d38/f61 0 2024-04-25T15:49:25.045 INFO:tasks.workunit.client.0.smithi110.stdout:4/410: readlink d13/d24/l5f 0 2024-04-25T15:49:25.046 INFO:tasks.workunit.client.0.smithi110.stdout:7/300: chown d1b/d29/d46 32 1 2024-04-25T15:49:25.048 INFO:tasks.workunit.client.0.smithi110.stdout:8/428: symlink d14/d21/d26/d6a/d7e/l8c 0 2024-04-25T15:49:25.049 INFO:tasks.workunit.client.0.smithi110.stdout:4/411: chown d13/d24/d2d/d36 0 1 2024-04-25T15:49:25.051 INFO:tasks.workunit.client.0.smithi110.stdout:5/305: mknod d4/d1d/d1f/d35/d3d/c68 0 2024-04-25T15:49:25.053 INFO:tasks.workunit.client.0.smithi110.stdout:8/429: mkdir d14/d4c/d55/d8d 0 2024-04-25T15:49:25.055 INFO:tasks.workunit.client.0.smithi110.stdout:4/412: fdatasync d13/d24/d2d/d36/f4d 0 2024-04-25T15:49:25.055 INFO:tasks.workunit.client.0.smithi110.stdout:5/306: symlink d4/d1d/l69 0 2024-04-25T15:49:25.056 INFO:tasks.workunit.client.0.smithi110.stdout:1/428: dwrite d0/d14/d5f/d6f/f6e [0,4194304] 0 2024-04-25T15:49:25.056 INFO:tasks.workunit.client.0.smithi110.stdout:6/290: dwrite d1/db/d22/f38 [0,4194304] 0 2024-04-25T15:49:25.061 INFO:tasks.workunit.client.0.smithi110.stdout:5/307: read d4/d1d/d1f/d35/f39 [1831282,55874] 0 2024-04-25T15:49:25.062 INFO:tasks.workunit.client.0.smithi110.stdout:8/430: dread - d14/d21/d23/f2e zero size 2024-04-25T15:49:25.063 INFO:tasks.workunit.client.0.smithi110.stdout:1/429: read d0/d25/f41 [977308,11804] 0 2024-04-25T15:49:25.064 INFO:tasks.workunit.client.0.smithi110.stdout:6/291: unlink d1/f1d 0 2024-04-25T15:49:25.067 INFO:tasks.workunit.client.0.smithi110.stdout:7/301: write d1b/f36 [4006925,104630] 0 2024-04-25T15:49:25.069 INFO:tasks.workunit.client.0.smithi110.stdout:7/302: write f4 [4412230,124169] 0 2024-04-25T15:49:25.072 INFO:tasks.workunit.client.0.smithi110.stdout:5/308: truncate d4/d1d/d1f/d35/d3d/d52/f55 883113 0 2024-04-25T15:49:25.073 INFO:tasks.workunit.client.0.smithi110.stdout:7/303: truncate d1b/d29/f60 1022094 0 2024-04-25T15:49:25.073 INFO:tasks.workunit.client.0.smithi110.stdout:7/304: fdatasync f19 0 2024-04-25T15:49:25.073 INFO:tasks.workunit.client.0.smithi110.stdout:4/413: link d13/d24/f58 d13/d24/d3c/d56/f82 0 2024-04-25T15:49:25.074 INFO:tasks.workunit.client.0.smithi110.stdout:6/292: symlink d1/db/d22/d33/d43/l6e 0 2024-04-25T15:49:25.075 INFO:tasks.workunit.client.0.smithi110.stdout:8/431: dwrite d14/f5c [0,4194304] 0 2024-04-25T15:49:25.078 INFO:tasks.workunit.client.0.smithi110.stdout:6/293: fdatasync d1/db/d22/f2c 0 2024-04-25T15:49:25.079 INFO:tasks.workunit.client.0.smithi110.stdout:4/414: rename d13/d24/d2d/d55/l67 to d13/d24/d2d/d5e/l83 0 2024-04-25T15:49:25.081 INFO:tasks.workunit.client.0.smithi110.stdout:7/305: mknod d1b/d29/d42/c63 0 2024-04-25T15:49:25.082 INFO:tasks.workunit.client.0.smithi110.stdout:8/432: write f5 [4714380,38565] 0 2024-04-25T15:49:25.083 INFO:tasks.workunit.client.0.smithi110.stdout:6/294: truncate d1/dd/d3c/f5d 5416 0 2024-04-25T15:49:25.086 INFO:tasks.workunit.client.0.smithi110.stdout:6/295: write d1/d48/f6c [750152,92788] 0 2024-04-25T15:49:25.089 INFO:tasks.workunit.client.0.smithi110.stdout:6/296: readlink d1/d12/l4f 0 2024-04-25T15:49:25.089 INFO:tasks.workunit.client.0.smithi110.stdout:7/306: creat d1b/d29/d42/f64 x:0 0 0 2024-04-25T15:49:25.090 INFO:tasks.workunit.client.0.smithi110.stdout:5/309: dwrite d4/d1d/d1f/d35/d3d/f58 [4194304,4194304] 0 2024-04-25T15:49:25.091 INFO:tasks.workunit.client.0.smithi110.stdout:8/433: dread d14/f2d [0,4194304] 0 2024-04-25T15:49:25.092 INFO:tasks.workunit.client.0.smithi110.stdout:5/310: chown d4/c7 776 1 2024-04-25T15:49:25.093 INFO:tasks.workunit.client.0.smithi110.stdout:8/434: dread - d14/d21/d26/f7a zero size 2024-04-25T15:49:25.093 INFO:tasks.workunit.client.0.smithi110.stdout:6/297: write d1/db/d22/d33/d43/f4e [515074,27343] 0 2024-04-25T15:49:25.097 INFO:tasks.workunit.client.0.smithi110.stdout:5/311: unlink d4/d1d/d1f/d35/d3d/l40 0 2024-04-25T15:49:25.099 INFO:tasks.workunit.client.0.smithi110.stdout:8/435: mknod d14/d4c/d55/c8e 0 2024-04-25T15:49:25.099 INFO:tasks.workunit.client.0.smithi110.stdout:1/430: dread d0/d23/f55 [0,4194304] 0 2024-04-25T15:49:25.100 INFO:tasks.workunit.client.0.smithi110.stdout:7/307: getdents d1b/d29/d46 0 2024-04-25T15:49:25.102 INFO:tasks.workunit.client.0.smithi110.stdout:6/298: creat d1/db/d22/d33/d43/f6f x:0 0 0 2024-04-25T15:49:25.103 INFO:tasks.workunit.client.0.smithi110.stdout:6/299: dread - d1/db/d22/f57 zero size 2024-04-25T15:49:25.105 INFO:tasks.workunit.client.0.smithi110.stdout:8/436: mkdir d14/d32/d36/d48/d8f 0 2024-04-25T15:49:25.107 INFO:tasks.workunit.client.0.smithi110.stdout:7/308: mknod d1b/d29/d38/c65 0 2024-04-25T15:49:25.107 INFO:tasks.workunit.client.0.smithi110.stdout:5/312: getdents d4/d15/d51 0 2024-04-25T15:49:25.109 INFO:tasks.workunit.client.0.smithi110.stdout:5/313: dread - d4/d1d/f65 zero size 2024-04-25T15:49:25.110 INFO:tasks.workunit.client.0.smithi110.stdout:6/300: mknod d1/d48/c70 0 2024-04-25T15:49:25.113 INFO:tasks.workunit.client.0.smithi110.stdout:7/309: symlink d1b/d29/d46/l66 0 2024-04-25T15:49:25.115 INFO:tasks.workunit.client.0.smithi110.stdout:8/437: dread d14/f1d [0,4194304] 0 2024-04-25T15:49:25.115 INFO:tasks.workunit.client.0.smithi110.stdout:4/415: dread d13/d1c/f4c [0,4194304] 0 2024-04-25T15:49:25.119 INFO:tasks.workunit.client.0.smithi110.stdout:7/310: mkdir d1b/d29/d67 0 2024-04-25T15:49:25.121 INFO:tasks.workunit.client.0.smithi110.stdout:1/431: getdents d0/d25 0 2024-04-25T15:49:25.123 INFO:tasks.workunit.client.0.smithi110.stdout:5/314: dread d4/d1d/d1f/f32 [0,4194304] 0 2024-04-25T15:49:25.124 INFO:tasks.workunit.client.0.smithi110.stdout:6/301: mknod d1/db/d22/d33/d63/c71 0 2024-04-25T15:49:25.125 INFO:tasks.workunit.client.0.smithi110.stdout:5/315: write d4/d1d/f65 [706307,29206] 0 2024-04-25T15:49:25.126 INFO:tasks.workunit.client.0.smithi110.stdout:4/416: creat d13/d24/d2d/d44/d65/f84 x:0 0 0 2024-04-25T15:49:25.126 INFO:tasks.workunit.client.0.smithi110.stdout:1/432: stat d0/d51/d6b/c70 0 2024-04-25T15:49:25.130 INFO:tasks.workunit.client.0.smithi110.stdout:8/438: getdents d14/d32/d36/d48 0 2024-04-25T15:49:25.131 INFO:tasks.workunit.client.0.smithi110.stdout:5/316: creat d4/d1d/d1f/d35/f6a x:0 0 0 2024-04-25T15:49:25.133 INFO:tasks.workunit.client.0.smithi110.stdout:4/417: creat d13/d24/d2d/d36/d51/f85 x:0 0 0 2024-04-25T15:49:25.133 INFO:tasks.workunit.client.0.smithi110.stdout:6/302: read - d1/db/d24/d2d/f34 zero size 2024-04-25T15:49:25.134 INFO:tasks.workunit.client.0.smithi110.stdout:5/317: mkdir d4/d1d/d1f/d56/d6b 0 2024-04-25T15:49:25.136 INFO:tasks.workunit.client.0.smithi110.stdout:8/439: symlink d14/d4c/d71/l90 0 2024-04-25T15:49:25.138 INFO:tasks.workunit.client.0.smithi110.stdout:1/433: link d0/d14/c73 d0/d14/d5f/d7d/d72/c85 0 2024-04-25T15:49:25.139 INFO:tasks.workunit.client.0.smithi110.stdout:8/440: creat d14/d4c/f91 x:0 0 0 2024-04-25T15:49:25.140 INFO:tasks.workunit.client.0.smithi110.stdout:8/441: chown d14/d4c/d55/f7d 1073 1 2024-04-25T15:49:25.140 INFO:tasks.workunit.client.0.smithi110.stdout:4/418: symlink d13/l86 0 2024-04-25T15:49:25.143 INFO:tasks.workunit.client.0.smithi110.stdout:6/303: link d1/fe d1/db/d22/d27/f72 0 2024-04-25T15:49:25.144 INFO:tasks.workunit.client.0.smithi110.stdout:1/434: symlink d0/d25/d33/l86 0 2024-04-25T15:49:25.145 INFO:tasks.workunit.client.0.smithi110.stdout:1/435: chown d0/d14 0 1 2024-04-25T15:49:25.146 INFO:tasks.workunit.client.0.smithi110.stdout:4/419: symlink d13/d24/l87 0 2024-04-25T15:49:25.147 INFO:tasks.workunit.client.0.smithi110.stdout:4/420: fsync ff 0 2024-04-25T15:49:25.148 INFO:tasks.workunit.client.0.smithi110.stdout:1/436: symlink d0/d14/d5f/l87 0 2024-04-25T15:49:25.150 INFO:tasks.workunit.client.0.smithi110.stdout:4/421: read d13/d24/d2d/f4f [1538480,22495] 0 2024-04-25T15:49:25.150 INFO:tasks.workunit.client.0.smithi110.stdout:1/437: truncate d0/d25/d33/f47 5124098 0 2024-04-25T15:49:25.153 INFO:tasks.workunit.client.0.smithi110.stdout:4/422: truncate d13/d24/d2d/d36/d51/f85 665176 0 2024-04-25T15:49:25.156 INFO:tasks.workunit.client.0.smithi110.stdout:5/318: rename d4/d1d/d1f/d35/d3d/d52/f5f to d4/d15/d51/f6c 0 2024-04-25T15:49:25.158 INFO:tasks.workunit.client.0.smithi110.stdout:4/423: rename d13/d1c to d13/d24/d3c/d56/d88 0 2024-04-25T15:49:25.211 INFO:tasks.workunit.client.0.smithi110.stdout:9/270: sync 2024-04-25T15:49:25.233 INFO:tasks.workunit.client.0.smithi110.stdout:4/424: mknod d13/d24/d2d/d44/c89 0 2024-04-25T15:49:25.234 INFO:tasks.workunit.client.0.smithi110.stdout:6/304: truncate d1/db/d22/d33/f56 3192735 0 2024-04-25T15:49:25.235 INFO:tasks.workunit.client.0.smithi110.stdout:6/305: readlink d1/db/d22/l2e 0 2024-04-25T15:49:25.235 INFO:tasks.workunit.client.0.smithi110.stdout:4/425: read d13/d24/d26/f49 [4124490,127016] 0 2024-04-25T15:49:25.236 INFO:tasks.workunit.client.0.smithi110.stdout:4/426: chown d13/d24/d3c/d56/d88 28 1 2024-04-25T15:49:25.237 INFO:tasks.workunit.client.0.smithi110.stdout:1/438: write d0/d23/f13 [1497426,26987] 0 2024-04-25T15:49:25.241 INFO:tasks.workunit.client.0.smithi110.stdout:5/319: creat d4/f6d x:0 0 0 2024-04-25T15:49:25.244 INFO:tasks.workunit.client.0.smithi110.stdout:6/306: dwrite d1/d48/f6c [0,4194304] 0 2024-04-25T15:49:25.245 INFO:tasks.workunit.client.0.smithi110.stdout:5/320: creat d4/d1d/d1f/f6e x:0 0 0 2024-04-25T15:49:25.245 INFO:tasks.workunit.client.0.smithi110.stdout:6/307: dread - d1/d12/f5b zero size 2024-04-25T15:49:25.247 INFO:tasks.workunit.client.0.smithi110.stdout:1/439: dwrite d0/fa [0,4194304] 0 2024-04-25T15:49:25.248 INFO:tasks.workunit.client.0.smithi110.stdout:6/308: chown d1/db/d22/d33/d43 11635 1 2024-04-25T15:49:25.248 INFO:tasks.workunit.client.0.smithi110.stdout:4/427: rmdir d13/d24/d3c 39 2024-04-25T15:49:25.248 INFO:tasks.workunit.client.0.smithi110.stdout:5/321: chown d4/d1d/d1f/d35/f6a 273245 1 2024-04-25T15:49:25.249 INFO:tasks.workunit.client.0.smithi110.stdout:5/322: dread - d4/d1d/d1f/d35/f6a zero size 2024-04-25T15:49:25.250 INFO:tasks.workunit.client.0.smithi110.stdout:4/428: read - d13/d24/d2d/d44/d65/f84 zero size 2024-04-25T15:49:25.252 INFO:tasks.workunit.client.0.smithi110.stdout:5/323: stat d4/d1d/d1f/d35/d3d/f58 0 2024-04-25T15:49:25.256 INFO:tasks.workunit.client.0.smithi110.stdout:6/309: rename d1/d12/f68 to d1/d12/d3d/f73 0 2024-04-25T15:49:25.260 INFO:tasks.workunit.client.0.smithi110.stdout:6/310: mknod d1/db/d22/d27/c74 0 2024-04-25T15:49:25.264 INFO:tasks.workunit.client.0.smithi110.stdout:1/440: dread d0/d25/f3d [0,4194304] 0 2024-04-25T15:49:25.265 INFO:tasks.workunit.client.0.smithi110.stdout:4/429: dread ff [0,4194304] 0 2024-04-25T15:49:25.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/430: stat d13/d24/d3c/d56/d88/c80 0 2024-04-25T15:49:25.275 INFO:tasks.workunit.client.0.smithi110.stdout:4/431: write d13/d24/d3c/d56/f82 [2400175,98165] 0 2024-04-25T15:49:25.278 INFO:tasks.workunit.client.0.smithi110.stdout:4/432: mknod d13/d24/d2d/d44/d65/c8a 0 2024-04-25T15:49:25.279 INFO:tasks.workunit.client.0.smithi110.stdout:6/311: dread d1/f9 [0,4194304] 0 2024-04-25T15:49:25.280 INFO:tasks.workunit.client.0.smithi110.stdout:4/433: chown d13/d24/d2d/c6f 16351 1 2024-04-25T15:49:25.280 INFO:tasks.workunit.client.0.smithi110.stdout:1/441: dwrite d0/d23/f5a [0,4194304] 0 2024-04-25T15:49:25.283 INFO:tasks.workunit.client.0.smithi110.stdout:6/312: mkdir d1/db/d24/d69/d75 0 2024-04-25T15:49:25.283 INFO:tasks.workunit.client.0.smithi110.stdout:6/313: readlink d1/db/l35 0 2024-04-25T15:49:25.284 INFO:tasks.workunit.client.0.smithi110.stdout:4/434: mknod d13/d24/d2d/d5e/c8b 0 2024-04-25T15:49:25.285 INFO:tasks.workunit.client.0.smithi110.stdout:6/314: dread - d1/db/d22/d33/d43/f6a zero size 2024-04-25T15:49:25.285 INFO:tasks.workunit.client.0.smithi110.stdout:4/435: truncate fa 6225003 0 2024-04-25T15:49:25.285 INFO:tasks.workunit.client.0.smithi110.stdout:1/442: mkdir d0/d14/d5f/d7d/d72/d88 0 2024-04-25T15:49:25.288 INFO:tasks.workunit.client.0.smithi110.stdout:6/315: mkdir d1/db/d22/d27/d76 0 2024-04-25T15:49:25.291 INFO:tasks.workunit.client.0.smithi110.stdout:6/316: write d1/f9 [2185381,62847] 0 2024-04-25T15:49:25.292 INFO:tasks.workunit.client.0.smithi110.stdout:1/443: dwrite d0/f27 [0,4194304] 0 2024-04-25T15:49:25.293 INFO:tasks.workunit.client.0.smithi110.stdout:6/317: chown d1/db/d22/d33/d43/f4e 393 1 2024-04-25T15:49:25.293 INFO:tasks.workunit.client.0.smithi110.stdout:1/444: read d0/d23/f5a [692548,20805] 0 2024-04-25T15:49:25.294 INFO:tasks.workunit.client.0.smithi110.stdout:1/445: fdatasync d0/d23/f3b 0 2024-04-25T15:49:25.297 INFO:tasks.workunit.client.0.smithi110.stdout:1/446: link d0/d14/c59 d0/d51/c89 0 2024-04-25T15:49:25.297 INFO:tasks.workunit.client.0.smithi110.stdout:1/447: chown d0/l68 38145934 1 2024-04-25T15:49:25.301 INFO:tasks.workunit.client.0.smithi110.stdout:1/448: rename d0/f15 to d0/d23/f8a 0 2024-04-25T15:49:25.304 INFO:tasks.workunit.client.0.smithi110.stdout:1/449: unlink d0/d14/c5d 0 2024-04-25T15:49:25.304 INFO:tasks.workunit.client.0.smithi110.stdout:1/450: chown d0/d14/d5f/f65 47598190 1 2024-04-25T15:49:25.306 INFO:tasks.workunit.client.0.smithi110.stdout:1/451: creat d0/d14/d5f/d7d/d72/d61/f8b x:0 0 0 2024-04-25T15:49:25.308 INFO:tasks.workunit.client.0.smithi110.stdout:1/452: symlink d0/d14/d5f/d7d/l8c 0 2024-04-25T15:49:25.309 INFO:tasks.workunit.client.0.smithi110.stdout:1/453: mknod d0/d51/d6b/c8d 0 2024-04-25T15:49:25.310 INFO:tasks.workunit.client.0.smithi110.stdout:1/454: chown d0/d23/f5c 940 1 2024-04-25T15:49:25.311 INFO:tasks.workunit.client.0.smithi110.stdout:1/455: mkdir d0/d25/d33/d8e 0 2024-04-25T15:49:25.312 INFO:tasks.workunit.client.0.smithi110.stdout:1/456: write d0/d14/f1f [2169929,7054] 0 2024-04-25T15:49:25.318 INFO:tasks.workunit.client.0.smithi110.stdout:1/457: dwrite d0/d51/d6b/f7c [0,4194304] 0 2024-04-25T15:49:25.321 INFO:tasks.workunit.client.0.smithi110.stdout:1/458: getdents d0/d14/d5f/d6f 0 2024-04-25T15:49:25.322 INFO:tasks.workunit.client.0.smithi110.stdout:1/459: chown d0/d23 76 1 2024-04-25T15:49:25.322 INFO:tasks.workunit.client.0.smithi110.stdout:1/460: readlink d0/d14/d5f/d7d/d72/l5b 0 2024-04-25T15:49:25.324 INFO:tasks.workunit.client.0.smithi110.stdout:1/461: read d0/d14/f7a [94960,32741] 0 2024-04-25T15:49:25.326 INFO:tasks.workunit.client.0.smithi110.stdout:1/462: read d0/d25/f41 [114583,16191] 0 2024-04-25T15:49:25.326 INFO:tasks.workunit.client.0.smithi110.stdout:1/463: dread - d0/d23/f83 zero size 2024-04-25T15:49:25.333 INFO:tasks.workunit.client.0.smithi110.stdout:1/464: read d0/d14/f62 [2708406,67069] 0 2024-04-25T15:49:25.333 INFO:tasks.workunit.client.0.smithi110.stdout:1/465: chown d0/d14/l3c 684878371 1 2024-04-25T15:49:25.337 INFO:tasks.workunit.client.0.smithi110.stdout:1/466: getdents d0/d14 0 2024-04-25T15:49:25.338 INFO:tasks.workunit.client.0.smithi110.stdout:1/467: dread d0/d23/f30 [0,4194304] 0 2024-04-25T15:49:25.339 INFO:tasks.workunit.client.0.smithi110.stdout:1/468: creat d0/d14/d81/f8f x:0 0 0 2024-04-25T15:49:25.340 INFO:tasks.workunit.client.0.smithi110.stdout:1/469: chown d0/c28 2890 1 2024-04-25T15:49:25.340 INFO:tasks.workunit.client.0.smithi110.stdout:1/470: write d0/d14/f57 [879634,66611] 0 2024-04-25T15:49:25.341 INFO:tasks.workunit.client.0.smithi110.stdout:1/471: write d0/d14/f4b [2384964,54527] 0 2024-04-25T15:49:25.346 INFO:tasks.workunit.client.0.smithi110.stdout:1/472: truncate d0/f3f 2795278 0 2024-04-25T15:49:25.346 INFO:tasks.workunit.client.0.smithi110.stdout:1/473: chown d0/d25 94179633 1 2024-04-25T15:49:25.348 INFO:tasks.workunit.client.0.smithi110.stdout:1/474: truncate d0/d25/f41 909641 0 2024-04-25T15:49:25.362 INFO:tasks.workunit.client.0.smithi110.stdout:1/475: dread d0/d25/d33/f50 [0,4194304] 0 2024-04-25T15:49:25.362 INFO:tasks.workunit.client.0.smithi110.stdout:1/476: readlink d0/d25/d33/l42 0 2024-04-25T15:49:25.364 INFO:tasks.workunit.client.0.smithi110.stdout:1/477: rename d0/d25/c34 to d0/d14/d81/c90 0 2024-04-25T15:49:25.365 INFO:tasks.workunit.client.0.smithi110.stdout:1/478: chown d0/cf 1342 1 2024-04-25T15:49:25.391 INFO:tasks.workunit.client.0.smithi110.stdout:3/429: sync 2024-04-25T15:49:25.391 INFO:tasks.workunit.client.0.smithi110.stdout:0/371: sync 2024-04-25T15:49:25.392 INFO:tasks.workunit.client.0.smithi110.stdout:0/372: chown d1/d47/f86 0 1 2024-04-25T15:49:25.393 INFO:tasks.workunit.client.0.smithi110.stdout:0/373: write d1/f19 [2454309,30122] 0 2024-04-25T15:49:25.394 INFO:tasks.workunit.client.0.smithi110.stdout:3/430: unlink d0/f12 0 2024-04-25T15:49:25.395 INFO:tasks.workunit.client.0.smithi110.stdout:3/431: write d0/d7/f29 [3427939,20991] 0 2024-04-25T15:49:25.417 INFO:tasks.workunit.client.0.smithi110.stdout:0/374: dwrite d1/d23/d30/f37 [0,4194304] 0 2024-04-25T15:49:25.422 INFO:tasks.workunit.client.0.smithi110.stdout:3/432: dwrite d0/d2e/f71 [0,4194304] 0 2024-04-25T15:49:25.428 INFO:tasks.workunit.client.0.smithi110.stdout:3/433: rename d0/d7/d1c/d2f/d85 to d0/d16/d8c/d90/d92 0 2024-04-25T15:49:25.432 INFO:tasks.workunit.client.0.smithi110.stdout:3/434: creat d0/d16/d8c/d90/d92/f93 x:0 0 0 2024-04-25T15:49:25.436 INFO:tasks.workunit.client.0.smithi110.stdout:3/435: creat d0/d7/d1c/f94 x:0 0 0 2024-04-25T15:49:25.436 INFO:tasks.workunit.client.0.smithi110.stdout:0/375: write d1/d23/d30/d33/f4b [450372,62603] 0 2024-04-25T15:49:25.440 INFO:tasks.workunit.client.0.smithi110.stdout:3/436: dread d0/d7/d1c/f22 [0,4194304] 0 2024-04-25T15:49:25.440 INFO:tasks.workunit.client.0.smithi110.stdout:0/376: unlink d1/d23/d30/f37 0 2024-04-25T15:49:25.442 INFO:tasks.workunit.client.0.smithi110.stdout:3/437: mknod d0/d7/c95 0 2024-04-25T15:49:25.443 INFO:tasks.workunit.client.0.smithi110.stdout:3/438: fsync d0/d7/f68 0 2024-04-25T15:49:25.447 INFO:tasks.workunit.client.0.smithi110.stdout:0/377: dwrite d1/f8b [0,4194304] 0 2024-04-25T15:49:25.448 INFO:tasks.workunit.client.0.smithi110.stdout:0/378: write d1/f38 [46495,122856] 0 2024-04-25T15:49:25.449 INFO:tasks.workunit.client.0.smithi110.stdout:3/439: link d0/d7/d1c/c91 d0/d16/d8c/d90/c96 0 2024-04-25T15:49:25.456 INFO:tasks.workunit.client.0.smithi110.stdout:3/440: dwrite d0/d2e/f76 [0,4194304] 0 2024-04-25T15:49:25.458 INFO:tasks.workunit.client.0.smithi110.stdout:3/441: write d0/d7/f17 [6518425,49252] 0 2024-04-25T15:49:25.460 INFO:tasks.workunit.client.0.smithi110.stdout:3/442: dread - d0/d16/d8c/d90/d92/f93 zero size 2024-04-25T15:49:25.462 INFO:tasks.workunit.client.0.smithi110.stdout:3/443: symlink d0/d16/d8c/d90/l97 0 2024-04-25T15:49:25.464 INFO:tasks.workunit.client.0.smithi110.stdout:3/444: mknod d0/d16/d8c/d90/c98 0 2024-04-25T15:49:25.467 INFO:tasks.workunit.client.0.smithi110.stdout:3/445: mkdir d0/d16/d8c/d90/d99 0 2024-04-25T15:49:25.469 INFO:tasks.workunit.client.0.smithi110.stdout:3/446: creat d0/d7/d1c/d50/f9a x:0 0 0 2024-04-25T15:49:25.471 INFO:tasks.workunit.client.0.smithi110.stdout:0/379: dwrite d1/f2a [0,4194304] 0 2024-04-25T15:49:25.471 INFO:tasks.workunit.client.0.smithi110.stdout:3/447: mknod d0/d7/d1c/c9b 0 2024-04-25T15:49:25.472 INFO:tasks.workunit.client.0.smithi110.stdout:3/448: write d0/d7/f68 [4765023,6865] 0 2024-04-25T15:49:25.475 INFO:tasks.workunit.client.0.smithi110.stdout:0/380: rename d1/d47/f52 to d1/d47/f95 0 2024-04-25T15:49:25.478 INFO:tasks.workunit.client.0.smithi110.stdout:3/449: link d0/d7/d40/l7f d0/d7/d7d/l9c 0 2024-04-25T15:49:25.479 INFO:tasks.workunit.client.0.smithi110.stdout:3/450: mknod d0/d16/d8c/c9d 0 2024-04-25T15:49:25.480 INFO:tasks.workunit.client.0.smithi110.stdout:3/451: chown d0/d7/l4a 3973 1 2024-04-25T15:49:25.482 INFO:tasks.workunit.client.0.smithi110.stdout:0/381: getdents d1/d47 0 2024-04-25T15:49:25.485 INFO:tasks.workunit.client.0.smithi110.stdout:0/382: unlink d1/d6c/d7f/c57 0 2024-04-25T15:49:25.486 INFO:tasks.workunit.client.0.smithi110.stdout:0/383: creat d1/d20/f96 x:0 0 0 2024-04-25T15:49:25.488 INFO:tasks.workunit.client.0.smithi110.stdout:0/384: rename d1/d23/c34 to d1/d23/d30/d33/d35/c97 0 2024-04-25T15:49:25.488 INFO:tasks.workunit.client.0.smithi110.stdout:0/385: chown d1/l1e 2170 1 2024-04-25T15:49:25.489 INFO:tasks.workunit.client.0.smithi110.stdout:0/386: rmdir d1/d23/d30/d33/d35 39 2024-04-25T15:49:25.491 INFO:tasks.workunit.client.0.smithi110.stdout:0/387: creat d1/d23/d30/d33/d5b/f98 x:0 0 0 2024-04-25T15:49:25.493 INFO:tasks.workunit.client.0.smithi110.stdout:0/388: dread d1/d47/f8e [0,4194304] 0 2024-04-25T15:49:25.494 INFO:tasks.workunit.client.0.smithi110.stdout:0/389: mkdir d1/d47/d54/d99 0 2024-04-25T15:49:25.497 INFO:tasks.workunit.client.0.smithi110.stdout:0/390: symlink d1/d23/l9a 0 2024-04-25T15:49:25.498 INFO:tasks.workunit.client.0.smithi110.stdout:2/292: sync 2024-04-25T15:49:25.498 INFO:tasks.workunit.client.0.smithi110.stdout:7/311: sync 2024-04-25T15:49:25.500 INFO:tasks.workunit.client.0.smithi110.stdout:2/293: write de/d40/d4f/f49 [377474,3422] 0 2024-04-25T15:49:25.507 INFO:tasks.workunit.client.0.smithi110.stdout:2/294: getdents de/d11 0 2024-04-25T15:49:25.507 INFO:tasks.workunit.client.0.smithi110.stdout:7/312: dwrite d1b/d29/d38/f5b [0,4194304] 0 2024-04-25T15:49:25.514 INFO:tasks.workunit.client.0.smithi110.stdout:2/295: mkdir de/d40/d4f/d53 0 2024-04-25T15:49:25.514 INFO:tasks.workunit.client.0.smithi110.stdout:7/313: dwrite d1b/f26 [0,4194304] 0 2024-04-25T15:49:25.517 INFO:tasks.workunit.client.0.smithi110.stdout:7/314: creat d1b/d29/f68 x:0 0 0 2024-04-25T15:49:25.519 INFO:tasks.workunit.client.0.smithi110.stdout:7/315: chown d1b/d29/d38/d62 800 1 2024-04-25T15:49:25.522 INFO:tasks.workunit.client.0.smithi110.stdout:7/316: rename d1b/d29/d46/c4e to d1b/d29/d38/c69 0 2024-04-25T15:49:25.526 INFO:tasks.workunit.client.0.smithi110.stdout:2/296: dread de/d40/f35 [0,4194304] 0 2024-04-25T15:49:25.529 INFO:tasks.workunit.client.0.smithi110.stdout:2/297: symlink de/d11/d50/l54 0 2024-04-25T15:49:25.529 INFO:tasks.workunit.client.0.smithi110.stdout:7/317: dwrite d1b/d29/f60 [0,4194304] 0 2024-04-25T15:49:25.531 INFO:tasks.workunit.client.0.smithi110.stdout:2/298: fdatasync de/d40/f3e 0 2024-04-25T15:49:25.531 INFO:tasks.workunit.client.0.smithi110.stdout:2/299: readlink de/d40/l46 0 2024-04-25T15:49:25.534 INFO:tasks.workunit.client.0.smithi110.stdout:7/318: fsync f17 0 2024-04-25T15:49:25.536 INFO:tasks.workunit.client.0.smithi110.stdout:7/319: write d1b/d29/d46/f47 [3325132,85531] 0 2024-04-25T15:49:25.536 INFO:tasks.workunit.client.0.smithi110.stdout:2/300: mkdir de/d40/d55 0 2024-04-25T15:49:25.541 INFO:tasks.workunit.client.0.smithi110.stdout:7/320: creat d1b/d29/d46/f6a x:0 0 0 2024-04-25T15:49:25.542 INFO:tasks.workunit.client.0.smithi110.stdout:7/321: fsync f4 0 2024-04-25T15:49:25.544 INFO:tasks.workunit.client.0.smithi110.stdout:7/322: chown d1b/d29/d46/l66 164162 1 2024-04-25T15:49:25.544 INFO:tasks.workunit.client.0.smithi110.stdout:2/301: dwrite de/d40/f3d [0,4194304] 0 2024-04-25T15:49:25.545 INFO:tasks.workunit.client.0.smithi110.stdout:7/323: readlink d1b/d29/l5a 0 2024-04-25T15:49:25.549 INFO:tasks.workunit.client.0.smithi110.stdout:7/324: creat d1b/d29/d38/d55/f6b x:0 0 0 2024-04-25T15:49:25.551 INFO:tasks.workunit.client.0.smithi110.stdout:2/302: rmdir de/d40/d55 0 2024-04-25T15:49:25.552 INFO:tasks.workunit.client.0.smithi110.stdout:2/303: write de/d40/d4f/f49 [1456002,108729] 0 2024-04-25T15:49:25.557 INFO:tasks.workunit.client.0.smithi110.stdout:2/304: rmdir de/d40/d4f/d53 0 2024-04-25T15:49:25.557 INFO:tasks.workunit.client.0.smithi110.stdout:7/325: dwrite d1b/d29/f2b [0,4194304] 0 2024-04-25T15:49:25.561 INFO:tasks.workunit.client.0.smithi110.stdout:7/326: rmdir d1b/d29/d42 39 2024-04-25T15:49:25.564 INFO:tasks.workunit.client.0.smithi110.stdout:2/305: dwrite de/d40/d4f/f4e [0,4194304] 0 2024-04-25T15:49:25.565 INFO:tasks.workunit.client.0.smithi110.stdout:2/306: dread - de/d40/d4f/f47 zero size 2024-04-25T15:49:25.566 INFO:tasks.workunit.client.0.smithi110.stdout:7/327: creat d1b/d29/d42/f6c x:0 0 0 2024-04-25T15:49:25.568 INFO:tasks.workunit.client.0.smithi110.stdout:7/328: truncate d1b/d29/d46/f6a 902379 0 2024-04-25T15:49:25.570 INFO:tasks.workunit.client.0.smithi110.stdout:2/307: creat de/f56 x:0 0 0 2024-04-25T15:49:25.575 INFO:tasks.workunit.client.0.smithi110.stdout:2/308: symlink de/d11/d52/l57 0 2024-04-25T15:49:25.577 INFO:tasks.workunit.client.0.smithi110.stdout:2/309: mknod de/d40/d4f/c58 0 2024-04-25T15:49:25.579 INFO:tasks.workunit.client.0.smithi110.stdout:7/329: dread d1b/f4b [0,4194304] 0 2024-04-25T15:49:25.584 INFO:tasks.workunit.client.0.smithi110.stdout:7/330: dwrite d1b/d29/f57 [0,4194304] 0 2024-04-25T15:49:25.584 INFO:tasks.workunit.client.0.smithi110.stdout:7/331: stat d1b/f45 0 2024-04-25T15:49:25.595 INFO:tasks.workunit.client.0.smithi110.stdout:2/310: rmdir de 39 2024-04-25T15:49:25.598 INFO:tasks.workunit.client.0.smithi110.stdout:2/311: rmdir de/d11/d50 39 2024-04-25T15:49:25.602 INFO:tasks.workunit.client.0.smithi110.stdout:2/312: rename de/f2c to de/d11/d52/f59 0 2024-04-25T15:49:25.603 INFO:tasks.workunit.client.0.smithi110.stdout:2/313: dread - de/f4b zero size 2024-04-25T15:49:25.603 INFO:tasks.workunit.client.0.smithi110.stdout:7/332: dread d1b/d29/f37 [0,4194304] 0 2024-04-25T15:49:25.608 INFO:tasks.workunit.client.0.smithi110.stdout:7/333: dread d1b/f31 [0,4194304] 0 2024-04-25T15:49:25.611 INFO:tasks.workunit.client.0.smithi110.stdout:2/314: dwrite de/d40/f3d [4194304,4194304] 0 2024-04-25T15:49:25.616 INFO:tasks.workunit.client.0.smithi110.stdout:2/315: mknod de/d11/d50/c5a 0 2024-04-25T15:49:25.618 INFO:tasks.workunit.client.0.smithi110.stdout:2/316: mkdir de/d40/d5b 0 2024-04-25T15:49:25.623 INFO:tasks.workunit.client.0.smithi110.stdout:2/317: write de/d11/f3a [3468084,90537] 0 2024-04-25T15:49:25.624 INFO:tasks.workunit.client.0.smithi110.stdout:2/318: write de/d40/d4f/f4e [4192070,60027] 0 2024-04-25T15:49:25.625 INFO:tasks.workunit.client.0.smithi110.stdout:2/319: chown de/d40/d4f/f4e 901 1 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 -- 172.21.15.110:0/409090289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fcf60 msgr2=0x7f6aac0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 --2- 172.21.15.110:0/409090289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fcf60 0x7f6aac0fd3d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0004940 tx=0x7f6aa0009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 -- 172.21.15.110:0/409090289 shutdown_connections 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 --2- 172.21.15.110:0/409090289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fcf60 0x7f6aac0fd3d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 --2- 172.21.15.110:0/409090289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aac0fbcb0 0x7f6aac0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 -- 172.21.15.110:0/409090289 >> 172.21.15.110:0/409090289 conn(0x7f6aac0f7430 msgr2=0x7f6aac0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 -- 172.21.15.110:0/409090289 shutdown_connections 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 -- 172.21.15.110:0/409090289 wait complete. 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 Processor -- start 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.644+0000 7f6ab1c80700 1 -- start start 2024-04-25T15:49:25.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6ab1c80700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fbcb0 0x7f6aac110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6ab1c80700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aac0fcf60 0x7f6aac112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6ab1c80700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac066340 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6ab1c80700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac066480 con 0x7f6aac0fcf60 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6aab7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fbcb0 0x7f6aac110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6aab7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fbcb0 0x7f6aac110680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35896/0 (socket says 172.21.15.110:35896) 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6aab7fe700 1 -- 172.21.15.110:0/4160554101 learned_addr learned my addr 172.21.15.110:0/4160554101 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6aaaffd700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aac0fcf60 0x7f6aac112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6aab7fe700 1 -- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aac0fcf60 msgr2=0x7f6aac112bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6aab7fe700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aac0fcf60 0x7f6aac112bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:25.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6aab7fe700 1 -- 172.21.15.110:0/4160554101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa0005040 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.645+0000 7f6aab7fe700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fbcb0 0x7f6aac110680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00eb40 tx=0x7f6a9c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:25.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.646+0000 7f6a93fff700 1 -- 172.21.15.110:0/4160554101 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9c007cc0 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.646+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac113170 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.646+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac113660 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.646+0000 7f6a93fff700 1 -- 172.21.15.110:0/4160554101 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a9c00f040 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.646+0000 7f6a93fff700 1 -- 172.21.15.110:0/4160554101 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a9c00ca10 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.647+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a980052e0 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.648+0000 7f6a93fff700 1 -- 172.21.15.110:0/4160554101 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f6a9c00cc80 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.648+0000 7f6a93fff700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f6a9406fcb0 0x7f6a94072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:25.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.648+0000 7f6a93fff700 1 -- 172.21.15.110:0/4160554101 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f6a9c08f570 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.649 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.649+0000 7f6aaaffd700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f6a9406fcb0 0x7f6a94072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:25.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.649+0000 7f6aaaffd700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f6a9406fcb0 0x7f6a94072160 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009b30 tx=0x7f6aa0009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:25.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.652+0000 7f6a93fff700 1 -- 172.21.15.110:0/4160554101 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f6a9c058960 con 0x7f6aac0fbcb0 2024-04-25T15:49:25.689 INFO:tasks.workunit.client.0.smithi110.stdout:8/442: sync 2024-04-25T15:49:25.693 INFO:tasks.workunit.client.0.smithi110.stdout:8/443: creat d14/d21/f92 x:0 0 0 2024-04-25T15:49:25.698 INFO:tasks.workunit.client.0.smithi110.stdout:8/444: dwrite d14/d4c/f57 [0,4194304] 0 2024-04-25T15:49:25.712 INFO:tasks.workunit.client.0.smithi110.stdout:8/445: mknod d14/d4c/c93 0 2024-04-25T15:49:25.716 INFO:tasks.workunit.client.0.smithi110.stdout:8/446: rename d14/d4c/d71/f7f to d14/f94 0 2024-04-25T15:49:25.716 INFO:tasks.workunit.client.0.smithi110.stdout:8/447: read - d14/d21/d26/f4a zero size 2024-04-25T15:49:25.719 INFO:tasks.workunit.client.0.smithi110.stdout:8/448: rename d14/f6e to d14/d4c/d55/d8d/f95 0 2024-04-25T15:49:25.721 INFO:tasks.workunit.client.0.smithi110.stdout:8/449: creat d14/d4c/d55/f96 x:0 0 0 2024-04-25T15:49:25.722 INFO:tasks.workunit.client.0.smithi110.stdout:8/450: write d14/d4c/d55/f88 [284689,50602] 0 2024-04-25T15:49:25.785 INFO:tasks.workunit.client.0.smithi110.stdout:8/451: dwrite d14/f1b [4194304,4194304] 0 2024-04-25T15:49:25.788 INFO:tasks.workunit.client.0.smithi110.stdout:8/452: rename d14/d21/d26/c3b to d14/d4c/d55/c97 0 2024-04-25T15:49:25.792 INFO:tasks.workunit.client.0.smithi110.stdout:8/453: rename d14/f15 to d14/d21/d26/f98 0 2024-04-25T15:49:25.793 INFO:tasks.workunit.client.0.smithi110.stdout:8/454: chown d14/f16 5026408 1 2024-04-25T15:49:25.866 INFO:tasks.workunit.client.1.smithi193.stderr:++ readlink -f fsstress 2024-04-25T15:49:25.868 INFO:tasks.workunit.client.1.smithi193.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-25T15:49:25.868 INFO:tasks.workunit.client.1.smithi193.stderr:+ popd 2024-04-25T15:49:25.869 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-25T15:49:25.869 INFO:tasks.workunit.client.1.smithi193.stderr:+ popd 2024-04-25T15:49:25.869 INFO:tasks.workunit.client.1.smithi193.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-25T15:49:25.869 INFO:tasks.workunit.client.1.smithi193.stderr:++ mktemp -d -p . 2024-04-25T15:49:25.872 INFO:tasks.workunit.client.1.smithi193.stderr:+ T=./tmp.Vx2iyfLbHL 2024-04-25T15:49:25.872 INFO:tasks.workunit.client.1.smithi193.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.Vx2iyfLbHL -l 1 -n 1000 -p 10 -v 2024-04-25T15:49:25.875 INFO:tasks.workunit.client.1.smithi193.stdout:seed = 1713841460 2024-04-25T15:49:25.881 INFO:tasks.workunit.client.1.smithi193.stdout:1/0: dread - no filename 2024-04-25T15:49:25.881 INFO:tasks.workunit.client.1.smithi193.stdout:1/1: fsync - no filename 2024-04-25T15:49:25.881 INFO:tasks.workunit.client.1.smithi193.stdout:1/2: fsync - no filename 2024-04-25T15:49:25.882 INFO:tasks.workunit.client.1.smithi193.stdout:0/0: mkdir d0 0 2024-04-25T15:49:25.882 INFO:tasks.workunit.client.1.smithi193.stdout:0/1: truncate - no filename 2024-04-25T15:49:25.882 INFO:tasks.workunit.client.1.smithi193.stdout:0/2: write - no filename 2024-04-25T15:49:25.882 INFO:tasks.workunit.client.1.smithi193.stdout:0/3: dwrite - no filename 2024-04-25T15:49:25.882 INFO:tasks.workunit.client.1.smithi193.stdout:0/4: unlink - no file 2024-04-25T15:49:25.883 INFO:tasks.workunit.client.1.smithi193.stdout:2/0: dread - no filename 2024-04-25T15:49:25.884 INFO:tasks.workunit.client.1.smithi193.stdout:0/5: mkdir d0/d1 0 2024-04-25T15:49:25.884 INFO:tasks.workunit.client.1.smithi193.stdout:0/6: stat d0/d1 0 2024-04-25T15:49:25.884 INFO:tasks.workunit.client.1.smithi193.stdout:0/7: dread - no filename 2024-04-25T15:49:25.884 INFO:tasks.workunit.client.1.smithi193.stdout:0/8: dwrite - no filename 2024-04-25T15:49:25.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.884+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a98000bf0 con 0x7f6a9406fcb0 2024-04-25T15:49:25.885 INFO:tasks.workunit.client.1.smithi193.stdout:3/0: truncate - no filename 2024-04-25T15:49:25.885 INFO:tasks.workunit.client.1.smithi193.stdout:3/1: truncate - no filename 2024-04-25T15:49:25.885 INFO:tasks.workunit.client.1.smithi193.stdout:3/2: truncate - no filename 2024-04-25T15:49:25.886 INFO:tasks.workunit.client.1.smithi193.stdout:0/9: symlink d0/l2 0 2024-04-25T15:49:25.886 INFO:tasks.workunit.client.1.smithi193.stdout:2/1: mkdir d0 0 2024-04-25T15:49:25.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.886+0000 7f6a93fff700 1 -- 172.21.15.110:0/4160554101 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f6a98000bf0 con 0x7f6a9406fcb0 2024-04-25T15:49:25.887 INFO:tasks.workunit.client.1.smithi193.stdout:4/0: rename - no filename 2024-04-25T15:49:25.887 INFO:tasks.workunit.client.1.smithi193.stdout:0/10: mkdir d0/d3 0 2024-04-25T15:49:25.888 INFO:tasks.workunit.client.1.smithi193.stdout:0/11: dwrite - no filename 2024-04-25T15:49:25.888 INFO:tasks.workunit.client.1.smithi193.stdout:0/12: write - no filename 2024-04-25T15:49:25.888 INFO:tasks.workunit.client.1.smithi193.stdout:0/13: write - no filename 2024-04-25T15:49:25.888 INFO:tasks.workunit.client.1.smithi193.stdout:3/3: creat f0 x:0 0 0 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f6a9406fcb0 msgr2=0x7f6a94072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f6a9406fcb0 0x7f6a94072160 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6aa0009b30 tx=0x7f6aa0009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fbcb0 msgr2=0x7f6aac110680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fbcb0 0x7f6aac110680 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c00eb40 tx=0x7f6a9c00b040 comp rx=0 tx=0).stop 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 shutdown_connections 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f6a9406fcb0 0x7f6a94072160 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6aac0fbcb0 0x7f6aac110680 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 --2- 172.21.15.110:0/4160554101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6aac0fcf60 0x7f6aac112bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:25.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.888+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 >> 172.21.15.110:0/4160554101 conn(0x7f6aac0f7430 msgr2=0x7f6aac0f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:25.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.889+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 shutdown_connections 2024-04-25T15:49:25.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:25.889+0000 7f6ab1c80700 1 -- 172.21.15.110:0/4160554101 wait complete. 2024-04-25T15:49:25.890 INFO:tasks.workunit.client.1.smithi193.stdout:5/0: dread - no filename 2024-04-25T15:49:25.890 INFO:tasks.workunit.client.1.smithi193.stdout:2/2: mknod d0/c1 0 2024-04-25T15:49:25.891 INFO:tasks.workunit.client.1.smithi193.stdout:0/14: symlink d0/l4 0 2024-04-25T15:49:25.891 INFO:tasks.workunit.client.1.smithi193.stdout:3/4: creat f1 x:0 0 0 2024-04-25T15:49:25.891 INFO:tasks.workunit.client.1.smithi193.stdout:4/1: creat f0 x:0 0 0 2024-04-25T15:49:25.892 INFO:tasks.workunit.client.1.smithi193.stdout:3/5: chown f1 1000 0 2024-04-25T15:49:25.892 INFO:tasks.workunit.client.1.smithi193.stdout:3/6: stat f1 0 2024-04-25T15:49:25.892 INFO:tasks.workunit.client.1.smithi193.stdout:4/2: dread - f0 zero size 2024-04-25T15:49:25.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/3: symlink d0/l2 0 2024-04-25T15:49:25.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/4: truncate - no filename 2024-04-25T15:49:25.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/5: dread - no filename 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:2/6: dwrite - no filename 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/0: dwrite - no filename 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/1: write - no filename 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/2: read - no filename 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/3: rename - no filename 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/4: fdatasync - no filename 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/5: fdatasync - no filename 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:3/7: truncate f0 819447 0 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:3/8: dread - f1 zero size 2024-04-25T15:49:25.893 INFO:tasks.workunit.client.1.smithi193.stdout:0/15: rename d0/d1 to d0/d3/d5 0 2024-04-25T15:49:25.894 INFO:tasks.workunit.client.1.smithi193.stdout:0/16: read - no filename 2024-04-25T15:49:25.894 INFO:tasks.workunit.client.1.smithi193.stdout:3/9: read f0 [514557,29374] 0 2024-04-25T15:49:25.894 INFO:tasks.workunit.client.1.smithi193.stdout:7/0: rename - no filename 2024-04-25T15:49:25.895 INFO:tasks.workunit.client.1.smithi193.stdout:7/1: chown . 12 1 2024-04-25T15:49:25.895 INFO:tasks.workunit.client.1.smithi193.stdout:2/7: creat d0/f3 x:0 0 0 2024-04-25T15:49:25.896 INFO:tasks.workunit.client.1.smithi193.stdout:0/17: unlink d0/l4 0 2024-04-25T15:49:25.897 INFO:tasks.workunit.client.1.smithi193.stdout:6/6: mknod c0 0 2024-04-25T15:49:25.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/7: chown c0 539 1 2024-04-25T15:49:25.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/8: write - no filename 2024-04-25T15:49:25.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/9: chown c0 3862 1 2024-04-25T15:49:25.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/10: dread - no filename 2024-04-25T15:49:25.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/11: fsync - no filename 2024-04-25T15:49:25.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/12: dwrite - no filename 2024-04-25T15:49:25.899 INFO:tasks.workunit.client.1.smithi193.stdout:8/0: chown . 916 1 2024-04-25T15:49:25.899 INFO:tasks.workunit.client.1.smithi193.stdout:8/1: write - no filename 2024-04-25T15:49:25.900 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:49:25.903 INFO:tasks.workunit.client.1.smithi193.stdout:2/8: symlink d0/l4 0 2024-04-25T15:49:25.903 INFO:tasks.workunit.client.1.smithi193.stdout:2/9: write d0/f3 [1011302,39973] 0 2024-04-25T15:49:25.904 INFO:tasks.workunit.client.1.smithi193.stdout:6/13: symlink l1 0 2024-04-25T15:49:25.905 INFO:tasks.workunit.client.1.smithi193.stdout:9/0: truncate - no filename 2024-04-25T15:49:25.906 INFO:tasks.workunit.client.1.smithi193.stdout:3/10: rename f1 to f2 0 2024-04-25T15:49:25.908 INFO:tasks.workunit.client.1.smithi193.stdout:7/2: creat f0 x:0 0 0 2024-04-25T15:49:25.909 INFO:tasks.workunit.client.1.smithi193.stdout:0/18: getdents d0/d3/d5 0 2024-04-25T15:49:25.909 INFO:tasks.workunit.client.1.smithi193.stdout:0/19: rename d0/d3 to d0/d3/d6 22 2024-04-25T15:49:25.911 INFO:tasks.workunit.client.1.smithi193.stdout:8/2: creat f0 x:0 0 0 2024-04-25T15:49:25.912 INFO:tasks.workunit.client.1.smithi193.stdout:8/3: write f0 [979262,34944] 0 2024-04-25T15:49:25.914 INFO:tasks.workunit.client.1.smithi193.stdout:7/3: rename f0 to f1 0 2024-04-25T15:49:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:0/20: mkdir d0/d3/d5/d7 0 2024-04-25T15:49:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:0/21: fsync - no filename 2024-04-25T15:49:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:0/22: read - no filename 2024-04-25T15:49:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:0/23: dwrite - no filename 2024-04-25T15:49:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:0/24: fdatasync - no filename 2024-04-25T15:49:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:0/25: fdatasync - no filename 2024-04-25T15:49:25.916 INFO:tasks.workunit.client.1.smithi193.stdout:4/3: fsync f0 0 2024-04-25T15:49:25.917 INFO:tasks.workunit.client.1.smithi193.stdout:0/26: readlink d0/l2 0 2024-04-25T15:49:25.917 INFO:tasks.workunit.client.1.smithi193.stdout:9/1: getdents . 0 2024-04-25T15:49:25.917 INFO:tasks.workunit.client.1.smithi193.stdout:9/2: dread - no filename 2024-04-25T15:49:25.917 INFO:tasks.workunit.client.1.smithi193.stdout:9/3: dread - no filename 2024-04-25T15:49:25.918 INFO:tasks.workunit.client.1.smithi193.stdout:0/27: stat d0/d3 0 2024-04-25T15:49:25.918 INFO:tasks.workunit.client.1.smithi193.stdout:8/4: mknod c1 0 2024-04-25T15:49:25.918 INFO:tasks.workunit.client.1.smithi193.stdout:8/5: rmdir - no directory 2024-04-25T15:49:25.920 INFO:tasks.workunit.client.1.smithi193.stdout:3/11: dwrite f0 [0,4194304] 0 2024-04-25T15:49:25.925 INFO:tasks.workunit.client.1.smithi193.stdout:0/28: symlink d0/l8 0 2024-04-25T15:49:25.925 INFO:tasks.workunit.client.1.smithi193.stdout:0/29: fsync - no filename 2024-04-25T15:49:25.928 INFO:tasks.workunit.client.1.smithi193.stdout:2/10: getdents d0 0 2024-04-25T15:49:25.928 INFO:tasks.workunit.client.1.smithi193.stdout:8/6: dwrite f0 [0,4194304] 0 2024-04-25T15:49:25.928 INFO:tasks.workunit.client.1.smithi193.stdout:6/14: getdents . 0 2024-04-25T15:49:25.928 INFO:tasks.workunit.client.1.smithi193.stdout:6/15: write - no filename 2024-04-25T15:49:25.929 INFO:tasks.workunit.client.1.smithi193.stdout:4/4: link f0 f1 0 2024-04-25T15:49:25.929 INFO:tasks.workunit.client.1.smithi193.stdout:3/12: creat f3 x:0 0 0 2024-04-25T15:49:25.930 INFO:tasks.workunit.client.1.smithi193.stdout:4/5: chown f0 14 1 2024-04-25T15:49:25.930 INFO:tasks.workunit.client.1.smithi193.stdout:4/6: readlink - no filename 2024-04-25T15:49:25.930 INFO:tasks.workunit.client.1.smithi193.stdout:9/4: symlink l0 0 2024-04-25T15:49:25.931 INFO:tasks.workunit.client.1.smithi193.stdout:9/5: write - no filename 2024-04-25T15:49:25.931 INFO:tasks.workunit.client.1.smithi193.stdout:9/6: rmdir - no directory 2024-04-25T15:49:25.933 INFO:tasks.workunit.client.1.smithi193.stdout:2/11: mknod d0/c5 0 2024-04-25T15:49:25.934 INFO:tasks.workunit.client.1.smithi193.stdout:8/7: dread f0 [0,4194304] 0 2024-04-25T15:49:25.936 INFO:tasks.workunit.client.1.smithi193.stdout:3/13: creat f4 x:0 0 0 2024-04-25T15:49:25.937 INFO:tasks.workunit.client.1.smithi193.stdout:0/30: mknod d0/d3/d5/d7/c9 0 2024-04-25T15:49:25.938 INFO:tasks.workunit.client.1.smithi193.stdout:6/16: getdents . 0 2024-04-25T15:49:25.939 INFO:tasks.workunit.client.1.smithi193.stdout:2/12: creat d0/f6 x:0 0 0 2024-04-25T15:49:25.941 INFO:tasks.workunit.client.1.smithi193.stdout:8/8: write f0 [2445912,17012] 0 2024-04-25T15:49:25.941 INFO:tasks.workunit.client.1.smithi193.stdout:0/31: rename d0/l8 to d0/la 0 2024-04-25T15:49:25.942 INFO:tasks.workunit.client.1.smithi193.stdout:6/17: symlink l2 0 2024-04-25T15:49:25.943 INFO:tasks.workunit.client.1.smithi193.stdout:2/13: creat d0/f7 x:0 0 0 2024-04-25T15:49:25.943 INFO:tasks.workunit.client.1.smithi193.stdout:9/7: link l0 l1 0 2024-04-25T15:49:25.943 INFO:tasks.workunit.client.1.smithi193.stdout:9/8: dwrite - no filename 2024-04-25T15:49:25.943 INFO:tasks.workunit.client.1.smithi193.stdout:9/9: write - no filename 2024-04-25T15:49:25.944 INFO:tasks.workunit.client.1.smithi193.stdout:9/10: dread - no filename 2024-04-25T15:49:25.944 INFO:tasks.workunit.client.1.smithi193.stdout:9/11: fdatasync - no filename 2024-04-25T15:49:25.944 INFO:tasks.workunit.client.1.smithi193.stdout:9/12: fdatasync - no filename 2024-04-25T15:49:25.944 INFO:tasks.workunit.client.1.smithi193.stdout:6/18: chown l2 427 1 2024-04-25T15:49:25.944 INFO:tasks.workunit.client.1.smithi193.stdout:6/19: dwrite - no filename 2024-04-25T15:49:25.945 INFO:tasks.workunit.client.1.smithi193.stdout:8/9: mkdir d2 0 2024-04-25T15:49:25.946 INFO:tasks.workunit.client.1.smithi193.stdout:3/14: fsync f0 0 2024-04-25T15:49:25.947 INFO:tasks.workunit.client.1.smithi193.stdout:4/7: dwrite f1 [0,4194304] 0 2024-04-25T15:49:25.947 INFO:tasks.workunit.client.1.smithi193.stdout:6/20: chown l1 15244117 1 2024-04-25T15:49:25.947 INFO:tasks.workunit.client.1.smithi193.stdout:2/14: symlink d0/l8 0 2024-04-25T15:49:25.947 INFO:tasks.workunit.client.1.smithi193.stdout:0/32: creat d0/d3/d5/d7/fb x:0 0 0 2024-04-25T15:49:25.950 INFO:tasks.workunit.client.1.smithi193.stdout:4/8: creat f2 x:0 0 0 2024-04-25T15:49:25.952 INFO:tasks.workunit.client.1.smithi193.stdout:6/21: rename c0 to c3 0 2024-04-25T15:49:25.952 INFO:tasks.workunit.client.1.smithi193.stdout:6/22: write - no filename 2024-04-25T15:49:25.952 INFO:tasks.workunit.client.1.smithi193.stdout:6/23: dwrite - no filename 2024-04-25T15:49:25.953 INFO:tasks.workunit.client.1.smithi193.stdout:3/15: rename f0 to f5 0 2024-04-25T15:49:25.953 INFO:tasks.workunit.client.1.smithi193.stdout:8/10: mkdir d2/d3 0 2024-04-25T15:49:25.954 INFO:tasks.workunit.client.1.smithi193.stdout:9/13: link l0 l2 0 2024-04-25T15:49:25.958 INFO:tasks.workunit.client.1.smithi193.stdout:3/16: symlink l6 0 2024-04-25T15:49:25.960 INFO:tasks.workunit.client.1.smithi193.stdout:2/15: dwrite d0/f6 [0,4194304] 0 2024-04-25T15:49:25.964 INFO:tasks.workunit.client.1.smithi193.stdout:0/33: dwrite d0/d3/d5/d7/fb [0,4194304] 0 2024-04-25T15:49:25.965 INFO:tasks.workunit.client.1.smithi193.stdout:3/17: rename l6 to l7 0 2024-04-25T15:49:25.966 INFO:tasks.workunit.client.1.smithi193.stdout:9/14: chown l1 48321 1 2024-04-25T15:49:25.966 INFO:tasks.workunit.client.1.smithi193.stdout:9/15: dwrite - no filename 2024-04-25T15:49:25.966 INFO:tasks.workunit.client.1.smithi193.stdout:0/34: chown d0/d3/d5/d7/fb 8354069 1 2024-04-25T15:49:25.968 INFO:tasks.workunit.client.1.smithi193.stdout:2/16: mkdir d0/d9 0 2024-04-25T15:49:25.968 INFO:tasks.workunit.client.1.smithi193.stdout:9/16: chown l0 911059 1 2024-04-25T15:49:25.968 INFO:tasks.workunit.client.1.smithi193.stdout:4/9: fdatasync f0 0 2024-04-25T15:49:25.968 INFO:tasks.workunit.client.1.smithi193.stdout:9/17: readlink l2 0 2024-04-25T15:49:25.969 INFO:tasks.workunit.client.1.smithi193.stdout:8/11: dread f0 [0,4194304] 0 2024-04-25T15:49:25.970 INFO:tasks.workunit.client.1.smithi193.stdout:0/35: creat d0/fc x:0 0 0 2024-04-25T15:49:25.971 INFO:tasks.workunit.client.1.smithi193.stdout:0/36: chown d0/d3/d5 15264 1 2024-04-25T15:49:25.972 INFO:tasks.workunit.client.1.smithi193.stdout:6/24: chown c3 148201 1 2024-04-25T15:49:25.972 INFO:tasks.workunit.client.0.smithi110.stdout:9/271: sync 2024-04-25T15:49:25.973 INFO:tasks.workunit.client.1.smithi193.stdout:4/10: write f1 [235907,43163] 0 2024-04-25T15:49:25.973 INFO:tasks.workunit.client.1.smithi193.stdout:0/37: chown d0/d3/d5/d7 26039237 1 2024-04-25T15:49:25.973 INFO:tasks.workunit.client.1.smithi193.stdout:9/18: mknod c3 0 2024-04-25T15:49:25.973 INFO:tasks.workunit.client.1.smithi193.stdout:9/19: dread - no filename 2024-04-25T15:49:25.974 INFO:tasks.workunit.client.1.smithi193.stdout:9/20: fdatasync - no filename 2024-04-25T15:49:25.974 INFO:tasks.workunit.client.1.smithi193.stdout:6/25: chown l1 816822259 1 2024-04-25T15:49:25.974 INFO:tasks.workunit.client.1.smithi193.stdout:4/11: dread - f2 zero size 2024-04-25T15:49:25.975 INFO:tasks.workunit.client.1.smithi193.stdout:9/21: chown l1 96344471 1 2024-04-25T15:49:25.975 INFO:tasks.workunit.client.1.smithi193.stdout:9/22: dwrite - no filename 2024-04-25T15:49:25.975 INFO:tasks.workunit.client.1.smithi193.stdout:9/23: truncate - no filename 2024-04-25T15:49:25.975 INFO:tasks.workunit.client.1.smithi193.stdout:9/24: fsync - no filename 2024-04-25T15:49:25.975 INFO:tasks.workunit.client.1.smithi193.stdout:4/12: chown f2 249424704 1 2024-04-25T15:49:25.977 INFO:tasks.workunit.client.0.smithi110.stdout:9/272: mkdir d9/db/df/d2c/d3c/d56 0 2024-04-25T15:49:25.977 INFO:tasks.workunit.client.1.smithi193.stdout:4/13: write f1 [3263939,58160] 0 2024-04-25T15:49:25.978 INFO:tasks.workunit.client.1.smithi193.stdout:6/26: mknod c4 0 2024-04-25T15:49:25.980 INFO:tasks.workunit.client.1.smithi193.stdout:9/25: creat f4 x:0 0 0 2024-04-25T15:49:25.982 INFO:tasks.workunit.client.1.smithi193.stdout:3/18: chown l7 773 1 2024-04-25T15:49:25.982 INFO:tasks.workunit.client.1.smithi193.stdout:9/26: stat c3 0 2024-04-25T15:49:25.982 INFO:tasks.workunit.client.1.smithi193.stdout:4/14: symlink l3 0 2024-04-25T15:49:25.983 INFO:tasks.workunit.client.0.smithi110.stdout:9/273: dwrite d9/d44/f53 [0,4194304] 0 2024-04-25T15:49:25.983 INFO:tasks.workunit.client.1.smithi193.stdout:6/27: unlink c4 0 2024-04-25T15:49:25.984 INFO:tasks.workunit.client.0.smithi110.stdout:9/274: fsync d9/db/df/d25/f37 0 2024-04-25T15:49:25.985 INFO:tasks.workunit.client.1.smithi193.stdout:8/12: dwrite f0 [0,4194304] 0 2024-04-25T15:49:25.985 INFO:tasks.workunit.client.0.smithi110.stdout:9/275: truncate d9/d2d/f2f 130644 0 2024-04-25T15:49:25.986 INFO:tasks.workunit.client.1.smithi193.stdout:3/19: write f4 [686214,7958] 0 2024-04-25T15:49:25.987 INFO:tasks.workunit.client.1.smithi193.stdout:2/17: fdatasync d0/f6 0 2024-04-25T15:49:25.987 INFO:tasks.workunit.client.1.smithi193.stdout:0/38: dwrite d0/fc [0,4194304] 0 2024-04-25T15:49:25.988 INFO:tasks.workunit.client.0.smithi110.stdout:9/276: symlink d9/db/df/d2c/d3c/d56/l57 0 2024-04-25T15:49:25.989 INFO:tasks.workunit.client.1.smithi193.stdout:9/27: creat f5 x:0 0 0 2024-04-25T15:49:25.989 INFO:tasks.workunit.client.0.smithi110.stdout:9/277: mkdir d9/db/d58 0 2024-04-25T15:49:25.990 INFO:tasks.workunit.client.1.smithi193.stdout:6/28: unlink c3 0 2024-04-25T15:49:25.992 INFO:tasks.workunit.client.1.smithi193.stdout:9/28: write f4 [419697,59165] 0 2024-04-25T15:49:25.994 INFO:tasks.workunit.client.0.smithi110.stdout:9/278: mknod d9/db/df/d2c/d3c/d56/c59 0 2024-04-25T15:49:25.995 INFO:tasks.workunit.client.1.smithi193.stdout:3/20: mkdir d8 0 2024-04-25T15:49:25.997 INFO:tasks.workunit.client.1.smithi193.stdout:2/18: creat d0/fa x:0 0 0 2024-04-25T15:49:25.997 INFO:tasks.workunit.client.1.smithi193.stdout:8/13: dread f0 [0,4194304] 0 2024-04-25T15:49:25.998 INFO:tasks.workunit.client.1.smithi193.stdout:6/29: creat f5 x:0 0 0 2024-04-25T15:49:25.999 INFO:tasks.workunit.client.1.smithi193.stdout:6/30: readlink l2 0 2024-04-25T15:49:26.002 INFO:tasks.workunit.client.1.smithi193.stdout:8/14: rename f0 to d2/f4 0 2024-04-25T15:49:26.003 INFO:tasks.workunit.client.1.smithi193.stdout:9/29: dwrite f5 [0,4194304] 0 2024-04-25T15:49:26.004 INFO:tasks.workunit.client.1.smithi193.stdout:4/15: getdents . 0 2024-04-25T15:49:26.005 INFO:tasks.workunit.client.1.smithi193.stdout:3/21: getdents d8 0 2024-04-25T15:49:26.010 INFO:tasks.workunit.client.1.smithi193.stdout:9/30: unlink f5 0 2024-04-25T15:49:26.011 INFO:tasks.workunit.client.1.smithi193.stdout:4/16: creat f4 x:0 0 0 2024-04-25T15:49:26.011 INFO:tasks.workunit.client.1.smithi193.stdout:4/17: stat f1 0 2024-04-25T15:49:26.011 INFO:tasks.workunit.client.1.smithi193.stdout:4/18: readlink l3 0 2024-04-25T15:49:26.016 INFO:tasks.workunit.client.1.smithi193.stdout:9/31: creat f6 x:0 0 0 2024-04-25T15:49:26.017 INFO:tasks.workunit.client.1.smithi193.stdout:2/19: link d0/l4 d0/d9/lb 0 2024-04-25T15:49:26.020 INFO:tasks.workunit.client.1.smithi193.stdout:3/22: symlink d8/l9 0 2024-04-25T15:49:26.021 INFO:tasks.workunit.client.1.smithi193.stdout:8/15: unlink d2/f4 0 2024-04-25T15:49:26.021 INFO:tasks.workunit.client.1.smithi193.stdout:8/16: dwrite - no filename 2024-04-25T15:49:26.021 INFO:tasks.workunit.client.1.smithi193.stdout:8/17: chown d2 35467 1 2024-04-25T15:49:26.022 INFO:tasks.workunit.client.1.smithi193.stdout:9/32: symlink l7 0 2024-04-25T15:49:26.022 INFO:tasks.workunit.client.1.smithi193.stdout:9/33: read - f6 zero size 2024-04-25T15:49:26.024 INFO:tasks.workunit.client.1.smithi193.stdout:6/31: getdents . 0 2024-04-25T15:49:26.024 INFO:tasks.workunit.client.1.smithi193.stdout:4/19: fsync f4 0 2024-04-25T15:49:26.025 INFO:tasks.workunit.client.1.smithi193.stdout:3/23: mknod d8/ca 0 2024-04-25T15:49:26.026 INFO:tasks.workunit.client.0.smithi110.stdout:9/279: dread d9/f24 [0,4194304] 0 2024-04-25T15:49:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.026+0000 7f359d387700 1 -- 172.21.15.110:0/521597429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35980fff70 msgr2=0x7f3598102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.026+0000 7f359d387700 1 --2- 172.21.15.110:0/521597429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35980fff70 0x7f3598102350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f358c004940 tx=0x7f358c009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.026+0000 7f359d387700 1 -- 172.21.15.110:0/521597429 shutdown_connections 2024-04-25T15:49:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.026+0000 7f359d387700 1 --2- 172.21.15.110:0/521597429 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35980fff70 0x7f3598102350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.026+0000 7f359d387700 1 --2- 172.21.15.110:0/521597429 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3598066af0 0x7f35980ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.026+0000 7f359d387700 1 -- 172.21.15.110:0/521597429 >> 172.21.15.110:0/521597429 conn(0x7f35980f7440 msgr2=0x7f35980f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.026+0000 7f359d387700 1 -- 172.21.15.110:0/521597429 shutdown_connections 2024-04-25T15:49:26.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.026+0000 7f359d387700 1 -- 172.21.15.110:0/521597429 wait complete. 2024-04-25T15:49:26.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.027+0000 7f359d387700 1 Processor -- start 2024-04-25T15:49:26.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.027+0000 7f359d387700 1 -- start start 2024-04-25T15:49:26.029 INFO:tasks.workunit.client.0.smithi110.stdout:9/280: dread d9/d2d/f2f [0,4194304] 0 2024-04-25T15:49:26.029 INFO:tasks.workunit.client.1.smithi193.stdout:9/34: symlink l8 0 2024-04-25T15:49:26.029 INFO:tasks.workunit.client.1.smithi193.stdout:9/35: fsync f4 0 2024-04-25T15:49:26.029 INFO:tasks.workunit.client.1.smithi193.stdout:2/20: stat d0/l4 0 2024-04-25T15:49:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.027+0000 7f359d387700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598066af0 0x7f359810daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f359d387700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35980fff70 0x7f359810dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f359d387700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359810e670 con 0x7f3598066af0 2024-04-25T15:49:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f359d387700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359810ee10 con 0x7f35980fff70 2024-04-25T15:49:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f35967fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35980fff70 0x7f359810dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f35967fc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35980fff70 0x7f359810dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:37548/0 (socket says 172.21.15.110:37548) 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f35967fc700 1 -- 172.21.15.110:0/1899310401 learned_addr learned my addr 172.21.15.110:0/1899310401 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f3596ffd700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598066af0 0x7f359810daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f35967fc700 1 -- 172.21.15.110:0/1899310401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598066af0 msgr2=0x7f359810daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f35967fc700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598066af0 0x7f359810daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.028+0000 7f35967fc700 1 -- 172.21.15.110:0/1899310401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c005040 con 0x7f35980fff70 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.029+0000 7f35967fc700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35980fff70 0x7f359810dfe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f358c00b3b0 tx=0x7f358c009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.029+0000 7f357f7fe700 1 -- 172.21.15.110:0/1899310401 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358c00e040 con 0x7f35980fff70 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.029+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359810f090 con 0x7f35980fff70 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.029+0000 7f357f7fe700 1 -- 172.21.15.110:0/1899310401 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f358c00bbf0 con 0x7f35980fff70 2024-04-25T15:49:26.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.029+0000 7f357f7fe700 1 -- 172.21.15.110:0/1899310401 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358c01c630 con 0x7f35980fff70 2024-04-25T15:49:26.031 INFO:tasks.workunit.client.0.smithi110.stdout:9/281: truncate d9/d13/f3f 932782 0 2024-04-25T15:49:26.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.029+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359810f650 con 0x7f35980fff70 2024-04-25T15:49:26.032 INFO:tasks.workunit.client.0.smithi110.stdout:5/324: sync 2024-04-25T15:49:26.032 INFO:tasks.workunit.client.0.smithi110.stdout:4/436: sync 2024-04-25T15:49:26.032 INFO:tasks.workunit.client.0.smithi110.stdout:4/437: write f8 [3950857,31412] 0 2024-04-25T15:49:26.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.030+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359805f5c0 con 0x7f35980fff70 2024-04-25T15:49:26.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.031+0000 7f357f7fe700 1 -- 172.21.15.110:0/1899310401 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f358c013420 con 0x7f35980fff70 2024-04-25T15:49:26.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.032+0000 7f357f7fe700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f358006fcb0 0x7f3580072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.032+0000 7f357f7fe700 1 -- 172.21.15.110:0/1899310401 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f358c0903b0 con 0x7f35980fff70 2024-04-25T15:49:26.033 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.032+0000 7f3596ffd700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f358006fcb0 0x7f3580072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.033+0000 7f3596ffd700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f358006fcb0 0x7f3580072160 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3598066830 tx=0x7f35880091a0 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:26.034 INFO:tasks.workunit.client.1.smithi193.stdout:6/32: rename f5 to f6 0 2024-04-25T15:49:26.034 INFO:tasks.workunit.client.1.smithi193.stdout:2/21: dread d0/f3 [0,4194304] 0 2024-04-25T15:49:26.034 INFO:tasks.workunit.client.1.smithi193.stdout:3/24: fdatasync f5 0 2024-04-25T15:49:26.038 INFO:tasks.workunit.client.1.smithi193.stdout:8/18: creat d2/d3/f5 x:0 0 0 2024-04-25T15:49:26.038 INFO:tasks.workunit.client.1.smithi193.stdout:9/36: creat f9 x:0 0 0 2024-04-25T15:49:26.038 INFO:tasks.workunit.client.0.smithi110.stdout:4/438: creat d13/d24/d2d/d36/d51/f8c x:0 0 0 2024-04-25T15:49:26.039 INFO:tasks.workunit.client.0.smithi110.stdout:5/325: dwrite d4/f44 [0,4194304] 0 2024-04-25T15:49:26.039 INFO:tasks.workunit.client.0.smithi110.stdout:9/282: fsync d9/db/df/d2c/f36 0 2024-04-25T15:49:26.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.038+0000 7f357f7fe700 1 -- 172.21.15.110:0/1899310401 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f358c0597f0 con 0x7f35980fff70 2024-04-25T15:49:26.040 INFO:tasks.workunit.client.1.smithi193.stdout:2/22: dwrite d0/f3 [0,4194304] 0 2024-04-25T15:49:26.050 INFO:tasks.workunit.client.1.smithi193.stdout:3/25: rmdir d8 39 2024-04-25T15:49:26.051 INFO:tasks.workunit.client.1.smithi193.stdout:2/23: dwrite d0/f6 [4194304,4194304] 0 2024-04-25T15:49:26.052 INFO:tasks.workunit.client.0.smithi110.stdout:4/439: dread d13/d24/d3c/d56/d88/f2c [0,4194304] 0 2024-04-25T15:49:26.052 INFO:tasks.workunit.client.1.smithi193.stdout:2/24: readlink d0/d9/lb 0 2024-04-25T15:49:26.054 INFO:tasks.workunit.client.1.smithi193.stdout:9/37: mknod ca 0 2024-04-25T15:49:26.055 INFO:tasks.workunit.client.0.smithi110.stdout:9/283: mkdir d9/db/d58/d5a 0 2024-04-25T15:49:26.057 INFO:tasks.workunit.client.0.smithi110.stdout:4/440: fsync d13/d24/d26/f49 0 2024-04-25T15:49:26.057 INFO:tasks.workunit.client.1.smithi193.stdout:4/20: link l3 l5 0 2024-04-25T15:49:26.059 INFO:tasks.workunit.client.0.smithi110.stdout:9/284: write d9/d13/f1c [2539743,92462] 0 2024-04-25T15:49:26.060 INFO:tasks.workunit.client.1.smithi193.stdout:3/26: unlink l7 0 2024-04-25T15:49:26.061 INFO:tasks.workunit.client.1.smithi193.stdout:3/27: chown f4 4330255 1 2024-04-25T15:49:26.061 INFO:tasks.workunit.client.0.smithi110.stdout:5/326: dwrite d4/d1d/d1f/d35/f4c [0,4194304] 0 2024-04-25T15:49:26.061 INFO:tasks.workunit.client.1.smithi193.stdout:3/28: stat f2 0 2024-04-25T15:49:26.062 INFO:tasks.workunit.client.1.smithi193.stdout:3/29: fsync f3 0 2024-04-25T15:49:26.065 INFO:tasks.workunit.client.0.smithi110.stdout:4/441: symlink d13/d24/d2d/d55/l8d 0 2024-04-25T15:49:26.065 INFO:tasks.workunit.client.1.smithi193.stdout:4/21: dwrite f1 [0,4194304] 0 2024-04-25T15:49:26.066 INFO:tasks.workunit.client.1.smithi193.stdout:2/25: creat d0/fc x:0 0 0 2024-04-25T15:49:26.066 INFO:tasks.workunit.client.0.smithi110.stdout:4/442: chown d13/d24/d2d/d36/d51/f54 955136 1 2024-04-25T15:49:26.069 INFO:tasks.workunit.client.0.smithi110.stdout:9/285: mkdir d9/d13/d5b 0 2024-04-25T15:49:26.070 INFO:tasks.workunit.client.1.smithi193.stdout:9/38: creat fb x:0 0 0 2024-04-25T15:49:26.070 INFO:tasks.workunit.client.1.smithi193.stdout:9/39: chown l7 6257 1 2024-04-25T15:49:26.072 INFO:tasks.workunit.client.1.smithi193.stdout:9/40: dread - f9 zero size 2024-04-25T15:49:26.072 INFO:tasks.workunit.client.1.smithi193.stdout:9/41: readlink l2 0 2024-04-25T15:49:26.072 INFO:tasks.workunit.client.1.smithi193.stdout:9/42: rmdir - no directory 2024-04-25T15:49:26.072 INFO:tasks.workunit.client.1.smithi193.stdout:9/43: rmdir - no directory 2024-04-25T15:49:26.072 INFO:tasks.workunit.client.0.smithi110.stdout:5/327: creat d4/d1d/d1f/d35/d3d/d52/f6f x:0 0 0 2024-04-25T15:49:26.073 INFO:tasks.workunit.client.1.smithi193.stdout:9/44: truncate f4 539111 0 2024-04-25T15:49:26.073 INFO:tasks.workunit.client.0.smithi110.stdout:8/455: sync 2024-04-25T15:49:26.073 INFO:tasks.workunit.client.0.smithi110.stdout:3/452: sync 2024-04-25T15:49:26.073 INFO:tasks.workunit.client.0.smithi110.stdout:0/391: sync 2024-04-25T15:49:26.073 INFO:tasks.workunit.client.0.smithi110.stdout:1/479: sync 2024-04-25T15:49:26.074 INFO:tasks.workunit.client.0.smithi110.stdout:6/318: sync 2024-04-25T15:49:26.074 INFO:tasks.workunit.client.0.smithi110.stdout:7/334: sync 2024-04-25T15:49:26.074 INFO:tasks.workunit.client.0.smithi110.stdout:2/320: sync 2024-04-25T15:49:26.075 INFO:tasks.workunit.client.0.smithi110.stdout:8/456: fsync d14/f5c 0 2024-04-25T15:49:26.082 INFO:tasks.workunit.client.0.smithi110.stdout:5/328: rename d4/d15/d51/f6c to d4/d1d/f70 0 2024-04-25T15:49:26.083 INFO:tasks.workunit.client.0.smithi110.stdout:5/329: fdatasync d4/d2c/d2a/f5c 0 2024-04-25T15:49:26.084 INFO:tasks.workunit.client.0.smithi110.stdout:1/480: creat d0/f91 x:0 0 0 2024-04-25T15:49:26.085 INFO:tasks.workunit.client.0.smithi110.stdout:1/481: stat d0/d14/d5f/l87 0 2024-04-25T15:49:26.086 INFO:tasks.workunit.client.1.smithi193.stdout:4/22: mknod c6 0 2024-04-25T15:49:26.088 INFO:tasks.workunit.client.1.smithi193.stdout:2/26: creat d0/fd x:0 0 0 2024-04-25T15:49:26.089 INFO:tasks.workunit.client.1.smithi193.stdout:6/33: getdents . 0 2024-04-25T15:49:26.089 INFO:tasks.workunit.client.1.smithi193.stdout:6/34: stat f6 0 2024-04-25T15:49:26.093 INFO:tasks.workunit.client.1.smithi193.stdout:3/30: mkdir d8/db 0 2024-04-25T15:49:26.095 INFO:tasks.workunit.client.1.smithi193.stdout:8/19: getdents d2 0 2024-04-25T15:49:26.097 INFO:tasks.workunit.client.0.smithi110.stdout:4/443: dread d13/d24/d2d/d44/f60 [0,4194304] 0 2024-04-25T15:49:26.097 INFO:tasks.workunit.client.0.smithi110.stdout:0/392: rmdir d1/d23/d30/d33/d5b/d72/d77 39 2024-04-25T15:49:26.098 INFO:tasks.workunit.client.0.smithi110.stdout:0/393: fsync d1/d23/d30/d33/d5b/f98 0 2024-04-25T15:49:26.100 INFO:tasks.workunit.client.0.smithi110.stdout:3/453: creat d0/d7/d1c/d2f/d33/f9e x:0 0 0 2024-04-25T15:49:26.102 INFO:tasks.workunit.client.1.smithi193.stdout:8/20: dwrite d2/d3/f5 [0,4194304] 0 2024-04-25T15:49:26.102 INFO:tasks.workunit.client.1.smithi193.stdout:2/27: symlink d0/le 0 2024-04-25T15:49:26.102 INFO:tasks.workunit.client.1.smithi193.stdout:2/28: readlink d0/l8 0 2024-04-25T15:49:26.103 INFO:tasks.workunit.client.1.smithi193.stdout:2/29: fsync d0/f6 0 2024-04-25T15:49:26.103 INFO:tasks.workunit.client.1.smithi193.stdout:8/21: truncate d2/d3/f5 4778110 0 2024-04-25T15:49:26.103 INFO:tasks.workunit.client.1.smithi193.stdout:2/30: dread - d0/fa zero size 2024-04-25T15:49:26.104 INFO:tasks.workunit.client.1.smithi193.stdout:2/31: truncate d0/f7 564137 0 2024-04-25T15:49:26.107 INFO:tasks.workunit.client.1.smithi193.stdout:3/31: creat d8/fc x:0 0 0 2024-04-25T15:49:26.107 INFO:tasks.workunit.client.1.smithi193.stdout:3/32: read - f3 zero size 2024-04-25T15:49:26.109 INFO:tasks.workunit.client.0.smithi110.stdout:8/457: dwrite d14/d32/d36/f84 [0,4194304] 0 2024-04-25T15:49:26.112 INFO:tasks.workunit.client.1.smithi193.stdout:8/22: dwrite d2/d3/f5 [0,4194304] 0 2024-04-25T15:49:26.112 INFO:tasks.workunit.client.0.smithi110.stdout:5/330: rmdir d4/d1d 39 2024-04-25T15:49:26.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/23: write d2/d3/f5 [3765853,81382] 0 2024-04-25T15:49:26.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/24: readlink - no filename 2024-04-25T15:49:26.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/25: readlink - no filename 2024-04-25T15:49:26.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/26: readlink - no filename 2024-04-25T15:49:26.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/27: readlink - no filename 2024-04-25T15:49:26.117 INFO:tasks.workunit.client.1.smithi193.stdout:6/35: getdents . 0 2024-04-25T15:49:26.117 INFO:tasks.workunit.client.0.smithi110.stdout:8/458: dwrite f6 [0,4194304] 0 2024-04-25T15:49:26.118 INFO:tasks.workunit.client.0.smithi110.stdout:8/459: read d14/f1d [897421,81299] 0 2024-04-25T15:49:26.119 INFO:tasks.workunit.client.0.smithi110.stdout:6/319: symlink d1/db/d24/d2d/l77 0 2024-04-25T15:49:26.121 INFO:tasks.workunit.client.0.smithi110.stdout:2/321: mkdir de/d40/d5b/d5c 0 2024-04-25T15:49:26.122 INFO:tasks.workunit.client.0.smithi110.stdout:6/320: truncate d1/db/d24/d32/f5a 823111 0 2024-04-25T15:49:26.129 INFO:tasks.workunit.client.0.smithi110.stdout:6/321: dwrite d1/f26 [0,4194304] 0 2024-04-25T15:49:26.130 INFO:tasks.workunit.client.0.smithi110.stdout:4/444: creat d13/d24/d3c/d56/f8e x:0 0 0 2024-04-25T15:49:26.131 INFO:tasks.workunit.client.0.smithi110.stdout:6/322: read d1/db/d24/d32/f5a [814614,94499] 0 2024-04-25T15:49:26.132 INFO:tasks.workunit.client.0.smithi110.stdout:3/454: mkdir d0/d2e/d9f 0 2024-04-25T15:49:26.134 INFO:tasks.workunit.client.1.smithi193.stdout:2/32: mkdir d0/df 0 2024-04-25T15:49:26.135 INFO:tasks.workunit.client.1.smithi193.stdout:3/33: mknod d8/cd 0 2024-04-25T15:49:26.135 INFO:tasks.workunit.client.1.smithi193.stdout:3/34: chown d8 200869 1 2024-04-25T15:49:26.135 INFO:tasks.workunit.client.1.smithi193.stdout:2/33: write d0/fd [504772,113159] 0 2024-04-25T15:49:26.136 INFO:tasks.workunit.client.1.smithi193.stdout:2/34: stat d0/le 0 2024-04-25T15:49:26.138 INFO:tasks.workunit.client.1.smithi193.stdout:8/28: rmdir d2 39 2024-04-25T15:49:26.142 INFO:tasks.workunit.client.1.smithi193.stdout:9/45: getdents . 0 2024-04-25T15:49:26.142 INFO:tasks.workunit.client.0.smithi110.stdout:8/460: rmdir d14/d21/d26 39 2024-04-25T15:49:26.143 INFO:tasks.workunit.client.1.smithi193.stdout:9/46: write fb [307528,14361] 0 2024-04-25T15:49:26.144 INFO:tasks.workunit.client.1.smithi193.stdout:4/23: getdents . 0 2024-04-25T15:49:26.144 INFO:tasks.workunit.client.1.smithi193.stdout:9/47: read - f9 zero size 2024-04-25T15:49:26.146 INFO:tasks.workunit.client.1.smithi193.stdout:2/35: dwrite d0/f7 [0,4194304] 0 2024-04-25T15:49:26.146 INFO:tasks.workunit.client.1.smithi193.stdout:9/48: write fb [711104,123939] 0 2024-04-25T15:49:26.146 INFO:tasks.workunit.client.1.smithi193.stdout:3/35: mknod d8/ce 0 2024-04-25T15:49:26.146 INFO:tasks.workunit.client.1.smithi193.stdout:9/49: dread - f6 zero size 2024-04-25T15:49:26.147 INFO:tasks.workunit.client.1.smithi193.stdout:9/50: truncate f9 572228 0 2024-04-25T15:49:26.147 INFO:tasks.workunit.client.1.smithi193.stdout:9/51: stat f6 0 2024-04-25T15:49:26.147 INFO:tasks.workunit.client.1.smithi193.stdout:9/52: fsync f9 0 2024-04-25T15:49:26.148 INFO:tasks.workunit.client.0.smithi110.stdout:2/322: unlink de/d11/f19 0 2024-04-25T15:49:26.149 INFO:tasks.workunit.client.0.smithi110.stdout:2/323: fdatasync de/d11/f21 0 2024-04-25T15:49:26.151 INFO:tasks.workunit.client.0.smithi110.stdout:4/445: creat d13/d24/d3c/d56/d88/f8f x:0 0 0 2024-04-25T15:49:26.152 INFO:tasks.workunit.client.0.smithi110.stdout:6/323: creat d1/d12/d3d/f78 x:0 0 0 2024-04-25T15:49:26.157 INFO:tasks.workunit.client.0.smithi110.stdout:9/286: getdents d9 0 2024-04-25T15:49:26.157 INFO:tasks.workunit.client.0.smithi110.stdout:3/455: creat d0/d7/d1c/d2f/d4c/fa0 x:0 0 0 2024-04-25T15:49:26.159 INFO:tasks.workunit.client.1.smithi193.stdout:8/29: fsync d2/d3/f5 0 2024-04-25T15:49:26.160 INFO:tasks.workunit.client.0.smithi110.stdout:5/331: mknod d4/d1d/d1f/d56/c71 0 2024-04-25T15:49:26.160 INFO:tasks.workunit.client.0.smithi110.stdout:5/332: fdatasync d4/d2c/f4f 0 2024-04-25T15:49:26.161 INFO:tasks.workunit.client.0.smithi110.stdout:1/482: creat d0/d23/f92 x:0 0 0 2024-04-25T15:49:26.162 INFO:tasks.workunit.client.0.smithi110.stdout:1/483: readlink d0/d25/d33/l38 0 2024-04-25T15:49:26.167 INFO:tasks.workunit.client.1.smithi193.stdout:9/53: creat fc x:0 0 0 2024-04-25T15:49:26.171 INFO:tasks.workunit.client.1.smithi193.stdout:6/36: link l1 l7 0 2024-04-25T15:49:26.174 INFO:tasks.workunit.client.0.smithi110.stdout:2/324: symlink de/d40/d5b/l5d 0 2024-04-25T15:49:26.178 INFO:tasks.workunit.client.1.smithi193.stdout:2/36: creat d0/df/f10 x:0 0 0 2024-04-25T15:49:26.178 INFO:tasks.workunit.client.1.smithi193.stdout:2/37: fsync d0/fa 0 2024-04-25T15:49:26.179 INFO:tasks.workunit.client.1.smithi193.stdout:2/38: fdatasync d0/fc 0 2024-04-25T15:49:26.179 INFO:tasks.workunit.client.1.smithi193.stdout:2/39: fdatasync d0/fd 0 2024-04-25T15:49:26.179 INFO:tasks.workunit.client.1.smithi193.stdout:2/40: chown d0/fd 3 1 2024-04-25T15:49:26.180 INFO:tasks.workunit.client.1.smithi193.stdout:2/41: fdatasync d0/fd 0 2024-04-25T15:49:26.180 INFO:tasks.workunit.client.1.smithi193.stdout:8/30: symlink d2/l6 0 2024-04-25T15:49:26.181 INFO:tasks.workunit.client.0.smithi110.stdout:4/446: dwrite d13/d24/d3c/d56/f66 [0,4194304] 0 2024-04-25T15:49:26.184 INFO:tasks.workunit.client.0.smithi110.stdout:3/456: mknod d0/d7/d1c/d2f/d33/ca1 0 2024-04-25T15:49:26.185 INFO:tasks.workunit.client.0.smithi110.stdout:9/287: symlink d9/db/df/l5c 0 2024-04-25T15:49:26.186 INFO:tasks.workunit.client.0.smithi110.stdout:9/288: stat d9/d13/l18 0 2024-04-25T15:49:26.186 INFO:tasks.workunit.client.0.smithi110.stdout:3/457: read - d0/d7/d1c/d50/f9a zero size 2024-04-25T15:49:26.187 INFO:tasks.workunit.client.0.smithi110.stdout:6/324: dwrite d1/db/d24/d2d/f34 [0,4194304] 0 2024-04-25T15:49:26.188 INFO:tasks.workunit.client.1.smithi193.stdout:6/37: creat f8 x:0 0 0 2024-04-25T15:49:26.188 INFO:tasks.workunit.client.1.smithi193.stdout:6/38: truncate f6 217990 0 2024-04-25T15:49:26.191 INFO:tasks.workunit.client.0.smithi110.stdout:7/335: link d1b/c20 d1b/d29/c6d 0 2024-04-25T15:49:26.194 INFO:tasks.workunit.client.1.smithi193.stdout:2/42: creat d0/f11 x:0 0 0 2024-04-25T15:49:26.194 INFO:tasks.workunit.client.0.smithi110.stdout:8/461: mknod d14/c99 0 2024-04-25T15:49:26.196 INFO:tasks.workunit.client.1.smithi193.stdout:8/31: creat d2/d3/f7 x:0 0 0 2024-04-25T15:49:26.196 INFO:tasks.workunit.client.0.smithi110.stdout:8/462: dread - d14/d4c/d55/f96 zero size 2024-04-25T15:49:26.197 INFO:tasks.workunit.client.0.smithi110.stdout:3/458: dwrite d0/d2e/f4f [0,4194304] 0 2024-04-25T15:49:26.198 INFO:tasks.workunit.client.1.smithi193.stdout:8/32: dread d2/d3/f5 [4194304,4194304] 0 2024-04-25T15:49:26.202 INFO:tasks.workunit.client.1.smithi193.stdout:4/24: truncate f1 4065432 0 2024-04-25T15:49:26.204 INFO:tasks.workunit.client.1.smithi193.stdout:6/39: symlink l9 0 2024-04-25T15:49:26.205 INFO:tasks.workunit.client.0.smithi110.stdout:4/447: mkdir d13/d24/d2d/d44/d90 0 2024-04-25T15:49:26.208 INFO:tasks.workunit.client.1.smithi193.stdout:8/33: creat d2/f8 x:0 0 0 2024-04-25T15:49:26.209 INFO:tasks.workunit.client.0.smithi110.stdout:8/463: dwrite d14/f83 [0,4194304] 0 2024-04-25T15:49:26.211 INFO:tasks.workunit.client.0.smithi110.stdout:8/464: write d14/d21/f92 [467214,109709] 0 2024-04-25T15:49:26.212 INFO:tasks.workunit.client.1.smithi193.stdout:3/36: getdents d8 0 2024-04-25T15:49:26.215 INFO:tasks.workunit.client.0.smithi110.stdout:9/289: rename d9/db/f15 to d9/d2d/f5d 0 2024-04-25T15:49:26.218 INFO:tasks.workunit.client.1.smithi193.stdout:2/43: fsync d0/f11 0 2024-04-25T15:49:26.220 INFO:tasks.workunit.client.0.smithi110.stdout:5/333: mknod d4/c72 0 2024-04-25T15:49:26.220 INFO:tasks.workunit.client.0.smithi110.stdout:9/290: stat d9/db/df/d25/c26 0 2024-04-25T15:49:26.224 INFO:tasks.workunit.client.1.smithi193.stdout:8/34: creat d2/f9 x:0 0 0 2024-04-25T15:49:26.224 INFO:tasks.workunit.client.0.smithi110.stdout:9/291: read d9/d2d/f30 [2212828,90565] 0 2024-04-25T15:49:26.228 INFO:tasks.workunit.client.0.smithi110.stdout:3/459: unlink d0/d7/d1c/d2f/d4c/f78 0 2024-04-25T15:49:26.228 INFO:tasks.workunit.client.0.smithi110.stdout:2/325: dwrite de/f28 [0,4194304] 0 2024-04-25T15:49:26.229 INFO:tasks.workunit.client.0.smithi110.stdout:9/292: write d9/db/df/f4f [761565,25817] 0 2024-04-25T15:49:26.229 INFO:tasks.workunit.client.0.smithi110.stdout:3/460: fdatasync d0/d7/f17 0 2024-04-25T15:49:26.230 INFO:tasks.workunit.client.1.smithi193.stdout:3/37: creat d8/ff x:0 0 0 2024-04-25T15:49:26.230 INFO:tasks.workunit.client.1.smithi193.stdout:3/38: chown d8/db 24421 1 2024-04-25T15:49:26.233 INFO:tasks.workunit.client.0.smithi110.stdout:2/326: read - de/d40/f36 zero size 2024-04-25T15:49:26.234 INFO:tasks.workunit.client.0.smithi110.stdout:6/325: mkdir d1/d79 0 2024-04-25T15:49:26.234 INFO:tasks.workunit.client.1.smithi193.stdout:9/54: getdents . 0 2024-04-25T15:49:26.235 INFO:tasks.workunit.client.0.smithi110.stdout:6/326: stat d1/l7 0 2024-04-25T15:49:26.239 INFO:tasks.workunit.client.1.smithi193.stdout:2/44: rename d0/l8 to d0/l12 0 2024-04-25T15:49:26.241 INFO:tasks.workunit.client.1.smithi193.stdout:9/55: dwrite fc [0,4194304] 0 2024-04-25T15:49:26.241 INFO:tasks.workunit.client.1.smithi193.stdout:9/56: readlink l1 0 2024-04-25T15:49:26.243 INFO:tasks.workunit.client.1.smithi193.stdout:6/40: rename l7 to la 0 2024-04-25T15:49:26.245 INFO:tasks.workunit.client.0.smithi110.stdout:7/336: dwrite d1b/d29/d38/f54 [0,4194304] 0 2024-04-25T15:49:26.246 INFO:tasks.workunit.client.0.smithi110.stdout:7/337: write f19 [5091670,55759] 0 2024-04-25T15:49:26.249 INFO:tasks.workunit.client.1.smithi193.stdout:9/57: dwrite fc [0,4194304] 0 2024-04-25T15:49:26.249 INFO:tasks.workunit.client.1.smithi193.stdout:9/58: rmdir - no directory 2024-04-25T15:49:26.254 INFO:tasks.workunit.client.1.smithi193.stdout:4/25: write f1 [2353129,42195] 0 2024-04-25T15:49:26.258 INFO:tasks.workunit.client.1.smithi193.stdout:9/59: dwrite fb [0,4194304] 0 2024-04-25T15:49:26.262 INFO:tasks.workunit.client.1.smithi193.stdout:2/45: symlink d0/l13 0 2024-04-25T15:49:26.268 INFO:tasks.workunit.client.1.smithi193.stdout:3/39: dwrite f2 [0,4194304] 0 2024-04-25T15:49:26.270 INFO:tasks.workunit.client.1.smithi193.stdout:3/40: write f3 [325764,122598] 0 2024-04-25T15:49:26.270 INFO:tasks.workunit.client.1.smithi193.stdout:3/41: chown d8/ff 3 1 2024-04-25T15:49:26.271 INFO:tasks.workunit.client.1.smithi193.stdout:3/42: readlink d8/l9 0 2024-04-25T15:49:26.272 INFO:tasks.workunit.client.1.smithi193.stdout:8/35: rename d2/f9 to d2/fa 0 2024-04-25T15:49:26.273 INFO:tasks.workunit.client.0.smithi110.stdout:5/334: creat d4/d15/f73 x:0 0 0 2024-04-25T15:49:26.273 INFO:tasks.workunit.client.0.smithi110.stdout:5/335: truncate d4/f10 4279019 0 2024-04-25T15:49:26.278 INFO:tasks.workunit.client.1.smithi193.stdout:4/26: symlink l7 0 2024-04-25T15:49:26.278 INFO:tasks.workunit.client.1.smithi193.stdout:4/27: dread - f4 zero size 2024-04-25T15:49:26.279 INFO:tasks.workunit.client.1.smithi193.stdout:4/28: write f2 [311050,123194] 0 2024-04-25T15:49:26.284 INFO:tasks.workunit.client.1.smithi193.stdout:6/41: unlink la 0 2024-04-25T15:49:26.284 INFO:tasks.workunit.client.0.smithi110.stdout:9/293: dread - d9/f39 zero size 2024-04-25T15:49:26.284 INFO:tasks.workunit.client.1.smithi193.stdout:6/42: write f8 [660753,20997] 0 2024-04-25T15:49:26.285 INFO:tasks.workunit.client.1.smithi193.stdout:3/43: creat d8/f10 x:0 0 0 2024-04-25T15:49:26.286 INFO:tasks.workunit.client.1.smithi193.stdout:3/44: truncate d8/ff 348328 0 2024-04-25T15:49:26.286 INFO:tasks.workunit.client.1.smithi193.stdout:6/43: read f8 [429080,14136] 0 2024-04-25T15:49:26.288 INFO:tasks.workunit.client.1.smithi193.stdout:8/36: unlink d2/f8 0 2024-04-25T15:49:26.289 INFO:tasks.workunit.client.0.smithi110.stdout:3/461: mknod d0/d7/d1c/d50/ca2 0 2024-04-25T15:49:26.290 INFO:tasks.workunit.client.1.smithi193.stdout:2/46: mkdir d0/d9/d14 0 2024-04-25T15:49:26.293 INFO:tasks.workunit.client.1.smithi193.stdout:8/37: dwrite d2/d3/f7 [0,4194304] 0 2024-04-25T15:49:26.296 INFO:tasks.workunit.client.0.smithi110.stdout:5/336: dread d4/d2c/f27 [0,4194304] 0 2024-04-25T15:49:26.296 INFO:tasks.workunit.client.0.smithi110.stdout:3/462: dwrite d0/d7/d1c/d50/f59 [0,4194304] 0 2024-04-25T15:49:26.297 INFO:tasks.workunit.client.1.smithi193.stdout:3/45: rename f3 to d8/f11 0 2024-04-25T15:49:26.299 INFO:tasks.workunit.client.1.smithi193.stdout:6/44: creat fb x:0 0 0 2024-04-25T15:49:26.300 INFO:tasks.workunit.client.1.smithi193.stdout:6/45: dread - fb zero size 2024-04-25T15:49:26.300 INFO:tasks.workunit.client.1.smithi193.stdout:6/46: write fb [155596,128715] 0 2024-04-25T15:49:26.304 INFO:tasks.workunit.client.0.smithi110.stdout:2/327: symlink de/l5e 0 2024-04-25T15:49:26.305 INFO:tasks.workunit.client.0.smithi110.stdout:2/328: chown de/d11/c51 243749 1 2024-04-25T15:49:26.307 INFO:tasks.workunit.client.1.smithi193.stdout:2/47: mkdir d0/d9/d15 0 2024-04-25T15:49:26.308 INFO:tasks.workunit.client.0.smithi110.stdout:5/337: dread d4/d2c/d2a/f5c [0,4194304] 0 2024-04-25T15:49:26.311 INFO:tasks.workunit.client.0.smithi110.stdout:6/327: creat d1/d79/f7a x:0 0 0 2024-04-25T15:49:26.313 INFO:tasks.workunit.client.0.smithi110.stdout:6/328: dread d1/dd/d3c/f51 [0,4194304] 0 2024-04-25T15:49:26.313 INFO:tasks.workunit.client.1.smithi193.stdout:8/38: mkdir d2/d3/db 0 2024-04-25T15:49:26.314 INFO:tasks.workunit.client.0.smithi110.stdout:4/448: rename d13/d24/d3c/d56/c6d to d13/d24/d2d/c91 0 2024-04-25T15:49:26.315 INFO:tasks.workunit.client.0.smithi110.stdout:4/449: dread - d13/d24/d2d/d44/d65/f84 zero size 2024-04-25T15:49:26.319 INFO:tasks.workunit.client.1.smithi193.stdout:2/48: mkdir d0/df/d16 0 2024-04-25T15:49:26.320 INFO:tasks.workunit.client.1.smithi193.stdout:2/49: chown d0/f7 35 1 2024-04-25T15:49:26.320 INFO:tasks.workunit.client.1.smithi193.stdout:3/46: mkdir d8/db/d12 0 2024-04-25T15:49:26.321 INFO:tasks.workunit.client.0.smithi110.stdout:5/338: symlink d4/d1d/l74 0 2024-04-25T15:49:26.321 INFO:tasks.workunit.client.1.smithi193.stdout:3/47: readlink d8/l9 0 2024-04-25T15:49:26.321 INFO:tasks.workunit.client.1.smithi193.stdout:6/47: dread f8 [0,4194304] 0 2024-04-25T15:49:26.321 INFO:tasks.workunit.client.1.smithi193.stdout:6/48: readlink l9 0 2024-04-25T15:49:26.323 INFO:tasks.workunit.client.0.smithi110.stdout:5/339: read d4/d1d/d1f/f47 [3954183,33170] 0 2024-04-25T15:49:26.323 INFO:tasks.workunit.client.1.smithi193.stdout:2/50: mknod d0/c17 0 2024-04-25T15:49:26.324 INFO:tasks.workunit.client.1.smithi193.stdout:2/51: truncate d0/df/f10 908627 0 2024-04-25T15:49:26.325 INFO:tasks.workunit.client.1.smithi193.stdout:2/52: dread - d0/fc zero size 2024-04-25T15:49:26.325 INFO:tasks.workunit.client.0.smithi110.stdout:8/465: rename d14/d32/d36/c39 to d14/d21/d26/d6a/d7e/c9a 0 2024-04-25T15:49:26.325 INFO:tasks.workunit.client.1.smithi193.stdout:2/53: read d0/df/f10 [254223,77456] 0 2024-04-25T15:49:26.326 INFO:tasks.workunit.client.0.smithi110.stdout:4/450: rmdir d13/d24/d26 39 2024-04-25T15:49:26.328 INFO:tasks.workunit.client.0.smithi110.stdout:4/451: truncate d13/d24/d2d/d44/d65/f72 356900 0 2024-04-25T15:49:26.328 INFO:tasks.workunit.client.0.smithi110.stdout:7/338: creat d1b/f6e x:0 0 0 2024-04-25T15:49:26.330 INFO:tasks.workunit.client.0.smithi110.stdout:4/452: write d13/d24/d2d/d44/f73 [433735,64573] 0 2024-04-25T15:49:26.331 INFO:tasks.workunit.client.0.smithi110.stdout:3/463: mknod d0/d7/d40/ca3 0 2024-04-25T15:49:26.332 INFO:tasks.workunit.client.0.smithi110.stdout:3/464: write d0/d7/f8b [782103,11314] 0 2024-04-25T15:49:26.333 INFO:tasks.workunit.client.0.smithi110.stdout:2/329: mkdir de/d40/d5b/d5c/d5f 0 2024-04-25T15:49:26.336 INFO:tasks.workunit.client.1.smithi193.stdout:3/48: rename d8/db to d8/d13 0 2024-04-25T15:49:26.338 INFO:tasks.workunit.client.0.smithi110.stdout:5/340: mkdir d4/d1d/d1f/d35/d3d/d52/d75 0 2024-04-25T15:49:26.339 INFO:tasks.workunit.client.0.smithi110.stdout:6/329: write d1/db/d24/d32/f5a [1633101,69928] 0 2024-04-25T15:49:26.340 INFO:tasks.workunit.client.0.smithi110.stdout:5/341: dread - d4/d1d/d1f/f6e zero size 2024-04-25T15:49:26.340 INFO:tasks.workunit.client.1.smithi193.stdout:3/49: fsync d8/f11 0 2024-04-25T15:49:26.340 INFO:tasks.workunit.client.1.smithi193.stdout:3/50: fdatasync f4 0 2024-04-25T15:49:26.341 INFO:tasks.workunit.client.1.smithi193.stdout:3/51: dread - d8/f10 zero size 2024-04-25T15:49:26.341 INFO:tasks.workunit.client.1.smithi193.stdout:3/52: write d8/f10 [660435,102670] 0 2024-04-25T15:49:26.345 INFO:tasks.workunit.client.1.smithi193.stdout:3/53: mkdir d8/d13/d14 0 2024-04-25T15:49:26.347 INFO:tasks.workunit.client.0.smithi110.stdout:3/465: unlink d0/d2e/f3b 0 2024-04-25T15:49:26.348 INFO:tasks.workunit.client.0.smithi110.stdout:4/453: dread d13/d24/d3c/d56/d88/f47 [0,4194304] 0 2024-04-25T15:49:26.349 INFO:tasks.workunit.client.0.smithi110.stdout:8/466: symlink d14/d21/l9b 0 2024-04-25T15:49:26.351 INFO:tasks.workunit.client.1.smithi193.stdout:3/54: dwrite f4 [0,4194304] 0 2024-04-25T15:49:26.352 INFO:tasks.workunit.client.0.smithi110.stdout:4/454: dread d13/d24/d3c/d56/d88/f2c [0,4194304] 0 2024-04-25T15:49:26.353 INFO:tasks.workunit.client.0.smithi110.stdout:4/455: chown d13/d24/d3c/d56/d88/f41 134 1 2024-04-25T15:49:26.356 INFO:tasks.workunit.client.0.smithi110.stdout:8/467: mknod d14/d4c/c9c 0 2024-04-25T15:49:26.356 INFO:tasks.workunit.client.0.smithi110.stdout:7/339: link d1b/c20 d1b/d29/d46/c6f 0 2024-04-25T15:49:26.357 INFO:tasks.workunit.client.0.smithi110.stdout:2/330: link de/d11/l1e de/d11/d50/l60 0 2024-04-25T15:49:26.359 INFO:tasks.workunit.client.0.smithi110.stdout:2/331: write de/f28 [4035,41855] 0 2024-04-25T15:49:26.360 INFO:tasks.workunit.client.0.smithi110.stdout:2/332: chown de/f28 22825 1 2024-04-25T15:49:26.360 INFO:tasks.workunit.client.0.smithi110.stdout:2/333: readlink de/d40/d5b/l5d 0 2024-04-25T15:49:26.361 INFO:tasks.workunit.client.0.smithi110.stdout:4/456: mkdir d13/d24/d2d/d7d/d92 0 2024-04-25T15:49:26.362 INFO:tasks.workunit.client.0.smithi110.stdout:4/457: write d13/f77 [653812,11286] 0 2024-04-25T15:49:26.363 INFO:tasks.workunit.client.0.smithi110.stdout:6/330: creat d1/db/d24/d2d/f7b x:0 0 0 2024-04-25T15:49:26.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/55: rmdir d8 39 2024-04-25T15:49:26.370 INFO:tasks.workunit.client.0.smithi110.stdout:6/331: dwrite d1/d12/d3d/f78 [0,4194304] 0 2024-04-25T15:49:26.372 INFO:tasks.workunit.client.0.smithi110.stdout:3/466: creat d0/d7/d1c/d2f/fa4 x:0 0 0 2024-04-25T15:49:26.372 INFO:tasks.workunit.client.0.smithi110.stdout:8/468: dwrite d14/f1d [0,4194304] 0 2024-04-25T15:49:26.373 INFO:tasks.workunit.client.0.smithi110.stdout:6/332: truncate d1/db/f62 310154 0 2024-04-25T15:49:26.374 INFO:tasks.workunit.client.1.smithi193.stdout:3/56: creat d8/f15 x:0 0 0 2024-04-25T15:49:26.378 INFO:tasks.workunit.client.0.smithi110.stdout:2/334: fsync de/d40/f33 0 2024-04-25T15:49:26.380 INFO:tasks.workunit.client.0.smithi110.stdout:4/458: rmdir d13/d24/d2d/d44 39 2024-04-25T15:49:26.383 INFO:tasks.workunit.client.1.smithi193.stdout:3/57: unlink f5 0 2024-04-25T15:49:26.385 INFO:tasks.workunit.client.0.smithi110.stdout:5/342: getdents d4/d1d/d1f/d35/d3d 0 2024-04-25T15:49:26.385 INFO:tasks.workunit.client.1.smithi193.stdout:3/58: read d8/f11 [319334,20273] 0 2024-04-25T15:49:26.386 INFO:tasks.workunit.client.0.smithi110.stdout:5/343: chown d4/d1d/d1f/c4a 118534969 1 2024-04-25T15:49:26.387 INFO:tasks.workunit.client.0.smithi110.stdout:4/459: dwrite d13/d24/d2d/d5e/f7b [0,4194304] 0 2024-04-25T15:49:26.389 INFO:tasks.workunit.client.1.smithi193.stdout:3/59: rename f4 to d8/d13/d12/f16 0 2024-04-25T15:49:26.393 INFO:tasks.workunit.client.0.smithi110.stdout:3/467: unlink d0/d2e/f31 0 2024-04-25T15:49:26.394 INFO:tasks.workunit.client.0.smithi110.stdout:3/468: dread - d0/d7/d1c/f94 zero size 2024-04-25T15:49:26.394 INFO:tasks.workunit.client.0.smithi110.stdout:8/469: mkdir d14/d4c/d71/d9d 0 2024-04-25T15:49:26.395 INFO:tasks.workunit.client.0.smithi110.stdout:6/333: creat d1/dd/d3c/f7c x:0 0 0 2024-04-25T15:49:26.395 INFO:tasks.workunit.client.0.smithi110.stdout:8/470: read - d14/d21/f62 zero size 2024-04-25T15:49:26.396 INFO:tasks.workunit.client.0.smithi110.stdout:6/334: fdatasync d1/dd/f21 0 2024-04-25T15:49:26.397 INFO:tasks.workunit.client.1.smithi193.stdout:3/60: creat d8/d13/d14/f17 x:0 0 0 2024-04-25T15:49:26.403 INFO:tasks.workunit.client.0.smithi110.stdout:4/460: mknod d13/d24/d2d/d55/c93 0 2024-04-25T15:49:26.406 INFO:tasks.workunit.client.0.smithi110.stdout:2/335: dwrite de/d11/d52/f59 [4194304,4194304] 0 2024-04-25T15:49:26.406 INFO:tasks.workunit.client.0.smithi110.stdout:4/461: fdatasync d13/d24/d2d/d36/d51/f54 0 2024-04-25T15:49:26.406 INFO:tasks.workunit.client.0.smithi110.stdout:7/340: rmdir d1b/d29/d38/d62 0 2024-04-25T15:49:26.407 INFO:tasks.workunit.client.0.smithi110.stdout:3/469: mkdir d0/d16/d8c/d90/da5 0 2024-04-25T15:49:26.409 INFO:tasks.workunit.client.1.smithi193.stdout:3/61: rmdir d8/d13/d14 39 2024-04-25T15:49:26.410 INFO:tasks.workunit.client.0.smithi110.stdout:8/471: symlink d14/d4c/d55/l9e 0 2024-04-25T15:49:26.412 INFO:tasks.workunit.client.0.smithi110.stdout:6/335: creat d1/dd/d3c/f7d x:0 0 0 2024-04-25T15:49:26.414 INFO:tasks.workunit.client.0.smithi110.stdout:5/344: dread d4/d1d/d1f/d35/f39 [0,4194304] 0 2024-04-25T15:49:26.415 INFO:tasks.workunit.client.1.smithi193.stdout:3/62: truncate d8/d13/d14/f17 628110 0 2024-04-25T15:49:26.415 INFO:tasks.workunit.client.0.smithi110.stdout:5/345: dread d4/d2c/f4f [0,4194304] 0 2024-04-25T15:49:26.420 INFO:tasks.workunit.client.0.smithi110.stdout:4/462: creat d13/d24/d2d/d5e/f94 x:0 0 0 2024-04-25T15:49:26.423 INFO:tasks.workunit.client.0.smithi110.stdout:3/470: unlink d0/d7/d1c/l20 0 2024-04-25T15:49:26.423 INFO:tasks.workunit.client.0.smithi110.stdout:3/471: chown d0/d7/l60 228 1 2024-04-25T15:49:26.423 INFO:tasks.workunit.client.1.smithi193.stdout:3/63: dwrite d8/d13/d12/f16 [0,4194304] 0 2024-04-25T15:49:26.424 INFO:tasks.workunit.client.1.smithi193.stdout:3/64: chown d8/l9 332 1 2024-04-25T15:49:26.424 INFO:tasks.workunit.client.0.smithi110.stdout:8/472: rmdir d14/d21/d23 39 2024-04-25T15:49:26.424 INFO:tasks.workunit.client.1.smithi193.stdout:3/65: chown d8/d13/d12 252329157 1 2024-04-25T15:49:26.428 INFO:tasks.workunit.client.1.smithi193.stdout:3/66: dread d8/f11 [0,4194304] 0 2024-04-25T15:49:26.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.429+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f359810f8d0 con 0x7f358006fcb0 2024-04-25T15:49:26.430 INFO:tasks.workunit.client.0.smithi110.stdout:4/463: creat d13/d24/d2d/d36/f95 x:0 0 0 2024-04-25T15:49:26.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.430+0000 7f357f7fe700 1 -- 172.21.15.110:0/1899310401 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f359810f8d0 con 0x7f358006fcb0 2024-04-25T15:49:26.431 INFO:tasks.workunit.client.0.smithi110.stdout:4/464: stat d13/d24/d3c/d56/d88/f2c 0 2024-04-25T15:49:26.432 INFO:tasks.workunit.client.0.smithi110.stdout:4/465: chown d13/d24/d2d/d5e/f71 1 1 2024-04-25T15:49:26.432 INFO:tasks.workunit.client.0.smithi110.stdout:3/472: rmdir d0/d2e 39 2024-04-25T15:49:26.434 INFO:tasks.workunit.client.1.smithi193.stdout:3/67: creat d8/d13/d14/f18 x:0 0 0 2024-04-25T15:49:26.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f358006fcb0 msgr2=0x7f3580072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f358006fcb0 0x7f3580072160 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3598066830 tx=0x7f35880091a0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35980fff70 msgr2=0x7f359810dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35980fff70 0x7f359810dfe0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f358c00b3b0 tx=0x7f358c009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 shutdown_connections 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f358006fcb0 0x7f3580072160 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3598066af0 0x7f359810daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 --2- 172.21.15.110:0/1899310401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35980fff70 0x7f359810dfe0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 >> 172.21.15.110:0/1899310401 conn(0x7f35980f7440 msgr2=0x7f35980f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 shutdown_connections 2024-04-25T15:49:26.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.434+0000 7f359d387700 1 -- 172.21.15.110:0/1899310401 wait complete. 2024-04-25T15:49:26.435 INFO:tasks.workunit.client.0.smithi110.stdout:7/341: link c5 d1b/d29/d38/c70 0 2024-04-25T15:49:26.436 INFO:tasks.workunit.client.1.smithi193.stdout:3/68: mknod d8/d13/d14/c19 0 2024-04-25T15:49:26.439 INFO:tasks.workunit.client.1.smithi193.stdout:3/69: mknod d8/d13/d14/c1a 0 2024-04-25T15:49:26.439 INFO:tasks.workunit.client.1.smithi193.stdout:3/70: chown d8/fc 260601787 1 2024-04-25T15:49:26.440 INFO:tasks.workunit.client.0.smithi110.stdout:5/346: creat d4/d1d/d1f/d35/f76 x:0 0 0 2024-04-25T15:49:26.440 INFO:tasks.workunit.client.0.smithi110.stdout:2/336: getdents de/d40/d5b 0 2024-04-25T15:49:26.441 INFO:tasks.workunit.client.1.smithi193.stdout:3/71: mknod d8/c1b 0 2024-04-25T15:49:26.442 INFO:tasks.workunit.client.0.smithi110.stdout:7/342: rmdir d1b/d29/d42 39 2024-04-25T15:49:26.442 INFO:tasks.workunit.client.0.smithi110.stdout:3/473: rmdir d0/d2e 39 2024-04-25T15:49:26.443 INFO:tasks.workunit.client.0.smithi110.stdout:4/466: creat d13/d24/d2d/d7d/d92/f96 x:0 0 0 2024-04-25T15:49:26.444 INFO:tasks.workunit.client.0.smithi110.stdout:4/467: dread - d13/d24/d2d/f62 zero size 2024-04-25T15:49:26.444 INFO:tasks.workunit.client.0.smithi110.stdout:6/336: link d1/d48/c70 d1/db/d24/d2d/c7e 0 2024-04-25T15:49:26.446 INFO:tasks.workunit.client.0.smithi110.stdout:6/337: write d1/db/d24/d32/f5a [2669332,59462] 0 2024-04-25T15:49:26.450 INFO:tasks.workunit.client.0.smithi110.stdout:5/347: mknod d4/d1d/d1f/d56/d6b/c77 0 2024-04-25T15:49:26.450 INFO:tasks.workunit.client.0.smithi110.stdout:3/474: dwrite d0/d7/f68 [4194304,4194304] 0 2024-04-25T15:49:26.452 INFO:tasks.workunit.client.0.smithi110.stdout:2/337: truncate de/d11/f21 2617083 0 2024-04-25T15:49:26.453 INFO:tasks.workunit.client.0.smithi110.stdout:7/343: mknod d1b/c71 0 2024-04-25T15:49:26.455 INFO:tasks.workunit.client.0.smithi110.stdout:7/344: truncate d1b/f48 292110 0 2024-04-25T15:49:26.455 INFO:tasks.workunit.client.0.smithi110.stdout:6/338: creat d1/db/d24/d32/f7f x:0 0 0 2024-04-25T15:49:26.455 INFO:tasks.workunit.client.0.smithi110.stdout:7/345: chown f19 185380024 1 2024-04-25T15:49:26.456 INFO:tasks.workunit.client.0.smithi110.stdout:4/468: symlink d13/d24/l97 0 2024-04-25T15:49:26.459 INFO:tasks.workunit.client.0.smithi110.stdout:5/348: fsync d4/d1d/d1f/d35/d3d/d52/f6f 0 2024-04-25T15:49:26.460 INFO:tasks.workunit.client.0.smithi110.stdout:5/349: dread - d4/d1d/d1f/d35/f76 zero size 2024-04-25T15:49:26.461 INFO:tasks.workunit.client.0.smithi110.stdout:2/338: rename de/l31 to de/d40/d5b/d5c/l61 0 2024-04-25T15:49:26.462 INFO:tasks.workunit.client.0.smithi110.stdout:2/339: chown de/d40/l37 3 1 2024-04-25T15:49:26.463 INFO:tasks.workunit.client.0.smithi110.stdout:2/340: fdatasync de/d11/f3a 0 2024-04-25T15:49:26.464 INFO:tasks.workunit.client.0.smithi110.stdout:4/469: creat d13/d24/d2d/d5e/f98 x:0 0 0 2024-04-25T15:49:26.467 INFO:tasks.workunit.client.0.smithi110.stdout:5/350: creat d4/d15/d51/f78 x:0 0 0 2024-04-25T15:49:26.468 INFO:tasks.workunit.client.0.smithi110.stdout:7/346: rename f4 to d1b/d29/d42/f72 0 2024-04-25T15:49:26.469 INFO:tasks.workunit.client.0.smithi110.stdout:5/351: write d4/d1d/d1f/d35/d3d/d52/f6f [366542,12832] 0 2024-04-25T15:49:26.471 INFO:tasks.workunit.client.0.smithi110.stdout:5/352: write d4/d1d/f34 [2137266,1963] 0 2024-04-25T15:49:26.472 INFO:tasks.workunit.client.0.smithi110.stdout:2/341: creat de/d11/d52/f62 x:0 0 0 2024-04-25T15:49:26.473 INFO:tasks.workunit.client.0.smithi110.stdout:2/342: write de/d40/d4f/f4c [896912,61384] 0 2024-04-25T15:49:26.475 INFO:tasks.workunit.client.0.smithi110.stdout:2/343: truncate de/d40/f36 856062 0 2024-04-25T15:49:26.478 INFO:tasks.workunit.client.0.smithi110.stdout:3/475: truncate d0/d7/f68 6392379 0 2024-04-25T15:49:26.478 INFO:tasks.workunit.client.0.smithi110.stdout:6/339: dread d1/d48/f6c [0,4194304] 0 2024-04-25T15:49:26.479 INFO:tasks.workunit.client.0.smithi110.stdout:7/347: symlink d1b/d29/d38/d55/l73 0 2024-04-25T15:49:26.479 INFO:tasks.workunit.client.0.smithi110.stdout:3/476: readlink d0/d16/l51 0 2024-04-25T15:49:26.480 INFO:tasks.workunit.client.0.smithi110.stdout:5/353: mknod d4/d15/c79 0 2024-04-25T15:49:26.482 INFO:tasks.workunit.client.0.smithi110.stdout:4/470: mkdir d13/d24/d2d/d44/d65/d99 0 2024-04-25T15:49:26.482 INFO:tasks.workunit.client.0.smithi110.stdout:7/348: write d1b/d29/d42/f6c [201393,125320] 0 2024-04-25T15:49:26.484 INFO:tasks.workunit.client.0.smithi110.stdout:7/349: fdatasync d1b/d29/f2b 0 2024-04-25T15:49:26.485 INFO:tasks.workunit.client.0.smithi110.stdout:2/344: dwrite de/d40/d4f/f4e [0,4194304] 0 2024-04-25T15:49:26.488 INFO:tasks.workunit.client.0.smithi110.stdout:5/354: creat d4/d1d/d1f/d35/d3d/d52/f7a x:0 0 0 2024-04-25T15:49:26.488 INFO:tasks.workunit.client.0.smithi110.stdout:4/471: creat d13/d24/d2d/d7d/d92/f9a x:0 0 0 2024-04-25T15:49:26.492 INFO:tasks.workunit.client.0.smithi110.stdout:6/340: dwrite d1/db/d24/d2d/f7b [0,4194304] 0 2024-04-25T15:49:26.493 INFO:tasks.workunit.client.0.smithi110.stdout:4/472: dread d13/d24/d2d/d36/d51/f85 [0,4194304] 0 2024-04-25T15:49:26.498 INFO:tasks.workunit.client.0.smithi110.stdout:2/345: dread de/d40/f33 [0,4194304] 0 2024-04-25T15:49:26.499 INFO:tasks.workunit.client.0.smithi110.stdout:7/350: rmdir d1b/d29/d38 39 2024-04-25T15:49:26.502 INFO:tasks.workunit.client.0.smithi110.stdout:5/355: truncate d4/d2c/f19 1409527 0 2024-04-25T15:49:26.505 INFO:tasks.workunit.client.0.smithi110.stdout:4/473: write d13/d24/d3c/d56/d88/f2c [5003312,121083] 0 2024-04-25T15:49:26.506 INFO:tasks.workunit.client.0.smithi110.stdout:2/346: rename de/d11/d52/f59 to de/d40/d5b/d5c/d5f/f63 0 2024-04-25T15:49:26.509 INFO:tasks.workunit.client.0.smithi110.stdout:6/341: dwrite d1/d12/d3d/f73 [0,4194304] 0 2024-04-25T15:49:26.511 INFO:tasks.workunit.client.0.smithi110.stdout:5/356: rename d4/d15/l2b to d4/d1d/d1f/l7b 0 2024-04-25T15:49:26.514 INFO:tasks.workunit.client.0.smithi110.stdout:7/351: dwrite d1b/d29/f2b [0,4194304] 0 2024-04-25T15:49:26.517 INFO:tasks.workunit.client.0.smithi110.stdout:2/347: dread de/d40/d4f/f4c [0,4194304] 0 2024-04-25T15:49:26.517 INFO:tasks.workunit.client.0.smithi110.stdout:4/474: creat d13/d24/d26/f9b x:0 0 0 2024-04-25T15:49:26.518 INFO:tasks.workunit.client.0.smithi110.stdout:4/475: chown d13/d24/d2d/d36/d51/f85 736985006 1 2024-04-25T15:49:26.522 INFO:tasks.workunit.client.0.smithi110.stdout:7/352: dread - d1b/d29/d38/f61 zero size 2024-04-25T15:49:26.523 INFO:tasks.workunit.client.0.smithi110.stdout:7/353: write d1b/d29/d46/f6a [1926448,121366] 0 2024-04-25T15:49:26.524 INFO:tasks.workunit.client.0.smithi110.stdout:4/476: fdatasync d13/f1a 0 2024-04-25T15:49:26.526 INFO:tasks.workunit.client.0.smithi110.stdout:6/342: getdents d1/db/d22/d33/d63 0 2024-04-25T15:49:26.527 INFO:tasks.workunit.client.0.smithi110.stdout:7/354: dread d1b/d29/d42/f6c [0,4194304] 0 2024-04-25T15:49:26.529 INFO:tasks.workunit.client.0.smithi110.stdout:4/477: dread d13/d24/d3c/d56/d88/f47 [0,4194304] 0 2024-04-25T15:49:26.530 INFO:tasks.workunit.client.0.smithi110.stdout:4/478: fdatasync d13/d24/d3c/d56/f8e 0 2024-04-25T15:49:26.531 INFO:tasks.workunit.client.0.smithi110.stdout:7/355: unlink d1b/d29/l58 0 2024-04-25T15:49:26.531 INFO:tasks.workunit.client.0.smithi110.stdout:4/479: readlink d13/l18 0 2024-04-25T15:49:26.533 INFO:tasks.workunit.client.0.smithi110.stdout:6/343: rename d1/dd/l18 to d1/dd/d3c/l80 0 2024-04-25T15:49:26.535 INFO:tasks.workunit.client.0.smithi110.stdout:2/348: link de/d11/l1e de/d11/d50/l64 0 2024-04-25T15:49:26.536 INFO:tasks.workunit.client.0.smithi110.stdout:7/356: rename d1b/l1e to d1b/d29/d42/l74 0 2024-04-25T15:49:26.538 INFO:tasks.workunit.client.0.smithi110.stdout:2/349: symlink de/d40/d4f/l65 0 2024-04-25T15:49:26.539 INFO:tasks.workunit.client.0.smithi110.stdout:6/344: creat d1/db/d22/d27/d76/f81 x:0 0 0 2024-04-25T15:49:26.541 INFO:tasks.workunit.client.0.smithi110.stdout:2/350: write de/d40/f35 [2036291,65528] 0 2024-04-25T15:49:26.543 INFO:tasks.workunit.client.0.smithi110.stdout:2/351: write de/d11/f42 [4656632,86355] 0 2024-04-25T15:49:26.546 INFO:tasks.workunit.client.0.smithi110.stdout:7/357: dwrite d1b/d29/d38/d55/f6b [0,4194304] 0 2024-04-25T15:49:26.546 INFO:tasks.workunit.client.0.smithi110.stdout:6/345: mkdir d1/db/d24/d69/d75/d82 0 2024-04-25T15:49:26.548 INFO:tasks.workunit.client.0.smithi110.stdout:7/358: write d1b/d29/d46/f6a [156268,78944] 0 2024-04-25T15:49:26.559 INFO:tasks.workunit.client.0.smithi110.stdout:4/480: dread d13/d24/d2d/d7d/f81 [0,4194304] 0 2024-04-25T15:49:26.562 INFO:tasks.workunit.client.0.smithi110.stdout:4/481: dread d13/d24/d3c/d56/f66 [0,4194304] 0 2024-04-25T15:49:26.568 INFO:tasks.workunit.client.0.smithi110.stdout:2/352: getdents de/d40/d5b/d5c 0 2024-04-25T15:49:26.570 INFO:tasks.workunit.client.0.smithi110.stdout:4/482: dwrite d13/d24/d26/f9b [0,4194304] 0 2024-04-25T15:49:26.572 INFO:tasks.workunit.client.0.smithi110.stdout:7/359: getdents d1b/d29 0 2024-04-25T15:49:26.575 INFO:tasks.workunit.client.0.smithi110.stdout:4/483: write d13/d24/f45 [1043665,11477] 0 2024-04-25T15:49:26.575 INFO:tasks.workunit.client.0.smithi110.stdout:2/353: dread de/d40/d4f/f4c [0,4194304] 0 2024-04-25T15:49:26.576 INFO:tasks.workunit.client.0.smithi110.stdout:6/346: dwrite d1/db/d24/d2d/f3a [0,4194304] 0 2024-04-25T15:49:26.578 INFO:tasks.workunit.client.0.smithi110.stdout:6/347: dread - d1/db/d22/d33/d43/f60 zero size 2024-04-25T15:49:26.579 INFO:tasks.workunit.client.0.smithi110.stdout:7/360: truncate d1b/d29/d38/f3d 5241915 0 2024-04-25T15:49:26.580 INFO:tasks.workunit.client.0.smithi110.stdout:2/354: mkdir de/d40/d5b/d66 0 2024-04-25T15:49:26.582 INFO:tasks.workunit.client.0.smithi110.stdout:6/348: symlink d1/db/d24/d69/d75/d82/l83 0 2024-04-25T15:49:26.582 INFO:tasks.workunit.client.0.smithi110.stdout:2/355: fsync de/d11/d52/f62 0 2024-04-25T15:49:26.585 INFO:tasks.workunit.client.0.smithi110.stdout:7/361: getdents d1b/d29/d38/d55 0 2024-04-25T15:49:26.588 INFO:tasks.workunit.client.0.smithi110.stdout:2/356: rename de/c4a to de/d11/d52/c67 0 2024-04-25T15:49:26.590 INFO:tasks.workunit.client.0.smithi110.stdout:6/349: unlink d1/db/d22/d33/l6b 0 2024-04-25T15:49:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.595+0000 7ff424fbc700 1 -- 172.21.15.110:0/2596209747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fce60 msgr2=0x7ff4200fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.595+0000 7ff424fbc700 1 --2- 172.21.15.110:0/2596209747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fce60 0x7ff4200fd270 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff414004940 tx=0x7ff414009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:26.596 INFO:tasks.workunit.client.0.smithi110.stdout:7/362: link d1b/l1d d1b/l75 0 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.595+0000 7ff424fbc700 1 -- 172.21.15.110:0/2596209747 shutdown_connections 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.595+0000 7ff424fbc700 1 --2- 172.21.15.110:0/2596209747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4200fdfd0 0x7ff4200fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.595+0000 7ff424fbc700 1 --2- 172.21.15.110:0/2596209747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fce60 0x7ff4200fd270 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.595+0000 7ff424fbc700 1 -- 172.21.15.110:0/2596209747 >> 172.21.15.110:0/2596209747 conn(0x7ff4200f8500 msgr2=0x7ff4200fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.595+0000 7ff424fbc700 1 -- 172.21.15.110:0/2596209747 shutdown_connections 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.595+0000 7ff424fbc700 1 -- 172.21.15.110:0/2596209747 wait complete. 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff424fbc700 1 Processor -- start 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff424fbc700 1 -- start start 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff424fbc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4200fce60 0x7ff42010d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff424fbc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fdfd0 0x7ff42010df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff424fbc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42010e500 con 0x7ff4200fce60 2024-04-25T15:49:26.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff424fbc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42010ed50 con 0x7ff4200fdfd0 2024-04-25T15:49:26.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff41e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4200fce60 0x7ff42010d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff41e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4200fce60 0x7ff42010d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35932/0 (socket says 172.21.15.110:35932) 2024-04-25T15:49:26.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff41e59c700 1 -- 172.21.15.110:0/3987039743 learned_addr learned my addr 172.21.15.110:0/3987039743 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:26.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.596+0000 7ff41dd9b700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fdfd0 0x7ff42010df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff41dd9b700 1 -- 172.21.15.110:0/3987039743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4200fce60 msgr2=0x7ff42010d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff41dd9b700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4200fce60 0x7ff42010d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff41dd9b700 1 -- 172.21.15.110:0/3987039743 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff414005040 con 0x7ff4200fdfd0 2024-04-25T15:49:26.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff41e59c700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4200fce60 0x7ff42010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:49:26.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff41dd9b700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fdfd0 0x7ff42010df00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff410009fd0 tx=0x7ff41000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:26.599 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff40effd700 1 -- 172.21.15.110:0/3987039743 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff410004e90 con 0x7ff4200fdfd0 2024-04-25T15:49:26.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42010efd0 con 0x7ff4200fdfd0 2024-04-25T15:49:26.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff40effd700 1 -- 172.21.15.110:0/3987039743 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff41000f040 con 0x7ff4200fdfd0 2024-04-25T15:49:26.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.597+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42010f510 con 0x7ff4200fdfd0 2024-04-25T15:49:26.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.598+0000 7ff40effd700 1 -- 172.21.15.110:0/3987039743 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41000ca50 con 0x7ff4200fdfd0 2024-04-25T15:49:26.600 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.599+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42005f5c0 con 0x7ff4200fdfd0 2024-04-25T15:49:26.600 INFO:tasks.workunit.client.0.smithi110.stdout:2/357: write f8 [2622548,12395] 0 2024-04-25T15:49:26.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.600+0000 7ff40effd700 1 -- 172.21.15.110:0/3987039743 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7ff410007cc0 con 0x7ff4200fdfd0 2024-04-25T15:49:26.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.600+0000 7ff40effd700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff40806fcb0 0x7ff408072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.601+0000 7ff40effd700 1 -- 172.21.15.110:0/3987039743 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff41008f250 con 0x7ff4200fdfd0 2024-04-25T15:49:26.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.601+0000 7ff41e59c700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff40806fcb0 0x7ff408072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.601+0000 7ff41e59c700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff40806fcb0 0x7ff408072160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b0d0 tx=0x7ff414009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:26.603 INFO:tasks.workunit.client.0.smithi110.stdout:7/363: dwrite f19 [0,4194304] 0 2024-04-25T15:49:26.606 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.605+0000 7ff40effd700 1 -- 172.21.15.110:0/3987039743 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7ff410050930 con 0x7ff4200fdfd0 2024-04-25T15:49:26.608 INFO:tasks.workunit.client.0.smithi110.stdout:7/364: rmdir d1b/d29/d67 0 2024-04-25T15:49:26.609 INFO:tasks.workunit.client.0.smithi110.stdout:2/358: dwrite de/d40/d4f/f47 [0,4194304] 0 2024-04-25T15:49:26.610 INFO:tasks.workunit.client.0.smithi110.stdout:6/350: dread d1/db/d22/f38 [0,4194304] 0 2024-04-25T15:49:26.611 INFO:tasks.workunit.client.0.smithi110.stdout:2/359: write de/d40/f36 [12560,121410] 0 2024-04-25T15:49:26.617 INFO:tasks.workunit.client.1.smithi193.stdout:3/72: creat d8/d13/d12/f1c x:0 0 0 2024-04-25T15:49:26.618 INFO:tasks.workunit.client.1.smithi193.stdout:3/73: write f2 [3342279,9132] 0 2024-04-25T15:49:26.619 INFO:tasks.workunit.client.0.smithi110.stdout:6/351: symlink d1/db/d22/d33/l84 0 2024-04-25T15:49:26.620 INFO:tasks.workunit.client.0.smithi110.stdout:2/360: dwrite de/f28 [4194304,4194304] 0 2024-04-25T15:49:26.622 INFO:tasks.workunit.client.0.smithi110.stdout:6/352: fsync d1/db/f19 0 2024-04-25T15:49:26.624 INFO:tasks.workunit.client.0.smithi110.stdout:2/361: write de/d40/d4f/f49 [842403,103552] 0 2024-04-25T15:49:26.625 INFO:tasks.workunit.client.1.smithi193.stdout:3/74: dwrite d8/f15 [0,4194304] 0 2024-04-25T15:49:26.625 INFO:tasks.workunit.client.0.smithi110.stdout:6/353: rmdir d1/db/d22/d27 39 2024-04-25T15:49:26.627 INFO:tasks.workunit.client.1.smithi193.stdout:3/75: read - d8/d13/d12/f1c zero size 2024-04-25T15:49:26.627 INFO:tasks.workunit.client.1.smithi193.stdout:3/76: fdatasync d8/d13/d14/f18 0 2024-04-25T15:49:26.628 INFO:tasks.workunit.client.0.smithi110.stdout:6/354: read - d1/db/d22/f55 zero size 2024-04-25T15:49:26.628 INFO:tasks.workunit.client.0.smithi110.stdout:7/365: dwrite d1b/d29/f68 [0,4194304] 0 2024-04-25T15:49:26.631 INFO:tasks.workunit.client.0.smithi110.stdout:2/362: dread de/d40/d4f/f4e [0,4194304] 0 2024-04-25T15:49:26.634 INFO:tasks.workunit.client.0.smithi110.stdout:7/366: write d1b/d29/d46/f51 [5215687,28739] 0 2024-04-25T15:49:26.635 INFO:tasks.workunit.client.0.smithi110.stdout:6/355: creat d1/db/d24/d69/d75/d82/f85 x:0 0 0 2024-04-25T15:49:26.636 INFO:tasks.workunit.client.0.smithi110.stdout:6/356: dread - d1/d12/d3d/f4a zero size 2024-04-25T15:49:26.637 INFO:tasks.workunit.client.0.smithi110.stdout:7/367: mknod d1b/d29/c76 0 2024-04-25T15:49:26.643 INFO:tasks.workunit.client.0.smithi110.stdout:2/363: link de/l5e de/d40/d5b/d5c/l68 0 2024-04-25T15:49:26.649 INFO:tasks.workunit.client.0.smithi110.stdout:2/364: dread de/f2d [0,4194304] 0 2024-04-25T15:49:26.649 INFO:tasks.workunit.client.0.smithi110.stdout:7/368: dwrite d1b/d29/d38/d55/f59 [0,4194304] 0 2024-04-25T15:49:26.651 INFO:tasks.workunit.client.0.smithi110.stdout:2/365: dread - de/d11/d52/f62 zero size 2024-04-25T15:49:26.652 INFO:tasks.workunit.client.1.smithi193.stdout:1/3: sync 2024-04-25T15:49:26.652 INFO:tasks.workunit.client.1.smithi193.stdout:1/4: read - no filename 2024-04-25T15:49:26.652 INFO:tasks.workunit.client.1.smithi193.stdout:1/5: unlink - no file 2024-04-25T15:49:26.652 INFO:tasks.workunit.client.1.smithi193.stdout:1/6: truncate - no filename 2024-04-25T15:49:26.652 INFO:tasks.workunit.client.1.smithi193.stdout:1/7: dread - no filename 2024-04-25T15:49:26.652 INFO:tasks.workunit.client.1.smithi193.stdout:1/8: write - no filename 2024-04-25T15:49:26.652 INFO:tasks.workunit.client.1.smithi193.stdout:1/9: dwrite - no filename 2024-04-25T15:49:26.660 INFO:tasks.workunit.client.0.smithi110.stdout:2/366: rename de/d40/l3c to de/d11/l69 0 2024-04-25T15:49:26.665 INFO:tasks.workunit.client.0.smithi110.stdout:2/367: dwrite de/d11/f39 [0,4194304] 0 2024-04-25T15:49:26.673 INFO:tasks.workunit.client.0.smithi110.stdout:2/368: dwrite de/d40/d4f/f4c [0,4194304] 0 2024-04-25T15:49:26.679 INFO:tasks.workunit.client.0.smithi110.stdout:2/369: dread de/d40/f3d [4194304,4194304] 0 2024-04-25T15:49:26.684 INFO:tasks.workunit.client.0.smithi110.stdout:2/370: mknod de/d40/d4f/c6a 0 2024-04-25T15:49:26.690 INFO:tasks.workunit.client.0.smithi110.stdout:2/371: link de/d40/d5b/d5c/l61 de/d40/l6b 0 2024-04-25T15:49:26.692 INFO:tasks.workunit.client.0.smithi110.stdout:2/372: readlink de/d40/l6b 0 2024-04-25T15:49:26.694 INFO:tasks.workunit.client.0.smithi110.stdout:2/373: mknod de/d40/d5b/d5c/d5f/c6c 0 2024-04-25T15:49:26.699 INFO:tasks.workunit.client.0.smithi110.stdout:2/374: dwrite de/d40/d4f/f4c [0,4194304] 0 2024-04-25T15:49:26.704 INFO:tasks.workunit.client.0.smithi110.stdout:2/375: mkdir de/d11/d52/d6d 0 2024-04-25T15:49:26.758 INFO:tasks.workunit.client.1.smithi193.stdout:3/77: truncate d8/d13/d12/f16 252364 0 2024-04-25T15:49:26.758 INFO:tasks.workunit.client.1.smithi193.stdout:3/78: fdatasync d8/f15 0 2024-04-25T15:49:26.770 INFO:tasks.workunit.client.0.smithi110.stdout:7/369: truncate d1b/d29/f5d 926376 0 2024-04-25T15:49:26.778 INFO:tasks.workunit.client.0.smithi110.stdout:7/370: link d1b/d29/d42/c63 d1b/d29/d42/c77 0 2024-04-25T15:49:26.780 INFO:tasks.workunit.client.0.smithi110.stdout:7/371: truncate f17 222386 0 2024-04-25T15:49:26.781 INFO:tasks.workunit.client.0.smithi110.stdout:7/372: stat d1b/f6e 0 2024-04-25T15:49:26.781 INFO:tasks.workunit.client.0.smithi110.stdout:0/394: sync 2024-04-25T15:49:26.781 INFO:tasks.workunit.client.0.smithi110.stdout:7/373: chown d1b/d29/d38/f54 815 1 2024-04-25T15:49:26.787 INFO:tasks.workunit.client.0.smithi110.stdout:0/395: creat d1/d23/d30/d33/d5b/f9b x:0 0 0 2024-04-25T15:49:26.788 INFO:tasks.workunit.client.1.smithi193.stdout:5/1: sync 2024-04-25T15:49:26.788 INFO:tasks.workunit.client.1.smithi193.stdout:7/4: sync 2024-04-25T15:49:26.791 INFO:tasks.workunit.client.0.smithi110.stdout:7/374: rename d1b/d29/d46/f52 to d1b/d29/d42/f78 0 2024-04-25T15:49:26.791 INFO:tasks.workunit.client.0.smithi110.stdout:7/375: dread - d1b/d29/d38/f5e zero size 2024-04-25T15:49:26.794 INFO:tasks.workunit.client.0.smithi110.stdout:0/396: creat d1/d47/d54/f9c x:0 0 0 2024-04-25T15:49:26.801 INFO:tasks.workunit.client.0.smithi110.stdout:0/397: symlink d1/d20/d69/l9d 0 2024-04-25T15:49:26.802 INFO:tasks.workunit.client.0.smithi110.stdout:0/398: chown d1/d23/d30/d33/l4a 10713394 1 2024-04-25T15:49:26.805 INFO:tasks.workunit.client.0.smithi110.stdout:0/399: unlink d1/d47/d54/l56 0 2024-04-25T15:49:26.810 INFO:tasks.workunit.client.0.smithi110.stdout:0/400: chown d1/d23/d30/d33/d35/l55 5 1 2024-04-25T15:49:26.811 INFO:tasks.workunit.client.0.smithi110.stdout:0/401: chown d1/d23/d30/d33/d5b/d83/f8a 110 1 2024-04-25T15:49:26.819 INFO:tasks.workunit.client.1.smithi193.stdout:0/39: sync 2024-04-25T15:49:26.819 INFO:tasks.workunit.client.1.smithi193.stdout:1/10: sync 2024-04-25T15:49:26.819 INFO:tasks.workunit.client.1.smithi193.stdout:1/11: rename - no filename 2024-04-25T15:49:26.819 INFO:tasks.workunit.client.1.smithi193.stdout:9/60: sync 2024-04-25T15:49:26.819 INFO:tasks.workunit.client.1.smithi193.stdout:1/12: write - no filename 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/13: link - no file 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:6/49: sync 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/14: readlink - no filename 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:8/39: sync 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/15: unlink - no file 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/16: rmdir - no directory 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:1/17: link - no file 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:4/29: sync 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:2/54: sync 2024-04-25T15:49:26.820 INFO:tasks.workunit.client.1.smithi193.stdout:4/30: write f4 [164226,89561] 0 2024-04-25T15:49:26.821 INFO:tasks.workunit.client.1.smithi193.stdout:6/50: dread f6 [0,4194304] 0 2024-04-25T15:49:26.825 INFO:tasks.workunit.client.1.smithi193.stdout:0/40: dread d0/fc [0,4194304] 0 2024-04-25T15:49:26.826 INFO:tasks.workunit.client.1.smithi193.stdout:3/79: creat d8/d13/d14/f1d x:0 0 0 2024-04-25T15:49:26.827 INFO:tasks.workunit.client.1.smithi193.stdout:6/51: dwrite fb [0,4194304] 0 2024-04-25T15:49:26.857 INFO:tasks.workunit.client.0.smithi110.stdout:9/294: sync 2024-04-25T15:49:26.857 INFO:tasks.workunit.client.0.smithi110.stdout:1/484: sync 2024-04-25T15:49:26.858 INFO:tasks.workunit.client.0.smithi110.stdout:1/485: stat d0/d14/d5f/d6f/c6a 0 2024-04-25T15:49:26.860 INFO:tasks.workunit.client.0.smithi110.stdout:9/295: read f8 [6831041,32182] 0 2024-04-25T15:49:26.860 INFO:tasks.workunit.client.0.smithi110.stdout:9/296: chown d9 5788720 1 2024-04-25T15:49:26.861 INFO:tasks.workunit.client.1.smithi193.stdout:7/5: unlink f1 0 2024-04-25T15:49:26.862 INFO:tasks.workunit.client.0.smithi110.stdout:1/486: symlink d0/d25/d33/l93 0 2024-04-25T15:49:26.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.873+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff42010f7f0 con 0x7ff40806fcb0 2024-04-25T15:49:26.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.883+0000 7ff40effd700 1 -- 172.21.15.110:0/3987039743 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7ff42010f7f0 con 0x7ff40806fcb0 2024-04-25T15:49:26.883 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (7m) 43s ago 8m 24.3M - 0.25.0 c8568f914cd2 b995d416435c 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (8m) 43s ago 8m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (7m) 45s ago 7m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (6m) 43s ago 8m 85.9M - 9.4.7 954c08fa6188 6cf7f080ae79 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (2m) 43s ago 2m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (3m) 43s ago 3m 86.6M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (3m) 45s ago 3m 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (2m) 45s ago 2m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:9283 running (9m) 43s ago 9m 363M - 17.2.7-902-gc61472c8 0de407c1b3af c623b709ef43 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (62s) 45s ago 7m 558M - 19.0.0-3291-g994e2f20 d53c96779a29 82852af5489a 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (9m) 43s ago 9m 49.0M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (7m) 45s ago 7m 40.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:49:26.884 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 starting - - - - 2024-04-25T15:49:26.885 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 45s ago 7m 15.3M - 1.5.0 0da6a335fe13 13afa4d53352 2024-04-25T15:49:26.885 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (6m) 43s ago 6m 61.7M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:49:26.885 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (6m) 43s ago 6m 66.0M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:49:26.885 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (5m) 43s ago 5m 56.3M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:49:26.885 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (5m) 45s ago 5m 76.0M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:49:26.885 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (4m) 45s ago 4m 64.8M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:49:26.885 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (4m) 45s ago 4m 69.0M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:49:26.885 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (6m) 43s ago 7m 54.9M - 2.43.0 a07b618ecd1d ce7fd55a0368 2024-04-25T15:49:26.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff40806fcb0 msgr2=0x7ff408072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff40806fcb0 0x7ff408072160 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff41400b0d0 tx=0x7ff414009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:26.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fdfd0 msgr2=0x7ff42010df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fdfd0 0x7ff42010df00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff410009fd0 tx=0x7ff41000b040 comp rx=0 tx=0).stop 2024-04-25T15:49:26.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 shutdown_connections 2024-04-25T15:49:26.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff40806fcb0 0x7ff408072160 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff4200fce60 0x7ff42010d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 --2- 172.21.15.110:0/3987039743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4200fdfd0 0x7ff42010df00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 >> 172.21.15.110:0/3987039743 conn(0x7ff4200f8500 msgr2=0x7ff4200f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:26.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 shutdown_connections 2024-04-25T15:49:26.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.885+0000 7ff424fbc700 1 -- 172.21.15.110:0/3987039743 wait complete. 2024-04-25T15:49:26.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.984+0000 7f800f19d700 1 -- 172.21.15.110:0/304453470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fbcc0 msgr2=0x7f80080fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.984+0000 7f800f19d700 1 --2- 172.21.15.110:0/304453470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fbcc0 0x7f80080fc0d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc004940 tx=0x7f7ffc009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.984+0000 7f800f19d700 1 -- 172.21.15.110:0/304453470 shutdown_connections 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.984+0000 7f800f19d700 1 --2- 172.21.15.110:0/304453470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80080fcf70 0x7f80080fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.984+0000 7f800f19d700 1 --2- 172.21.15.110:0/304453470 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fbcc0 0x7f80080fc0d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.984+0000 7f800f19d700 1 -- 172.21.15.110:0/304453470 >> 172.21.15.110:0/304453470 conn(0x7f80080f7440 msgr2=0x7f80080f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.984+0000 7f800f19d700 1 -- 172.21.15.110:0/304453470 shutdown_connections 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.984+0000 7f800f19d700 1 -- 172.21.15.110:0/304453470 wait complete. 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f800f19d700 1 Processor -- start 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f800f19d700 1 -- start start 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f800f19d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80080fbcc0 0x7f800810c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f800f19d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fcf70 0x7f800810cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f800f19d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800810d400 con 0x7f80080fcf70 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f800f19d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8008111da0 con 0x7f80080fbcc0 2024-04-25T15:49:26.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f8007fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fcf70 0x7f800810cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f8007fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fcf70 0x7f800810cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35942/0 (socket says 172.21.15.110:35942) 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f8007fff700 1 -- 172.21.15.110:0/1630184803 learned_addr learned my addr 172.21.15.110:0/1630184803 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f800cf39700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80080fbcc0 0x7f800810c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f8007fff700 1 -- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80080fbcc0 msgr2=0x7f800810c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f8007fff700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80080fbcc0 0x7f800810c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f8007fff700 1 -- 172.21.15.110:0/1630184803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc005040 con 0x7f80080fcf70 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.985+0000 7f800cf39700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80080fbcc0 0x7f800810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.986+0000 7f8007fff700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fcf70 0x7f800810cd70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400b790 tx=0x7f7ff4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.986+0000 7f80057fa700 1 -- 172.21.15.110:0/1630184803 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ff400f770 con 0x7f80080fcf70 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.986+0000 7f80057fa700 1 -- 172.21.15.110:0/1630184803 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7ff400fda0 con 0x7f80080fcf70 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.986+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8008112000 con 0x7f80080fcf70 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.986+0000 7f80057fa700 1 -- 172.21.15.110:0/1630184803 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ff400d310 con 0x7f80080fcf70 2024-04-25T15:49:26.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.986+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8008112540 con 0x7f80080fcf70 2024-04-25T15:49:26.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.987+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800804c710 con 0x7f80080fcf70 2024-04-25T15:49:26.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.987+0000 7f80057fa700 1 -- 172.21.15.110:0/1630184803 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f7ff4010420 con 0x7f80080fcf70 2024-04-25T15:49:26.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.988+0000 7f80057fa700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7ff806fcc0 0x7f7ff8072170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:26.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.988+0000 7f80057fa700 1 -- 172.21.15.110:0/1630184803 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f7ff408f640 con 0x7f80080fcf70 2024-04-25T15:49:26.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.988+0000 7f800cf39700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7ff806fcc0 0x7f7ff8072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:26.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.988+0000 7f800cf39700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7ff806fcc0 0x7f7ff8072170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00b3b0 tx=0x7f7ffc009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:26.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:26.993+0000 7f80057fa700 1 -- 172.21.15.110:0/1630184803 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f7ff4058b30 con 0x7f80080fcf70 2024-04-25T15:49:26.994 INFO:tasks.workunit.client.0.smithi110.stdout:1/487: stat d0/d25/c39 0 2024-04-25T15:49:26.995 INFO:tasks.workunit.client.0.smithi110.stdout:1/488: write d0/d14/f1f [1019366,2024] 0 2024-04-25T15:49:26.997 INFO:tasks.workunit.client.0.smithi110.stdout:1/489: write d0/d14/d5f/d7d/d72/d61/f8b [723719,100606] 0 2024-04-25T15:49:26.999 INFO:tasks.workunit.client.0.smithi110.stdout:2/376: truncate de/d40/d4f/f4c 2665407 0 2024-04-25T15:49:27.000 INFO:tasks.workunit.client.0.smithi110.stdout:2/377: fsync de/d40/f35 0 2024-04-25T15:49:27.001 INFO:tasks.workunit.client.1.smithi193.stdout:4/31: mknod c8 0 2024-04-25T15:49:27.001 INFO:tasks.workunit.client.1.smithi193.stdout:4/32: read f0 [2490113,39651] 0 2024-04-25T15:49:27.002 INFO:tasks.workunit.client.0.smithi110.stdout:1/490: mkdir d0/d25/d33/d94 0 2024-04-25T15:49:27.003 INFO:tasks.workunit.client.1.smithi193.stdout:4/33: write f1 [381277,125695] 0 2024-04-25T15:49:27.003 INFO:tasks.workunit.client.1.smithi193.stdout:4/34: read f4 [55978,26024] 0 2024-04-25T15:49:27.007 INFO:tasks.workunit.client.0.smithi110.stdout:2/378: rename de/d11 to de/d40/d6e 0 2024-04-25T15:49:27.007 INFO:tasks.workunit.client.0.smithi110.stdout:2/379: chown de/d40/l4d 30010 1 2024-04-25T15:49:27.008 INFO:tasks.workunit.client.1.smithi193.stdout:0/41: mkdir d0/d3/d5/d7/dd 0 2024-04-25T15:49:27.009 INFO:tasks.workunit.client.0.smithi110.stdout:2/380: read f8 [2200497,36532] 0 2024-04-25T15:49:27.015 INFO:tasks.workunit.client.1.smithi193.stdout:5/2: symlink l0 0 2024-04-25T15:49:27.015 INFO:tasks.workunit.client.1.smithi193.stdout:5/3: readlink l0 0 2024-04-25T15:49:27.019 INFO:tasks.workunit.client.1.smithi193.stdout:1/18: creat f0 x:0 0 0 2024-04-25T15:49:27.020 INFO:tasks.workunit.client.0.smithi110.stdout:1/491: link d0/d14/d81/c90 d0/d14/d5f/d7d/d72/c95 0 2024-04-25T15:49:27.025 INFO:tasks.workunit.client.0.smithi110.stdout:1/492: mknod d0/d14/d5f/d7d/d72/c96 0 2024-04-25T15:49:27.025 INFO:tasks.workunit.client.0.smithi110.stdout:1/493: write d0/d14/d5f/d7d/f67 [550148,25580] 0 2024-04-25T15:49:27.027 INFO:tasks.workunit.client.0.smithi110.stdout:1/494: truncate d0/d14/f4b 5130270 0 2024-04-25T15:49:27.028 INFO:tasks.workunit.client.0.smithi110.stdout:1/495: write d0/d23/f5c [1245414,89189] 0 2024-04-25T15:49:27.028 INFO:tasks.workunit.client.1.smithi193.stdout:8/40: truncate d2/d3/f5 2083097 0 2024-04-25T15:49:27.030 INFO:tasks.workunit.client.0.smithi110.stdout:8/473: sync 2024-04-25T15:49:27.031 INFO:tasks.workunit.client.0.smithi110.stdout:8/474: dread - d14/d4c/d55/f7d zero size 2024-04-25T15:49:27.033 INFO:tasks.workunit.client.1.smithi193.stdout:3/80: symlink d8/d13/l1e 0 2024-04-25T15:49:27.035 INFO:tasks.workunit.client.1.smithi193.stdout:0/42: dread d0/fc [0,4194304] 0 2024-04-25T15:49:27.035 INFO:tasks.workunit.client.1.smithi193.stdout:3/81: dread d8/ff [0,4194304] 0 2024-04-25T15:49:27.035 INFO:tasks.workunit.client.1.smithi193.stdout:3/82: chown f2 1 1 2024-04-25T15:49:27.035 INFO:tasks.workunit.client.0.smithi110.stdout:8/475: dread - d14/d21/d23/f2e zero size 2024-04-25T15:49:27.038 INFO:tasks.workunit.client.1.smithi193.stdout:5/4: rename l0 to l1 0 2024-04-25T15:49:27.038 INFO:tasks.workunit.client.1.smithi193.stdout:5/5: dread - no filename 2024-04-25T15:49:27.038 INFO:tasks.workunit.client.1.smithi193.stdout:5/6: fsync - no filename 2024-04-25T15:49:27.038 INFO:tasks.workunit.client.1.smithi193.stdout:5/7: write - no filename 2024-04-25T15:49:27.039 INFO:tasks.workunit.client.0.smithi110.stdout:1/496: dread d0/d23/f5a [0,4194304] 0 2024-04-25T15:49:27.040 INFO:tasks.workunit.client.0.smithi110.stdout:1/497: write d0/d23/f92 [646463,117697] 0 2024-04-25T15:49:27.040 INFO:tasks.workunit.client.0.smithi110.stdout:8/476: mknod d14/d32/d36/d48/c9f 0 2024-04-25T15:49:27.041 INFO:tasks.workunit.client.0.smithi110.stdout:8/477: write d14/d21/f62 [361151,82901] 0 2024-04-25T15:49:27.042 INFO:tasks.workunit.client.1.smithi193.stdout:1/19: rename f0 to f1 0 2024-04-25T15:49:27.045 INFO:tasks.workunit.client.0.smithi110.stdout:1/498: fsync d0/d14/f62 0 2024-04-25T15:49:27.047 INFO:tasks.workunit.client.0.smithi110.stdout:8/478: mknod d14/d32/d36/d48/ca0 0 2024-04-25T15:49:27.048 INFO:tasks.workunit.client.1.smithi193.stdout:8/41: symlink d2/d3/lc 0 2024-04-25T15:49:27.059 INFO:tasks.workunit.client.1.smithi193.stdout:1/20: mknod c2 0 2024-04-25T15:49:27.060 INFO:tasks.workunit.client.1.smithi193.stdout:0/43: dwrite d0/fc [0,4194304] 0 2024-04-25T15:49:27.060 INFO:tasks.workunit.client.1.smithi193.stdout:8/42: rename d2/d3/f7 to d2/d3/fd 0 2024-04-25T15:49:27.061 INFO:tasks.workunit.client.1.smithi193.stdout:8/43: readlink d2/d3/lc 0 2024-04-25T15:49:27.062 INFO:tasks.workunit.client.0.smithi110.stdout:8/479: dread d14/d32/d36/d48/f66 [0,4194304] 0 2024-04-25T15:49:27.065 INFO:tasks.workunit.client.1.smithi193.stdout:1/21: symlink l3 0 2024-04-25T15:49:27.069 INFO:tasks.workunit.client.1.smithi193.stdout:0/44: creat d0/d3/fe x:0 0 0 2024-04-25T15:49:27.069 INFO:tasks.workunit.client.1.smithi193.stdout:0/45: chown d0/fc 30258 1 2024-04-25T15:49:27.071 INFO:tasks.workunit.client.0.smithi110.stdout:8/480: creat d14/d4c/d71/d9d/fa1 x:0 0 0 2024-04-25T15:49:27.076 INFO:tasks.workunit.client.1.smithi193.stdout:0/46: creat d0/ff x:0 0 0 2024-04-25T15:49:27.077 INFO:tasks.workunit.client.1.smithi193.stdout:0/47: write d0/fc [316857,99718] 0 2024-04-25T15:49:27.083 INFO:tasks.workunit.client.1.smithi193.stdout:0/48: dwrite d0/ff [0,4194304] 0 2024-04-25T15:49:27.086 INFO:tasks.workunit.client.1.smithi193.stdout:0/49: symlink d0/d3/d5/d7/dd/l10 0 2024-04-25T15:49:27.089 INFO:tasks.workunit.client.1.smithi193.stdout:0/50: symlink d0/d3/d5/l11 0 2024-04-25T15:49:27.105 INFO:tasks.workunit.client.0.smithi110.stdout:7/376: rmdir d1b 39 2024-04-25T15:49:27.107 INFO:tasks.workunit.client.0.smithi110.stdout:7/377: chown d1b/c20 5 1 2024-04-25T15:49:27.116 INFO:tasks.workunit.client.1.smithi193.stdout:3/83: sync 2024-04-25T15:49:27.121 INFO:tasks.workunit.client.1.smithi193.stdout:3/84: dwrite d8/f11 [0,4194304] 0 2024-04-25T15:49:27.122 INFO:tasks.workunit.client.1.smithi193.stdout:3/85: fsync d8/d13/d14/f18 0 2024-04-25T15:49:27.122 INFO:tasks.workunit.client.1.smithi193.stdout:7/6: getdents . 0 2024-04-25T15:49:27.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/7: dwrite - no filename 2024-04-25T15:49:27.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/8: dwrite - no filename 2024-04-25T15:49:27.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/9: write - no filename 2024-04-25T15:49:27.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/10: dwrite - no filename 2024-04-25T15:49:27.125 INFO:tasks.workunit.client.0.smithi110.stdout:9/297: write d9/db/df/d2c/f36 [6284646,33738] 0 2024-04-25T15:49:27.125 INFO:tasks.workunit.client.1.smithi193.stdout:7/11: creat f2 x:0 0 0 2024-04-25T15:49:27.131 INFO:tasks.workunit.client.1.smithi193.stdout:2/55: dwrite d0/df/f10 [0,4194304] 0 2024-04-25T15:49:27.132 INFO:tasks.workunit.client.1.smithi193.stdout:7/12: link f2 f3 0 2024-04-25T15:49:27.133 INFO:tasks.workunit.client.1.smithi193.stdout:7/13: stat f3 0 2024-04-25T15:49:27.133 INFO:tasks.workunit.client.1.smithi193.stdout:7/14: rmdir - no directory 2024-04-25T15:49:27.133 INFO:tasks.workunit.client.0.smithi110.stdout:9/298: dwrite d9/d13/f1c [0,4194304] 0 2024-04-25T15:49:27.134 INFO:tasks.workunit.client.0.smithi110.stdout:2/381: dwrite de/d40/d4f/f4c [0,4194304] 0 2024-04-25T15:49:27.142 INFO:tasks.workunit.client.0.smithi110.stdout:2/382: dwrite de/f56 [0,4194304] 0 2024-04-25T15:49:27.143 INFO:tasks.workunit.client.1.smithi193.stdout:7/15: mknod c4 0 2024-04-25T15:49:27.143 INFO:tasks.workunit.client.1.smithi193.stdout:2/56: getdents d0/d9/d15 0 2024-04-25T15:49:27.143 INFO:tasks.workunit.client.1.smithi193.stdout:7/16: chown c4 121926941 1 2024-04-25T15:49:27.143 INFO:tasks.workunit.client.1.smithi193.stdout:2/57: write d0/fd [1530686,22766] 0 2024-04-25T15:49:27.149 INFO:tasks.workunit.client.1.smithi193.stdout:7/17: mknod c5 0 2024-04-25T15:49:27.149 INFO:tasks.workunit.client.0.smithi110.stdout:9/299: mkdir d9/d44/d5e 0 2024-04-25T15:49:27.150 INFO:tasks.workunit.client.1.smithi193.stdout:7/18: dread - f3 zero size 2024-04-25T15:49:27.151 INFO:tasks.workunit.client.0.smithi110.stdout:9/300: dread - d9/db/df/d25/f37 zero size 2024-04-25T15:49:27.151 INFO:tasks.workunit.client.1.smithi193.stdout:2/58: mknod d0/df/c18 0 2024-04-25T15:49:27.152 INFO:tasks.workunit.client.0.smithi110.stdout:9/301: dread - d9/f39 zero size 2024-04-25T15:49:27.153 INFO:tasks.workunit.client.1.smithi193.stdout:2/59: read - d0/fc zero size 2024-04-25T15:49:27.155 INFO:tasks.workunit.client.1.smithi193.stdout:9/61: truncate fc 1582707 0 2024-04-25T15:49:27.158 INFO:tasks.workunit.client.0.smithi110.stdout:9/302: creat d9/f5f x:0 0 0 2024-04-25T15:49:27.158 INFO:tasks.workunit.client.1.smithi193.stdout:6/52: dwrite f8 [0,4194304] 0 2024-04-25T15:49:27.160 INFO:tasks.workunit.client.0.smithi110.stdout:2/383: mkdir de/d6f 0 2024-04-25T15:49:27.172 INFO:tasks.workunit.client.1.smithi193.stdout:2/60: creat d0/df/f19 x:0 0 0 2024-04-25T15:49:27.173 INFO:tasks.workunit.client.0.smithi110.stdout:2/384: truncate de/d40/f3e 186770 0 2024-04-25T15:49:27.173 INFO:tasks.workunit.client.1.smithi193.stdout:0/51: sync 2024-04-25T15:49:27.174 INFO:tasks.workunit.client.1.smithi193.stdout:0/52: write d0/d3/fe [237951,112954] 0 2024-04-25T15:49:27.176 INFO:tasks.workunit.client.0.smithi110.stdout:2/385: mkdir de/d40/d70 0 2024-04-25T15:49:27.178 INFO:tasks.workunit.client.1.smithi193.stdout:5/8: rename l1 to l2 0 2024-04-25T15:49:27.179 INFO:tasks.workunit.client.1.smithi193.stdout:5/9: dwrite - no filename 2024-04-25T15:49:27.182 INFO:tasks.workunit.client.1.smithi193.stdout:2/61: mknod d0/df/c1a 0 2024-04-25T15:49:27.182 INFO:tasks.workunit.client.1.smithi193.stdout:4/35: truncate f1 3504842 0 2024-04-25T15:49:27.185 INFO:tasks.workunit.client.1.smithi193.stdout:2/62: readlink d0/l12 0 2024-04-25T15:49:27.186 INFO:tasks.workunit.client.1.smithi193.stdout:8/44: rmdir d2 39 2024-04-25T15:49:27.186 INFO:tasks.workunit.client.1.smithi193.stdout:3/86: sync 2024-04-25T15:49:27.188 INFO:tasks.workunit.client.1.smithi193.stdout:0/53: link d0/d3/fe d0/f12 0 2024-04-25T15:49:27.191 INFO:tasks.workunit.client.1.smithi193.stdout:2/63: rename d0/f3 to d0/d9/d15/f1b 0 2024-04-25T15:49:27.192 INFO:tasks.workunit.client.1.smithi193.stdout:2/64: write d0/fd [1167125,125095] 0 2024-04-25T15:49:27.192 INFO:tasks.workunit.client.1.smithi193.stdout:1/22: getdents . 0 2024-04-25T15:49:27.195 INFO:tasks.workunit.client.1.smithi193.stdout:3/87: rmdir d8 39 2024-04-25T15:49:27.196 INFO:tasks.workunit.client.0.smithi110.stdout:8/481: rmdir d14/d4c 39 2024-04-25T15:49:27.197 INFO:tasks.workunit.client.1.smithi193.stdout:2/65: dwrite d0/f7 [0,4194304] 0 2024-04-25T15:49:27.197 INFO:tasks.workunit.client.1.smithi193.stdout:0/54: fdatasync d0/f12 0 2024-04-25T15:49:27.198 INFO:tasks.workunit.client.1.smithi193.stdout:0/55: write d0/f12 [94658,106220] 0 2024-04-25T15:49:27.201 INFO:tasks.workunit.client.0.smithi110.stdout:8/482: creat d14/d21/d26/d6a/d7e/fa2 x:0 0 0 2024-04-25T15:49:27.204 INFO:tasks.workunit.client.1.smithi193.stdout:0/56: dwrite d0/f12 [0,4194304] 0 2024-04-25T15:49:27.205 INFO:tasks.workunit.client.1.smithi193.stdout:0/57: rename d0/d3/d5 to d0/d3/d5/d13 22 2024-04-25T15:49:27.208 INFO:tasks.workunit.client.0.smithi110.stdout:8/483: creat d14/d21/d26/d6a/d7e/fa3 x:0 0 0 2024-04-25T15:49:27.208 INFO:tasks.workunit.client.0.smithi110.stdout:3/477: sync 2024-04-25T15:49:27.209 INFO:tasks.workunit.client.0.smithi110.stdout:8/484: rename d14 to d14/d32/d36/d48/d8f/da4 22 2024-04-25T15:49:27.213 INFO:tasks.workunit.client.0.smithi110.stdout:8/485: dread d14/d32/d36/f84 [0,4194304] 0 2024-04-25T15:49:27.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/23: dwrite f1 [0,4194304] 0 2024-04-25T15:49:27.215 INFO:tasks.workunit.client.1.smithi193.stdout:2/66: creat d0/df/f1c x:0 0 0 2024-04-25T15:49:27.218 INFO:tasks.workunit.client.1.smithi193.stdout:2/67: write d0/df/f10 [4108443,36953] 0 2024-04-25T15:49:27.221 INFO:tasks.workunit.client.1.smithi193.stdout:2/68: dread - d0/df/f1c zero size 2024-04-25T15:49:27.223 INFO:tasks.workunit.client.1.smithi193.stdout:3/88: dwrite d8/fc [0,4194304] 0 2024-04-25T15:49:27.223 INFO:tasks.workunit.client.1.smithi193.stdout:0/58: creat d0/d3/d5/d7/f14 x:0 0 0 2024-04-25T15:49:27.223 INFO:tasks.workunit.client.1.smithi193.stdout:0/59: readlink d0/l2 0 2024-04-25T15:49:27.224 INFO:tasks.workunit.client.1.smithi193.stdout:3/89: fsync d8/d13/d12/f1c 0 2024-04-25T15:49:27.224 INFO:tasks.workunit.client.1.smithi193.stdout:1/24: dread f1 [0,4194304] 0 2024-04-25T15:49:27.230 INFO:tasks.workunit.client.0.smithi110.stdout:8/486: dwrite d14/f1c [0,4194304] 0 2024-04-25T15:49:27.235 INFO:tasks.workunit.client.0.smithi110.stdout:8/487: dread d14/f1d [0,4194304] 0 2024-04-25T15:49:27.236 INFO:tasks.workunit.client.1.smithi193.stdout:7/19: truncate f2 439728 0 2024-04-25T15:49:27.239 INFO:tasks.workunit.client.1.smithi193.stdout:6/53: write f8 [4429962,60352] 0 2024-04-25T15:49:27.240 INFO:tasks.workunit.client.1.smithi193.stdout:6/54: dread f6 [0,4194304] 0 2024-04-25T15:49:27.241 INFO:tasks.workunit.client.1.smithi193.stdout:6/55: write f8 [3845660,31212] 0 2024-04-25T15:49:27.243 INFO:tasks.workunit.client.1.smithi193.stdout:1/25: mknod c4 0 2024-04-25T15:49:27.248 INFO:tasks.workunit.client.0.smithi110.stdout:2/386: dread de/d40/f3e [0,4194304] 0 2024-04-25T15:49:27.249 INFO:tasks.workunit.client.0.smithi110.stdout:3/478: dread d0/d7/f17 [4194304,4194304] 0 2024-04-25T15:49:27.252 INFO:tasks.workunit.client.1.smithi193.stdout:5/10: readlink l2 0 2024-04-25T15:49:27.252 INFO:tasks.workunit.client.0.smithi110.stdout:5/357: sync 2024-04-25T15:49:27.252 INFO:tasks.workunit.client.0.smithi110.stdout:4/484: sync 2024-04-25T15:49:27.252 INFO:tasks.workunit.client.0.smithi110.stdout:6/357: sync 2024-04-25T15:49:27.253 INFO:tasks.workunit.client.0.smithi110.stdout:5/358: fdatasync d4/f44 0 2024-04-25T15:49:27.255 INFO:tasks.workunit.client.1.smithi193.stdout:4/36: read f1 [2915298,101024] 0 2024-04-25T15:49:27.259 INFO:tasks.workunit.client.0.smithi110.stdout:6/358: dread d1/db/d24/f29 [0,4194304] 0 2024-04-25T15:49:27.260 INFO:tasks.workunit.client.0.smithi110.stdout:6/359: truncate d1/dd/d3c/f7d 591301 0 2024-04-25T15:49:27.263 INFO:tasks.workunit.client.1.smithi193.stdout:6/56: symlink lc 0 2024-04-25T15:49:27.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/57: write fb [802623,32501] 0 2024-04-25T15:49:27.265 INFO:tasks.workunit.client.0.smithi110.stdout:9/303: write d9/db/df/d25/f51 [388142,96891] 0 2024-04-25T15:49:27.267 INFO:tasks.workunit.client.0.smithi110.stdout:8/488: creat d14/d32/fa5 x:0 0 0 2024-04-25T15:49:27.268 INFO:tasks.workunit.client.1.smithi193.stdout:6/58: dread fb [0,4194304] 0 2024-04-25T15:49:27.270 INFO:tasks.workunit.client.1.smithi193.stdout:1/26: dwrite f1 [0,4194304] 0 2024-04-25T15:49:27.273 INFO:tasks.workunit.client.1.smithi193.stdout:5/11: creat f3 x:0 0 0 2024-04-25T15:49:27.279 INFO:tasks.workunit.client.1.smithi193.stdout:7/20: chown f2 55954169 1 2024-04-25T15:49:27.280 INFO:tasks.workunit.client.1.smithi193.stdout:8/45: dwrite d2/d3/f5 [0,4194304] 0 2024-04-25T15:49:27.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/46: rename d2/d3 to d2/d3/de 22 2024-04-25T15:49:27.282 INFO:tasks.workunit.client.1.smithi193.stdout:9/62: dwrite fc [0,4194304] 0 2024-04-25T15:49:27.283 INFO:tasks.workunit.client.0.smithi110.stdout:4/485: rename f8 to d13/d24/d2d/d44/d65/d99/f9c 0 2024-04-25T15:49:27.285 INFO:tasks.workunit.client.0.smithi110.stdout:4/486: write d13/d24/d3c/d56/d88/f20 [1153203,41320] 0 2024-04-25T15:49:27.290 INFO:tasks.workunit.client.1.smithi193.stdout:9/63: dwrite f4 [0,4194304] 0 2024-04-25T15:49:27.290 INFO:tasks.workunit.client.0.smithi110.stdout:9/304: truncate f8 4616808 0 2024-04-25T15:49:27.293 INFO:tasks.workunit.client.1.smithi193.stdout:9/64: dread fb [0,4194304] 0 2024-04-25T15:49:27.294 INFO:tasks.workunit.client.0.smithi110.stdout:8/489: mkdir d14/d21/d26/d6a/d7e/da6 0 2024-04-25T15:49:27.295 INFO:tasks.workunit.client.0.smithi110.stdout:8/490: dread - d14/d21/d23/f60 zero size 2024-04-25T15:49:27.296 INFO:tasks.workunit.client.1.smithi193.stdout:3/90: rename d8/d13/d12 to d8/d13/d1f 0 2024-04-25T15:49:27.296 INFO:tasks.workunit.client.1.smithi193.stdout:6/59: mkdir dd 0 2024-04-25T15:49:27.297 INFO:tasks.workunit.client.0.smithi110.stdout:2/387: symlink de/d40/d6e/l71 0 2024-04-25T15:49:27.299 INFO:tasks.workunit.client.1.smithi193.stdout:1/27: mknod c5 0 2024-04-25T15:49:27.299 INFO:tasks.workunit.client.0.smithi110.stdout:8/491: read d14/d21/f2a [1226808,102649] 0 2024-04-25T15:49:27.302 INFO:tasks.workunit.client.1.smithi193.stdout:0/60: link d0/d3/d5/l11 d0/d3/d5/d7/l15 0 2024-04-25T15:49:27.305 INFO:tasks.workunit.client.1.smithi193.stdout:1/28: dwrite f1 [0,4194304] 0 2024-04-25T15:49:27.306 INFO:tasks.workunit.client.1.smithi193.stdout:5/12: rename f3 to f4 0 2024-04-25T15:49:27.306 INFO:tasks.workunit.client.1.smithi193.stdout:5/13: rmdir - no directory 2024-04-25T15:49:27.310 INFO:tasks.workunit.client.1.smithi193.stdout:0/61: dwrite d0/fc [0,4194304] 0 2024-04-25T15:49:27.311 INFO:tasks.workunit.client.0.smithi110.stdout:9/305: dread d9/db/df/f33 [0,4194304] 0 2024-04-25T15:49:27.311 INFO:tasks.workunit.client.1.smithi193.stdout:0/62: chown d0/d3 80472 1 2024-04-25T15:49:27.315 INFO:tasks.workunit.client.1.smithi193.stdout:4/37: dwrite f0 [0,4194304] 0 2024-04-25T15:49:27.315 INFO:tasks.workunit.client.0.smithi110.stdout:4/487: symlink d13/d24/d2d/d55/l9d 0 2024-04-25T15:49:27.317 INFO:tasks.workunit.client.1.smithi193.stdout:2/69: getdents d0 0 2024-04-25T15:49:27.317 INFO:tasks.workunit.client.0.smithi110.stdout:2/388: fdatasync de/d40/d5b/d5c/d5f/f63 0 2024-04-25T15:49:27.319 INFO:tasks.workunit.client.0.smithi110.stdout:2/389: write de/d40/d4f/f47 [1015328,36633] 0 2024-04-25T15:49:27.321 INFO:tasks.workunit.client.0.smithi110.stdout:7/378: write d1b/d29/f5d [1801169,1911] 0 2024-04-25T15:49:27.322 INFO:tasks.workunit.client.0.smithi110.stdout:8/492: rmdir d14/d21/d26/d6a 39 2024-04-25T15:49:27.323 INFO:tasks.workunit.client.0.smithi110.stdout:8/493: dread - d14/d21/d26/f4a zero size 2024-04-25T15:49:27.324 INFO:tasks.workunit.client.1.smithi193.stdout:8/47: rename d2/d3/fd to d2/d3/ff 0 2024-04-25T15:49:27.330 INFO:tasks.workunit.client.0.smithi110.stdout:5/359: creat d4/f7c x:0 0 0 2024-04-25T15:49:27.332 INFO:tasks.workunit.client.0.smithi110.stdout:9/306: chown d9/db/c43 601 1 2024-04-25T15:49:27.334 INFO:tasks.workunit.client.0.smithi110.stdout:4/488: symlink d13/d24/d2d/d44/d65/d99/l9e 0 2024-04-25T15:49:27.338 INFO:tasks.workunit.client.0.smithi110.stdout:9/307: dwrite d9/db/df/f21 [0,4194304] 0 2024-04-25T15:49:27.340 INFO:tasks.workunit.client.1.smithi193.stdout:9/65: unlink c3 0 2024-04-25T15:49:27.342 INFO:tasks.workunit.client.0.smithi110.stdout:2/390: creat de/d40/f72 x:0 0 0 2024-04-25T15:49:27.343 INFO:tasks.workunit.client.1.smithi193.stdout:6/60: rename f8 to dd/fe 0 2024-04-25T15:49:27.346 INFO:tasks.workunit.client.0.smithi110.stdout:7/379: write d1b/d29/f57 [4326086,10231] 0 2024-04-25T15:49:27.347 INFO:tasks.workunit.client.0.smithi110.stdout:9/308: dwrite d9/db/df/d2c/d3c/f49 [0,4194304] 0 2024-04-25T15:49:27.348 INFO:tasks.workunit.client.1.smithi193.stdout:9/66: dwrite f9 [0,4194304] 0 2024-04-25T15:49:27.348 INFO:tasks.workunit.client.1.smithi193.stdout:3/91: creat d8/d13/d14/f20 x:0 0 0 2024-04-25T15:49:27.350 INFO:tasks.workunit.client.1.smithi193.stdout:3/92: dread - d8/d13/d14/f1d zero size 2024-04-25T15:49:27.353 INFO:tasks.workunit.client.0.smithi110.stdout:9/309: dread d9/db/df/d25/f51 [0,4194304] 0 2024-04-25T15:49:27.354 INFO:tasks.workunit.client.1.smithi193.stdout:3/93: dread d8/f11 [0,4194304] 0 2024-04-25T15:49:27.354 INFO:tasks.workunit.client.1.smithi193.stdout:3/94: truncate d8/f11 4815379 0 2024-04-25T15:49:27.356 INFO:tasks.workunit.client.0.smithi110.stdout:3/479: link d0/d2e/f39 d0/d7/fa6 0 2024-04-25T15:49:27.362 INFO:tasks.workunit.client.1.smithi193.stdout:5/14: stat l2 0 2024-04-25T15:49:27.366 INFO:tasks.workunit.client.0.smithi110.stdout:2/391: creat de/d40/d5b/d5c/d5f/f73 x:0 0 0 2024-04-25T15:49:27.368 INFO:tasks.workunit.client.1.smithi193.stdout:7/21: dread f3 [0,4194304] 0 2024-04-25T15:49:27.368 INFO:tasks.workunit.client.1.smithi193.stdout:7/22: chown f2 46 1 2024-04-25T15:49:27.370 INFO:tasks.workunit.client.1.smithi193.stdout:0/63: mkdir d0/d3/d5/d7/dd/d16 0 2024-04-25T15:49:27.372 INFO:tasks.workunit.client.1.smithi193.stdout:4/38: creat f9 x:0 0 0 2024-04-25T15:49:27.374 INFO:tasks.workunit.client.1.smithi193.stdout:8/48: fdatasync d2/fa 0 2024-04-25T15:49:27.376 INFO:tasks.workunit.client.1.smithi193.stdout:0/64: dwrite d0/d3/fe [0,4194304] 0 2024-04-25T15:49:27.382 INFO:tasks.workunit.client.0.smithi110.stdout:2/392: symlink de/d40/d6e/d50/l74 0 2024-04-25T15:49:27.383 INFO:tasks.workunit.client.1.smithi193.stdout:8/49: dwrite d2/d3/f5 [0,4194304] 0 2024-04-25T15:49:27.383 INFO:tasks.workunit.client.1.smithi193.stdout:8/50: rename d2/d3 to d2/d3/db/d10 22 2024-04-25T15:49:27.384 INFO:tasks.workunit.client.1.smithi193.stdout:6/61: creat dd/ff x:0 0 0 2024-04-25T15:49:27.384 INFO:tasks.workunit.client.1.smithi193.stdout:6/62: chown dd 352250776 1 2024-04-25T15:49:27.390 INFO:tasks.workunit.client.1.smithi193.stdout:9/67: mknod cd 0 2024-04-25T15:49:27.390 INFO:tasks.workunit.client.0.smithi110.stdout:8/494: dwrite d14/d32/d36/f84 [4194304,4194304] 0 2024-04-25T15:49:27.393 INFO:tasks.workunit.client.1.smithi193.stdout:9/68: dread fb [0,4194304] 0 2024-04-25T15:49:27.394 INFO:tasks.workunit.client.0.smithi110.stdout:2/393: dwrite de/d40/d6e/d52/f62 [0,4194304] 0 2024-04-25T15:49:27.398 INFO:tasks.workunit.client.1.smithi193.stdout:3/95: creat d8/d13/d14/f21 x:0 0 0 2024-04-25T15:49:27.403 INFO:tasks.workunit.client.0.smithi110.stdout:5/360: rename d4/d1d/f1e to d4/d1d/d1f/d35/f7d 0 2024-04-25T15:49:27.404 INFO:tasks.workunit.client.1.smithi193.stdout:5/15: write f4 [238361,26755] 0 2024-04-25T15:49:27.408 INFO:tasks.workunit.client.0.smithi110.stdout:7/380: creat d1b/f79 x:0 0 0 2024-04-25T15:49:27.411 INFO:tasks.workunit.client.0.smithi110.stdout:3/480: dwrite d0/d16/f3a [0,4194304] 0 2024-04-25T15:49:27.412 INFO:tasks.workunit.client.1.smithi193.stdout:4/39: creat fa x:0 0 0 2024-04-25T15:49:27.415 INFO:tasks.workunit.client.1.smithi193.stdout:2/70: symlink d0/d9/d14/l1d 0 2024-04-25T15:49:27.415 INFO:tasks.workunit.client.0.smithi110.stdout:9/310: fsync d9/d2d/f5d 0 2024-04-25T15:49:27.416 INFO:tasks.workunit.client.0.smithi110.stdout:9/311: chown d9/d2d/f30 2 1 2024-04-25T15:49:27.418 INFO:tasks.workunit.client.0.smithi110.stdout:2/394: symlink de/d40/d5b/d5c/l75 0 2024-04-25T15:49:27.418 INFO:tasks.workunit.client.0.smithi110.stdout:2/395: readlink de/d40/d6e/d52/l57 0 2024-04-25T15:49:27.421 INFO:tasks.workunit.client.1.smithi193.stdout:0/65: readlink d0/la 0 2024-04-25T15:49:27.423 INFO:tasks.workunit.client.1.smithi193.stdout:0/66: write d0/ff [1961315,34644] 0 2024-04-25T15:49:27.423 INFO:tasks.workunit.client.1.smithi193.stdout:2/71: dwrite d0/fd [0,4194304] 0 2024-04-25T15:49:27.424 INFO:tasks.workunit.client.1.smithi193.stdout:8/51: creat d2/f11 x:0 0 0 2024-04-25T15:49:27.425 INFO:tasks.workunit.client.0.smithi110.stdout:5/361: unlink d4/d2c/f4f 0 2024-04-25T15:49:27.429 INFO:tasks.workunit.client.1.smithi193.stdout:1/29: truncate f1 157959 0 2024-04-25T15:49:27.430 INFO:tasks.workunit.client.1.smithi193.stdout:9/69: mknod ce 0 2024-04-25T15:49:27.431 INFO:tasks.workunit.client.0.smithi110.stdout:3/481: symlink d0/d2e/d88/la7 0 2024-04-25T15:49:27.432 INFO:tasks.workunit.client.1.smithi193.stdout:0/67: dwrite d0/fc [0,4194304] 0 2024-04-25T15:49:27.435 INFO:tasks.workunit.client.1.smithi193.stdout:9/70: dread f4 [0,4194304] 0 2024-04-25T15:49:27.435 INFO:tasks.workunit.client.1.smithi193.stdout:9/71: readlink l8 0 2024-04-25T15:49:27.435 INFO:tasks.workunit.client.0.smithi110.stdout:7/381: dread d1b/d29/d38/f5b [0,4194304] 0 2024-04-25T15:49:27.435 INFO:tasks.workunit.client.1.smithi193.stdout:9/72: write f6 [860918,96922] 0 2024-04-25T15:49:27.436 INFO:tasks.workunit.client.0.smithi110.stdout:8/495: readlink d14/d21/d26/d6a/d7e/l85 0 2024-04-25T15:49:27.437 INFO:tasks.workunit.client.1.smithi193.stdout:3/96: creat d8/d13/d1f/f22 x:0 0 0 2024-04-25T15:49:27.437 INFO:tasks.workunit.client.1.smithi193.stdout:3/97: truncate d8/f10 1595352 0 2024-04-25T15:49:27.439 INFO:tasks.workunit.client.1.smithi193.stdout:5/16: creat f5 x:0 0 0 2024-04-25T15:49:27.442 INFO:tasks.workunit.client.1.smithi193.stdout:4/40: fdatasync f4 0 2024-04-25T15:49:27.443 INFO:tasks.workunit.client.0.smithi110.stdout:5/362: dread d4/d1d/d1f/d35/d3d/f5d [4194304,4194304] 0 2024-04-25T15:49:27.443 INFO:tasks.workunit.client.1.smithi193.stdout:4/41: write f2 [1360502,12312] 0 2024-04-25T15:49:27.447 INFO:tasks.workunit.client.1.smithi193.stdout:6/63: write fb [3076767,75711] 0 2024-04-25T15:49:27.450 INFO:tasks.workunit.client.1.smithi193.stdout:4/42: dwrite f2 [0,4194304] 0 2024-04-25T15:49:27.451 INFO:tasks.workunit.client.0.smithi110.stdout:1/499: sync 2024-04-25T15:49:27.451 INFO:tasks.workunit.client.0.smithi110.stdout:0/402: sync 2024-04-25T15:49:27.452 INFO:tasks.workunit.client.1.smithi193.stdout:4/43: write f0 [1301994,47999] 0 2024-04-25T15:49:27.452 INFO:tasks.workunit.client.0.smithi110.stdout:0/403: readlink d1/d23/d30/d33/l4a 0 2024-04-25T15:49:27.456 INFO:tasks.workunit.client.1.smithi193.stdout:8/52: rename d2/l6 to d2/d3/db/l12 0 2024-04-25T15:49:27.458 INFO:tasks.workunit.client.0.smithi110.stdout:7/382: creat d1b/d29/d42/f7a x:0 0 0 2024-04-25T15:49:27.462 INFO:tasks.workunit.client.0.smithi110.stdout:2/396: symlink de/d6f/l76 0 2024-04-25T15:49:27.463 INFO:tasks.workunit.client.0.smithi110.stdout:2/397: fdatasync de/f24 0 2024-04-25T15:49:27.464 INFO:tasks.workunit.client.1.smithi193.stdout:0/68: unlink d0/d3/d5/d7/c9 0 2024-04-25T15:49:27.466 INFO:tasks.workunit.client.0.smithi110.stdout:5/363: creat d4/d1d/d1f/d56/f7e x:0 0 0 2024-04-25T15:49:27.469 INFO:tasks.workunit.client.0.smithi110.stdout:4/489: rename d13/d24/d2d/d36/d51/f85 to d13/d24/f9f 0 2024-04-25T15:49:27.470 INFO:tasks.workunit.client.1.smithi193.stdout:3/98: fdatasync d8/f11 0 2024-04-25T15:49:27.471 INFO:tasks.workunit.client.0.smithi110.stdout:1/500: creat d0/d14/d5f/d7d/f97 x:0 0 0 2024-04-25T15:49:27.475 INFO:tasks.workunit.client.0.smithi110.stdout:9/312: dwrite f8 [0,4194304] 0 2024-04-25T15:49:27.476 INFO:tasks.workunit.client.0.smithi110.stdout:9/313: stat d9/db/c43 0 2024-04-25T15:49:27.480 INFO:tasks.workunit.client.0.smithi110.stdout:8/496: creat d14/d4c/d71/d9d/fa7 x:0 0 0 2024-04-25T15:49:27.481 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.480+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8008104830 con 0x7f80080fcf70 2024-04-25T15:49:27.482 INFO:tasks.workunit.client.1.smithi193.stdout:6/64: write dd/fe [562916,103464] 0 2024-04-25T15:49:27.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.482+0000 7f80057fa700 1 -- 172.21.15.110:0/1630184803 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f7ff40584d0 con 0x7f80080fcf70 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:49:27.483 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:49:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:49:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-25T15:49:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:49:27.484 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:49:27.484 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:49:27.486 INFO:tasks.workunit.client.1.smithi193.stdout:7/23: sync 2024-04-25T15:49:27.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.485+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7ff806fcc0 msgr2=0x7f7ff8072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:27.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.485+0000 7f800f19d700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7ff806fcc0 0x7f7ff8072170 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc00b3b0 tx=0x7f7ffc009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:27.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.485+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fcf70 msgr2=0x7f800810cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:27.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.485+0000 7f800f19d700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fcf70 0x7f800810cd70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400b790 tx=0x7f7ff4009250 comp rx=0 tx=0).stop 2024-04-25T15:49:27.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.486+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 shutdown_connections 2024-04-25T15:49:27.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.486+0000 7f800f19d700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7ff806fcc0 0x7f7ff8072170 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:27.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.486+0000 7f800f19d700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f80080fbcc0 0x7f800810c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:27.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.486+0000 7f800f19d700 1 --2- 172.21.15.110:0/1630184803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80080fcf70 0x7f800810cd70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:27.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.486+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 >> 172.21.15.110:0/1630184803 conn(0x7f80080f7440 msgr2=0x7f80080f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:27.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.486+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 shutdown_connections 2024-04-25T15:49:27.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.486+0000 7f800f19d700 1 -- 172.21.15.110:0/1630184803 wait complete. 2024-04-25T15:49:27.488 INFO:tasks.workunit.client.0.smithi110.stdout:8/497: dwrite d14/f83 [4194304,4194304] 0 2024-04-25T15:49:27.489 INFO:tasks.workunit.client.0.smithi110.stdout:2/398: mknod de/d40/d6e/d50/c77 0 2024-04-25T15:49:27.492 INFO:tasks.workunit.client.1.smithi193.stdout:4/44: dwrite f4 [0,4194304] 0 2024-04-25T15:49:27.492 INFO:tasks.workunit.client.0.smithi110.stdout:1/501: dread d0/d25/d33/f47 [0,4194304] 0 2024-04-25T15:49:27.492 INFO:tasks.workunit.client.1.smithi193.stdout:8/53: mknod d2/d3/c13 0 2024-04-25T15:49:27.494 INFO:tasks.workunit.client.1.smithi193.stdout:8/54: write d2/f11 [1003216,13887] 0 2024-04-25T15:49:27.500 INFO:tasks.workunit.client.0.smithi110.stdout:9/314: chown d9/d13/f54 37886 1 2024-04-25T15:49:27.501 INFO:tasks.workunit.client.1.smithi193.stdout:4/45: dwrite f9 [0,4194304] 0 2024-04-25T15:49:27.501 INFO:tasks.workunit.client.1.smithi193.stdout:4/46: stat c6 0 2024-04-25T15:49:27.502 INFO:tasks.workunit.client.1.smithi193.stdout:4/47: dread - fa zero size 2024-04-25T15:49:27.502 INFO:tasks.workunit.client.1.smithi193.stdout:4/48: rmdir - no directory 2024-04-25T15:49:27.502 INFO:tasks.workunit.client.1.smithi193.stdout:0/69: creat d0/f17 x:0 0 0 2024-04-25T15:49:27.502 INFO:tasks.workunit.client.1.smithi193.stdout:0/70: dread - d0/f17 zero size 2024-04-25T15:49:27.503 INFO:tasks.workunit.client.0.smithi110.stdout:6/360: sync 2024-04-25T15:49:27.503 INFO:tasks.workunit.client.0.smithi110.stdout:3/482: link d0/d7/d1c/d2f/d33/l34 d0/d16/d8c/d90/la8 0 2024-04-25T15:49:27.503 INFO:tasks.workunit.client.1.smithi193.stdout:0/71: write d0/d3/fe [1837952,18250] 0 2024-04-25T15:49:27.504 INFO:tasks.workunit.client.0.smithi110.stdout:6/361: readlink d1/db/d22/d33/d43/l41 0 2024-04-25T15:49:27.505 INFO:tasks.workunit.client.1.smithi193.stdout:0/72: write d0/d3/fe [4946064,75796] 0 2024-04-25T15:49:27.505 INFO:tasks.workunit.client.0.smithi110.stdout:3/483: write d0/d7/f6c [65418,51503] 0 2024-04-25T15:49:27.506 INFO:tasks.workunit.client.1.smithi193.stdout:4/49: dread f2 [0,4194304] 0 2024-04-25T15:49:27.506 INFO:tasks.workunit.client.1.smithi193.stdout:0/73: chown d0/d3/d5/d7/fb 4686163 1 2024-04-25T15:49:27.507 INFO:tasks.workunit.client.1.smithi193.stdout:2/72: truncate d0/fd 1286361 0 2024-04-25T15:49:27.507 INFO:tasks.workunit.client.1.smithi193.stdout:4/50: truncate f2 4477890 0 2024-04-25T15:49:27.508 INFO:tasks.workunit.client.0.smithi110.stdout:3/484: chown d0/d7/d1c/d50/c62 42 1 2024-04-25T15:49:27.509 INFO:tasks.workunit.client.0.smithi110.stdout:7/383: getdents d1b/d29/d38/d55 0 2024-04-25T15:49:27.514 INFO:tasks.workunit.client.1.smithi193.stdout:6/65: mknod dd/c10 0 2024-04-25T15:49:27.516 INFO:tasks.workunit.client.0.smithi110.stdout:4/490: dwrite d13/d24/f2a [0,4194304] 0 2024-04-25T15:49:27.517 INFO:tasks.workunit.client.1.smithi193.stdout:6/66: write dd/fe [2416083,114838] 0 2024-04-25T15:49:27.519 INFO:tasks.workunit.client.1.smithi193.stdout:0/74: dread d0/d3/fe [0,4194304] 0 2024-04-25T15:49:27.521 INFO:tasks.workunit.client.1.smithi193.stdout:5/17: sync 2024-04-25T15:49:27.521 INFO:tasks.workunit.client.1.smithi193.stdout:1/30: sync 2024-04-25T15:49:27.521 INFO:tasks.workunit.client.1.smithi193.stdout:2/73: dwrite d0/df/f1c [0,4194304] 0 2024-04-25T15:49:27.521 INFO:tasks.workunit.client.1.smithi193.stdout:6/67: read f6 [109744,67308] 0 2024-04-25T15:49:27.521 INFO:tasks.workunit.client.1.smithi193.stdout:4/51: dwrite fa [0,4194304] 0 2024-04-25T15:49:27.522 INFO:tasks.workunit.client.0.smithi110.stdout:1/502: creat d0/d51/f98 x:0 0 0 2024-04-25T15:49:27.522 INFO:tasks.workunit.client.1.smithi193.stdout:5/18: write f5 [320156,22509] 0 2024-04-25T15:49:27.523 INFO:tasks.workunit.client.0.smithi110.stdout:0/404: dwrite d1/f1c [0,4194304] 0 2024-04-25T15:49:27.524 INFO:tasks.workunit.client.1.smithi193.stdout:0/75: chown d0/d3/d5/d7/dd 19564236 1 2024-04-25T15:49:27.524 INFO:tasks.workunit.client.1.smithi193.stdout:5/19: write f4 [968640,81390] 0 2024-04-25T15:49:27.526 INFO:tasks.workunit.client.0.smithi110.stdout:9/315: unlink d9/db/l40 0 2024-04-25T15:49:27.526 INFO:tasks.workunit.client.1.smithi193.stdout:0/76: truncate d0/f17 289769 0 2024-04-25T15:49:27.526 INFO:tasks.workunit.client.0.smithi110.stdout:0/405: chown d1/d47/d54/f9c 411624 1 2024-04-25T15:49:27.527 INFO:tasks.workunit.client.1.smithi193.stdout:9/73: truncate f4 2891343 0 2024-04-25T15:49:27.528 INFO:tasks.workunit.client.1.smithi193.stdout:0/77: dread - d0/d3/d5/d7/f14 zero size 2024-04-25T15:49:27.531 INFO:tasks.workunit.client.0.smithi110.stdout:4/491: dwrite d13/d24/d2d/d36/f4d [0,4194304] 0 2024-04-25T15:49:27.535 INFO:tasks.workunit.client.0.smithi110.stdout:7/384: mknod d1b/d29/d46/c7b 0 2024-04-25T15:49:27.546 INFO:tasks.workunit.client.0.smithi110.stdout:8/498: rename d14/d32/f3f to d14/d21/d26/d6a/d7e/da6/fa8 0 2024-04-25T15:49:27.553 INFO:tasks.workunit.client.0.smithi110.stdout:8/499: dwrite f6 [0,4194304] 0 2024-04-25T15:49:27.555 INFO:tasks.workunit.client.1.smithi193.stdout:8/55: dwrite d2/fa [0,4194304] 0 2024-04-25T15:49:27.558 INFO:tasks.workunit.client.1.smithi193.stdout:1/31: creat f6 x:0 0 0 2024-04-25T15:49:27.563 INFO:tasks.workunit.client.1.smithi193.stdout:4/52: mkdir db 0 2024-04-25T15:49:27.564 INFO:tasks.workunit.client.1.smithi193.stdout:8/56: dread d2/d3/f5 [0,4194304] 0 2024-04-25T15:49:27.564 INFO:tasks.workunit.client.0.smithi110.stdout:8/500: dwrite f5 [4194304,4194304] 0 2024-04-25T15:49:27.566 INFO:tasks.workunit.client.1.smithi193.stdout:2/74: unlink d0/df/c1a 0 2024-04-25T15:49:27.567 INFO:tasks.workunit.client.1.smithi193.stdout:2/75: chown d0/le 410866993 1 2024-04-25T15:49:27.570 INFO:tasks.workunit.client.1.smithi193.stdout:6/68: creat dd/f11 x:0 0 0 2024-04-25T15:49:27.573 INFO:tasks.workunit.client.0.smithi110.stdout:6/362: dwrite d1/dd/f17 [4194304,4194304] 0 2024-04-25T15:49:27.576 INFO:tasks.workunit.client.0.smithi110.stdout:7/385: symlink d1b/d29/d46/l7c 0 2024-04-25T15:49:27.577 INFO:tasks.workunit.client.0.smithi110.stdout:5/364: getdents d4/d1d/d1f/d56 0 2024-04-25T15:49:27.579 INFO:tasks.workunit.client.0.smithi110.stdout:7/386: dread d1b/d29/d42/f78 [0,4194304] 0 2024-04-25T15:49:27.579 INFO:tasks.workunit.client.0.smithi110.stdout:5/365: dread - d4/d1d/d1f/d56/f7e zero size 2024-04-25T15:49:27.579 INFO:tasks.workunit.client.0.smithi110.stdout:2/399: truncate de/d40/d4f/f4c 1182892 0 2024-04-25T15:49:27.582 INFO:tasks.workunit.client.0.smithi110.stdout:5/366: fsync d4/f6d 0 2024-04-25T15:49:27.584 INFO:tasks.workunit.client.0.smithi110.stdout:8/501: dread d14/d21/f27 [0,4194304] 0 2024-04-25T15:49:27.585 INFO:tasks.workunit.client.0.smithi110.stdout:5/367: fsync d4/d15/f1a 0 2024-04-25T15:49:27.588 INFO:tasks.workunit.client.1.smithi193.stdout:7/24: link c5 c6 0 2024-04-25T15:49:27.589 INFO:tasks.workunit.client.0.smithi110.stdout:5/368: dread - d4/f6d zero size 2024-04-25T15:49:27.593 INFO:tasks.workunit.client.0.smithi110.stdout:1/503: dwrite d0/d25/d33/f48 [0,4194304] 0 2024-04-25T15:49:27.594 INFO:tasks.workunit.client.0.smithi110.stdout:4/492: symlink d13/d24/d2d/d44/d90/la0 0 2024-04-25T15:49:27.595 INFO:tasks.workunit.client.0.smithi110.stdout:9/316: dwrite d9/db/f1b [4194304,4194304] 0 2024-04-25T15:49:27.603 INFO:tasks.workunit.client.0.smithi110.stdout:3/485: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:27.603 INFO:tasks.workunit.client.0.smithi110.stdout:5/369: dwrite d4/f10 [0,4194304] 0 2024-04-25T15:49:27.606 INFO:tasks.workunit.client.1.smithi193.stdout:2/76: symlink d0/d9/d14/l1e 0 2024-04-25T15:49:27.607 INFO:tasks.workunit.client.0.smithi110.stdout:6/363: symlink d1/d12/l86 0 2024-04-25T15:49:27.611 INFO:tasks.workunit.client.1.smithi193.stdout:6/69: rmdir dd 39 2024-04-25T15:49:27.616 INFO:tasks.workunit.client.0.smithi110.stdout:9/317: dwrite d9/db/df/f4f [0,4194304] 0 2024-04-25T15:49:27.623 INFO:tasks.workunit.client.0.smithi110.stdout:1/504: dread d0/d14/f1d [0,4194304] 0 2024-04-25T15:49:27.624 INFO:tasks.workunit.client.1.smithi193.stdout:0/78: mkdir d0/d3/d5/d7/dd/d16/d18 0 2024-04-25T15:49:27.625 INFO:tasks.workunit.client.0.smithi110.stdout:8/502: symlink d14/d21/d26/d6a/la9 0 2024-04-25T15:49:27.626 INFO:tasks.workunit.client.0.smithi110.stdout:9/318: dread d9/d2d/f38 [0,4194304] 0 2024-04-25T15:49:27.626 INFO:tasks.workunit.client.1.smithi193.stdout:3/99: getdents d8 0 2024-04-25T15:49:27.627 INFO:tasks.workunit.client.1.smithi193.stdout:3/100: write d8/d13/d14/f18 [915558,54107] 0 2024-04-25T15:49:27.628 INFO:tasks.workunit.client.0.smithi110.stdout:7/387: dwrite d1b/d29/d38/f53 [0,4194304] 0 2024-04-25T15:49:27.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/503: dread - d14/d32/d36/d48/f69 zero size 2024-04-25T15:49:27.634 INFO:tasks.workunit.client.1.smithi193.stdout:9/74: truncate fc 2786177 0 2024-04-25T15:49:27.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/53: creat db/fc x:0 0 0 2024-04-25T15:49:27.640 INFO:tasks.workunit.client.0.smithi110.stdout:6/364: dread d1/f9 [0,4194304] 0 2024-04-25T15:49:27.641 INFO:tasks.workunit.client.1.smithi193.stdout:7/25: write f3 [212163,60185] 0 2024-04-25T15:49:27.643 INFO:tasks.workunit.client.1.smithi193.stdout:8/57: unlink d2/d3/f5 0 2024-04-25T15:49:27.646 INFO:tasks.workunit.client.1.smithi193.stdout:8/58: dread d2/fa [0,4194304] 0 2024-04-25T15:49:27.646 INFO:tasks.workunit.client.1.smithi193.stdout:2/77: rename d0/l13 to d0/d9/l1f 0 2024-04-25T15:49:27.647 INFO:tasks.workunit.client.1.smithi193.stdout:1/32: write f1 [1145047,5546] 0 2024-04-25T15:49:27.647 INFO:tasks.workunit.client.0.smithi110.stdout:2/400: truncate de/d40/d6e/f3a 3559727 0 2024-04-25T15:49:27.648 INFO:tasks.workunit.client.1.smithi193.stdout:2/78: truncate d0/fc 958235 0 2024-04-25T15:49:27.648 INFO:tasks.workunit.client.1.smithi193.stdout:2/79: chown d0/d9/d15 12387030 1 2024-04-25T15:49:27.649 INFO:tasks.workunit.client.0.smithi110.stdout:5/370: dread d4/d2c/f19 [0,4194304] 0 2024-04-25T15:49:27.650 INFO:tasks.workunit.client.1.smithi193.stdout:5/20: link l2 l6 0 2024-04-25T15:49:27.653 INFO:tasks.workunit.client.0.smithi110.stdout:1/505: creat d0/d51/f99 x:0 0 0 2024-04-25T15:49:27.656 INFO:tasks.workunit.client.0.smithi110.stdout:4/493: symlink d13/la1 0 2024-04-25T15:49:27.658 INFO:tasks.workunit.client.0.smithi110.stdout:7/388: mkdir d1b/d29/d38/d55/d7d 0 2024-04-25T15:49:27.660 INFO:tasks.workunit.client.0.smithi110.stdout:8/504: creat d14/d4c/faa x:0 0 0 2024-04-25T15:49:27.662 INFO:tasks.workunit.client.0.smithi110.stdout:7/389: dread d1b/d29/d42/f78 [0,4194304] 0 2024-04-25T15:49:27.663 INFO:tasks.workunit.client.0.smithi110.stdout:1/506: dwrite d0/d51/f98 [0,4194304] 0 2024-04-25T15:49:27.664 INFO:tasks.workunit.client.1.smithi193.stdout:4/54: rmdir db 39 2024-04-25T15:49:27.664 INFO:tasks.workunit.client.0.smithi110.stdout:7/390: fsync d1b/d29/d46/f47 0 2024-04-25T15:49:27.665 INFO:tasks.workunit.client.1.smithi193.stdout:7/26: creat f7 x:0 0 0 2024-04-25T15:49:27.666 INFO:tasks.workunit.client.1.smithi193.stdout:6/70: dwrite dd/f11 [0,4194304] 0 2024-04-25T15:49:27.667 INFO:tasks.workunit.client.1.smithi193.stdout:6/71: chown lc 10098 1 2024-04-25T15:49:27.670 INFO:tasks.workunit.client.1.smithi193.stdout:6/72: dread dd/f11 [0,4194304] 0 2024-04-25T15:49:27.670 INFO:tasks.workunit.client.1.smithi193.stdout:6/73: fsync fb 0 2024-04-25T15:49:27.671 INFO:tasks.workunit.client.1.smithi193.stdout:8/59: symlink d2/l14 0 2024-04-25T15:49:27.672 INFO:tasks.workunit.client.0.smithi110.stdout:3/486: truncate d0/d7/f69 1731019 0 2024-04-25T15:49:27.673 INFO:tasks.workunit.client.0.smithi110.stdout:3/487: chown d0/d7/d1c/d2f/d4c 3855 1 2024-04-25T15:49:27.675 INFO:tasks.workunit.client.1.smithi193.stdout:2/80: rename d0/fa to d0/df/d16/f20 0 2024-04-25T15:49:27.675 INFO:tasks.workunit.client.1.smithi193.stdout:2/81: stat d0/d9 0 2024-04-25T15:49:27.679 INFO:tasks.workunit.client.1.smithi193.stdout:5/21: mknod c7 0 2024-04-25T15:49:27.680 INFO:tasks.workunit.client.1.smithi193.stdout:5/22: read f5 [126848,76340] 0 2024-04-25T15:49:27.681 INFO:tasks.workunit.client.0.smithi110.stdout:2/401: mknod de/d40/c78 0 2024-04-25T15:49:27.683 INFO:tasks.workunit.client.0.smithi110.stdout:2/402: write de/d40/d4f/f47 [1643473,14152] 0 2024-04-25T15:49:27.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.685+0000 7fcf09c7c700 1 -- 172.21.15.110:0/728559136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 msgr2=0x7fcefc097f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:27.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.685+0000 7fcf09c7c700 1 --2- 172.21.15.110:0/728559136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 0x7fcefc097f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fcef0004880 tx=0x7fcef0009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:27.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.685+0000 7fcf09c7c700 1 -- 172.21.15.110:0/728559136 shutdown_connections 2024-04-25T15:49:27.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.685+0000 7fcf09c7c700 1 --2- 172.21.15.110:0/728559136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 0x7fcefc097f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:27.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.685+0000 7fcf09c7c700 1 --2- 172.21.15.110:0/728559136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcefc006d40 0x7fcefc0955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:27.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.685+0000 7fcf09c7c700 1 -- 172.21.15.110:0/728559136 >> 172.21.15.110:0/728559136 conn(0x7fcefc08f240 msgr2=0x7fcefc091670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:27.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.686+0000 7fcf09c7c700 1 -- 172.21.15.110:0/728559136 shutdown_connections 2024-04-25T15:49:27.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.686+0000 7fcf09c7c700 1 -- 172.21.15.110:0/728559136 wait complete. 2024-04-25T15:49:27.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.687+0000 7fcf09c7c700 1 Processor -- start 2024-04-25T15:49:27.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.687+0000 7fcf09c7c700 1 -- start start 2024-04-25T15:49:27.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.687+0000 7fcf09c7c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcefc006d40 0x7fcefc0abb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:27.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.687+0000 7fcf09c7c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 0x7fcefc0ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:27.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf09c7c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0ac650 con 0x7fcefc006d40 2024-04-25T15:49:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf09c7c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcefc0ac7c0 con 0x7fcefc095b00 2024-04-25T15:49:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf08c7a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcefc006d40 0x7fcefc0abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf08c7a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcefc006d40 0x7fcefc0abb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35950/0 (socket says 172.21.15.110:35950) 2024-04-25T15:49:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf08c7a700 1 -- 172.21.15.110:0/1873777944 learned_addr learned my addr 172.21.15.110:0/1873777944 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf03fff700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 0x7fcefc0ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:27.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf08c7a700 1 -- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 msgr2=0x7fcefc0ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:27.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf08c7a700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 0x7fcefc0ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:27.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf08c7a700 1 -- 172.21.15.110:0/1873777944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef0005040 con 0x7fcefc006d40 2024-04-25T15:49:27.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.688+0000 7fcf03fff700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 0x7fcefc0ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:49:27.690 INFO:tasks.workunit.client.0.smithi110.stdout:4/494: mkdir d13/d24/d2d/d7d/da2 0 2024-04-25T15:49:27.690 INFO:tasks.workunit.client.0.smithi110.stdout:2/403: dread de/d40/d4f/f4e [0,4194304] 0 2024-04-25T15:49:27.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.689+0000 7fcf08c7a700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcefc006d40 0x7fcefc0abb10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcef800eb40 tx=0x7fcef800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:27.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.689+0000 7fcf017fa700 1 -- 172.21.15.110:0/1873777944 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcef8007cc0 con 0x7fcefc006d40 2024-04-25T15:49:27.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.689+0000 7fcf017fa700 1 -- 172.21.15.110:0/1873777944 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcef800f040 con 0x7fcefc006d40 2024-04-25T15:49:27.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.689+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcefc0b1170 con 0x7fcefc006d40 2024-04-25T15:49:27.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.689+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcefc0b1730 con 0x7fcefc006d40 2024-04-25T15:49:27.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.689+0000 7fcf017fa700 1 -- 172.21.15.110:0/1873777944 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcef800cb20 con 0x7fcefc006d40 2024-04-25T15:49:27.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.692+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcee80052e0 con 0x7fcefc006d40 2024-04-25T15:49:27.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.692+0000 7fcf017fa700 1 -- 172.21.15.110:0/1873777944 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7fcef800cc80 con 0x7fcefc006d40 2024-04-25T15:49:27.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.693+0000 7fcf017fa700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fcef406fcf0 0x7fcef40721a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:27.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.693+0000 7fcf03fff700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fcef406fcf0 0x7fcef40721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:27.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.693+0000 7fcf017fa700 1 -- 172.21.15.110:0/1873777944 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fcef80584e0 con 0x7fcefc006d40 2024-04-25T15:49:27.694 INFO:tasks.workunit.client.0.smithi110.stdout:6/365: dwrite d1/db/d24/f29 [0,4194304] 0 2024-04-25T15:49:27.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.694+0000 7fcf03fff700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fcef406fcf0 0x7fcef40721a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcef00137c0 tx=0x7fcef0009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:27.696 INFO:tasks.workunit.client.0.smithi110.stdout:9/319: creat d9/db/df/d55/f60 x:0 0 0 2024-04-25T15:49:27.697 INFO:tasks.workunit.client.0.smithi110.stdout:9/320: chown d9/db/f11 2950 1 2024-04-25T15:49:27.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:27.701+0000 7fcf017fa700 1 -- 172.21.15.110:0/1873777944 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fcef8094060 con 0x7fcefc006d40 2024-04-25T15:49:27.703 INFO:tasks.workunit.client.0.smithi110.stdout:4/495: dread fd [0,4194304] 0 2024-04-25T15:49:27.703 INFO:tasks.workunit.client.0.smithi110.stdout:3/488: rmdir d0/d7/d1c/d2f/d33 39 2024-04-25T15:49:27.707 INFO:tasks.workunit.client.0.smithi110.stdout:5/371: mkdir d4/d1d/d1f/d35/d7f 0 2024-04-25T15:49:27.713 INFO:tasks.workunit.client.0.smithi110.stdout:2/404: rename de/d40/d5b/d5c/d5f/c6c to de/d40/d6e/d52/c79 0 2024-04-25T15:49:27.756 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:27 smithi193 ceph-mon[33747]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:49:27.756 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:27 smithi193 ceph-mon[33747]: pgmap v30: 65 pgs: 65 active+clean; 859 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 17 MiB/s rd, 105 MiB/s wr, 285 op/s 2024-04-25T15:49:27.756 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:27 smithi193 ceph-mon[33747]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:49:27.758 INFO:tasks.workunit.client.0.smithi110.stdout:8/505: mkdir d14/d21/d26/dab 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:8/506: chown d14/d32/d36/l5f 501 1 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:7/391: symlink d1b/d29/l7e 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:3/489: symlink d0/d16/la9 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:8/507: dread d14/d21/f67 [4194304,4194304] 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:5/372: mknod d4/d15/d51/c80 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:3/490: dread d0/d7/d1c/d50/f6d [0,4194304] 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:2/405: creat de/d40/d5b/d5c/d5f/f7a x:0 0 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:6/366: symlink d1/db/d24/d2d/l87 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:9/321: mkdir d9/d13/d5b/d61 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:6/367: dread d1/dd/d3c/f7d [0,4194304] 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:6/368: write d1/dd/d3c/f7d [435343,87607] 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:7/392: rename d1b/d29/d42 to d1b/d29/d38/d55/d7f 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:1/507: link d0/d25/l46 d0/d14/d5f/d7d/d72/d61/l9a 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:9/322: dread d9/d13/f27 [0,4194304] 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:9/323: write d9/d13/f27 [383851,129335] 0 2024-04-25T15:49:27.759 INFO:tasks.workunit.client.0.smithi110.stdout:8/508: fdatasync ff 0 2024-04-25T15:49:27.760 INFO:tasks.workunit.client.0.smithi110.stdout:1/508: dwrite d0/d14/d5f/d7d/d72/d61/f8b [0,4194304] 0 2024-04-25T15:49:27.760 INFO:tasks.workunit.client.0.smithi110.stdout:1/509: write d0/d25/d33/f48 [912105,112152] 0 2024-04-25T15:49:27.760 INFO:tasks.workunit.client.0.smithi110.stdout:3/491: mknod d0/d7/caa 0 2024-04-25T15:49:27.760 INFO:tasks.workunit.client.0.smithi110.stdout:2/406: creat de/d40/d6e/d50/f7b x:0 0 0 2024-04-25T15:49:27.760 INFO:tasks.workunit.client.0.smithi110.stdout:6/369: write d1/db/d22/f38 [4880272,66423] 0 2024-04-25T15:49:27.762 INFO:tasks.workunit.client.1.smithi193.stdout:1/33: sync 2024-04-25T15:49:27.763 INFO:tasks.workunit.client.0.smithi110.stdout:5/373: rename d4/d2c/l12 to d4/d2c/d2a/l81 0 2024-04-25T15:49:27.765 INFO:tasks.workunit.client.0.smithi110.stdout:5/374: read d4/f44 [2963328,14651] 0 2024-04-25T15:49:27.765 INFO:tasks.workunit.client.0.smithi110.stdout:7/393: creat d1b/d29/d46/f80 x:0 0 0 2024-04-25T15:49:27.766 INFO:tasks.workunit.client.0.smithi110.stdout:5/375: dread - d4/f7c zero size 2024-04-25T15:49:27.767 INFO:tasks.workunit.client.0.smithi110.stdout:6/370: dwrite d1/db/d24/f66 [0,4194304] 0 2024-04-25T15:49:27.767 INFO:tasks.workunit.client.0.smithi110.stdout:4/496: creat d13/fa3 x:0 0 0 2024-04-25T15:49:27.769 INFO:tasks.workunit.client.0.smithi110.stdout:6/371: fdatasync d1/db/d22/d33/d43/f60 0 2024-04-25T15:49:27.770 INFO:tasks.workunit.client.0.smithi110.stdout:9/324: unlink d9/db/c4b 0 2024-04-25T15:49:27.777 INFO:tasks.workunit.client.0.smithi110.stdout:1/510: rmdir d0/d14 39 2024-04-25T15:49:27.781 INFO:tasks.workunit.client.0.smithi110.stdout:5/376: dread d4/d1d/d1f/f32 [0,4194304] 0 2024-04-25T15:49:27.789 INFO:tasks.workunit.client.0.smithi110.stdout:4/497: unlink d13/d24/d2d/d44/d65/f84 0 2024-04-25T15:49:27.802 INFO:tasks.workunit.client.0.smithi110.stdout:5/377: readlink d4/d2c/l28 0 2024-04-25T15:49:27.802 INFO:tasks.workunit.client.1.smithi193.stdout:6/74: sync 2024-04-25T15:49:27.803 INFO:tasks.workunit.client.0.smithi110.stdout:5/378: chown d4/d1d/d1f/l41 5351560 1 2024-04-25T15:49:27.804 INFO:tasks.workunit.client.0.smithi110.stdout:4/498: dread d13/d24/d3c/d56/f66 [0,4194304] 0 2024-04-25T15:49:27.805 INFO:tasks.workunit.client.1.smithi193.stdout:6/75: dread fb [0,4194304] 0 2024-04-25T15:49:27.809 INFO:tasks.workunit.client.0.smithi110.stdout:9/325: symlink d9/l62 0 2024-04-25T15:49:27.815 INFO:tasks.workunit.client.0.smithi110.stdout:2/407: dwrite de/d40/f3e [0,4194304] 0 2024-04-25T15:49:27.817 INFO:tasks.workunit.client.0.smithi110.stdout:1/511: rename d0/d25/c2a to d0/d14/d5f/d7d/d72/c9b 0 2024-04-25T15:49:27.820 INFO:tasks.workunit.client.0.smithi110.stdout:5/379: fsync d4/d2c/f27 0 2024-04-25T15:49:27.832 INFO:tasks.workunit.client.0.smithi110.stdout:8/509: dwrite d14/d21/d26/f29 [0,4194304] 0 2024-04-25T15:49:27.835 INFO:tasks.workunit.client.0.smithi110.stdout:3/492: getdents d0/d7 0 2024-04-25T15:49:27.837 INFO:tasks.workunit.client.0.smithi110.stdout:3/493: truncate d0/d16/f3d 993927 0 2024-04-25T15:49:27.839 INFO:tasks.workunit.client.0.smithi110.stdout:8/510: dwrite d14/f1b [4194304,4194304] 0 2024-04-25T15:49:27.844 INFO:tasks.workunit.client.0.smithi110.stdout:1/512: mkdir d0/d14/d5f/d7d/d9c 0 2024-04-25T15:49:27.846 INFO:tasks.workunit.client.0.smithi110.stdout:5/380: creat d4/d1d/d1f/d35/d3d/d52/f82 x:0 0 0 2024-04-25T15:49:27.847 INFO:tasks.workunit.client.0.smithi110.stdout:5/381: write d4/d1d/d1f/f42 [305891,8296] 0 2024-04-25T15:49:27.849 INFO:tasks.workunit.client.0.smithi110.stdout:4/499: unlink d13/d24/d2d/d5e/l83 0 2024-04-25T15:49:27.851 INFO:tasks.workunit.client.0.smithi110.stdout:7/394: truncate d1b/d29/f60 644827 0 2024-04-25T15:49:27.854 INFO:tasks.workunit.client.0.smithi110.stdout:5/382: dwrite d4/f6d [0,4194304] 0 2024-04-25T15:49:27.862 INFO:tasks.workunit.client.0.smithi110.stdout:5/383: dwrite d4/d1d/d1f/d35/d3d/f53 [0,4194304] 0 2024-04-25T15:49:27.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:27 smithi110 ceph-mon[26344]: from='client.14702 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:49:27.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:27 smithi110 ceph-mon[26344]: pgmap v30: 65 pgs: 65 active+clean; 859 MiB data, 4.0 GiB used, 532 GiB / 536 GiB avail; 17 MiB/s rd, 105 MiB/s wr, 285 op/s 2024-04-25T15:49:27.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:27 smithi110 ceph-mon[26344]: from='client.24435 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:49:27.863 INFO:tasks.workunit.client.0.smithi110.stdout:5/384: dread - d4/d1d/f70 zero size 2024-04-25T15:49:27.865 INFO:tasks.workunit.client.0.smithi110.stdout:8/511: dread d14/d21/f27 [0,4194304] 0 2024-04-25T15:49:27.866 INFO:tasks.workunit.client.0.smithi110.stdout:2/408: rename de/d40/d5b/d5c/l61 to de/d40/d6e/d52/d6d/l7c 0 2024-04-25T15:49:27.867 INFO:tasks.workunit.client.0.smithi110.stdout:8/512: fdatasync d14/d21/d26/d6a/d7e/fa3 0 2024-04-25T15:49:27.869 INFO:tasks.workunit.client.0.smithi110.stdout:1/513: stat d0/d14/l3c 0 2024-04-25T15:49:27.870 INFO:tasks.workunit.client.0.smithi110.stdout:4/500: fsync d13/d24/d2d/d36/f5a 0 2024-04-25T15:49:27.875 INFO:tasks.workunit.client.0.smithi110.stdout:7/395: write d1b/d29/d38/f5b [3570659,670] 0 2024-04-25T15:49:27.876 INFO:tasks.workunit.client.0.smithi110.stdout:6/372: link d1/db/d22/d27/f72 d1/db/f88 0 2024-04-25T15:49:27.877 INFO:tasks.workunit.client.0.smithi110.stdout:9/326: creat d9/d13/d5b/d61/f63 x:0 0 0 2024-04-25T15:49:27.878 INFO:tasks.workunit.client.0.smithi110.stdout:1/514: dwrite d0/d51/f52 [4194304,4194304] 0 2024-04-25T15:49:27.883 INFO:tasks.workunit.client.0.smithi110.stdout:5/385: chown d4/d1d/d1f/d35/c36 6452 1 2024-04-25T15:49:27.888 INFO:tasks.workunit.client.0.smithi110.stdout:0/406: sync 2024-04-25T15:49:27.891 INFO:tasks.workunit.client.0.smithi110.stdout:4/501: dread d13/d24/d3c/d56/d88/f4c [0,4194304] 0 2024-04-25T15:49:27.897 INFO:tasks.workunit.client.0.smithi110.stdout:7/396: chown d1b/d29/c2a 26923997 1 2024-04-25T15:49:27.899 INFO:tasks.workunit.client.0.smithi110.stdout:6/373: chown d1/db/d24/d2d/c7e 11 1 2024-04-25T15:49:27.903 INFO:tasks.workunit.client.0.smithi110.stdout:1/515: creat d0/d14/d81/f9d x:0 0 0 2024-04-25T15:49:27.904 INFO:tasks.workunit.client.0.smithi110.stdout:1/516: stat d0/f27 0 2024-04-25T15:49:27.906 INFO:tasks.workunit.client.0.smithi110.stdout:6/374: dwrite d1/dd/f21 [4194304,4194304] 0 2024-04-25T15:49:27.912 INFO:tasks.workunit.client.0.smithi110.stdout:2/409: dread f8 [0,4194304] 0 2024-04-25T15:49:27.913 INFO:tasks.workunit.client.0.smithi110.stdout:4/502: mkdir d13/d24/d2d/d44/da4 0 2024-04-25T15:49:27.913 INFO:tasks.workunit.client.0.smithi110.stdout:2/410: fdatasync de/d40/f35 0 2024-04-25T15:49:27.915 INFO:tasks.workunit.client.0.smithi110.stdout:8/513: mkdir d14/d21/d26/dab/dac 0 2024-04-25T15:49:27.921 INFO:tasks.workunit.client.0.smithi110.stdout:5/386: mkdir d4/d2c/d83 0 2024-04-25T15:49:27.928 INFO:tasks.workunit.client.0.smithi110.stdout:2/411: dread de/d40/d4f/f47 [0,4194304] 0 2024-04-25T15:49:27.929 INFO:tasks.workunit.client.0.smithi110.stdout:2/412: chown de/d40/d4f/c6a 560186 1 2024-04-25T15:49:27.936 INFO:tasks.workunit.client.0.smithi110.stdout:8/514: creat d14/d4c/d55/d8d/fad x:0 0 0 2024-04-25T15:49:27.939 INFO:tasks.workunit.client.0.smithi110.stdout:6/375: dread d1/db/f88 [0,4194304] 0 2024-04-25T15:49:27.940 INFO:tasks.workunit.client.0.smithi110.stdout:9/327: symlink d9/d44/d5e/l64 0 2024-04-25T15:49:27.943 INFO:tasks.workunit.client.0.smithi110.stdout:1/517: symlink d0/d25/d33/d94/l9e 0 2024-04-25T15:49:27.949 INFO:tasks.workunit.client.0.smithi110.stdout:0/407: write d1/d23/f44 [859264,70154] 0 2024-04-25T15:49:27.951 INFO:tasks.workunit.client.0.smithi110.stdout:3/494: rename d0/d7/d40/l7f to d0/lab 0 2024-04-25T15:49:27.954 INFO:tasks.workunit.client.0.smithi110.stdout:4/503: unlink d13/d24/d2d/c91 0 2024-04-25T15:49:27.957 INFO:tasks.workunit.client.0.smithi110.stdout:8/515: mkdir d14/d4c/d71/dae 0 2024-04-25T15:49:27.958 INFO:tasks.workunit.client.0.smithi110.stdout:8/516: truncate d14/d21/d26/f7a 150615 0 2024-04-25T15:49:27.959 INFO:tasks.workunit.client.0.smithi110.stdout:8/517: fsync f5 0 2024-04-25T15:49:27.959 INFO:tasks.workunit.client.0.smithi110.stdout:6/376: chown d1/c28 178484 1 2024-04-25T15:49:27.960 INFO:tasks.workunit.client.0.smithi110.stdout:7/397: truncate d1b/f31 412438 0 2024-04-25T15:49:27.962 INFO:tasks.workunit.client.0.smithi110.stdout:9/328: rename d9/db/d58/d5a to d9/db/d58/d5a/d65 22 2024-04-25T15:49:27.963 INFO:tasks.workunit.client.0.smithi110.stdout:1/518: symlink d0/d14/d5f/d7d/d72/l9f 0 2024-04-25T15:49:27.965 INFO:tasks.workunit.client.0.smithi110.stdout:5/387: symlink d4/l84 0 2024-04-25T15:49:27.967 INFO:tasks.workunit.client.0.smithi110.stdout:6/377: dwrite d1/db/f62 [0,4194304] 0 2024-04-25T15:49:27.970 INFO:tasks.workunit.client.0.smithi110.stdout:3/495: creat d0/d7/d1c/d2f/d4c/fac x:0 0 0 2024-04-25T15:49:27.971 INFO:tasks.workunit.client.0.smithi110.stdout:4/504: mknod d13/d24/d2d/d36/ca5 0 2024-04-25T15:49:27.976 INFO:tasks.workunit.client.0.smithi110.stdout:8/518: mknod d14/d32/d36/d48/caf 0 2024-04-25T15:49:27.986 INFO:tasks.workunit.client.0.smithi110.stdout:5/388: readlink d4/d2c/d2a/l81 0 2024-04-25T15:49:27.993 INFO:tasks.workunit.client.0.smithi110.stdout:0/408: mkdir d1/d23/d9e 0 2024-04-25T15:49:27.998 INFO:tasks.workunit.client.0.smithi110.stdout:8/519: creat d14/d4c/d71/d9d/fb0 x:0 0 0 2024-04-25T15:49:27.999 INFO:tasks.workunit.client.0.smithi110.stdout:2/413: dwrite de/d40/d4f/f4c [0,4194304] 0 2024-04-25T15:49:27.999 INFO:tasks.workunit.client.0.smithi110.stdout:8/520: readlink d14/d21/l9b 0 2024-04-25T15:49:28.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/521: stat ld 0 2024-04-25T15:49:28.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/522: stat d14/d21/d26/f7a 0 2024-04-25T15:49:28.000 INFO:tasks.workunit.client.0.smithi110.stdout:7/398: symlink d1b/d29/l81 0 2024-04-25T15:49:28.006 INFO:tasks.workunit.client.0.smithi110.stdout:1/519: creat d0/d14/d5f/d7d/d9c/fa0 x:0 0 0 2024-04-25T15:49:28.009 INFO:tasks.workunit.client.0.smithi110.stdout:8/523: dwrite d14/d32/d36/f84 [8388608,4194304] 0 2024-04-25T15:49:28.010 INFO:tasks.workunit.client.0.smithi110.stdout:5/389: mkdir d4/d1d/d85 0 2024-04-25T15:49:28.022 INFO:tasks.workunit.client.0.smithi110.stdout:3/496: mkdir d0/d16/d8c/d90/d99/dad 0 2024-04-25T15:49:28.024 INFO:tasks.workunit.client.0.smithi110.stdout:4/505: mkdir d13/d24/da6 0 2024-04-25T15:49:28.025 INFO:tasks.workunit.client.0.smithi110.stdout:4/506: fdatasync d13/d24/d2d/d5e/f7b 0 2024-04-25T15:49:28.037 INFO:tasks.workunit.client.0.smithi110.stdout:8/524: fsync d14/d21/f67 0 2024-04-25T15:49:28.038 INFO:tasks.workunit.client.0.smithi110.stdout:3/497: dread d0/d2e/f76 [0,4194304] 0 2024-04-25T15:49:28.039 INFO:tasks.workunit.client.0.smithi110.stdout:9/329: truncate d9/db/df/d2c/d3c/f49 191790 0 2024-04-25T15:49:28.040 INFO:tasks.workunit.client.0.smithi110.stdout:5/390: symlink d4/d1d/l86 0 2024-04-25T15:49:28.041 INFO:tasks.workunit.client.0.smithi110.stdout:5/391: write d4/d1d/d1f/d35/d3d/d52/f82 [991638,84596] 0 2024-04-25T15:49:28.043 INFO:tasks.workunit.client.0.smithi110.stdout:6/378: rename d1/dd/d3c/f7d to d1/db/d24/d2d/f89 0 2024-04-25T15:49:28.047 INFO:tasks.workunit.client.0.smithi110.stdout:0/409: symlink d1/d23/d30/d33/d5b/d72/l9f 0 2024-04-25T15:49:28.057 INFO:tasks.workunit.client.0.smithi110.stdout:1/520: write d0/d23/f30 [809709,15070] 0 2024-04-25T15:49:28.066 INFO:tasks.workunit.client.0.smithi110.stdout:8/525: creat d14/d21/d26/dab/fb1 x:0 0 0 2024-04-25T15:49:28.070 INFO:tasks.workunit.client.1.smithi193.stdout:7/27: creat f8 x:0 0 0 2024-04-25T15:49:28.071 INFO:tasks.workunit.client.0.smithi110.stdout:9/330: readlink d9/l28 0 2024-04-25T15:49:28.072 INFO:tasks.workunit.client.0.smithi110.stdout:9/331: dread - d9/d13/d5b/d61/f63 zero size 2024-04-25T15:49:28.072 INFO:tasks.workunit.client.0.smithi110.stdout:9/332: stat d9/db/df/l23 0 2024-04-25T15:49:28.075 INFO:tasks.workunit.client.0.smithi110.stdout:3/498: dread d0/f1e [0,4194304] 0 2024-04-25T15:49:28.075 INFO:tasks.workunit.client.1.smithi193.stdout:7/28: dwrite f3 [0,4194304] 0 2024-04-25T15:49:28.076 INFO:tasks.workunit.client.1.smithi193.stdout:7/29: dread - f8 zero size 2024-04-25T15:49:28.076 INFO:tasks.workunit.client.0.smithi110.stdout:6/379: symlink d1/d12/l8a 0 2024-04-25T15:49:28.077 INFO:tasks.workunit.client.1.smithi193.stdout:7/30: write f2 [3107346,113817] 0 2024-04-25T15:49:28.078 INFO:tasks.workunit.client.0.smithi110.stdout:6/380: dread - d1/d12/f5b zero size 2024-04-25T15:49:28.081 INFO:tasks.workunit.client.1.smithi193.stdout:9/75: write f4 [1689487,37285] 0 2024-04-25T15:49:28.081 INFO:tasks.workunit.client.1.smithi193.stdout:9/76: chown f4 377649300 1 2024-04-25T15:49:28.083 INFO:tasks.workunit.client.0.smithi110.stdout:4/507: symlink d13/d24/d2d/d44/da4/la7 0 2024-04-25T15:49:28.084 INFO:tasks.workunit.client.0.smithi110.stdout:4/508: write fa [1324712,109768] 0 2024-04-25T15:49:28.085 INFO:tasks.workunit.client.1.smithi193.stdout:2/82: symlink d0/d9/d15/l21 0 2024-04-25T15:49:28.086 INFO:tasks.workunit.client.0.smithi110.stdout:4/509: write d13/d24/d2d/d5e/f71 [635443,69695] 0 2024-04-25T15:49:28.086 INFO:tasks.workunit.client.0.smithi110.stdout:1/521: chown d0/d14/d5f/d7d/d72/c85 211251590 1 2024-04-25T15:49:28.090 INFO:tasks.workunit.client.1.smithi193.stdout:1/34: unlink c5 0 2024-04-25T15:49:28.091 INFO:tasks.workunit.client.1.smithi193.stdout:1/35: fsync f6 0 2024-04-25T15:49:28.091 INFO:tasks.workunit.client.0.smithi110.stdout:8/526: unlink d14/d32/d36/d48/c9f 0 2024-04-25T15:49:28.093 INFO:tasks.workunit.client.1.smithi193.stdout:6/76: mkdir dd/d12 0 2024-04-25T15:49:28.093 INFO:tasks.workunit.client.0.smithi110.stdout:1/522: read d0/d51/f79 [633519,118351] 0 2024-04-25T15:49:28.094 INFO:tasks.workunit.client.0.smithi110.stdout:9/333: rmdir d9/db 39 2024-04-25T15:49:28.097 INFO:tasks.workunit.client.0.smithi110.stdout:5/392: link d4/d1d/f70 d4/d2c/d2a/f87 0 2024-04-25T15:49:28.099 INFO:tasks.workunit.client.0.smithi110.stdout:3/499: mknod d0/d16/cae 0 2024-04-25T15:49:28.100 INFO:tasks.workunit.client.1.smithi193.stdout:4/55: mknod db/cd 0 2024-04-25T15:49:28.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.100+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcee8005860 con 0x7fcefc006d40 2024-04-25T15:49:28.101 INFO:tasks.workunit.client.0.smithi110.stdout:6/381: creat d1/db/d24/d32/f8b x:0 0 0 2024-04-25T15:49:28.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.100+0000 7fcf017fa700 1 -- 172.21.15.110:0/1873777944 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7fcef8057ec0 con 0x7fcefc006d40 2024-04-25T15:49:28.101 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:49:28.101 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:49:28.101 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:49:28.101 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:49:28.101 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:49:28.101 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:49:28.102 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:49:28.103 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:49:28.104 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:49:28.104 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:49:28.104 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:49:28.104 INFO:tasks.workunit.client.0.smithi110.stdout:0/410: symlink d1/d23/d30/d33/d5b/d72/la0 0 2024-04-25T15:49:28.104 INFO:tasks.workunit.client.0.smithi110.stdout:0/411: write d1/f19 [1081691,23377] 0 2024-04-25T15:49:28.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.103+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fcef406fcf0 msgr2=0x7fcef40721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:28.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.103+0000 7fcf09c7c700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fcef406fcf0 0x7fcef40721a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcef00137c0 tx=0x7fcef0009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:28.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.103+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcefc006d40 msgr2=0x7fcefc0abb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:28.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.103+0000 7fcf09c7c700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcefc006d40 0x7fcefc0abb10 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcef800eb40 tx=0x7fcef800b040 comp rx=0 tx=0).stop 2024-04-25T15:49:28.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.104+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 shutdown_connections 2024-04-25T15:49:28.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.104+0000 7fcf09c7c700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fcef406fcf0 0x7fcef40721a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.104+0000 7fcf09c7c700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcefc006d40 0x7fcefc0abb10 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.104+0000 7fcf09c7c700 1 --2- 172.21.15.110:0/1873777944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcefc095b00 0x7fcefc0ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.104+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 >> 172.21.15.110:0/1873777944 conn(0x7fcefc08f240 msgr2=0x7fcefc08f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:28.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.104+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 shutdown_connections 2024-04-25T15:49:28.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.104+0000 7fcf09c7c700 1 -- 172.21.15.110:0/1873777944 wait complete. 2024-04-25T15:49:28.106 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:49:28.106 INFO:tasks.workunit.client.0.smithi110.stdout:2/414: getdents de/d40/d6e/d52/d6d 0 2024-04-25T15:49:28.108 INFO:tasks.workunit.client.1.smithi193.stdout:7/31: creat f9 x:0 0 0 2024-04-25T15:49:28.108 INFO:tasks.workunit.client.0.smithi110.stdout:4/510: symlink d13/d24/d3c/la8 0 2024-04-25T15:49:28.113 INFO:tasks.workunit.client.1.smithi193.stdout:9/77: unlink l1 0 2024-04-25T15:49:28.114 INFO:tasks.workunit.client.1.smithi193.stdout:7/32: dwrite f9 [0,4194304] 0 2024-04-25T15:49:28.114 INFO:tasks.workunit.client.1.smithi193.stdout:9/78: write f4 [3670810,62281] 0 2024-04-25T15:49:28.115 INFO:tasks.workunit.client.0.smithi110.stdout:7/399: getdents d1b/d29/d46 0 2024-04-25T15:49:28.115 INFO:tasks.workunit.client.1.smithi193.stdout:7/33: fdatasync f7 0 2024-04-25T15:49:28.117 INFO:tasks.workunit.client.1.smithi193.stdout:2/83: rename d0/f7 to d0/d9/f22 0 2024-04-25T15:49:28.122 INFO:tasks.workunit.client.1.smithi193.stdout:7/34: dwrite f8 [0,4194304] 0 2024-04-25T15:49:28.123 INFO:tasks.workunit.client.1.smithi193.stdout:2/84: dread d0/df/f1c [0,4194304] 0 2024-04-25T15:49:28.123 INFO:tasks.workunit.client.1.smithi193.stdout:5/23: rename f5 to f8 0 2024-04-25T15:49:28.125 INFO:tasks.workunit.client.0.smithi110.stdout:1/523: dwrite d0/d23/f5a [0,4194304] 0 2024-04-25T15:49:28.127 INFO:tasks.workunit.client.0.smithi110.stdout:5/393: mknod d4/d1d/d1f/d56/d6b/c88 0 2024-04-25T15:49:28.129 INFO:tasks.workunit.client.0.smithi110.stdout:5/394: write d4/d2c/d2a/f46 [3387169,12517] 0 2024-04-25T15:49:28.130 INFO:tasks.workunit.client.1.smithi193.stdout:8/60: truncate d2/fa 661254 0 2024-04-25T15:49:28.130 INFO:tasks.workunit.client.1.smithi193.stdout:7/35: dwrite f8 [0,4194304] 0 2024-04-25T15:49:28.132 INFO:tasks.workunit.client.0.smithi110.stdout:5/395: truncate d4/d1d/d1f/d56/f7e 636084 0 2024-04-25T15:49:28.133 INFO:tasks.workunit.client.0.smithi110.stdout:3/500: read d0/d7/d40/f43 [261247,13762] 0 2024-04-25T15:49:28.133 INFO:tasks.workunit.client.1.smithi193.stdout:3/101: getdents d8/d13/d1f 0 2024-04-25T15:49:28.134 INFO:tasks.workunit.client.0.smithi110.stdout:5/396: write d4/f7c [518508,104932] 0 2024-04-25T15:49:28.135 INFO:tasks.workunit.client.0.smithi110.stdout:5/397: stat d4/d1d/d1f/d35/d3d/d52/f7a 0 2024-04-25T15:49:28.137 INFO:tasks.workunit.client.1.smithi193.stdout:7/36: dwrite f9 [0,4194304] 0 2024-04-25T15:49:28.140 INFO:tasks.workunit.client.1.smithi193.stdout:0/79: link d0/d3/d5/l11 d0/d3/d5/d7/dd/l19 0 2024-04-25T15:49:28.142 INFO:tasks.workunit.client.0.smithi110.stdout:0/412: mkdir d1/d23/d30/d33/d35/da1 0 2024-04-25T15:49:28.144 INFO:tasks.workunit.client.0.smithi110.stdout:2/415: mkdir de/d40/d6e/d52/d6d/d7d 0 2024-04-25T15:49:28.144 INFO:tasks.workunit.client.0.smithi110.stdout:0/413: fsync d1/d23/f44 0 2024-04-25T15:49:28.149 INFO:tasks.workunit.client.0.smithi110.stdout:4/511: rmdir d13/d24/d2d/d44 39 2024-04-25T15:49:28.150 INFO:tasks.workunit.client.0.smithi110.stdout:9/334: dread d9/db/df/f33 [0,4194304] 0 2024-04-25T15:49:28.152 INFO:tasks.workunit.client.0.smithi110.stdout:1/524: mknod d0/d25/d33/d94/ca1 0 2024-04-25T15:49:28.156 INFO:tasks.workunit.client.1.smithi193.stdout:2/85: chown d0/fd 8951 1 2024-04-25T15:49:28.158 INFO:tasks.workunit.client.1.smithi193.stdout:5/24: write f8 [949718,16932] 0 2024-04-25T15:49:28.158 INFO:tasks.workunit.client.1.smithi193.stdout:5/25: stat c7 0 2024-04-25T15:49:28.158 INFO:tasks.workunit.client.1.smithi193.stdout:5/26: readlink l6 0 2024-04-25T15:49:28.159 INFO:tasks.workunit.client.0.smithi110.stdout:5/398: creat d4/d1d/d1f/d56/d6b/f89 x:0 0 0 2024-04-25T15:49:28.161 INFO:tasks.workunit.client.1.smithi193.stdout:6/77: link dd/ff dd/f13 0 2024-04-25T15:49:28.162 INFO:tasks.workunit.client.0.smithi110.stdout:5/399: dread d4/d1d/d1f/f32 [0,4194304] 0 2024-04-25T15:49:28.163 INFO:tasks.workunit.client.1.smithi193.stdout:5/27: dread f4 [0,4194304] 0 2024-04-25T15:49:28.163 INFO:tasks.workunit.client.0.smithi110.stdout:5/400: fdatasync d4/d1d/f38 0 2024-04-25T15:49:28.168 INFO:tasks.workunit.client.0.smithi110.stdout:0/414: mknod d1/d23/d30/d33/d5b/ca2 0 2024-04-25T15:49:28.169 INFO:tasks.workunit.client.0.smithi110.stdout:5/401: dwrite d4/f7c [0,4194304] 0 2024-04-25T15:49:28.171 INFO:tasks.workunit.client.0.smithi110.stdout:5/402: chown d4/d1d/d1f/d56/d6b/c88 476344181 1 2024-04-25T15:49:28.175 INFO:tasks.workunit.client.1.smithi193.stdout:7/37: symlink la 0 2024-04-25T15:49:28.176 INFO:tasks.workunit.client.1.smithi193.stdout:7/38: write f9 [224544,42861] 0 2024-04-25T15:49:28.178 INFO:tasks.workunit.client.0.smithi110.stdout:8/527: creat d14/fb2 x:0 0 0 2024-04-25T15:49:28.180 INFO:tasks.workunit.client.1.smithi193.stdout:0/80: mkdir d0/d3/d5/d7/dd/d16/d1a 0 2024-04-25T15:49:28.185 INFO:tasks.workunit.client.0.smithi110.stdout:9/335: symlink d9/d13/d5b/d61/l66 0 2024-04-25T15:49:28.187 INFO:tasks.workunit.client.0.smithi110.stdout:1/525: mknod d0/d51/d6b/ca2 0 2024-04-25T15:49:28.190 INFO:tasks.workunit.client.0.smithi110.stdout:3/501: truncate d0/d7/d40/f61 90539 0 2024-04-25T15:49:28.192 INFO:tasks.workunit.client.0.smithi110.stdout:6/382: creat d1/db/f8c x:0 0 0 2024-04-25T15:49:28.193 INFO:tasks.workunit.client.1.smithi193.stdout:2/86: creat d0/df/d16/f23 x:0 0 0 2024-04-25T15:49:28.193 INFO:tasks.workunit.client.0.smithi110.stdout:8/528: dread d14/f80 [0,4194304] 0 2024-04-25T15:49:28.195 INFO:tasks.workunit.client.0.smithi110.stdout:8/529: dread d14/d21/d26/f7a [0,4194304] 0 2024-04-25T15:49:28.198 INFO:tasks.workunit.client.0.smithi110.stdout:2/416: rename de/d40/d6e/l1e to de/d40/d6e/d52/l7e 0 2024-04-25T15:49:28.199 INFO:tasks.workunit.client.1.smithi193.stdout:6/78: symlink dd/l14 0 2024-04-25T15:49:28.200 INFO:tasks.workunit.client.0.smithi110.stdout:2/417: chown de/d40/d4f/c58 3308 1 2024-04-25T15:49:28.200 INFO:tasks.workunit.client.1.smithi193.stdout:6/79: truncate dd/f11 4331992 0 2024-04-25T15:49:28.201 INFO:tasks.workunit.client.0.smithi110.stdout:2/418: write de/d40/d6e/d52/f62 [3975197,94038] 0 2024-04-25T15:49:28.202 INFO:tasks.workunit.client.0.smithi110.stdout:0/415: fdatasync d1/d23/f24 0 2024-04-25T15:49:28.202 INFO:tasks.workunit.client.0.smithi110.stdout:7/400: link d1b/d29/d46/c7b d1b/d29/d38/d55/d7d/c82 0 2024-04-25T15:49:28.203 INFO:tasks.workunit.client.0.smithi110.stdout:0/416: fdatasync d1/d47/d54/f6d 0 2024-04-25T15:49:28.204 INFO:tasks.workunit.client.0.smithi110.stdout:2/419: chown de/c48 1673654048 1 2024-04-25T15:49:28.207 INFO:tasks.workunit.client.0.smithi110.stdout:9/336: mkdir d9/d13/d5b/d61/d67 0 2024-04-25T15:49:28.208 INFO:tasks.workunit.client.1.smithi193.stdout:3/102: truncate d8/f15 1135148 0 2024-04-25T15:49:28.208 INFO:tasks.workunit.client.1.smithi193.stdout:3/103: write d8/d13/d14/f21 [501623,100934] 0 2024-04-25T15:49:28.209 INFO:tasks.workunit.client.0.smithi110.stdout:6/383: truncate d1/db/f19 3780311 0 2024-04-25T15:49:28.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/39: mknod cb 0 2024-04-25T15:49:28.212 INFO:tasks.workunit.client.0.smithi110.stdout:8/530: truncate d14/d21/f67 7047137 0 2024-04-25T15:49:28.213 INFO:tasks.workunit.client.1.smithi193.stdout:9/79: truncate f9 3418706 0 2024-04-25T15:49:28.213 INFO:tasks.workunit.client.0.smithi110.stdout:8/531: dread - d14/d4c/d71/d9d/fb0 zero size 2024-04-25T15:49:28.214 INFO:tasks.workunit.client.1.smithi193.stdout:2/87: mkdir d0/df/d24 0 2024-04-25T15:49:28.215 INFO:tasks.workunit.client.0.smithi110.stdout:8/532: dread d14/d4c/d55/d8d/f95 [0,4194304] 0 2024-04-25T15:49:28.217 INFO:tasks.workunit.client.0.smithi110.stdout:5/403: rename d4/d1d/d1f/d56/f67 to d4/d1d/d1f/f8a 0 2024-04-25T15:49:28.223 INFO:tasks.workunit.client.1.smithi193.stdout:1/36: getdents . 0 2024-04-25T15:49:28.223 INFO:tasks.workunit.client.0.smithi110.stdout:5/404: dwrite d4/d1d/d1f/f64 [0,4194304] 0 2024-04-25T15:49:28.227 INFO:tasks.workunit.client.1.smithi193.stdout:6/80: rename dd/ff to dd/f15 0 2024-04-25T15:49:28.233 INFO:tasks.workunit.client.0.smithi110.stdout:4/512: rmdir d13/d24/d26/d7f 0 2024-04-25T15:49:28.234 INFO:tasks.workunit.client.1.smithi193.stdout:3/104: mkdir d8/d13/d14/d23 0 2024-04-25T15:49:28.236 INFO:tasks.workunit.client.1.smithi193.stdout:7/40: creat fc x:0 0 0 2024-04-25T15:49:28.237 INFO:tasks.workunit.client.0.smithi110.stdout:9/337: creat d9/db/df/f68 x:0 0 0 2024-04-25T15:49:28.238 INFO:tasks.workunit.client.1.smithi193.stdout:4/56: getdents db 0 2024-04-25T15:49:28.238 INFO:tasks.workunit.client.1.smithi193.stdout:4/57: fsync f0 0 2024-04-25T15:49:28.239 INFO:tasks.workunit.client.0.smithi110.stdout:3/502: mkdir d0/d16/d8c/d90/d99/dad/daf 0 2024-04-25T15:49:28.240 INFO:tasks.workunit.client.0.smithi110.stdout:3/503: read d0/f70 [159362,4245] 0 2024-04-25T15:49:28.241 INFO:tasks.workunit.client.0.smithi110.stdout:6/384: mkdir d1/db/d24/d32/d8d 0 2024-04-25T15:49:28.242 INFO:tasks.workunit.client.0.smithi110.stdout:3/504: fdatasync d0/d7/d1c/f94 0 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:4/58: dread f2 [0,4194304] 0 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/37: mknod c7 0 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:4/59: fsync f0 0 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/38: stat f1 0 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/39: chown c2 3540909 1 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:4/60: write f4 [1325745,73481] 0 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/40: stat l3 0 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/41: truncate f1 2179498 0 2024-04-25T15:49:28.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/42: fsync f6 0 2024-04-25T15:49:28.245 INFO:tasks.workunit.client.1.smithi193.stdout:8/61: getdents d2/d3 0 2024-04-25T15:49:28.246 INFO:tasks.workunit.client.1.smithi193.stdout:3/105: symlink d8/d13/d1f/l24 0 2024-04-25T15:49:28.247 INFO:tasks.workunit.client.1.smithi193.stdout:7/41: unlink cb 0 2024-04-25T15:49:28.247 INFO:tasks.workunit.client.1.smithi193.stdout:7/42: write f9 [4267883,7471] 0 2024-04-25T15:49:28.248 INFO:tasks.workunit.client.1.smithi193.stdout:3/106: read d8/d13/d14/f18 [911779,91975] 0 2024-04-25T15:49:28.250 INFO:tasks.workunit.client.1.smithi193.stdout:4/61: creat db/fe x:0 0 0 2024-04-25T15:49:28.252 INFO:tasks.workunit.client.1.smithi193.stdout:5/28: getdents . 0 2024-04-25T15:49:28.252 INFO:tasks.workunit.client.1.smithi193.stdout:5/29: stat f4 0 2024-04-25T15:49:28.256 INFO:tasks.workunit.client.1.smithi193.stdout:7/43: rename f9 to fd 0 2024-04-25T15:49:28.257 INFO:tasks.workunit.client.1.smithi193.stdout:3/107: mknod d8/d13/d1f/c25 0 2024-04-25T15:49:28.257 INFO:tasks.workunit.client.1.smithi193.stdout:8/62: dwrite d2/d3/ff [0,4194304] 0 2024-04-25T15:49:28.258 INFO:tasks.workunit.client.1.smithi193.stdout:4/62: symlink db/lf 0 2024-04-25T15:49:28.261 INFO:tasks.workunit.client.1.smithi193.stdout:5/30: symlink l9 0 2024-04-25T15:49:28.261 INFO:tasks.workunit.client.1.smithi193.stdout:5/31: chown l6 410851 1 2024-04-25T15:49:28.261 INFO:tasks.workunit.client.1.smithi193.stdout:5/32: truncate f8 1881344 0 2024-04-25T15:49:28.264 INFO:tasks.workunit.client.0.smithi110.stdout:7/401: unlink l7 0 2024-04-25T15:49:28.268 INFO:tasks.workunit.client.0.smithi110.stdout:5/405: dwrite d4/d1d/d1f/d35/d3d/f5d [4194304,4194304] 0 2024-04-25T15:49:28.268 INFO:tasks.workunit.client.1.smithi193.stdout:3/108: creat d8/f26 x:0 0 0 2024-04-25T15:49:28.270 INFO:tasks.workunit.client.1.smithi193.stdout:8/63: mknod d2/c15 0 2024-04-25T15:49:28.270 INFO:tasks.workunit.client.1.smithi193.stdout:8/64: fsync d2/f11 0 2024-04-25T15:49:28.270 INFO:tasks.workunit.client.0.smithi110.stdout:2/420: truncate de/f24 4671769 0 2024-04-25T15:49:28.271 INFO:tasks.workunit.client.1.smithi193.stdout:9/80: sync 2024-04-25T15:49:28.271 INFO:tasks.workunit.client.1.smithi193.stdout:0/81: sync 2024-04-25T15:49:28.271 INFO:tasks.workunit.client.1.smithi193.stdout:4/63: symlink db/l10 0 2024-04-25T15:49:28.272 INFO:tasks.workunit.client.1.smithi193.stdout:0/82: dread - d0/d3/d5/d7/f14 zero size 2024-04-25T15:49:28.272 INFO:tasks.workunit.client.0.smithi110.stdout:9/338: mkdir d9/d13/d69 0 2024-04-25T15:49:28.274 INFO:tasks.workunit.client.1.smithi193.stdout:5/33: rename l6 to la 0 2024-04-25T15:49:28.274 INFO:tasks.workunit.client.1.smithi193.stdout:5/34: chown l9 55756307 1 2024-04-25T15:49:28.275 INFO:tasks.workunit.client.1.smithi193.stdout:9/81: dread f4 [0,4194304] 0 2024-04-25T15:49:28.275 INFO:tasks.workunit.client.0.smithi110.stdout:6/385: rename d1/db/f62 to d1/dd/f8e 0 2024-04-25T15:49:28.275 INFO:tasks.workunit.client.1.smithi193.stdout:9/82: chown fb 224 1 2024-04-25T15:49:28.278 INFO:tasks.workunit.client.0.smithi110.stdout:8/533: link d14/d21/d23/f7c d14/d32/d36/d48/fb3 0 2024-04-25T15:49:28.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.279+0000 7f5ef3241700 1 -- 172.21.15.110:0/1304296671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 msgr2=0x7f5eec0fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:28.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.279+0000 7f5ef3241700 1 --2- 172.21.15.110:0/1304296671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 0x7f5eec0fcb80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8004a50 tx=0x7f5ed800af40 comp rx=0 tx=0).stop 2024-04-25T15:49:28.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.279+0000 7f5ef3241700 1 -- 172.21.15.110:0/1304296671 shutdown_connections 2024-04-25T15:49:28.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.279+0000 7f5ef3241700 1 --2- 172.21.15.110:0/1304296671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 0x7f5eec0fcb80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.279+0000 7f5ef3241700 1 --2- 172.21.15.110:0/1304296671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eec0fb460 0x7f5eec0fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.279+0000 7f5ef3241700 1 -- 172.21.15.110:0/1304296671 >> 172.21.15.110:0/1304296671 conn(0x7f5eec0f6be0 msgr2=0x7f5eec0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:28.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.279+0000 7f5ef3241700 1 -- 172.21.15.110:0/1304296671 shutdown_connections 2024-04-25T15:49:28.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.279+0000 7f5ef3241700 1 -- 172.21.15.110:0/1304296671 wait complete. 2024-04-25T15:49:28.282 INFO:tasks.workunit.client.1.smithi193.stdout:8/65: creat d2/d3/f16 x:0 0 0 2024-04-25T15:49:28.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.280+0000 7f5ef3241700 1 Processor -- start 2024-04-25T15:49:28.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.280+0000 7f5ef3241700 1 -- start start 2024-04-25T15:49:28.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.280+0000 7f5ef3241700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eec0fb460 0x7f5eec0ff3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:28.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.280+0000 7f5ef3241700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 0x7f5eec0fda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:28.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.280+0000 7f5ef3241700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec0ffa40 con 0x7f5eec0fc710 2024-04-25T15:49:28.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.280+0000 7f5ef3241700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eec111630 con 0x7f5eec0fb460 2024-04-25T15:49:28.283 INFO:tasks.workunit.client.0.smithi110.stdout:0/417: creat d1/d23/d30/fa3 x:0 0 0 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5eebfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 0x7f5eec0fda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5ef0fdd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eec0fb460 0x7f5eec0ff3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5eebfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 0x7f5eec0fda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35966/0 (socket says 172.21.15.110:35966) 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5ef0fdd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eec0fb460 0x7f5eec0ff3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:37608/0 (socket says 172.21.15.110:37608) 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5eebfff700 1 -- 172.21.15.110:0/1659599058 learned_addr learned my addr 172.21.15.110:0/1659599058 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5eebfff700 1 -- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eec0fb460 msgr2=0x7f5eec0ff3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5eebfff700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eec0fb460 0x7f5eec0ff3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5eebfff700 1 -- 172.21.15.110:0/1659599058 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8005040 con 0x7f5eec0fc710 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5eebfff700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 0x7f5eec0fda50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8004f80 tx=0x7f5ed800af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5ee97fa700 1 -- 172.21.15.110:0/1659599058 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed800f070 con 0x7f5eec0fc710 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.281+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eec111880 con 0x7f5eec0fc710 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.282+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eec0fe1d0 con 0x7f5eec0fc710 2024-04-25T15:49:28.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.282+0000 7f5ee97fa700 1 -- 172.21.15.110:0/1659599058 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ed8026930 con 0x7f5eec0fc710 2024-04-25T15:49:28.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.282+0000 7f5ee97fa700 1 -- 172.21.15.110:0/1659599058 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ed801c660 con 0x7f5eec0fc710 2024-04-25T15:49:28.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.283+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed00052b0 con 0x7f5eec0fc710 2024-04-25T15:49:28.286 INFO:tasks.workunit.client.0.smithi110.stdout:5/406: mknod d4/d1d/c8b 0 2024-04-25T15:49:28.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.284+0000 7f5ee97fa700 1 -- 172.21.15.110:0/1659599058 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f5ed8026460 con 0x7f5eec0fc710 2024-04-25T15:49:28.286 INFO:tasks.workunit.client.0.smithi110.stdout:5/407: fdatasync d4/d2c/d2a/f5c 0 2024-04-25T15:49:28.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.285+0000 7f5ee97fa700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f5edc06fcc0 0x7f5edc072170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:28.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.285+0000 7f5ee97fa700 1 -- 172.21.15.110:0/1659599058 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f5ed8090ea0 con 0x7f5eec0fc710 2024-04-25T15:49:28.287 INFO:tasks.workunit.client.0.smithi110.stdout:5/408: chown d4/f22 750 1 2024-04-25T15:49:28.287 INFO:tasks.workunit.client.1.smithi193.stdout:4/64: symlink db/l11 0 2024-04-25T15:49:28.288 INFO:tasks.workunit.client.1.smithi193.stdout:0/83: rename d0/d3/d5/d7/dd/l19 to d0/d3/d5/d7/dd/l1b 0 2024-04-25T15:49:28.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.285+0000 7f5ef0fdd700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f5edc06fcc0 0x7f5edc072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:28.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.285+0000 7f5ef0fdd700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f5edc06fcc0 0x7f5edc072170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0ff180 tx=0x7f5ee000b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:28.290 INFO:tasks.workunit.client.1.smithi193.stdout:8/66: mknod d2/d3/c17 0 2024-04-25T15:49:28.290 INFO:tasks.workunit.client.1.smithi193.stdout:8/67: stat c1 0 2024-04-25T15:49:28.293 INFO:tasks.workunit.client.0.smithi110.stdout:6/386: symlink d1/db/d24/d69/d75/d82/l8f 0 2024-04-25T15:49:28.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.294+0000 7f5ee97fa700 1 -- 172.21.15.110:0/1659599058 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f5ed805a1c0 con 0x7f5eec0fc710 2024-04-25T15:49:28.295 INFO:tasks.workunit.client.0.smithi110.stdout:6/387: chown d1/d12/l86 0 1 2024-04-25T15:49:28.295 INFO:tasks.workunit.client.0.smithi110.stdout:9/339: dwrite d9/d13/f54 [0,4194304] 0 2024-04-25T15:49:28.295 INFO:tasks.workunit.client.1.smithi193.stdout:8/68: dwrite d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:28.297 INFO:tasks.workunit.client.1.smithi193.stdout:0/84: fdatasync d0/f12 0 2024-04-25T15:49:28.304 INFO:tasks.workunit.client.1.smithi193.stdout:8/69: dwrite d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:28.419 INFO:tasks.workunit.client.0.smithi110.stdout:8/534: creat d14/d4c/d55/fb4 x:0 0 0 2024-04-25T15:49:28.426 INFO:tasks.workunit.client.0.smithi110.stdout:7/402: mknod d1b/c83 0 2024-04-25T15:49:28.435 INFO:tasks.workunit.client.0.smithi110.stdout:0/418: truncate d1/d20/f6a 1590210 0 2024-04-25T15:49:28.449 INFO:tasks.workunit.client.0.smithi110.stdout:9/340: truncate d9/d13/f1a 5372011 0 2024-04-25T15:49:28.450 INFO:tasks.workunit.client.1.smithi193.stdout:8/70: chown d2/fa 4 1 2024-04-25T15:49:28.451 INFO:tasks.workunit.client.1.smithi193.stdout:8/71: read d2/d3/ff [4177761,109411] 0 2024-04-25T15:49:28.453 INFO:tasks.workunit.client.0.smithi110.stdout:8/535: unlink d14/d4c/faa 0 2024-04-25T15:49:28.454 INFO:tasks.workunit.client.1.smithi193.stdout:8/72: dread d2/d3/ff [0,4194304] 0 2024-04-25T15:49:28.455 INFO:tasks.workunit.client.1.smithi193.stdout:1/43: getdents . 0 2024-04-25T15:49:28.457 INFO:tasks.workunit.client.1.smithi193.stdout:1/44: dread - f6 zero size 2024-04-25T15:49:28.457 INFO:tasks.workunit.client.1.smithi193.stdout:6/81: write dd/f13 [374725,111244] 0 2024-04-25T15:49:28.457 INFO:tasks.workunit.client.1.smithi193.stdout:4/65: link db/cd db/c12 0 2024-04-25T15:49:28.458 INFO:tasks.workunit.client.1.smithi193.stdout:1/45: readlink l3 0 2024-04-25T15:49:28.458 INFO:tasks.workunit.client.1.smithi193.stdout:2/88: dwrite d0/fd [0,4194304] 0 2024-04-25T15:49:28.458 INFO:tasks.workunit.client.1.smithi193.stdout:1/46: stat c2 0 2024-04-25T15:49:28.458 INFO:tasks.workunit.client.0.smithi110.stdout:0/419: mkdir d1/d23/d30/d33/d5b/d78/da4 0 2024-04-25T15:49:28.459 INFO:tasks.workunit.client.0.smithi110.stdout:0/420: chown d1/d23/d30/d33/d35 14326804 1 2024-04-25T15:49:28.459 INFO:tasks.workunit.client.1.smithi193.stdout:1/47: chown c7 54 1 2024-04-25T15:49:28.460 INFO:tasks.workunit.client.0.smithi110.stdout:4/513: getdents d13/d24/d2d/d55 0 2024-04-25T15:49:28.461 INFO:tasks.workunit.client.1.smithi193.stdout:0/85: getdents d0/d3/d5/d7/dd/d16/d18 0 2024-04-25T15:49:28.468 INFO:tasks.workunit.client.0.smithi110.stdout:4/514: dwrite d13/d24/d2d/d5e/f71 [0,4194304] 0 2024-04-25T15:49:28.470 INFO:tasks.workunit.client.1.smithi193.stdout:8/73: mkdir d2/d18 0 2024-04-25T15:49:28.475 INFO:tasks.workunit.client.0.smithi110.stdout:8/536: creat d14/d4c/d71/fb5 x:0 0 0 2024-04-25T15:49:28.476 INFO:tasks.workunit.client.1.smithi193.stdout:4/66: mkdir db/d13 0 2024-04-25T15:49:28.477 INFO:tasks.workunit.client.0.smithi110.stdout:7/403: symlink d1b/l84 0 2024-04-25T15:49:28.478 INFO:tasks.workunit.client.0.smithi110.stdout:7/404: fsync d1b/d29/d46/f47 0 2024-04-25T15:49:28.479 INFO:tasks.workunit.client.1.smithi193.stdout:4/67: dread f0 [0,4194304] 0 2024-04-25T15:49:28.481 INFO:tasks.workunit.client.1.smithi193.stdout:2/89: mkdir d0/d9/d14/d25 0 2024-04-25T15:49:28.483 INFO:tasks.workunit.client.1.smithi193.stdout:1/48: symlink l8 0 2024-04-25T15:49:28.483 INFO:tasks.workunit.client.0.smithi110.stdout:5/409: link d4/d1d/l69 d4/d1d/l8c 0 2024-04-25T15:49:28.484 INFO:tasks.workunit.client.1.smithi193.stdout:1/49: chown c2 2 1 2024-04-25T15:49:28.484 INFO:tasks.workunit.client.1.smithi193.stdout:1/50: chown l3 4767279 1 2024-04-25T15:49:28.485 INFO:tasks.workunit.client.1.smithi193.stdout:4/68: dwrite fa [0,4194304] 0 2024-04-25T15:49:28.486 INFO:tasks.workunit.client.1.smithi193.stdout:4/69: fdatasync f9 0 2024-04-25T15:49:28.487 INFO:tasks.workunit.client.1.smithi193.stdout:0/86: rename d0/d3/d5/d7/f14 to d0/d3/d5/d7/dd/d16/f1c 0 2024-04-25T15:49:28.488 INFO:tasks.workunit.client.0.smithi110.stdout:2/421: getdents de/d40/d4f 0 2024-04-25T15:49:28.488 INFO:tasks.workunit.client.1.smithi193.stdout:0/87: stat d0/d3/d5/d7/dd/d16/f1c 0 2024-04-25T15:49:28.490 INFO:tasks.workunit.client.0.smithi110.stdout:2/422: read de/d40/f35 [2255125,70858] 0 2024-04-25T15:49:28.490 INFO:tasks.workunit.client.0.smithi110.stdout:5/410: dread d4/d1d/d1f/d35/d3d/f5d [4194304,4194304] 0 2024-04-25T15:49:28.490 INFO:tasks.workunit.client.0.smithi110.stdout:7/405: dread d1b/d29/d38/f5b [0,4194304] 0 2024-04-25T15:49:28.491 INFO:tasks.workunit.client.1.smithi193.stdout:4/70: dread f1 [0,4194304] 0 2024-04-25T15:49:28.491 INFO:tasks.workunit.client.1.smithi193.stdout:3/109: truncate d8/d13/d14/f18 1249869 0 2024-04-25T15:49:28.491 INFO:tasks.workunit.client.1.smithi193.stdout:3/110: fdatasync d8/d13/d14/f20 0 2024-04-25T15:49:28.492 INFO:tasks.workunit.client.1.smithi193.stdout:8/74: truncate d2/fa 129396 0 2024-04-25T15:49:28.495 INFO:tasks.workunit.client.1.smithi193.stdout:6/82: symlink dd/d12/l16 0 2024-04-25T15:49:28.496 INFO:tasks.workunit.client.1.smithi193.stdout:6/83: stat dd 0 2024-04-25T15:49:28.500 INFO:tasks.workunit.client.0.smithi110.stdout:8/537: unlink d14/d21/d23/f60 0 2024-04-25T15:49:28.501 INFO:tasks.workunit.client.0.smithi110.stdout:8/538: chown d14/d4c/f75 370 1 2024-04-25T15:49:28.504 INFO:tasks.workunit.client.1.smithi193.stdout:1/51: unlink c2 0 2024-04-25T15:49:28.504 INFO:tasks.workunit.client.1.smithi193.stdout:5/35: getdents . 0 2024-04-25T15:49:28.505 INFO:tasks.workunit.client.0.smithi110.stdout:0/421: truncate d1/f8 2226026 0 2024-04-25T15:49:28.507 INFO:tasks.workunit.client.1.smithi193.stdout:7/44: truncate fd 3563311 0 2024-04-25T15:49:28.507 INFO:tasks.workunit.client.1.smithi193.stdout:7/45: fsync fc 0 2024-04-25T15:49:28.510 INFO:tasks.workunit.client.1.smithi193.stdout:2/90: rename d0/fd to d0/df/f26 0 2024-04-25T15:49:28.511 INFO:tasks.workunit.client.1.smithi193.stdout:2/91: fdatasync d0/f11 0 2024-04-25T15:49:28.511 INFO:tasks.workunit.client.1.smithi193.stdout:2/92: chown d0/l12 27370 1 2024-04-25T15:49:28.511 INFO:tasks.workunit.client.1.smithi193.stdout:2/93: dread - d0/f11 zero size 2024-04-25T15:49:28.512 INFO:tasks.workunit.client.0.smithi110.stdout:3/505: dwrite d0/f1e [0,4194304] 0 2024-04-25T15:49:28.514 INFO:tasks.workunit.client.1.smithi193.stdout:0/88: creat d0/d3/d5/d7/dd/f1d x:0 0 0 2024-04-25T15:49:28.515 INFO:tasks.workunit.client.1.smithi193.stdout:0/89: truncate d0/ff 4859476 0 2024-04-25T15:49:28.522 INFO:tasks.workunit.client.0.smithi110.stdout:4/515: rmdir d13/d24/d2d/d44/d65 39 2024-04-25T15:49:28.524 INFO:tasks.workunit.client.1.smithi193.stdout:8/75: creat d2/d3/db/f19 x:0 0 0 2024-04-25T15:49:28.525 INFO:tasks.workunit.client.1.smithi193.stdout:8/76: write d2/d3/db/f19 [535835,110649] 0 2024-04-25T15:49:28.526 INFO:tasks.workunit.client.1.smithi193.stdout:8/77: chown d2/d18 10 1 2024-04-25T15:49:28.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/516: dwrite d13/d24/d2d/d36/f4d [0,4194304] 0 2024-04-25T15:49:28.531 INFO:tasks.workunit.client.1.smithi193.stdout:1/52: unlink c7 0 2024-04-25T15:49:28.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/84: dwrite dd/f11 [0,4194304] 0 2024-04-25T15:49:28.537 INFO:tasks.workunit.client.0.smithi110.stdout:8/539: symlink d14/d21/d26/dab/lb6 0 2024-04-25T15:49:28.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/94: rename d0/df/d16/f20 to d0/f27 0 2024-04-25T15:49:28.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/95: rename d0/d9 to d0/d9/d15/d28 22 2024-04-25T15:49:28.542 INFO:tasks.workunit.client.0.smithi110.stdout:8/540: dread d14/d32/d36/f84 [8388608,4194304] 0 2024-04-25T15:49:28.543 INFO:tasks.workunit.client.1.smithi193.stdout:9/83: dread f9 [0,4194304] 0 2024-04-25T15:49:28.545 INFO:tasks.workunit.client.0.smithi110.stdout:0/422: unlink d1/d23/d30/d33/c26 0 2024-04-25T15:49:28.546 INFO:tasks.workunit.client.1.smithi193.stdout:4/71: symlink db/d13/l14 0 2024-04-25T15:49:28.549 INFO:tasks.workunit.client.1.smithi193.stdout:4/72: dread f0 [0,4194304] 0 2024-04-25T15:49:28.550 INFO:tasks.workunit.client.1.smithi193.stdout:4/73: truncate fa 4740003 0 2024-04-25T15:49:28.550 INFO:tasks.workunit.client.1.smithi193.stdout:8/78: creat d2/d3/db/f1a x:0 0 0 2024-04-25T15:49:28.550 INFO:tasks.workunit.client.1.smithi193.stdout:4/74: stat db 0 2024-04-25T15:49:28.550 INFO:tasks.workunit.client.1.smithi193.stdout:4/75: chown f0 903287 1 2024-04-25T15:49:28.552 INFO:tasks.workunit.client.1.smithi193.stdout:1/53: mkdir d9 0 2024-04-25T15:49:28.552 INFO:tasks.workunit.client.1.smithi193.stdout:1/54: readlink l8 0 2024-04-25T15:49:28.552 INFO:tasks.workunit.client.1.smithi193.stdout:1/55: fdatasync f1 0 2024-04-25T15:49:28.552 INFO:tasks.workunit.client.1.smithi193.stdout:0/90: truncate d0/d3/fe 1641111 0 2024-04-25T15:49:28.554 INFO:tasks.workunit.client.1.smithi193.stdout:5/36: truncate f4 650596 0 2024-04-25T15:49:28.556 INFO:tasks.workunit.client.1.smithi193.stdout:0/91: dread d0/fc [0,4194304] 0 2024-04-25T15:49:28.556 INFO:tasks.workunit.client.0.smithi110.stdout:2/423: creat de/d40/d70/f7f x:0 0 0 2024-04-25T15:49:28.559 INFO:tasks.workunit.client.1.smithi193.stdout:2/96: creat d0/f29 x:0 0 0 2024-04-25T15:49:28.560 INFO:tasks.workunit.client.1.smithi193.stdout:2/97: truncate d0/df/f10 5201923 0 2024-04-25T15:49:28.560 INFO:tasks.workunit.client.0.smithi110.stdout:9/341: getdents d9/db/df/d4d 0 2024-04-25T15:49:28.562 INFO:tasks.workunit.client.1.smithi193.stdout:9/84: mkdir df 0 2024-04-25T15:49:28.563 INFO:tasks.workunit.client.0.smithi110.stdout:4/517: unlink d13/d24/d2d/d5e/f6b 0 2024-04-25T15:49:28.565 INFO:tasks.workunit.client.1.smithi193.stdout:8/79: mknod d2/c1b 0 2024-04-25T15:49:28.566 INFO:tasks.workunit.client.1.smithi193.stdout:8/80: write d2/f11 [1258385,126159] 0 2024-04-25T15:49:28.567 INFO:tasks.workunit.client.0.smithi110.stdout:6/388: dwrite d1/dd/d3c/f5d [0,4194304] 0 2024-04-25T15:49:28.577 INFO:tasks.workunit.client.0.smithi110.stdout:4/518: dread d13/f22 [0,4194304] 0 2024-04-25T15:49:28.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.586+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ed0000bc0 con 0x7f5edc06fcc0 2024-04-25T15:49:28.586 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:28 smithi110 ceph-mon[26344]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:49:28.587 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:28 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1630184803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:49:28.587 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:28 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1873777944' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:49:28.587 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:28 smithi110 ceph-mon[26344]: pgmap v31: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 29 MiB/s rd, 148 MiB/s wr, 358 op/s 2024-04-25T15:49:28.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.587+0000 7f5ee97fa700 1 -- 172.21.15.110:0/1659599058 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f5ed0000bc0 con 0x7f5edc06fcc0 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "1/21 daemons upgraded", 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:49:28.588 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:49:28.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f5edc06fcc0 msgr2=0x7f5edc072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:28.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f5edc06fcc0 0x7f5edc072170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0ff180 tx=0x7f5ee000b040 comp rx=0 tx=0).stop 2024-04-25T15:49:28.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 msgr2=0x7f5eec0fda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:28.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 0x7f5eec0fda50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8004f80 tx=0x7f5ed800af40 comp rx=0 tx=0).stop 2024-04-25T15:49:28.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 shutdown_connections 2024-04-25T15:49:28.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f5edc06fcc0 0x7f5edc072170 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eec0fb460 0x7f5eec0ff3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 --2- 172.21.15.110:0/1659599058 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5eec0fc710 0x7f5eec0fda50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 >> 172.21.15.110:0/1659599058 conn(0x7f5eec0f6be0 msgr2=0x7f5eec0f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 shutdown_connections 2024-04-25T15:49:28.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.589+0000 7f5ef3241700 1 -- 172.21.15.110:0/1659599058 wait complete. 2024-04-25T15:49:28.616 INFO:tasks.workunit.client.1.smithi193.stdout:7/46: sync 2024-04-25T15:49:28.616 INFO:tasks.workunit.client.1.smithi193.stdout:7/47: stat f3 0 2024-04-25T15:49:28.657 INFO:tasks.workunit.client.1.smithi193.stdout:4/76: rename f0 to db/f15 0 2024-04-25T15:49:28.672 INFO:tasks.workunit.client.1.smithi193.stdout:5/37: rename f8 to fb 0 2024-04-25T15:49:28.676 INFO:tasks.workunit.client.0.smithi110.stdout:0/423: truncate d1/d47/f61 34904 0 2024-04-25T15:49:28.679 INFO:tasks.workunit.client.1.smithi193.stdout:2/98: rmdir d0/d9/d15 39 2024-04-25T15:49:28.679 INFO:tasks.workunit.client.1.smithi193.stdout:2/99: read d0/df/f10 [294952,75124] 0 2024-04-25T15:49:28.680 INFO:tasks.workunit.client.1.smithi193.stdout:2/100: fdatasync d0/f11 0 2024-04-25T15:49:28.680 INFO:tasks.workunit.client.0.smithi110.stdout:8/541: dread d14/d21/d23/f25 [0,4194304] 0 2024-04-25T15:49:28.683 INFO:tasks.workunit.client.1.smithi193.stdout:9/85: unlink fb 0 2024-04-25T15:49:28.686 INFO:tasks.workunit.client.1.smithi193.stdout:8/81: mknod d2/d3/c1c 0 2024-04-25T15:49:28.688 INFO:tasks.workunit.client.1.smithi193.stdout:9/86: dwrite f6 [0,4194304] 0 2024-04-25T15:49:28.690 INFO:tasks.workunit.client.0.smithi110.stdout:5/411: write d4/d1d/f70 [463338,101930] 0 2024-04-25T15:49:28.691 INFO:tasks.workunit.client.0.smithi110.stdout:5/412: read d4/d1d/d1f/f32 [1233218,91274] 0 2024-04-25T15:49:28.692 INFO:tasks.workunit.client.0.smithi110.stdout:5/413: fsync d4/d1d/d1f/d35/f6a 0 2024-04-25T15:49:28.695 INFO:tasks.workunit.client.0.smithi110.stdout:8/542: dread f11 [0,4194304] 0 2024-04-25T15:49:28.695 INFO:tasks.workunit.client.0.smithi110.stdout:5/414: write d4/d1d/d1f/d35/d3d/d52/f7a [943027,40506] 0 2024-04-25T15:49:28.696 INFO:tasks.workunit.client.0.smithi110.stdout:5/415: stat d4/d1d/d1f/c23 0 2024-04-25T15:49:28.696 INFO:tasks.workunit.client.1.smithi193.stdout:4/77: creat db/d13/f16 x:0 0 0 2024-04-25T15:49:28.697 INFO:tasks.workunit.client.0.smithi110.stdout:7/406: rmdir d1b 39 2024-04-25T15:49:28.698 INFO:tasks.workunit.client.0.smithi110.stdout:5/416: write d4/d1d/d1f/f64 [4782968,60266] 0 2024-04-25T15:49:28.699 INFO:tasks.workunit.client.1.smithi193.stdout:3/111: dwrite d8/d13/d1f/f16 [0,4194304] 0 2024-04-25T15:49:28.700 INFO:tasks.workunit.client.1.smithi193.stdout:3/112: truncate d8/f26 690096 0 2024-04-25T15:49:28.700 INFO:tasks.workunit.client.1.smithi193.stdout:3/113: chown d8/d13/l1e 447 1 2024-04-25T15:49:28.701 INFO:tasks.workunit.client.1.smithi193.stdout:3/114: chown d8/d13/d1f/f1c 334987882 1 2024-04-25T15:49:28.702 INFO:tasks.workunit.client.1.smithi193.stdout:3/115: truncate d8/d13/d14/f1d 741031 0 2024-04-25T15:49:28.703 INFO:tasks.workunit.client.1.smithi193.stdout:3/116: chown d8/d13/d1f/f1c 772990 1 2024-04-25T15:49:28.704 INFO:tasks.workunit.client.0.smithi110.stdout:0/424: symlink d1/d20/d69/la5 0 2024-04-25T15:49:28.707 INFO:tasks.workunit.client.1.smithi193.stdout:3/117: dread d8/f10 [0,4194304] 0 2024-04-25T15:49:28.707 INFO:tasks.workunit.client.1.smithi193.stdout:3/118: readlink d8/d13/l1e 0 2024-04-25T15:49:28.711 INFO:tasks.workunit.client.0.smithi110.stdout:0/425: dwrite d1/d23/d30/fa3 [0,4194304] 0 2024-04-25T15:49:28.724 INFO:tasks.workunit.client.0.smithi110.stdout:5/417: rename d4/d1d/f38 to d4/d1d/d1f/d35/d7f/f8d 0 2024-04-25T15:49:28.730 INFO:tasks.workunit.client.0.smithi110.stdout:4/519: link d13/d24/d2d/d36/d51/f8c d13/d24/d2d/d5e/fa9 0 2024-04-25T15:49:28.731 INFO:tasks.workunit.client.0.smithi110.stdout:4/520: write d13/d24/d2d/d36/f5a [550436,82900] 0 2024-04-25T15:49:28.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:28 smithi193 ceph-mon[33747]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:49:28.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:28 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1630184803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:49:28.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:28 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1873777944' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:49:28.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:28 smithi193 ceph-mon[33747]: pgmap v31: 65 pgs: 65 active+clean; 1.1 GiB data, 5.0 GiB used, 531 GiB / 536 GiB avail; 29 MiB/s rd, 148 MiB/s wr, 358 op/s 2024-04-25T15:49:28.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.744+0000 7f8a03582700 1 -- 172.21.15.110:0/3917941365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89fc0fffa0 msgr2=0x7f89fc102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:28.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.744+0000 7f8a03582700 1 --2- 172.21.15.110:0/3917941365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89fc0fffa0 0x7f89fc102380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89f0004940 tx=0x7f89f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:28.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.744+0000 7f8a03582700 1 -- 172.21.15.110:0/3917941365 shutdown_connections 2024-04-25T15:49:28.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.744+0000 7f8a03582700 1 --2- 172.21.15.110:0/3917941365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89fc0fffa0 0x7f89fc102380 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f89f0004940 tx=0x7f89f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:28.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.744+0000 7f8a03582700 1 --2- 172.21.15.110:0/3917941365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89fc066200 0x7f89fc0ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.744+0000 7f8a03582700 1 -- 172.21.15.110:0/3917941365 >> 172.21.15.110:0/3917941365 conn(0x7f89fc0f7430 msgr2=0x7f89fc0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:28.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.745+0000 7f8a03582700 1 -- 172.21.15.110:0/3917941365 shutdown_connections 2024-04-25T15:49:28.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.745+0000 7f8a03582700 1 -- 172.21.15.110:0/3917941365 wait complete. 2024-04-25T15:49:28.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.745+0000 7f8a03582700 1 Processor -- start 2024-04-25T15:49:28.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.746+0000 7f8a03582700 1 -- start start 2024-04-25T15:49:28.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.746+0000 7f8a03582700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89fc066200 0x7f89fc10bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:28.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.746+0000 7f8a03582700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89fc10c0b0 0x7f89fc10cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:28.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.746+0000 7f8a03582700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc10c570 con 0x7f89fc10c0b0 2024-04-25T15:49:28.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.746+0000 7f8a03582700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89fc10c6e0 con 0x7f89fc066200 2024-04-25T15:49:28.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.746+0000 7f8a00b1d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89fc10c0b0 0x7f89fc10cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:28.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.746+0000 7f8a00b1d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89fc10c0b0 0x7f89fc10cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35988/0 (socket says 172.21.15.110:35988) 2024-04-25T15:49:28.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.746+0000 7f8a00b1d700 1 -- 172.21.15.110:0/1054259809 learned_addr learned my addr 172.21.15.110:0/1054259809 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:28.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.747+0000 7f8a00b1d700 1 -- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89fc066200 msgr2=0x7f89fc10bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:28.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.747+0000 7f8a00b1d700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89fc066200 0x7f89fc10bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.747+0000 7f8a00b1d700 1 -- 172.21.15.110:0/1054259809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f0005040 con 0x7f89fc10c0b0 2024-04-25T15:49:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.747+0000 7f8a00b1d700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89fc10c0b0 0x7f89fc10cf50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f89f0004940 tx=0x7f89f0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.747+0000 7f89edffb700 1 -- 172.21.15.110:0/1054259809 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89f000e070 con 0x7f89fc10c0b0 2024-04-25T15:49:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.747+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89fc10d550 con 0x7f89fc10c0b0 2024-04-25T15:49:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.747+0000 7f89edffb700 1 -- 172.21.15.110:0/1054259809 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89f0009710 con 0x7f89fc10c0b0 2024-04-25T15:49:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.747+0000 7f89edffb700 1 -- 172.21.15.110:0/1054259809 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89f000b650 con 0x7f89fc10c0b0 2024-04-25T15:49:28.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.748+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89fc10da60 con 0x7f89fc10c0b0 2024-04-25T15:49:28.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.750+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89fc1060b0 con 0x7f89fc10c0b0 2024-04-25T15:49:28.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.752+0000 7f89edffb700 1 -- 172.21.15.110:0/1054259809 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f89f001c400 con 0x7f89fc10c0b0 2024-04-25T15:49:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.752+0000 7f89edffb700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f89e806fd00 0x7f89e80721b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:28.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.753+0000 7f89edffb700 1 -- 172.21.15.110:0/1054259809 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f89f00911c0 con 0x7f89fc10c0b0 2024-04-25T15:49:28.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.753+0000 7f8a0131e700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f89e806fd00 0x7f89e80721b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:28.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.754+0000 7f8a0131e700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f89e806fd00 0x7f89e80721b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f89fc10b900 tx=0x7f89f800b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:28.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:28.759+0000 7f89edffb700 1 -- 172.21.15.110:0/1054259809 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f89f005a6c0 con 0x7f89fc10c0b0 2024-04-25T15:49:28.767 INFO:tasks.workunit.client.0.smithi110.stdout:2/424: link de/d40/d6e/f21 de/f80 0 2024-04-25T15:49:28.770 INFO:tasks.workunit.client.0.smithi110.stdout:0/426: getdents d1/d23 0 2024-04-25T15:49:28.773 INFO:tasks.workunit.client.0.smithi110.stdout:2/425: creat de/d40/d5b/d5c/d5f/f81 x:0 0 0 2024-04-25T15:49:28.776 INFO:tasks.workunit.client.0.smithi110.stdout:2/426: rmdir de/d6f 39 2024-04-25T15:49:28.777 INFO:tasks.workunit.client.0.smithi110.stdout:2/427: chown de/d40/d5b/d5c/d5f/f63 311179 1 2024-04-25T15:49:28.778 INFO:tasks.workunit.client.0.smithi110.stdout:0/427: symlink d1/d23/d9e/la6 0 2024-04-25T15:49:28.779 INFO:tasks.workunit.client.0.smithi110.stdout:0/428: dread - d1/d20/d69/f91 zero size 2024-04-25T15:49:28.779 INFO:tasks.workunit.client.0.smithi110.stdout:2/428: read de/f28 [6462597,26257] 0 2024-04-25T15:49:28.784 INFO:tasks.workunit.client.0.smithi110.stdout:2/429: fsync de/f2d 0 2024-04-25T15:49:28.794 INFO:tasks.workunit.client.0.smithi110.stdout:2/430: mknod de/d6f/c82 0 2024-04-25T15:49:28.798 INFO:tasks.workunit.client.0.smithi110.stdout:2/431: readlink de/d40/d6e/d52/d6d/l7c 0 2024-04-25T15:49:28.801 INFO:tasks.workunit.client.0.smithi110.stdout:2/432: mknod de/d40/d5b/d5c/c83 0 2024-04-25T15:49:28.802 INFO:tasks.workunit.client.0.smithi110.stdout:2/433: fdatasync de/d40/d6e/d52/f62 0 2024-04-25T15:49:28.803 INFO:tasks.workunit.client.0.smithi110.stdout:2/434: chown de/d40/d6e/d50 469239 1 2024-04-25T15:49:28.806 INFO:tasks.workunit.client.0.smithi110.stdout:2/435: dread f8 [0,4194304] 0 2024-04-25T15:49:28.807 INFO:tasks.workunit.client.0.smithi110.stdout:2/436: read - de/d40/d5b/d5c/d5f/f7a zero size 2024-04-25T15:49:28.807 INFO:tasks.workunit.client.0.smithi110.stdout:2/437: chown de/l2e 29767 1 2024-04-25T15:49:28.814 INFO:tasks.workunit.client.0.smithi110.stdout:2/438: dwrite de/d40/d5b/d5c/d5f/f7a [0,4194304] 0 2024-04-25T15:49:28.822 INFO:tasks.workunit.client.0.smithi110.stdout:2/439: dwrite de/f2d [0,4194304] 0 2024-04-25T15:49:28.829 INFO:tasks.workunit.client.0.smithi110.stdout:2/440: creat de/d40/d70/f84 x:0 0 0 2024-04-25T15:49:28.836 INFO:tasks.workunit.client.0.smithi110.stdout:2/441: creat de/d40/d6e/d52/d6d/d7d/f85 x:0 0 0 2024-04-25T15:49:28.837 INFO:tasks.workunit.client.0.smithi110.stdout:2/442: truncate de/d40/d5b/d5c/d5f/f81 290805 0 2024-04-25T15:49:28.869 INFO:tasks.workunit.client.0.smithi110.stdout:6/389: rename d1/db/d22/d27 to d1/db/d22/d90 0 2024-04-25T15:49:28.875 INFO:tasks.workunit.client.0.smithi110.stdout:3/506: mkdir d0/d7/d1c/db0 0 2024-04-25T15:49:28.891 INFO:tasks.workunit.client.0.smithi110.stdout:0/429: truncate d1/d23/d30/d33/f2d 2927766 0 2024-04-25T15:49:28.891 INFO:tasks.workunit.client.0.smithi110.stdout:0/430: readlink d1/l1e 0 2024-04-25T15:49:28.892 INFO:tasks.workunit.client.0.smithi110.stdout:0/431: write d1/d20/d69/f91 [980769,117827] 0 2024-04-25T15:49:28.893 INFO:tasks.workunit.client.0.smithi110.stdout:0/432: stat d1/d20/d69 0 2024-04-25T15:49:28.893 INFO:tasks.workunit.client.0.smithi110.stdout:8/543: mkdir d14/d32/d36/db7 0 2024-04-25T15:49:28.894 INFO:tasks.workunit.client.0.smithi110.stdout:1/526: sync 2024-04-25T15:49:28.898 INFO:tasks.workunit.client.1.smithi193.stdout:5/38: creat fc x:0 0 0 2024-04-25T15:49:28.899 INFO:tasks.workunit.client.0.smithi110.stdout:1/527: chown d0/d14/d5f/d7d/f97 1228 1 2024-04-25T15:49:28.902 INFO:tasks.workunit.client.0.smithi110.stdout:8/544: dwrite d14/d4c/f75 [0,4194304] 0 2024-04-25T15:49:28.912 INFO:tasks.workunit.client.0.smithi110.stdout:8/545: creat d14/d4c/d55/fb8 x:0 0 0 2024-04-25T15:49:28.914 INFO:tasks.workunit.client.0.smithi110.stdout:0/433: link d1/d20/c75 d1/d23/d30/d33/d35/da1/ca7 0 2024-04-25T15:49:28.916 INFO:tasks.workunit.client.0.smithi110.stdout:8/546: mkdir d14/d21/d26/d6a/db9 0 2024-04-25T15:49:28.923 INFO:tasks.workunit.client.1.smithi193.stdout:3/119: rmdir d8 39 2024-04-25T15:49:28.924 INFO:tasks.workunit.client.0.smithi110.stdout:0/434: dread d1/d20/f6a [0,4194304] 0 2024-04-25T15:49:28.924 INFO:tasks.workunit.client.1.smithi193.stdout:3/120: write f2 [4801974,24556] 0 2024-04-25T15:49:28.924 INFO:tasks.workunit.client.0.smithi110.stdout:1/528: getdents d0/d14 0 2024-04-25T15:49:28.931 INFO:tasks.workunit.client.1.smithi193.stdout:1/56: rmdir d9 0 2024-04-25T15:49:28.931 INFO:tasks.workunit.client.1.smithi193.stdout:1/57: dread - f6 zero size 2024-04-25T15:49:28.931 INFO:tasks.workunit.client.0.smithi110.stdout:1/529: dwrite d0/d14/d5f/d7d/f67 [0,4194304] 0 2024-04-25T15:49:28.933 INFO:tasks.workunit.client.0.smithi110.stdout:1/530: truncate d0/d14/d81/f8f 392004 0 2024-04-25T15:49:28.936 INFO:tasks.workunit.client.1.smithi193.stdout:5/39: symlink ld 0 2024-04-25T15:49:28.937 INFO:tasks.workunit.client.1.smithi193.stdout:0/92: write d0/d3/fe [397099,26827] 0 2024-04-25T15:49:28.938 INFO:tasks.workunit.client.0.smithi110.stdout:8/547: creat d14/d21/fba x:0 0 0 2024-04-25T15:49:28.939 INFO:tasks.workunit.client.0.smithi110.stdout:5/418: rmdir d4/d1d/d1f/d35/d3d 39 2024-04-25T15:49:28.941 INFO:tasks.workunit.client.1.smithi193.stdout:6/85: getdents dd 0 2024-04-25T15:49:28.943 INFO:tasks.workunit.client.1.smithi193.stdout:5/40: dread fb [0,4194304] 0 2024-04-25T15:49:28.943 INFO:tasks.workunit.client.1.smithi193.stdout:5/41: readlink l9 0 2024-04-25T15:49:28.944 INFO:tasks.workunit.client.1.smithi193.stdout:6/86: dread dd/f15 [0,4194304] 0 2024-04-25T15:49:28.945 INFO:tasks.workunit.client.0.smithi110.stdout:2/443: write de/d40/d4f/f47 [3856999,100339] 0 2024-04-25T15:49:28.945 INFO:tasks.workunit.client.1.smithi193.stdout:2/101: symlink d0/d9/d14/d25/l2a 0 2024-04-25T15:49:28.945 INFO:tasks.workunit.client.1.smithi193.stdout:6/87: stat dd/c10 0 2024-04-25T15:49:28.947 INFO:tasks.workunit.client.0.smithi110.stdout:5/419: dwrite d4/d1d/d1f/f6e [0,4194304] 0 2024-04-25T15:49:28.948 INFO:tasks.workunit.client.1.smithi193.stdout:6/88: dread dd/fe [0,4194304] 0 2024-04-25T15:49:28.950 INFO:tasks.workunit.client.0.smithi110.stdout:7/407: rmdir d1b/d29/d46 39 2024-04-25T15:49:28.952 INFO:tasks.workunit.client.0.smithi110.stdout:1/531: fsync d0/d14/d5f/d7d/f67 0 2024-04-25T15:49:28.955 INFO:tasks.workunit.client.1.smithi193.stdout:6/89: dwrite dd/f15 [0,4194304] 0 2024-04-25T15:49:28.956 INFO:tasks.workunit.client.1.smithi193.stdout:6/90: rename dd to dd/d17 22 2024-04-25T15:49:28.956 INFO:tasks.workunit.client.0.smithi110.stdout:1/532: chown d0/d23/f92 2 1 2024-04-25T15:49:28.956 INFO:tasks.workunit.client.0.smithi110.stdout:0/435: dread d1/f38 [0,4194304] 0 2024-04-25T15:49:28.956 INFO:tasks.workunit.client.1.smithi193.stdout:6/91: write dd/f15 [3397,116143] 0 2024-04-25T15:49:28.957 INFO:tasks.workunit.client.1.smithi193.stdout:9/87: symlink df/l10 0 2024-04-25T15:49:28.959 INFO:tasks.workunit.client.0.smithi110.stdout:5/420: readlink d4/d1d/l8c 0 2024-04-25T15:49:28.961 INFO:tasks.workunit.client.0.smithi110.stdout:5/421: chown d4/d1d/c8b 52563540 1 2024-04-25T15:49:28.964 INFO:tasks.workunit.client.0.smithi110.stdout:1/533: creat d0/d14/d5f/d6f/fa3 x:0 0 0 2024-04-25T15:49:28.965 INFO:tasks.workunit.client.0.smithi110.stdout:1/534: chown d0/d14/c44 400337 1 2024-04-25T15:49:28.967 INFO:tasks.workunit.client.1.smithi193.stdout:3/121: dread - d8/d13/d14/f20 zero size 2024-04-25T15:49:28.969 INFO:tasks.workunit.client.1.smithi193.stdout:1/58: rename l8 to la 0 2024-04-25T15:49:28.971 INFO:tasks.workunit.client.1.smithi193.stdout:0/93: rmdir d0/d3/d5 39 2024-04-25T15:49:28.973 INFO:tasks.workunit.client.0.smithi110.stdout:5/422: creat d4/d2c/d2a/f8e x:0 0 0 2024-04-25T15:49:28.973 INFO:tasks.workunit.client.0.smithi110.stdout:8/548: dread f5 [4194304,4194304] 0 2024-04-25T15:49:28.974 INFO:tasks.workunit.client.1.smithi193.stdout:1/59: dread f1 [0,4194304] 0 2024-04-25T15:49:28.974 INFO:tasks.workunit.client.0.smithi110.stdout:8/549: write ff [4412056,17220] 0 2024-04-25T15:49:28.976 INFO:tasks.workunit.client.0.smithi110.stdout:1/535: mknod d0/d51/ca4 0 2024-04-25T15:49:28.977 INFO:tasks.workunit.client.1.smithi193.stdout:5/42: readlink la 0 2024-04-25T15:49:28.979 INFO:tasks.workunit.client.0.smithi110.stdout:0/436: mkdir d1/d23/d30/d33/d5b/d83/d87/da8 0 2024-04-25T15:49:28.980 INFO:tasks.workunit.client.0.smithi110.stdout:2/444: dread de/d40/d6e/f39 [0,4194304] 0 2024-04-25T15:49:28.980 INFO:tasks.workunit.client.0.smithi110.stdout:0/437: fsync d1/d47/d54/f9c 0 2024-04-25T15:49:28.981 INFO:tasks.workunit.client.0.smithi110.stdout:7/408: symlink d1b/d29/d38/d55/d7f/l85 0 2024-04-25T15:49:28.983 INFO:tasks.workunit.client.1.smithi193.stdout:2/102: rename d0/df/f1c to d0/df/f2b 0 2024-04-25T15:49:28.987 INFO:tasks.workunit.client.0.smithi110.stdout:1/536: mknod d0/d14/d5f/d7d/d9c/ca5 0 2024-04-25T15:49:28.989 INFO:tasks.workunit.client.1.smithi193.stdout:6/92: rename l2 to dd/d12/l18 0 2024-04-25T15:49:28.991 INFO:tasks.workunit.client.0.smithi110.stdout:2/445: mkdir de/d40/d86 0 2024-04-25T15:49:28.992 INFO:tasks.workunit.client.0.smithi110.stdout:2/446: chown de/d40/d6e/d52 7430769 1 2024-04-25T15:49:28.992 INFO:tasks.workunit.client.1.smithi193.stdout:9/88: mkdir df/d11 0 2024-04-25T15:49:28.992 INFO:tasks.workunit.client.1.smithi193.stdout:9/89: chown l7 19372197 1 2024-04-25T15:49:28.993 INFO:tasks.workunit.client.0.smithi110.stdout:2/447: readlink de/d40/d5b/d5c/l75 0 2024-04-25T15:49:28.994 INFO:tasks.workunit.client.0.smithi110.stdout:0/438: write d1/d23/d30/d33/d35/f12 [21705,33987] 0 2024-04-25T15:49:28.994 INFO:tasks.workunit.client.0.smithi110.stdout:1/537: dwrite d0/d23/f5c [4194304,4194304] 0 2024-04-25T15:49:28.995 INFO:tasks.workunit.client.0.smithi110.stdout:1/538: readlink d0/d14/l3c 0 2024-04-25T15:49:28.997 INFO:tasks.workunit.client.0.smithi110.stdout:1/539: dread - d0/d14/d5f/d6f/f82 zero size 2024-04-25T15:49:28.997 INFO:tasks.workunit.client.1.smithi193.stdout:3/122: unlink d8/l9 0 2024-04-25T15:49:28.998 INFO:tasks.workunit.client.0.smithi110.stdout:2/448: write de/f56 [757266,82394] 0 2024-04-25T15:49:28.999 INFO:tasks.workunit.client.0.smithi110.stdout:0/439: dread - d1/d23/d30/d33/d5b/d83/f84 zero size 2024-04-25T15:49:29.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/550: mkdir d14/d21/d26/dab/dac/dbb 0 2024-04-25T15:49:29.004 INFO:tasks.workunit.client.1.smithi193.stdout:1/60: symlink lb 0 2024-04-25T15:49:29.004 INFO:tasks.workunit.client.1.smithi193.stdout:1/61: dread - f6 zero size 2024-04-25T15:49:29.004 INFO:tasks.workunit.client.1.smithi193.stdout:0/94: dread d0/fc [0,4194304] 0 2024-04-25T15:49:29.005 INFO:tasks.workunit.client.1.smithi193.stdout:1/62: write f6 [394157,48818] 0 2024-04-25T15:49:29.005 INFO:tasks.workunit.client.1.smithi193.stdout:1/63: chown f6 20057 1 2024-04-25T15:49:29.007 INFO:tasks.workunit.client.0.smithi110.stdout:8/551: dwrite f6 [0,4194304] 0 2024-04-25T15:49:29.008 INFO:tasks.workunit.client.0.smithi110.stdout:8/552: dread - d14/d4c/d55/fb4 zero size 2024-04-25T15:49:29.008 INFO:tasks.workunit.client.1.smithi193.stdout:5/43: creat fe x:0 0 0 2024-04-25T15:49:29.008 INFO:tasks.workunit.client.0.smithi110.stdout:8/553: fsync d14/f81 0 2024-04-25T15:49:29.016 INFO:tasks.workunit.client.1.smithi193.stdout:2/103: creat d0/df/f2c x:0 0 0 2024-04-25T15:49:29.017 INFO:tasks.workunit.client.1.smithi193.stdout:8/82: getdents d2 0 2024-04-25T15:49:29.018 INFO:tasks.workunit.client.0.smithi110.stdout:1/540: dwrite d0/d23/f55 [0,4194304] 0 2024-04-25T15:49:29.019 INFO:tasks.workunit.client.0.smithi110.stdout:0/440: chown d1/c13 4258245 1 2024-04-25T15:49:29.022 INFO:tasks.workunit.client.1.smithi193.stdout:6/93: unlink lc 0 2024-04-25T15:49:29.026 INFO:tasks.workunit.client.1.smithi193.stdout:9/90: fdatasync f9 0 2024-04-25T15:49:29.026 INFO:tasks.workunit.client.1.smithi193.stdout:6/94: dread dd/f11 [0,4194304] 0 2024-04-25T15:49:29.027 INFO:tasks.workunit.client.1.smithi193.stdout:9/91: chown f6 33329489 1 2024-04-25T15:49:29.028 INFO:tasks.workunit.client.1.smithi193.stdout:4/78: getdents db 0 2024-04-25T15:49:29.029 INFO:tasks.workunit.client.0.smithi110.stdout:8/554: mknod d14/d4c/d55/cbc 0 2024-04-25T15:49:29.031 INFO:tasks.workunit.client.1.smithi193.stdout:3/123: unlink d8/f26 0 2024-04-25T15:49:29.032 INFO:tasks.workunit.client.1.smithi193.stdout:3/124: dread - d8/d13/d1f/f1c zero size 2024-04-25T15:49:29.035 INFO:tasks.workunit.client.1.smithi193.stdout:0/95: creat d0/f1e x:0 0 0 2024-04-25T15:49:29.035 INFO:tasks.workunit.client.1.smithi193.stdout:4/79: dwrite db/fc [0,4194304] 0 2024-04-25T15:49:29.037 INFO:tasks.workunit.client.0.smithi110.stdout:8/555: dwrite d14/d21/d26/d6a/d7e/fa3 [0,4194304] 0 2024-04-25T15:49:29.037 INFO:tasks.workunit.client.1.smithi193.stdout:1/64: mkdir dc 0 2024-04-25T15:49:29.037 INFO:tasks.workunit.client.1.smithi193.stdout:1/65: chown lb 8632970 1 2024-04-25T15:49:29.039 INFO:tasks.workunit.client.0.smithi110.stdout:7/409: rename d1b/d29/c6d to d1b/d29/c86 0 2024-04-25T15:49:29.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/541: dread d0/d25/f41 [0,4194304] 0 2024-04-25T15:49:29.042 INFO:tasks.workunit.client.1.smithi193.stdout:5/44: creat ff x:0 0 0 2024-04-25T15:49:29.042 INFO:tasks.workunit.client.1.smithi193.stdout:5/45: write fc [555275,110954] 0 2024-04-25T15:49:29.049 INFO:tasks.workunit.client.0.smithi110.stdout:0/441: link d1/f8b d1/d47/fa9 0 2024-04-25T15:49:29.056 INFO:tasks.workunit.client.0.smithi110.stdout:2/449: getdents de/d6f 0 2024-04-25T15:49:29.057 INFO:tasks.workunit.client.0.smithi110.stdout:8/556: dwrite d14/d4c/d55/d8d/f95 [0,4194304] 0 2024-04-25T15:49:29.058 INFO:tasks.workunit.client.0.smithi110.stdout:2/450: read de/d40/d4f/f47 [3759462,116496] 0 2024-04-25T15:49:29.060 INFO:tasks.workunit.client.0.smithi110.stdout:2/451: stat de/d40/c41 0 2024-04-25T15:49:29.062 INFO:tasks.workunit.client.0.smithi110.stdout:2/452: read - de/d40/d5b/d5c/d5f/f73 zero size 2024-04-25T15:49:29.062 INFO:tasks.workunit.client.0.smithi110.stdout:1/542: chown d0/d14/d5f/d7d/d72/c9b 24054695 1 2024-04-25T15:49:29.063 INFO:tasks.workunit.client.0.smithi110.stdout:1/543: write d0/d23/f3b [2627868,42597] 0 2024-04-25T15:49:29.063 INFO:tasks.workunit.client.1.smithi193.stdout:3/125: mkdir d8/d13/d14/d27 0 2024-04-25T15:49:29.064 INFO:tasks.workunit.client.0.smithi110.stdout:1/544: chown d0/d25/d33/d8e 1457095 1 2024-04-25T15:49:29.068 INFO:tasks.workunit.client.1.smithi193.stdout:3/126: dread d8/d13/d1f/f16 [0,4194304] 0 2024-04-25T15:49:29.068 INFO:tasks.workunit.client.0.smithi110.stdout:8/557: dread d14/d21/f92 [0,4194304] 0 2024-04-25T15:49:29.069 INFO:tasks.workunit.client.0.smithi110.stdout:7/410: chown d1b/d29/d46/c7b 2371 1 2024-04-25T15:49:29.070 INFO:tasks.workunit.client.0.smithi110.stdout:7/411: read d1b/d29/d38/d55/d7f/f78 [485433,46060] 0 2024-04-25T15:49:29.071 INFO:tasks.workunit.client.1.smithi193.stdout:3/127: dread d8/d13/d14/f21 [0,4194304] 0 2024-04-25T15:49:29.071 INFO:tasks.workunit.client.1.smithi193.stdout:3/128: write f2 [4057572,4235] 0 2024-04-25T15:49:29.073 INFO:tasks.workunit.client.1.smithi193.stdout:3/129: dread d8/d13/d14/f1d [0,4194304] 0 2024-04-25T15:49:29.073 INFO:tasks.workunit.client.1.smithi193.stdout:3/130: chown d8/cd 116643 1 2024-04-25T15:49:29.077 INFO:tasks.workunit.client.0.smithi110.stdout:7/412: dwrite d1b/d29/d38/f53 [0,4194304] 0 2024-04-25T15:49:29.091 INFO:tasks.workunit.client.1.smithi193.stdout:0/96: mknod d0/d3/d5/d7/dd/c1f 0 2024-04-25T15:49:29.092 INFO:tasks.workunit.client.0.smithi110.stdout:7/413: rmdir d1b/d29/d38 39 2024-04-25T15:49:29.095 INFO:tasks.workunit.client.0.smithi110.stdout:1/545: creat d0/d25/d33/d8e/fa6 x:0 0 0 2024-04-25T15:49:29.098 INFO:tasks.workunit.client.1.smithi193.stdout:4/80: rename db/c12 to db/c17 0 2024-04-25T15:49:29.101 INFO:tasks.workunit.client.0.smithi110.stdout:7/414: readlink d1b/d29/l4d 0 2024-04-25T15:49:29.104 INFO:tasks.workunit.client.1.smithi193.stdout:1/66: mknod dc/cd 0 2024-04-25T15:49:29.105 INFO:tasks.workunit.client.0.smithi110.stdout:8/558: link d14/d21/d26/d6a/d7e/l86 d14/d32/lbd 0 2024-04-25T15:49:29.106 INFO:tasks.workunit.client.0.smithi110.stdout:8/559: write d14/f1c [4245241,28437] 0 2024-04-25T15:49:29.106 INFO:tasks.workunit.client.1.smithi193.stdout:6/95: write dd/f15 [4589455,83237] 0 2024-04-25T15:49:29.110 INFO:tasks.workunit.client.1.smithi193.stdout:1/67: dwrite f1 [0,4194304] 0 2024-04-25T15:49:29.111 INFO:tasks.workunit.client.1.smithi193.stdout:1/68: chown dc 265510 1 2024-04-25T15:49:29.112 INFO:tasks.workunit.client.0.smithi110.stdout:7/415: mkdir d1b/d29/d87 0 2024-04-25T15:49:29.113 INFO:tasks.workunit.client.0.smithi110.stdout:5/423: truncate d4/d1d/d1f/d35/d7f/f8d 220666 0 2024-04-25T15:49:29.114 INFO:tasks.workunit.client.0.smithi110.stdout:5/424: chown d4/d1d/d1f/c37 278270473 1 2024-04-25T15:49:29.117 INFO:tasks.workunit.client.0.smithi110.stdout:8/560: unlink d14/d4c/d55/cbc 0 2024-04-25T15:49:29.119 INFO:tasks.workunit.client.1.smithi193.stdout:7/48: truncate fd 4188140 0 2024-04-25T15:49:29.121 INFO:tasks.workunit.client.0.smithi110.stdout:5/425: dwrite d4/f44 [4194304,4194304] 0 2024-04-25T15:49:29.126 INFO:tasks.workunit.client.0.smithi110.stdout:5/426: dread d4/d1d/d1f/f6e [0,4194304] 0 2024-04-25T15:49:29.128 INFO:tasks.workunit.client.1.smithi193.stdout:4/81: creat db/d13/f18 x:0 0 0 2024-04-25T15:49:29.129 INFO:tasks.workunit.client.1.smithi193.stdout:9/92: write f9 [2344912,93729] 0 2024-04-25T15:49:29.131 INFO:tasks.workunit.client.0.smithi110.stdout:9/342: sync 2024-04-25T15:49:29.133 INFO:tasks.workunit.client.0.smithi110.stdout:7/416: creat d1b/f88 x:0 0 0 2024-04-25T15:49:29.134 INFO:tasks.workunit.client.1.smithi193.stdout:3/131: symlink d8/d13/d14/d23/l28 0 2024-04-25T15:49:29.135 INFO:tasks.workunit.client.1.smithi193.stdout:3/132: chown d8/d13/d14/c19 1639085 1 2024-04-25T15:49:29.138 INFO:tasks.workunit.client.0.smithi110.stdout:9/343: dwrite d9/db/df/d25/f37 [0,4194304] 0 2024-04-25T15:49:29.138 INFO:tasks.workunit.client.1.smithi193.stdout:2/104: getdents d0/d9 0 2024-04-25T15:49:29.139 INFO:tasks.workunit.client.1.smithi193.stdout:2/105: chown d0/d9/d14/l1e 58621761 1 2024-04-25T15:49:29.139 INFO:tasks.workunit.client.0.smithi110.stdout:9/344: chown d9/db/df/d25/l32 2 1 2024-04-25T15:49:29.141 INFO:tasks.workunit.client.1.smithi193.stdout:6/96: fsync dd/f11 0 2024-04-25T15:49:29.144 INFO:tasks.workunit.client.1.smithi193.stdout:6/97: dread dd/f11 [0,4194304] 0 2024-04-25T15:49:29.147 INFO:tasks.workunit.client.1.smithi193.stdout:8/83: getdents d2/d3 0 2024-04-25T15:49:29.150 INFO:tasks.workunit.client.1.smithi193.stdout:7/49: unlink c4 0 2024-04-25T15:49:29.150 INFO:tasks.workunit.client.1.smithi193.stdout:8/84: dread d2/d3/ff [0,4194304] 0 2024-04-25T15:49:29.152 INFO:tasks.workunit.client.1.smithi193.stdout:8/85: write d2/f11 [621143,109389] 0 2024-04-25T15:49:29.152 INFO:tasks.workunit.client.1.smithi193.stdout:0/97: unlink d0/d3/d5/d7/l15 0 2024-04-25T15:49:29.153 INFO:tasks.workunit.client.1.smithi193.stdout:4/82: symlink db/d13/l19 0 2024-04-25T15:49:29.154 INFO:tasks.workunit.client.1.smithi193.stdout:0/98: write d0/f12 [218128,14575] 0 2024-04-25T15:49:29.155 INFO:tasks.workunit.client.1.smithi193.stdout:9/93: rename l8 to df/l12 0 2024-04-25T15:49:29.157 INFO:tasks.workunit.client.1.smithi193.stdout:9/94: fsync f6 0 2024-04-25T15:49:29.157 INFO:tasks.workunit.client.1.smithi193.stdout:8/86: truncate d2/d3/db/f1a 164075 0 2024-04-25T15:49:29.158 INFO:tasks.workunit.client.1.smithi193.stdout:2/106: unlink d0/f27 0 2024-04-25T15:49:29.159 INFO:tasks.workunit.client.1.smithi193.stdout:5/46: getdents . 0 2024-04-25T15:49:29.160 INFO:tasks.workunit.client.1.smithi193.stdout:7/50: dwrite f3 [0,4194304] 0 2024-04-25T15:49:29.161 INFO:tasks.workunit.client.1.smithi193.stdout:6/98: mkdir dd/d12/d19 0 2024-04-25T15:49:29.162 INFO:tasks.workunit.client.1.smithi193.stdout:5/47: truncate ff 134912 0 2024-04-25T15:49:29.164 INFO:tasks.workunit.client.1.smithi193.stdout:4/83: dwrite f9 [4194304,4194304] 0 2024-04-25T15:49:29.165 INFO:tasks.workunit.client.1.smithi193.stdout:4/84: readlink l7 0 2024-04-25T15:49:29.165 INFO:tasks.workunit.client.1.smithi193.stdout:9/95: dread f9 [0,4194304] 0 2024-04-25T15:49:29.165 INFO:tasks.workunit.client.1.smithi193.stdout:3/133: creat d8/d13/d14/d27/f29 x:0 0 0 2024-04-25T15:49:29.165 INFO:tasks.workunit.client.1.smithi193.stdout:3/134: stat d8/d13/d14/d23/l28 0 2024-04-25T15:49:29.167 INFO:tasks.workunit.client.1.smithi193.stdout:0/99: symlink d0/d3/d5/l20 0 2024-04-25T15:49:29.169 INFO:tasks.workunit.client.1.smithi193.stdout:8/87: mknod d2/c1d 0 2024-04-25T15:49:29.191 INFO:tasks.workunit.client.0.smithi110.stdout:9/345: mkdir d9/db/df/d2c/d3c/d56/d6a 0 2024-04-25T15:49:29.192 INFO:tasks.workunit.client.0.smithi110.stdout:0/442: write d1/f4d [457443,78053] 0 2024-04-25T15:49:29.193 INFO:tasks.workunit.client.0.smithi110.stdout:0/443: readlink d1/d20/l39 0 2024-04-25T15:49:29.194 INFO:tasks.workunit.client.1.smithi193.stdout:2/107: mknod d0/d9/d14/c2d 0 2024-04-25T15:49:29.194 INFO:tasks.workunit.client.1.smithi193.stdout:5/48: mknod c10 0 2024-04-25T15:49:29.195 INFO:tasks.workunit.client.1.smithi193.stdout:4/85: rename db/l11 to db/d13/l1a 0 2024-04-25T15:49:29.196 INFO:tasks.workunit.client.1.smithi193.stdout:0/100: symlink d0/d3/d5/d7/l21 0 2024-04-25T15:49:29.196 INFO:tasks.workunit.client.1.smithi193.stdout:4/86: read - db/fe zero size 2024-04-25T15:49:29.197 INFO:tasks.workunit.client.0.smithi110.stdout:1/546: write d0/d25/f6 [1008444,36666] 0 2024-04-25T15:49:29.197 INFO:tasks.workunit.client.1.smithi193.stdout:4/87: stat db/f15 0 2024-04-25T15:49:29.197 INFO:tasks.workunit.client.1.smithi193.stdout:0/101: dread - d0/d3/d5/d7/dd/d16/f1c zero size 2024-04-25T15:49:29.197 INFO:tasks.workunit.client.1.smithi193.stdout:1/69: getdents dc 0 2024-04-25T15:49:29.198 INFO:tasks.workunit.client.0.smithi110.stdout:1/547: write d0/d14/d5f/d7d/f64 [1319890,104584] 0 2024-04-25T15:49:29.203 INFO:tasks.workunit.client.1.smithi193.stdout:0/102: dwrite d0/f12 [0,4194304] 0 2024-04-25T15:49:29.204 INFO:tasks.workunit.client.1.smithi193.stdout:8/88: truncate d2/d3/ff 4274608 0 2024-04-25T15:49:29.205 INFO:tasks.workunit.client.1.smithi193.stdout:8/89: fdatasync d2/d3/db/f19 0 2024-04-25T15:49:29.209 INFO:tasks.workunit.client.0.smithi110.stdout:5/427: getdents d4/d1d/d1f/d35 0 2024-04-25T15:49:29.210 INFO:tasks.workunit.client.1.smithi193.stdout:9/96: fsync fc 0 2024-04-25T15:49:29.213 INFO:tasks.workunit.client.0.smithi110.stdout:7/417: mkdir d1b/d29/d38/d55/d7d/d89 0 2024-04-25T15:49:29.214 INFO:tasks.workunit.client.0.smithi110.stdout:7/418: chown d1b/d29/d46/f5c 3209 1 2024-04-25T15:49:29.214 INFO:tasks.workunit.client.1.smithi193.stdout:3/135: symlink d8/d13/l2a 0 2024-04-25T15:49:29.215 INFO:tasks.workunit.client.1.smithi193.stdout:3/136: readlink d8/d13/l2a 0 2024-04-25T15:49:29.215 INFO:tasks.workunit.client.0.smithi110.stdout:7/419: rename d1b/d29/d46 to d1b/d29/d46/d8a 22 2024-04-25T15:49:29.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.216+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f89fc05f5c0 con 0x7f89fc10c0b0 2024-04-25T15:49:29.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.217+0000 7f89edffb700 1 -- 172.21.15.110:0/1054259809 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f89f005a060 con 0x7f89fc10c0b0 2024-04-25T15:49:29.218 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:49:29.219 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:49:29.219 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:49:29.219 INFO:tasks.workunit.client.1.smithi193.stdout:2/108: creat d0/d9/d15/f2e x:0 0 0 2024-04-25T15:49:29.220 INFO:tasks.workunit.client.0.smithi110.stdout:0/444: dread d1/d23/d30/d33/d35/f12 [0,4194304] 0 2024-04-25T15:49:29.222 INFO:tasks.workunit.client.0.smithi110.stdout:9/346: dwrite d9/db/df/d25/f37 [4194304,4194304] 0 2024-04-25T15:49:29.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.221+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f89e806fd00 msgr2=0x7f89e80721b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:29.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.221+0000 7f8a03582700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f89e806fd00 0x7f89e80721b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f89fc10b900 tx=0x7f89f800b040 comp rx=0 tx=0).stop 2024-04-25T15:49:29.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.221+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89fc10c0b0 msgr2=0x7f89fc10cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:29.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.221+0000 7f8a03582700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89fc10c0b0 0x7f89fc10cf50 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f89f0004940 tx=0x7f89f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:29.223 INFO:tasks.workunit.client.0.smithi110.stdout:1/548: unlink d0/d14/d5f/c66 0 2024-04-25T15:49:29.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.222+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 shutdown_connections 2024-04-25T15:49:29.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.222+0000 7f8a03582700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f89e806fd00 0x7f89e80721b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:29.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.222+0000 7f8a03582700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89fc066200 0x7f89fc10bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:29.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.222+0000 7f8a03582700 1 --2- 172.21.15.110:0/1054259809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f89fc10c0b0 0x7f89fc10cf50 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:29.224 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.222+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 >> 172.21.15.110:0/1054259809 conn(0x7f89fc0f7430 msgr2=0x7f89fc0fb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:29.224 INFO:tasks.workunit.client.0.smithi110.stdout:9/347: write f8 [4217273,117917] 0 2024-04-25T15:49:29.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.223+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 shutdown_connections 2024-04-25T15:49:29.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:29.223+0000 7f8a03582700 1 -- 172.21.15.110:0/1054259809 wait complete. 2024-04-25T15:49:29.225 INFO:tasks.workunit.client.0.smithi110.stdout:0/445: read d1/d23/d30/d33/d5b/d83/f5d [69885,51625] 0 2024-04-25T15:49:29.225 INFO:tasks.workunit.client.1.smithi193.stdout:4/88: symlink db/d13/l1b 0 2024-04-25T15:49:29.227 INFO:tasks.workunit.client.1.smithi193.stdout:2/109: dwrite d0/df/d16/f23 [0,4194304] 0 2024-04-25T15:49:29.227 INFO:tasks.workunit.client.1.smithi193.stdout:4/89: write db/fc [4996171,50976] 0 2024-04-25T15:49:29.234 INFO:tasks.workunit.client.1.smithi193.stdout:1/70: creat dc/fe x:0 0 0 2024-04-25T15:49:29.236 INFO:tasks.workunit.client.0.smithi110.stdout:5/428: dwrite d4/d1d/d1f/d35/f7d [4194304,4194304] 0 2024-04-25T15:49:29.238 INFO:tasks.workunit.client.0.smithi110.stdout:5/429: chown d4/d1d/d1f/d35/d3d/f53 1 1 2024-04-25T15:49:29.239 INFO:tasks.workunit.client.0.smithi110.stdout:5/430: fsync d4/d1d/d1f/d56/f7e 0 2024-04-25T15:49:29.240 INFO:tasks.workunit.client.0.smithi110.stdout:1/549: mknod d0/d25/d33/ca7 0 2024-04-25T15:49:29.249 INFO:tasks.workunit.client.0.smithi110.stdout:5/431: dwrite d4/d1d/d1f/d35/d3d/f53 [0,4194304] 0 2024-04-25T15:49:29.249 INFO:tasks.workunit.client.1.smithi193.stdout:8/90: rename c1 to d2/c1e 0 2024-04-25T15:49:29.258 INFO:tasks.workunit.client.1.smithi193.stdout:3/137: mknod d8/c2b 0 2024-04-25T15:49:29.258 INFO:tasks.workunit.client.1.smithi193.stdout:3/138: dread - d8/d13/d14/f20 zero size 2024-04-25T15:49:29.259 INFO:tasks.workunit.client.1.smithi193.stdout:3/139: dread d8/ff [0,4194304] 0 2024-04-25T15:49:29.260 INFO:tasks.workunit.client.1.smithi193.stdout:3/140: chown d8/c2b 233 1 2024-04-25T15:49:29.262 INFO:tasks.workunit.client.1.smithi193.stdout:4/90: fdatasync f2 0 2024-04-25T15:49:29.266 INFO:tasks.workunit.client.0.smithi110.stdout:7/420: rename d1b/d29/d38/d55/d7d to d1b/d29/d38/d8b 0 2024-04-25T15:49:29.267 INFO:tasks.workunit.client.1.smithi193.stdout:7/51: getdents . 0 2024-04-25T15:49:29.267 INFO:tasks.workunit.client.0.smithi110.stdout:7/421: write d1b/f36 [2136059,29510] 0 2024-04-25T15:49:29.268 INFO:tasks.workunit.client.1.smithi193.stdout:7/52: read - fc zero size 2024-04-25T15:49:29.269 INFO:tasks.workunit.client.1.smithi193.stdout:0/103: creat d0/d3/d5/d7/dd/d16/d18/f22 x:0 0 0 2024-04-25T15:49:29.270 INFO:tasks.workunit.client.1.smithi193.stdout:5/49: dwrite f4 [0,4194304] 0 2024-04-25T15:49:29.270 INFO:tasks.workunit.client.1.smithi193.stdout:1/71: rename dc to dc/df 22 2024-04-25T15:49:29.270 INFO:tasks.workunit.client.1.smithi193.stdout:5/50: readlink l2 0 2024-04-25T15:49:29.271 INFO:tasks.workunit.client.0.smithi110.stdout:7/422: read d1b/f36 [1025016,95089] 0 2024-04-25T15:49:29.271 INFO:tasks.workunit.client.0.smithi110.stdout:7/423: write d1b/f36 [3477526,88202] 0 2024-04-25T15:49:29.271 INFO:tasks.workunit.client.1.smithi193.stdout:5/51: fsync ff 0 2024-04-25T15:49:29.271 INFO:tasks.workunit.client.1.smithi193.stdout:5/52: readlink ld 0 2024-04-25T15:49:29.274 INFO:tasks.workunit.client.1.smithi193.stdout:9/97: mkdir df/d11/d13 0 2024-04-25T15:49:29.275 INFO:tasks.workunit.client.1.smithi193.stdout:7/53: dread f8 [0,4194304] 0 2024-04-25T15:49:29.276 INFO:tasks.workunit.client.1.smithi193.stdout:3/141: symlink d8/l2c 0 2024-04-25T15:49:29.277 INFO:tasks.workunit.client.0.smithi110.stdout:9/348: write d9/db/df/d25/f51 [1084299,53398] 0 2024-04-25T15:49:29.279 INFO:tasks.workunit.client.1.smithi193.stdout:5/53: dwrite fe [0,4194304] 0 2024-04-25T15:49:29.280 INFO:tasks.workunit.client.1.smithi193.stdout:5/54: dread ff [0,4194304] 0 2024-04-25T15:49:29.281 INFO:tasks.workunit.client.1.smithi193.stdout:6/99: sync 2024-04-25T15:49:29.282 INFO:tasks.workunit.client.1.smithi193.stdout:4/91: mknod db/c1c 0 2024-04-25T15:49:29.289 INFO:tasks.workunit.client.1.smithi193.stdout:0/104: mknod d0/c23 0 2024-04-25T15:49:29.292 INFO:tasks.workunit.client.1.smithi193.stdout:6/100: dwrite fb [0,4194304] 0 2024-04-25T15:49:29.292 INFO:tasks.workunit.client.1.smithi193.stdout:4/92: dwrite db/fe [0,4194304] 0 2024-04-25T15:49:29.296 INFO:tasks.workunit.client.1.smithi193.stdout:9/98: rmdir df/d11 39 2024-04-25T15:49:29.297 INFO:tasks.workunit.client.1.smithi193.stdout:9/99: chown f9 25802 1 2024-04-25T15:49:29.298 INFO:tasks.workunit.client.1.smithi193.stdout:7/54: creat fe x:0 0 0 2024-04-25T15:49:29.298 INFO:tasks.workunit.client.1.smithi193.stdout:7/55: readlink la 0 2024-04-25T15:49:29.300 INFO:tasks.workunit.client.1.smithi193.stdout:8/91: dread d2/d3/ff [0,4194304] 0 2024-04-25T15:49:29.304 INFO:tasks.workunit.client.1.smithi193.stdout:2/110: link d0/l12 d0/df/d24/l2f 0 2024-04-25T15:49:29.305 INFO:tasks.workunit.client.1.smithi193.stdout:2/111: write d0/fc [1839676,20534] 0 2024-04-25T15:49:29.312 INFO:tasks.workunit.client.1.smithi193.stdout:0/105: creat d0/d3/d5/f24 x:0 0 0 2024-04-25T15:49:29.313 INFO:tasks.workunit.client.0.smithi110.stdout:0/446: symlink d1/d23/d30/d33/d5b/d83/d87/da8/laa 0 2024-04-25T15:49:29.313 INFO:tasks.workunit.client.1.smithi193.stdout:0/106: chown d0/d3/d5/d7/dd/d16 379 1 2024-04-25T15:49:29.313 INFO:tasks.workunit.client.1.smithi193.stdout:1/72: symlink dc/l10 0 2024-04-25T15:49:29.314 INFO:tasks.workunit.client.1.smithi193.stdout:0/107: write d0/d3/d5/d7/dd/d16/d18/f22 [257412,87243] 0 2024-04-25T15:49:29.315 INFO:tasks.workunit.client.0.smithi110.stdout:5/432: creat d4/d1d/d1f/f8f x:0 0 0 2024-04-25T15:49:29.316 INFO:tasks.workunit.client.1.smithi193.stdout:6/101: creat dd/f1a x:0 0 0 2024-04-25T15:49:29.325 INFO:tasks.workunit.client.1.smithi193.stdout:8/92: mkdir d2/d1f 0 2024-04-25T15:49:29.325 INFO:tasks.workunit.client.1.smithi193.stdout:4/93: dwrite f1 [4194304,4194304] 0 2024-04-25T15:49:29.326 INFO:tasks.workunit.client.0.smithi110.stdout:5/433: dread d4/f6d [0,4194304] 0 2024-04-25T15:49:29.327 INFO:tasks.workunit.client.0.smithi110.stdout:0/447: symlink d1/d23/lab 0 2024-04-25T15:49:29.329 INFO:tasks.workunit.client.0.smithi110.stdout:1/550: dwrite d0/d14/f29 [0,4194304] 0 2024-04-25T15:49:29.330 INFO:tasks.workunit.client.0.smithi110.stdout:0/448: truncate d1/f4d 613592 0 2024-04-25T15:49:29.331 INFO:tasks.workunit.client.0.smithi110.stdout:5/434: read d4/d1d/f34 [3520527,123645] 0 2024-04-25T15:49:29.334 INFO:tasks.workunit.client.1.smithi193.stdout:2/112: rename d0/df/d16 to d0/d9/d30 0 2024-04-25T15:49:29.335 INFO:tasks.workunit.client.1.smithi193.stdout:2/113: write d0/fc [1760397,42997] 0 2024-04-25T15:49:29.338 INFO:tasks.workunit.client.1.smithi193.stdout:6/102: creat dd/f1b x:0 0 0 2024-04-25T15:49:29.341 INFO:tasks.workunit.client.1.smithi193.stdout:9/100: creat df/d11/f14 x:0 0 0 2024-04-25T15:49:29.341 INFO:tasks.workunit.client.1.smithi193.stdout:9/101: chown f4 7590 1 2024-04-25T15:49:29.343 INFO:tasks.workunit.client.1.smithi193.stdout:2/114: dwrite d0/fc [0,4194304] 0 2024-04-25T15:49:29.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/142: write d8/f15 [974567,111992] 0 2024-04-25T15:49:29.346 INFO:tasks.workunit.client.1.smithi193.stdout:8/93: mkdir d2/d3/d20 0 2024-04-25T15:49:29.348 INFO:tasks.workunit.client.0.smithi110.stdout:5/435: fsync d4/d1d/d1f/d35/f39 0 2024-04-25T15:49:29.349 INFO:tasks.workunit.client.0.smithi110.stdout:5/436: chown d4/c72 2 1 2024-04-25T15:49:29.350 INFO:tasks.workunit.client.1.smithi193.stdout:5/55: truncate f4 2245576 0 2024-04-25T15:49:29.350 INFO:tasks.workunit.client.1.smithi193.stdout:4/94: creat db/d13/f1d x:0 0 0 2024-04-25T15:49:29.353 INFO:tasks.workunit.client.1.smithi193.stdout:4/95: write db/fe [4729193,113507] 0 2024-04-25T15:49:29.353 INFO:tasks.workunit.client.1.smithi193.stdout:3/143: dwrite d8/d13/d1f/f22 [0,4194304] 0 2024-04-25T15:49:29.354 INFO:tasks.workunit.client.1.smithi193.stdout:3/144: readlink d8/d13/l2a 0 2024-04-25T15:49:29.354 INFO:tasks.workunit.client.1.smithi193.stdout:5/56: dread fb [0,4194304] 0 2024-04-25T15:49:29.357 INFO:tasks.workunit.client.0.smithi110.stdout:9/349: dwrite d9/f24 [0,4194304] 0 2024-04-25T15:49:29.357 INFO:tasks.workunit.client.0.smithi110.stdout:4/521: sync 2024-04-25T15:49:29.357 INFO:tasks.workunit.client.0.smithi110.stdout:6/390: sync 2024-04-25T15:49:29.358 INFO:tasks.workunit.client.0.smithi110.stdout:4/522: fsync d13/d24/f2a 0 2024-04-25T15:49:29.359 INFO:tasks.workunit.client.0.smithi110.stdout:9/350: chown d9/db/df/d4d 234866 1 2024-04-25T15:49:29.359 INFO:tasks.workunit.client.0.smithi110.stdout:6/391: chown d1/db/d24/f29 544957147 1 2024-04-25T15:49:29.361 INFO:tasks.workunit.client.0.smithi110.stdout:1/551: symlink d0/d23/la8 0 2024-04-25T15:49:29.368 INFO:tasks.workunit.client.1.smithi193.stdout:0/108: truncate d0/d3/d5/d7/fb 1561669 0 2024-04-25T15:49:29.370 INFO:tasks.workunit.client.0.smithi110.stdout:7/424: getdents d1b/d29/d38/d55/d7f 0 2024-04-25T15:49:29.371 INFO:tasks.workunit.client.1.smithi193.stdout:2/115: write d0/df/f26 [4220828,14901] 0 2024-04-25T15:49:29.373 INFO:tasks.workunit.client.0.smithi110.stdout:5/437: dwrite d4/d1d/d1f/f47 [0,4194304] 0 2024-04-25T15:49:29.373 INFO:tasks.workunit.client.1.smithi193.stdout:8/94: truncate d2/d3/ff 191783 0 2024-04-25T15:49:29.375 INFO:tasks.workunit.client.0.smithi110.stdout:0/449: mkdir d1/d23/d30/d33/d5b/dac 0 2024-04-25T15:49:29.376 INFO:tasks.workunit.client.0.smithi110.stdout:0/450: chown d1/f1c 8792769 1 2024-04-25T15:49:29.379 INFO:tasks.workunit.client.0.smithi110.stdout:1/552: dread d0/d23/f8a [0,4194304] 0 2024-04-25T15:49:29.380 INFO:tasks.workunit.client.0.smithi110.stdout:4/523: creat d13/d24/d2d/d55/faa x:0 0 0 2024-04-25T15:49:29.382 INFO:tasks.workunit.client.0.smithi110.stdout:9/351: creat d9/db/df/d55/f6b x:0 0 0 2024-04-25T15:49:29.382 INFO:tasks.workunit.client.0.smithi110.stdout:4/524: write d13/d24/d3c/d56/f8e [956105,72040] 0 2024-04-25T15:49:29.387 INFO:tasks.workunit.client.1.smithi193.stdout:5/57: rename fb to f11 0 2024-04-25T15:49:29.390 INFO:tasks.workunit.client.0.smithi110.stdout:7/425: read fb [925958,86230] 0 2024-04-25T15:49:29.391 INFO:tasks.workunit.client.1.smithi193.stdout:6/103: mkdir dd/d12/d19/d1c 0 2024-04-25T15:49:29.392 INFO:tasks.workunit.client.0.smithi110.stdout:5/438: mkdir d4/d15/d51/d90 0 2024-04-25T15:49:29.393 INFO:tasks.workunit.client.1.smithi193.stdout:7/56: getdents . 0 2024-04-25T15:49:29.394 INFO:tasks.workunit.client.1.smithi193.stdout:5/58: dwrite ff [0,4194304] 0 2024-04-25T15:49:29.397 INFO:tasks.workunit.client.0.smithi110.stdout:7/426: dwrite f14 [0,4194304] 0 2024-04-25T15:49:29.398 INFO:tasks.workunit.client.1.smithi193.stdout:3/145: write d8/f10 [2097170,39061] 0 2024-04-25T15:49:29.398 INFO:tasks.workunit.client.0.smithi110.stdout:1/553: mkdir d0/d25/d33/da9 0 2024-04-25T15:49:29.399 INFO:tasks.workunit.client.1.smithi193.stdout:0/109: chown d0/d3/d5/l11 265238 1 2024-04-25T15:49:29.400 INFO:tasks.workunit.client.0.smithi110.stdout:9/352: rmdir d9/db/df 39 2024-04-25T15:49:29.402 INFO:tasks.workunit.client.0.smithi110.stdout:6/392: dwrite d1/db/f1a [0,4194304] 0 2024-04-25T15:49:29.405 INFO:tasks.workunit.client.1.smithi193.stdout:1/73: sync 2024-04-25T15:49:29.405 INFO:tasks.workunit.client.1.smithi193.stdout:1/74: fsync f6 0 2024-04-25T15:49:29.410 INFO:tasks.workunit.client.0.smithi110.stdout:2/453: sync 2024-04-25T15:49:29.411 INFO:tasks.workunit.client.0.smithi110.stdout:3/507: sync 2024-04-25T15:49:29.412 INFO:tasks.workunit.client.0.smithi110.stdout:0/451: unlink d1/d6c/d7f/l48 0 2024-04-25T15:49:29.418 INFO:tasks.workunit.client.1.smithi193.stdout:7/57: mknod cf 0 2024-04-25T15:49:29.420 INFO:tasks.workunit.client.0.smithi110.stdout:0/452: dwrite d1/d23/d30/d33/d5b/d83/f84 [0,4194304] 0 2024-04-25T15:49:29.421 INFO:tasks.workunit.client.0.smithi110.stdout:6/393: rename d1/db/d24/d32 to d1/dd/d3c/d91 0 2024-04-25T15:49:29.422 INFO:tasks.workunit.client.1.smithi193.stdout:5/59: creat f12 x:0 0 0 2024-04-25T15:49:29.423 INFO:tasks.workunit.client.1.smithi193.stdout:6/104: dwrite f6 [0,4194304] 0 2024-04-25T15:49:29.424 INFO:tasks.workunit.client.1.smithi193.stdout:5/60: write fe [3504929,107859] 0 2024-04-25T15:49:29.424 INFO:tasks.workunit.client.1.smithi193.stdout:6/105: read dd/f11 [2084597,28584] 0 2024-04-25T15:49:29.425 INFO:tasks.workunit.client.0.smithi110.stdout:4/525: symlink d13/d24/d2d/lab 0 2024-04-25T15:49:29.426 INFO:tasks.workunit.client.1.smithi193.stdout:9/102: rmdir df/d11/d13 0 2024-04-25T15:49:29.428 INFO:tasks.workunit.client.0.smithi110.stdout:5/439: write d4/d1d/f34 [3598846,2921] 0 2024-04-25T15:49:29.430 INFO:tasks.workunit.client.1.smithi193.stdout:4/96: truncate db/fc 1853385 0 2024-04-25T15:49:29.431 INFO:tasks.workunit.client.1.smithi193.stdout:5/61: dwrite fe [0,4194304] 0 2024-04-25T15:49:29.431 INFO:tasks.workunit.client.1.smithi193.stdout:5/62: chown la 24499440 1 2024-04-25T15:49:29.435 INFO:tasks.workunit.client.0.smithi110.stdout:5/440: dwrite d4/d2c/f27 [0,4194304] 0 2024-04-25T15:49:29.438 INFO:tasks.workunit.client.0.smithi110.stdout:7/427: creat d1b/d29/d38/d8b/d89/f8c x:0 0 0 2024-04-25T15:49:29.438 INFO:tasks.workunit.client.1.smithi193.stdout:2/116: truncate d0/d9/d15/f1b 4516568 0 2024-04-25T15:49:29.439 INFO:tasks.workunit.client.1.smithi193.stdout:2/117: chown d0/df/d24 220543850 1 2024-04-25T15:49:29.443 INFO:tasks.workunit.client.1.smithi193.stdout:8/95: symlink d2/d3/d20/l21 0 2024-04-25T15:49:29.443 INFO:tasks.workunit.client.0.smithi110.stdout:6/394: mknod d1/d12/d3d/c92 0 2024-04-25T15:49:29.444 INFO:tasks.workunit.client.1.smithi193.stdout:8/96: read d2/d3/f16 [1684737,76250] 0 2024-04-25T15:49:29.444 INFO:tasks.workunit.client.0.smithi110.stdout:9/353: dwrite d9/d2d/f30 [0,4194304] 0 2024-04-25T15:49:29.446 INFO:tasks.workunit.client.1.smithi193.stdout:7/58: creat f10 x:0 0 0 2024-04-25T15:49:29.454 INFO:tasks.workunit.client.0.smithi110.stdout:1/554: creat d0/d14/d5f/faa x:0 0 0 2024-04-25T15:49:29.455 INFO:tasks.workunit.client.0.smithi110.stdout:1/555: chown d0/d25/f41 848097 1 2024-04-25T15:49:29.457 INFO:tasks.workunit.client.1.smithi193.stdout:6/106: creat dd/d12/d19/f1d x:0 0 0 2024-04-25T15:49:29.458 INFO:tasks.workunit.client.0.smithi110.stdout:5/441: read d4/f8 [43468,77095] 0 2024-04-25T15:49:29.459 INFO:tasks.workunit.client.0.smithi110.stdout:7/428: creat d1b/d29/d38/d55/f8d x:0 0 0 2024-04-25T15:49:29.462 INFO:tasks.workunit.client.0.smithi110.stdout:0/453: unlink d1/d23/d30/d33/d5b/d72/d77/c85 0 2024-04-25T15:49:29.464 INFO:tasks.workunit.client.0.smithi110.stdout:3/508: write d0/d16/f24 [2193182,38942] 0 2024-04-25T15:49:29.466 INFO:tasks.workunit.client.1.smithi193.stdout:4/97: unlink db/d13/f18 0 2024-04-25T15:49:29.467 INFO:tasks.workunit.client.0.smithi110.stdout:7/429: dwrite d1b/d29/d38/d8b/d89/f8c [0,4194304] 0 2024-04-25T15:49:29.467 INFO:tasks.workunit.client.1.smithi193.stdout:4/98: write f9 [1244501,35227] 0 2024-04-25T15:49:29.472 INFO:tasks.workunit.client.1.smithi193.stdout:2/118: fdatasync d0/df/f2b 0 2024-04-25T15:49:29.473 INFO:tasks.workunit.client.1.smithi193.stdout:1/75: symlink dc/l11 0 2024-04-25T15:49:29.474 INFO:tasks.workunit.client.1.smithi193.stdout:4/99: dwrite f9 [4194304,4194304] 0 2024-04-25T15:49:29.476 INFO:tasks.workunit.client.1.smithi193.stdout:8/97: unlink d2/l14 0 2024-04-25T15:49:29.477 INFO:tasks.workunit.client.1.smithi193.stdout:7/59: mknod c11 0 2024-04-25T15:49:29.478 INFO:tasks.workunit.client.0.smithi110.stdout:2/454: rename de/c3f to de/d40/d6e/d52/d6d/c87 0 2024-04-25T15:49:29.480 INFO:tasks.workunit.client.1.smithi193.stdout:4/100: dread f4 [0,4194304] 0 2024-04-25T15:49:29.480 INFO:tasks.workunit.client.1.smithi193.stdout:6/107: creat dd/d12/f1e x:0 0 0 2024-04-25T15:49:29.481 INFO:tasks.workunit.client.1.smithi193.stdout:6/108: chown dd/c10 56839 1 2024-04-25T15:49:29.481 INFO:tasks.workunit.client.1.smithi193.stdout:9/103: mknod df/c15 0 2024-04-25T15:49:29.483 INFO:tasks.workunit.client.0.smithi110.stdout:1/556: dread d0/d14/d5f/f65 [0,4194304] 0 2024-04-25T15:49:29.485 INFO:tasks.workunit.client.0.smithi110.stdout:1/557: write d0/d14/d5f/d6f/fa3 [823981,61373] 0 2024-04-25T15:49:29.485 INFO:tasks.workunit.client.1.smithi193.stdout:3/146: creat d8/d13/f2d x:0 0 0 2024-04-25T15:49:29.491 INFO:tasks.workunit.client.0.smithi110.stdout:7/430: fdatasync d1b/d29/d38/d8b/d89/f8c 0 2024-04-25T15:49:29.495 INFO:tasks.workunit.client.0.smithi110.stdout:9/354: creat d9/db/df/f6c x:0 0 0 2024-04-25T15:49:29.498 INFO:tasks.workunit.client.0.smithi110.stdout:7/431: dwrite d1b/f48 [0,4194304] 0 2024-04-25T15:49:29.500 INFO:tasks.workunit.client.0.smithi110.stdout:8/561: sync 2024-04-25T15:49:29.503 INFO:tasks.workunit.client.1.smithi193.stdout:5/63: rename f4 to f13 0 2024-04-25T15:49:29.505 INFO:tasks.workunit.client.1.smithi193.stdout:0/110: link d0/la d0/d3/d5/d7/dd/l25 0 2024-04-25T15:49:29.505 INFO:tasks.workunit.client.0.smithi110.stdout:9/355: dwrite d9/db/df/d2c/f36 [4194304,4194304] 0 2024-04-25T15:49:29.511 INFO:tasks.workunit.client.0.smithi110.stdout:3/509: unlink d0/d7/f29 0 2024-04-25T15:49:29.513 INFO:tasks.workunit.client.0.smithi110.stdout:6/395: dwrite d1/fe [0,4194304] 0 2024-04-25T15:49:29.516 INFO:tasks.workunit.client.0.smithi110.stdout:5/442: truncate d4/d1d/f65 169536 0 2024-04-25T15:49:29.517 INFO:tasks.workunit.client.0.smithi110.stdout:6/396: read d1/db/d24/d2d/f34 [241241,9949] 0 2024-04-25T15:49:29.518 INFO:tasks.workunit.client.1.smithi193.stdout:8/98: creat d2/d3/d20/f22 x:0 0 0 2024-04-25T15:49:29.519 INFO:tasks.workunit.client.1.smithi193.stdout:2/119: dwrite d0/d9/d30/f23 [4194304,4194304] 0 2024-04-25T15:49:29.519 INFO:tasks.workunit.client.0.smithi110.stdout:2/455: write de/f28 [5030352,96450] 0 2024-04-25T15:49:29.521 INFO:tasks.workunit.client.1.smithi193.stdout:2/120: write d0/df/f2b [2856762,119233] 0 2024-04-25T15:49:29.521 INFO:tasks.workunit.client.0.smithi110.stdout:1/558: dwrite d0/d14/f7a [0,4194304] 0 2024-04-25T15:49:29.521 INFO:tasks.workunit.client.1.smithi193.stdout:7/60: mkdir d12 0 2024-04-25T15:49:29.523 INFO:tasks.workunit.client.1.smithi193.stdout:4/101: mkdir db/d13/d1e 0 2024-04-25T15:49:29.528 INFO:tasks.workunit.client.0.smithi110.stdout:4/526: getdents d13/d24 0 2024-04-25T15:49:29.529 INFO:tasks.workunit.client.1.smithi193.stdout:3/147: dread d8/d13/d14/f1d [0,4194304] 0 2024-04-25T15:49:29.529 INFO:tasks.workunit.client.0.smithi110.stdout:1/559: dwrite d0/d14/f2c [4194304,4194304] 0 2024-04-25T15:49:29.533 INFO:tasks.workunit.client.1.smithi193.stdout:5/64: mknod c14 0 2024-04-25T15:49:29.537 INFO:tasks.workunit.client.1.smithi193.stdout:0/111: mkdir d0/d3/d5/d7/dd/d16/d26 0 2024-04-25T15:49:29.539 INFO:tasks.workunit.client.1.smithi193.stdout:0/112: readlink d0/d3/d5/d7/dd/l1b 0 2024-04-25T15:49:29.539 INFO:tasks.workunit.client.1.smithi193.stdout:0/113: chown d0/d3/d5/d7 47146 1 2024-04-25T15:49:29.541 INFO:tasks.workunit.client.1.smithi193.stdout:0/114: fsync d0/d3/d5/d7/dd/f1d 0 2024-04-25T15:49:29.543 INFO:tasks.workunit.client.1.smithi193.stdout:0/115: write d0/d3/d5/d7/dd/d16/d18/f22 [1196948,92633] 0 2024-04-25T15:49:29.549 INFO:tasks.workunit.client.0.smithi110.stdout:9/356: dread d9/db/df/d2c/f36 [0,4194304] 0 2024-04-25T15:49:29.551 INFO:tasks.workunit.client.0.smithi110.stdout:1/560: dread d0/d14/f4b [0,4194304] 0 2024-04-25T15:49:29.552 INFO:tasks.workunit.client.0.smithi110.stdout:6/397: write d1/d48/f6c [1975045,91131] 0 2024-04-25T15:49:29.553 INFO:tasks.workunit.client.1.smithi193.stdout:2/121: rename d0/df/d24 to d0/d9/d15/d31 0 2024-04-25T15:49:29.555 INFO:tasks.workunit.client.0.smithi110.stdout:2/456: mknod de/d40/d5b/c88 0 2024-04-25T15:49:29.556 INFO:tasks.workunit.client.1.smithi193.stdout:6/109: creat dd/d12/d19/d1c/f1f x:0 0 0 2024-04-25T15:49:29.557 INFO:tasks.workunit.client.0.smithi110.stdout:1/561: dread d0/d25/d33/f45 [0,4194304] 0 2024-04-25T15:49:29.558 INFO:tasks.workunit.client.0.smithi110.stdout:1/562: stat d0/d23/f13 0 2024-04-25T15:49:29.558 INFO:tasks.workunit.client.1.smithi193.stdout:9/104: mkdir df/d16 0 2024-04-25T15:49:29.558 INFO:tasks.workunit.client.1.smithi193.stdout:9/105: chown f4 435497764 1 2024-04-25T15:49:29.559 INFO:tasks.workunit.client.0.smithi110.stdout:4/527: truncate d13/d24/f9f 1035847 0 2024-04-25T15:49:29.560 INFO:tasks.workunit.client.0.smithi110.stdout:4/528: write d13/d24/f45 [3112347,39615] 0 2024-04-25T15:49:29.560 INFO:tasks.workunit.client.0.smithi110.stdout:7/432: mkdir d1b/d29/d87/d8e 0 2024-04-25T15:49:29.561 INFO:tasks.workunit.client.1.smithi193.stdout:5/65: rename ff to f15 0 2024-04-25T15:49:29.563 INFO:tasks.workunit.client.1.smithi193.stdout:1/76: creat dc/f12 x:0 0 0 2024-04-25T15:49:29.563 INFO:tasks.workunit.client.0.smithi110.stdout:8/562: symlink d14/d21/d26/d6a/db9/lbe 0 2024-04-25T15:49:29.565 INFO:tasks.workunit.client.0.smithi110.stdout:0/454: getdents d1/d23/d30/d33/d35 0 2024-04-25T15:49:29.566 INFO:tasks.workunit.client.1.smithi193.stdout:7/61: mkdir d12/d13 0 2024-04-25T15:49:29.566 INFO:tasks.workunit.client.0.smithi110.stdout:0/455: read - d1/d20/f96 zero size 2024-04-25T15:49:29.567 INFO:tasks.workunit.client.1.smithi193.stdout:7/62: read fd [140126,71615] 0 2024-04-25T15:49:29.567 INFO:tasks.workunit.client.1.smithi193.stdout:7/63: dread - fc zero size 2024-04-25T15:49:29.569 INFO:tasks.workunit.client.0.smithi110.stdout:3/510: symlink d0/d7/d1c/d2f/lb1 0 2024-04-25T15:49:29.571 INFO:tasks.workunit.client.1.smithi193.stdout:2/122: chown d0/d9/l1f 3162 1 2024-04-25T15:49:29.572 INFO:tasks.workunit.client.1.smithi193.stdout:2/123: chown d0/le 19240 1 2024-04-25T15:49:29.573 INFO:tasks.workunit.client.1.smithi193.stdout:0/116: rename d0/d3/d5/d7/dd/c1f to d0/c27 0 2024-04-25T15:49:29.575 INFO:tasks.workunit.client.0.smithi110.stdout:5/443: symlink d4/d1d/d1f/d35/d3d/d52/d75/l91 0 2024-04-25T15:49:29.578 INFO:tasks.workunit.client.0.smithi110.stdout:0/456: dread d1/f8d [4194304,4194304] 0 2024-04-25T15:49:29.578 INFO:tasks.workunit.client.1.smithi193.stdout:9/106: creat df/d11/f17 x:0 0 0 2024-04-25T15:49:29.584 INFO:tasks.workunit.client.0.smithi110.stdout:1/563: mknod d0/d14/d5f/d7d/cab 0 2024-04-25T15:49:29.585 INFO:tasks.workunit.client.0.smithi110.stdout:1/564: chown d0/d23/c6c 115 1 2024-04-25T15:49:29.585 INFO:tasks.workunit.client.1.smithi193.stdout:5/66: creat f16 x:0 0 0 2024-04-25T15:49:29.588 INFO:tasks.workunit.client.0.smithi110.stdout:7/433: mknod d1b/d29/d38/d55/c8f 0 2024-04-25T15:49:29.590 INFO:tasks.workunit.client.1.smithi193.stdout:4/102: dwrite db/fc [0,4194304] 0 2024-04-25T15:49:29.591 INFO:tasks.workunit.client.1.smithi193.stdout:5/67: dread fe [0,4194304] 0 2024-04-25T15:49:29.591 INFO:tasks.workunit.client.1.smithi193.stdout:4/103: write fa [2533807,41729] 0 2024-04-25T15:49:29.591 INFO:tasks.workunit.client.1.smithi193.stdout:4/104: stat db 0 2024-04-25T15:49:29.592 INFO:tasks.workunit.client.1.smithi193.stdout:7/64: truncate fd 247820 0 2024-04-25T15:49:29.596 INFO:tasks.workunit.client.0.smithi110.stdout:6/398: dwrite d1/d12/f49 [0,4194304] 0 2024-04-25T15:49:29.599 INFO:tasks.workunit.client.0.smithi110.stdout:1/565: dread d0/d14/d5f/d6f/f6e [0,4194304] 0 2024-04-25T15:49:29.599 INFO:tasks.workunit.client.1.smithi193.stdout:0/117: mkdir d0/d3/d5/d7/dd/d16/d18/d28 0 2024-04-25T15:49:29.600 INFO:tasks.workunit.client.1.smithi193.stdout:0/118: dread - d0/d3/d5/f24 zero size 2024-04-25T15:49:29.601 INFO:tasks.workunit.client.0.smithi110.stdout:3/511: read - d0/d16/f79 zero size 2024-04-25T15:49:29.601 INFO:tasks.workunit.client.0.smithi110.stdout:1/566: write d0/d14/d5f/d6f/f82 [733138,114998] 0 2024-04-25T15:49:29.603 INFO:tasks.workunit.client.0.smithi110.stdout:5/444: mkdir d4/d1d/d1f/d35/d7f/d92 0 2024-04-25T15:49:29.603 INFO:tasks.workunit.client.1.smithi193.stdout:3/148: link d8/d13/d14/c19 d8/d13/d14/c2e 0 2024-04-25T15:49:29.604 INFO:tasks.workunit.client.0.smithi110.stdout:5/445: write d4/d1d/d1f/d35/f6a [692645,108346] 0 2024-04-25T15:49:29.605 INFO:tasks.workunit.client.1.smithi193.stdout:0/119: dread d0/f12 [0,4194304] 0 2024-04-25T15:49:29.605 INFO:tasks.workunit.client.1.smithi193.stdout:9/107: dread f9 [0,4194304] 0 2024-04-25T15:49:29.607 INFO:tasks.workunit.client.1.smithi193.stdout:1/77: symlink dc/l13 0 2024-04-25T15:49:29.607 INFO:tasks.workunit.client.1.smithi193.stdout:0/120: write d0/d3/d5/d7/dd/d16/d18/f22 [749147,15826] 0 2024-04-25T15:49:29.611 INFO:tasks.workunit.client.1.smithi193.stdout:8/99: getdents d2/d3/d20 0 2024-04-25T15:49:29.614 INFO:tasks.workunit.client.1.smithi193.stdout:0/121: dwrite d0/f1e [0,4194304] 0 2024-04-25T15:49:29.615 INFO:tasks.workunit.client.1.smithi193.stdout:5/68: mkdir d17 0 2024-04-25T15:49:29.615 INFO:tasks.workunit.client.1.smithi193.stdout:0/122: read d0/d3/fe [1398856,66549] 0 2024-04-25T15:49:29.616 INFO:tasks.workunit.client.1.smithi193.stdout:0/123: write d0/d3/d5/f24 [613390,115556] 0 2024-04-25T15:49:29.617 INFO:tasks.workunit.client.0.smithi110.stdout:4/529: mknod d13/d24/d2d/d44/cac 0 2024-04-25T15:49:29.618 INFO:tasks.workunit.client.1.smithi193.stdout:0/124: chown d0/d3/d5/d7/dd/l25 0 1 2024-04-25T15:49:29.622 INFO:tasks.workunit.client.0.smithi110.stdout:8/563: link d14/d4c/d55/f88 d14/d4c/d55/fbf 0 2024-04-25T15:49:29.623 INFO:tasks.workunit.client.0.smithi110.stdout:8/564: chown d14/d21/d26/f98 7 1 2024-04-25T15:49:29.628 INFO:tasks.workunit.client.1.smithi193.stdout:3/149: mknod d8/d13/d14/c2f 0 2024-04-25T15:49:29.629 INFO:tasks.workunit.client.0.smithi110.stdout:3/512: dread d0/d7/d40/f61 [0,4194304] 0 2024-04-25T15:49:29.630 INFO:tasks.workunit.client.0.smithi110.stdout:1/567: rename d0/c28 to d0/d51/d6b/cac 0 2024-04-25T15:49:29.630 INFO:tasks.workunit.client.1.smithi193.stdout:3/150: dread d8/d13/d14/f17 [0,4194304] 0 2024-04-25T15:49:29.632 INFO:tasks.workunit.client.0.smithi110.stdout:5/446: mkdir d4/d1d/d1f/d35/d3d/d93 0 2024-04-25T15:49:29.633 INFO:tasks.workunit.client.1.smithi193.stdout:9/108: truncate f9 2367591 0 2024-04-25T15:49:29.633 INFO:tasks.workunit.client.1.smithi193.stdout:9/109: stat f6 0 2024-04-25T15:49:29.634 INFO:tasks.workunit.client.0.smithi110.stdout:0/457: mkdir d1/d23/d30/d33/d80/dad 0 2024-04-25T15:49:29.635 INFO:tasks.workunit.client.0.smithi110.stdout:7/434: getdents d1b/d29/d38/d55/d7f 0 2024-04-25T15:49:29.635 INFO:tasks.workunit.client.1.smithi193.stdout:3/151: dwrite d8/f11 [0,4194304] 0 2024-04-25T15:49:29.636 INFO:tasks.workunit.client.1.smithi193.stdout:7/65: creat d12/d13/f14 x:0 0 0 2024-04-25T15:49:29.637 INFO:tasks.workunit.client.1.smithi193.stdout:4/105: mkdir db/d13/d1e/d1f 0 2024-04-25T15:49:29.641 INFO:tasks.workunit.client.1.smithi193.stdout:0/125: unlink d0/ff 0 2024-04-25T15:49:29.642 INFO:tasks.workunit.client.0.smithi110.stdout:6/399: mknod d1/c93 0 2024-04-25T15:49:29.643 INFO:tasks.workunit.client.0.smithi110.stdout:6/400: chown d1/dd/d3c/f7c 150212 1 2024-04-25T15:49:29.646 INFO:tasks.workunit.client.0.smithi110.stdout:9/357: getdents d9/d13 0 2024-04-25T15:49:29.647 INFO:tasks.workunit.client.1.smithi193.stdout:0/126: dwrite d0/d3/d5/d7/dd/d16/d18/f22 [0,4194304] 0 2024-04-25T15:49:29.648 INFO:tasks.workunit.client.1.smithi193.stdout:0/127: write d0/f17 [1236442,112135] 0 2024-04-25T15:49:29.649 INFO:tasks.workunit.client.0.smithi110.stdout:3/513: mkdir d0/d7/d1c/d2f/d4c/db2 0 2024-04-25T15:49:29.651 INFO:tasks.workunit.client.1.smithi193.stdout:6/110: getdents dd 0 2024-04-25T15:49:29.651 INFO:tasks.workunit.client.0.smithi110.stdout:6/401: dwrite d1/dd/d3c/f5d [0,4194304] 0 2024-04-25T15:49:29.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/111: chown dd/d12/d19 501941948 1 2024-04-25T15:49:29.652 INFO:tasks.workunit.client.0.smithi110.stdout:1/568: write d0/d14/d5f/d6f/f6e [3113854,46249] 0 2024-04-25T15:49:29.656 INFO:tasks.workunit.client.0.smithi110.stdout:5/447: creat d4/d1d/d1f/f94 x:0 0 0 2024-04-25T15:49:29.659 INFO:tasks.workunit.client.0.smithi110.stdout:7/435: symlink d1b/d29/d38/d55/l90 0 2024-04-25T15:49:29.662 INFO:tasks.workunit.client.0.smithi110.stdout:4/530: creat d13/d24/d2d/d44/d65/fad x:0 0 0 2024-04-25T15:49:29.662 INFO:tasks.workunit.client.1.smithi193.stdout:5/69: mkdir d17/d18 0 2024-04-25T15:49:29.663 INFO:tasks.workunit.client.1.smithi193.stdout:3/152: unlink d8/c1b 0 2024-04-25T15:49:29.663 INFO:tasks.workunit.client.0.smithi110.stdout:8/565: mkdir d14/d21/d26/dc0 0 2024-04-25T15:49:29.664 INFO:tasks.workunit.client.1.smithi193.stdout:3/153: chown d8/d13/d14/f20 28 1 2024-04-25T15:49:29.665 INFO:tasks.workunit.client.0.smithi110.stdout:5/448: dwrite d4/d1d/d1f/d35/f4c [0,4194304] 0 2024-04-25T15:49:29.665 INFO:tasks.workunit.client.1.smithi193.stdout:4/106: truncate f4 3472092 0 2024-04-25T15:49:29.670 INFO:tasks.workunit.client.0.smithi110.stdout:6/402: mknod d1/db/d24/d69/d75/c94 0 2024-04-25T15:49:29.670 INFO:tasks.workunit.client.0.smithi110.stdout:1/569: creat d0/d14/fad x:0 0 0 2024-04-25T15:49:29.673 INFO:tasks.workunit.client.0.smithi110.stdout:0/458: fdatasync d1/d23/d30/d33/f2d 0 2024-04-25T15:49:29.674 INFO:tasks.workunit.client.0.smithi110.stdout:0/459: chown d1/l1e 6 1 2024-04-25T15:49:29.676 INFO:tasks.workunit.client.0.smithi110.stdout:6/403: dread d1/db/f88 [0,4194304] 0 2024-04-25T15:49:29.677 INFO:tasks.workunit.client.0.smithi110.stdout:6/404: dread - d1/db/d22/f57 zero size 2024-04-25T15:49:29.678 INFO:tasks.workunit.client.0.smithi110.stdout:6/405: write d1/db/f1a [1868142,40089] 0 2024-04-25T15:49:29.680 INFO:tasks.workunit.client.1.smithi193.stdout:2/124: link d0/d9/d15/l21 d0/d9/l32 0 2024-04-25T15:49:29.681 INFO:tasks.workunit.client.1.smithi193.stdout:6/112: rename dd/f1a to dd/d12/d19/d1c/f20 0 2024-04-25T15:49:29.681 INFO:tasks.workunit.client.0.smithi110.stdout:4/531: symlink d13/d24/d3c/d56/d88/lae 0 2024-04-25T15:49:29.681 INFO:tasks.workunit.client.0.smithi110.stdout:7/436: read f2 [109417,121635] 0 2024-04-25T15:49:29.681 INFO:tasks.workunit.client.1.smithi193.stdout:2/125: read - d0/df/f2c zero size 2024-04-25T15:49:29.682 INFO:tasks.workunit.client.0.smithi110.stdout:4/532: truncate d13/d24/d2d/d36/d51/f78 467418 0 2024-04-25T15:49:29.683 INFO:tasks.workunit.client.1.smithi193.stdout:2/126: write d0/fc [1861530,100315] 0 2024-04-25T15:49:29.683 INFO:tasks.workunit.client.0.smithi110.stdout:8/566: fsync d14/d32/d36/f3c 0 2024-04-25T15:49:29.685 INFO:tasks.workunit.client.0.smithi110.stdout:5/449: rmdir d4 39 2024-04-25T15:49:29.686 INFO:tasks.workunit.client.0.smithi110.stdout:3/514: dread d0/d7/f17 [0,4194304] 0 2024-04-25T15:49:29.687 INFO:tasks.workunit.client.1.smithi193.stdout:0/128: dwrite d0/d3/d5/d7/dd/d16/f1c [0,4194304] 0 2024-04-25T15:49:29.687 INFO:tasks.workunit.client.0.smithi110.stdout:9/358: symlink d9/l6d 0 2024-04-25T15:49:29.696 INFO:tasks.workunit.client.1.smithi193.stdout:3/154: mkdir d8/d30 0 2024-04-25T15:49:29.699 INFO:tasks.workunit.client.0.smithi110.stdout:3/515: dread d0/d2e/f4f [0,4194304] 0 2024-04-25T15:49:29.702 INFO:tasks.workunit.client.0.smithi110.stdout:6/406: mknod d1/db/d22/d33/d63/c95 0 2024-04-25T15:49:29.703 INFO:tasks.workunit.client.0.smithi110.stdout:6/407: fsync d1/dd/d3c/d91/f8b 0 2024-04-25T15:49:29.704 INFO:tasks.workunit.client.0.smithi110.stdout:7/437: mknod d1b/d29/d38/c91 0 2024-04-25T15:49:29.710 INFO:tasks.workunit.client.0.smithi110.stdout:7/438: dwrite f14 [0,4194304] 0 2024-04-25T15:49:29.712 INFO:tasks.workunit.client.0.smithi110.stdout:7/439: dread - d1b/d29/d38/d55/d7f/f7a zero size 2024-04-25T15:49:29.713 INFO:tasks.workunit.client.0.smithi110.stdout:7/440: chown d1b/d29/d38/d8b/d89/f8c 13096 1 2024-04-25T15:49:29.717 INFO:tasks.workunit.client.0.smithi110.stdout:4/533: rename d13/d24/d3c to d13/d24/d2d/d44/d65/d99/daf 0 2024-04-25T15:49:29.721 INFO:tasks.workunit.client.0.smithi110.stdout:7/441: dwrite d1b/f88 [0,4194304] 0 2024-04-25T15:49:29.723 INFO:tasks.workunit.client.0.smithi110.stdout:9/359: symlink d9/db/df/l6e 0 2024-04-25T15:49:29.724 INFO:tasks.workunit.client.1.smithi193.stdout:1/78: sync 2024-04-25T15:49:29.724 INFO:tasks.workunit.client.1.smithi193.stdout:8/100: sync 2024-04-25T15:49:29.725 INFO:tasks.workunit.client.1.smithi193.stdout:8/101: chown d2/d3/c13 141519 1 2024-04-25T15:49:29.726 INFO:tasks.workunit.client.0.smithi110.stdout:3/516: truncate d0/d7/d1c/d50/f6d 995270 0 2024-04-25T15:49:29.732 INFO:tasks.workunit.client.0.smithi110.stdout:8/567: symlink d14/d32/d36/d48/d8f/lc1 0 2024-04-25T15:49:29.732 INFO:tasks.workunit.client.0.smithi110.stdout:3/517: dwrite d0/d7/d1c/d2f/fa4 [0,4194304] 0 2024-04-25T15:49:29.733 INFO:tasks.workunit.client.1.smithi193.stdout:1/79: dwrite dc/fe [0,4194304] 0 2024-04-25T15:49:29.733 INFO:tasks.workunit.client.1.smithi193.stdout:1/80: readlink dc/l10 0 2024-04-25T15:49:29.744 INFO:tasks.workunit.client.0.smithi110.stdout:3/518: symlink d0/d7/lb3 0 2024-04-25T15:49:29.746 INFO:tasks.workunit.client.0.smithi110.stdout:0/460: rename d1/d23/d30/d33/l6f to d1/d47/d54/lae 0 2024-04-25T15:49:29.750 INFO:tasks.workunit.client.0.smithi110.stdout:7/442: unlink d1b/d29/l34 0 2024-04-25T15:49:29.752 INFO:tasks.workunit.client.0.smithi110.stdout:6/408: creat d1/db/d24/d2d/f96 x:0 0 0 2024-04-25T15:49:29.756 INFO:tasks.workunit.client.0.smithi110.stdout:5/450: rename d4/d15/f73 to d4/d2c/d2a/f95 0 2024-04-25T15:49:29.757 INFO:tasks.workunit.client.0.smithi110.stdout:5/451: write d4/d1d/d1f/d35/d3d/f53 [828973,17183] 0 2024-04-25T15:49:29.759 INFO:tasks.workunit.client.0.smithi110.stdout:4/534: creat d13/d24/d2d/d36/fb0 x:0 0 0 2024-04-25T15:49:29.761 INFO:tasks.workunit.client.0.smithi110.stdout:3/519: fdatasync d0/f2c 0 2024-04-25T15:49:29.767 INFO:tasks.workunit.client.0.smithi110.stdout:8/568: rename d14/d4c/c76 to d14/d21/d26/dab/dac/cc2 0 2024-04-25T15:49:29.771 INFO:tasks.workunit.client.0.smithi110.stdout:5/452: rmdir d4/d1d/d1f/d56 39 2024-04-25T15:49:29.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/453: write d4/d1d/d1f/d35/f7d [369782,34492] 0 2024-04-25T15:49:29.773 INFO:tasks.workunit.client.0.smithi110.stdout:5/454: fdatasync d4/d15/f1a 0 2024-04-25T15:49:29.774 INFO:tasks.workunit.client.0.smithi110.stdout:5/455: chown d4/d1d/d1f/d35/d3d/f5d 65712 1 2024-04-25T15:49:29.775 INFO:tasks.workunit.client.0.smithi110.stdout:9/360: getdents d9 0 2024-04-25T15:49:29.775 INFO:tasks.workunit.client.0.smithi110.stdout:5/456: chown d4/d1d/d1f/f32 77553670 1 2024-04-25T15:49:29.776 INFO:tasks.workunit.client.0.smithi110.stdout:5/457: chown d4/d1d 113 1 2024-04-25T15:49:29.776 INFO:tasks.workunit.client.0.smithi110.stdout:5/458: fsync d4/d1d/f70 0 2024-04-25T15:49:29.782 INFO:tasks.workunit.client.0.smithi110.stdout:0/461: rename d1/d20/c60 to d1/d47/d54/d99/caf 0 2024-04-25T15:49:29.784 INFO:tasks.workunit.client.0.smithi110.stdout:0/462: read d1/f4e [1505413,48414] 0 2024-04-25T15:49:29.784 INFO:tasks.workunit.client.0.smithi110.stdout:8/569: rmdir d14/d21/d23 39 2024-04-25T15:49:29.788 INFO:tasks.workunit.client.0.smithi110.stdout:4/535: symlink d13/d24/lb1 0 2024-04-25T15:49:29.790 INFO:tasks.workunit.client.0.smithi110.stdout:4/536: chown d13/d24/d2d/d44/d65/d99/daf/d56 3515 1 2024-04-25T15:49:29.794 INFO:tasks.workunit.client.0.smithi110.stdout:6/409: truncate d1/db/d24/d2d/f7b 1687914 0 2024-04-25T15:49:29.795 INFO:tasks.workunit.client.0.smithi110.stdout:6/410: write d1/d12/d3d/f73 [875418,125496] 0 2024-04-25T15:49:29.803 INFO:tasks.workunit.client.0.smithi110.stdout:8/570: rename d14/d21/d26/dab/lb6 to d14/d21/d26/dab/dac/dbb/lc3 0 2024-04-25T15:49:29.806 INFO:tasks.workunit.client.1.smithi193.stdout:1/81: sync 2024-04-25T15:49:29.807 INFO:tasks.workunit.client.1.smithi193.stdout:1/82: stat dc/l10 0 2024-04-25T15:49:29.807 INFO:tasks.workunit.client.1.smithi193.stdout:1/83: chown dc/f12 6 1 2024-04-25T15:49:29.809 INFO:tasks.workunit.client.0.smithi110.stdout:5/459: symlink d4/d1d/d1f/d56/d6b/l96 0 2024-04-25T15:49:29.810 INFO:tasks.workunit.client.0.smithi110.stdout:5/460: fsync d4/d15/d51/f78 0 2024-04-25T15:49:29.810 INFO:tasks.workunit.client.0.smithi110.stdout:6/411: dread d1/d48/f6c [0,4194304] 0 2024-04-25T15:49:29.816 INFO:tasks.workunit.client.0.smithi110.stdout:0/463: symlink d1/lb0 0 2024-04-25T15:49:29.819 INFO:tasks.workunit.client.0.smithi110.stdout:4/537: rename d13/d24/d2d/d44/d65/fad to d13/d24/d2d/d7d/d92/fb2 0 2024-04-25T15:49:29.820 INFO:tasks.workunit.client.0.smithi110.stdout:4/538: read d13/f22 [3734721,114166] 0 2024-04-25T15:49:29.821 INFO:tasks.workunit.client.0.smithi110.stdout:9/361: rmdir d9/db/df/d2c/d3c/d56/d6a 0 2024-04-25T15:49:29.822 INFO:tasks.workunit.client.0.smithi110.stdout:9/362: write d9/d13/f1c [1587741,36555] 0 2024-04-25T15:49:29.826 INFO:tasks.workunit.client.0.smithi110.stdout:6/412: creat d1/db/d22/d33/d63/f97 x:0 0 0 2024-04-25T15:49:29.826 INFO:tasks.workunit.client.0.smithi110.stdout:3/520: getdents d0/d16/d8c/d90/d99 0 2024-04-25T15:49:29.831 INFO:tasks.workunit.client.0.smithi110.stdout:9/363: mknod d9/d44/c6f 0 2024-04-25T15:49:29.832 INFO:tasks.workunit.client.0.smithi110.stdout:9/364: write d9/f5f [822173,105866] 0 2024-04-25T15:49:29.834 INFO:tasks.workunit.client.0.smithi110.stdout:9/365: truncate d9/db/df/d25/f37 9023527 0 2024-04-25T15:49:29.835 INFO:tasks.workunit.client.0.smithi110.stdout:6/413: symlink d1/dd/l98 0 2024-04-25T15:49:29.836 INFO:tasks.workunit.client.0.smithi110.stdout:6/414: chown d1/db/d22/d33/d63/c71 2496 1 2024-04-25T15:49:29.837 INFO:tasks.workunit.client.0.smithi110.stdout:3/521: mknod d0/d7/cb4 0 2024-04-25T15:49:29.837 INFO:tasks.workunit.client.0.smithi110.stdout:3/522: fsync d0/d7/f6c 0 2024-04-25T15:49:29.852 INFO:tasks.workunit.client.0.smithi110.stdout:3/523: link d0/d7/d1c/d50/f9a d0/d16/d8c/d90/d99/fb5 0 2024-04-25T15:49:29.854 INFO:tasks.workunit.client.0.smithi110.stdout:9/366: creat d9/db/df/d2c/f70 x:0 0 0 2024-04-25T15:49:29.855 INFO:tasks.workunit.client.0.smithi110.stdout:5/461: write d4/d1d/d1f/d35/d7f/f8d [60126,65836] 0 2024-04-25T15:49:29.856 INFO:tasks.workunit.client.0.smithi110.stdout:9/367: rmdir d9/d44 39 2024-04-25T15:49:29.860 INFO:tasks.workunit.client.0.smithi110.stdout:5/462: chown d4/d1d/d1f/l7b 4825116 1 2024-04-25T15:49:29.861 INFO:tasks.workunit.client.0.smithi110.stdout:0/464: write d1/d23/f89 [463162,33000] 0 2024-04-25T15:49:29.862 INFO:tasks.workunit.client.0.smithi110.stdout:0/465: readlink d1/d20/d69/l9d 0 2024-04-25T15:49:29.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:29 smithi110 ceph-mon[26344]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:49:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:29 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1054259809' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:49:29.863 INFO:tasks.workunit.client.0.smithi110.stdout:5/463: dread d4/d1d/d1f/d56/f7e [0,4194304] 0 2024-04-25T15:49:29.864 INFO:tasks.workunit.client.0.smithi110.stdout:3/524: link d0/d7/d1c/d50/l80 d0/d2e/lb6 0 2024-04-25T15:49:29.864 INFO:tasks.workunit.client.0.smithi110.stdout:6/415: write d1/db/d22/d33/d43/f53 [8722876,114816] 0 2024-04-25T15:49:29.870 INFO:tasks.workunit.client.0.smithi110.stdout:3/525: rmdir d0/d7/d7d 39 2024-04-25T15:49:29.874 INFO:tasks.workunit.client.0.smithi110.stdout:5/464: getdents d4/d1d/d1f/d35/d7f/d92 0 2024-04-25T15:49:29.878 INFO:tasks.workunit.client.0.smithi110.stdout:3/526: unlink d0/d7/d1c/d50/l5c 0 2024-04-25T15:49:29.880 INFO:tasks.workunit.client.0.smithi110.stdout:5/465: dwrite d4/d15/f1a [0,4194304] 0 2024-04-25T15:49:29.881 INFO:tasks.workunit.client.0.smithi110.stdout:3/527: chown d0/d7/d1c/d2f/d4c/fa0 3503 1 2024-04-25T15:49:29.881 INFO:tasks.workunit.client.0.smithi110.stdout:6/416: creat d1/dd/d3c/d91/d8d/f99 x:0 0 0 2024-04-25T15:49:29.882 INFO:tasks.workunit.client.0.smithi110.stdout:6/417: chown d1/dd/d3c/d91/f8b 26 1 2024-04-25T15:49:29.888 INFO:tasks.workunit.client.0.smithi110.stdout:3/528: chown d0/c3f 127 1 2024-04-25T15:49:29.889 INFO:tasks.workunit.client.0.smithi110.stdout:5/466: mkdir d4/d1d/d1f/d35/d3d/d52/d97 0 2024-04-25T15:49:29.894 INFO:tasks.workunit.client.0.smithi110.stdout:0/466: dwrite d1/f4e [4194304,4194304] 0 2024-04-25T15:49:29.895 INFO:tasks.workunit.client.0.smithi110.stdout:3/529: chown d0/cb 21202570 1 2024-04-25T15:49:29.896 INFO:tasks.workunit.client.0.smithi110.stdout:3/530: stat d0/d7/l48 0 2024-04-25T15:49:29.897 INFO:tasks.workunit.client.0.smithi110.stdout:9/368: getdents d9/db/df/d2c 0 2024-04-25T15:49:29.904 INFO:tasks.workunit.client.0.smithi110.stdout:6/418: link d1/db/d22/d33/d43/f60 d1/db/d22/d33/d63/f9a 0 2024-04-25T15:49:29.905 INFO:tasks.workunit.client.0.smithi110.stdout:6/419: chown d1/dd/f17 512867 1 2024-04-25T15:49:29.906 INFO:tasks.workunit.client.0.smithi110.stdout:5/467: dwrite d4/d1d/d1f/d35/d3d/d52/f55 [0,4194304] 0 2024-04-25T15:49:29.908 INFO:tasks.workunit.client.0.smithi110.stdout:0/467: creat d1/d23/d30/d33/d5b/d78/fb1 x:0 0 0 2024-04-25T15:49:29.909 INFO:tasks.workunit.client.0.smithi110.stdout:0/468: chown d1/d47/d54/c58 273050111 1 2024-04-25T15:49:29.910 INFO:tasks.workunit.client.0.smithi110.stdout:0/469: chown d1/d23/f89 252871 1 2024-04-25T15:49:29.910 INFO:tasks.workunit.client.0.smithi110.stdout:3/531: truncate d0/f2c 3783277 0 2024-04-25T15:49:29.912 INFO:tasks.workunit.client.0.smithi110.stdout:9/369: fdatasync d9/d44/f53 0 2024-04-25T15:49:29.917 INFO:tasks.workunit.client.0.smithi110.stdout:6/420: symlink d1/db/d22/d90/l9b 0 2024-04-25T15:49:29.922 INFO:tasks.workunit.client.0.smithi110.stdout:3/532: truncate d0/d7/fc 2513276 0 2024-04-25T15:49:29.930 INFO:tasks.workunit.client.0.smithi110.stdout:5/468: symlink d4/d1d/d85/l98 0 2024-04-25T15:49:29.930 INFO:tasks.workunit.client.0.smithi110.stdout:9/370: dread - d9/d13/d5b/d61/f63 zero size 2024-04-25T15:49:29.930 INFO:tasks.workunit.client.0.smithi110.stdout:6/421: dread d1/dd/f8e [0,4194304] 0 2024-04-25T15:49:29.931 INFO:tasks.workunit.client.0.smithi110.stdout:6/422: chown d1/l7 573 1 2024-04-25T15:49:29.932 INFO:tasks.workunit.client.0.smithi110.stdout:6/423: chown d1/c93 1347 1 2024-04-25T15:49:29.934 INFO:tasks.workunit.client.0.smithi110.stdout:0/470: write d1/d23/d30/d33/d5b/d83/f68 [35304,91548] 0 2024-04-25T15:49:29.934 INFO:tasks.workunit.client.0.smithi110.stdout:6/424: read d1/db/d22/d33/d43/f53 [7717727,35028] 0 2024-04-25T15:49:29.936 INFO:tasks.workunit.client.0.smithi110.stdout:9/371: fdatasync d9/d13/f1a 0 2024-04-25T15:49:29.938 INFO:tasks.workunit.client.0.smithi110.stdout:6/425: write d1/db/d22/d33/d43/f6f [766397,67020] 0 2024-04-25T15:49:29.938 INFO:tasks.workunit.client.0.smithi110.stdout:5/469: dwrite d4/d1d/d1f/d35/d3d/f53 [0,4194304] 0 2024-04-25T15:49:29.939 INFO:tasks.workunit.client.0.smithi110.stdout:6/426: chown d1/db/d24/d2d/l54 463 1 2024-04-25T15:49:29.949 INFO:tasks.workunit.client.1.smithi193.stdout:4/107: rename db/d13/l14 to db/l20 0 2024-04-25T15:49:29.950 INFO:tasks.workunit.client.0.smithi110.stdout:5/470: mkdir d4/d15/d51/d99 0 2024-04-25T15:49:29.956 INFO:tasks.workunit.client.0.smithi110.stdout:3/533: rename d0/d2e/d88/la7 to d0/d7/d1c/d2f/lb7 0 2024-04-25T15:49:29.956 INFO:tasks.workunit.client.0.smithi110.stdout:3/534: chown d0/d7/d40/f61 5067 1 2024-04-25T15:49:29.957 INFO:tasks.workunit.client.1.smithi193.stdout:0/129: symlink d0/l29 0 2024-04-25T15:49:29.958 INFO:tasks.workunit.client.1.smithi193.stdout:5/70: symlink d17/d18/l19 0 2024-04-25T15:49:29.959 INFO:tasks.workunit.client.0.smithi110.stdout:0/471: mknod d1/d23/d9e/cb2 0 2024-04-25T15:49:29.959 INFO:tasks.workunit.client.1.smithi193.stdout:7/66: link cf d12/d13/c15 0 2024-04-25T15:49:29.959 INFO:tasks.workunit.client.1.smithi193.stdout:7/67: dread - f10 zero size 2024-04-25T15:49:29.962 INFO:tasks.workunit.client.0.smithi110.stdout:5/471: chown d4/d1d/d1f/d35/c4b 85944 1 2024-04-25T15:49:29.963 INFO:tasks.workunit.client.1.smithi193.stdout:7/68: dread f8 [0,4194304] 0 2024-04-25T15:49:29.964 INFO:tasks.workunit.client.1.smithi193.stdout:8/102: chown d2/d3/ff 1786229 1 2024-04-25T15:49:29.964 INFO:tasks.workunit.client.1.smithi193.stdout:8/103: stat d2/d3/db/f1a 0 2024-04-25T15:49:29.965 INFO:tasks.workunit.client.0.smithi110.stdout:3/535: stat d0/l2b 0 2024-04-25T15:49:29.965 INFO:tasks.workunit.client.0.smithi110.stdout:3/536: write d0/d7/d1c/d2f/d4c/fa0 [894197,68782] 0 2024-04-25T15:49:29.966 INFO:tasks.workunit.client.0.smithi110.stdout:3/537: readlink d0/d16/d8c/d90/l97 0 2024-04-25T15:49:29.967 INFO:tasks.workunit.client.1.smithi193.stdout:8/104: dread d2/d3/db/f19 [0,4194304] 0 2024-04-25T15:49:29.970 INFO:tasks.workunit.client.1.smithi193.stdout:6/113: unlink l1 0 2024-04-25T15:49:29.970 INFO:tasks.workunit.client.0.smithi110.stdout:5/472: rmdir d4/d1d/d1f/d35 39 2024-04-25T15:49:29.971 INFO:tasks.workunit.client.0.smithi110.stdout:5/473: truncate d4/d1d/d1f/f8f 12953 0 2024-04-25T15:49:29.972 INFO:tasks.workunit.client.1.smithi193.stdout:3/155: write d8/d13/d14/f1d [707805,10629] 0 2024-04-25T15:49:29.972 INFO:tasks.workunit.client.0.smithi110.stdout:6/427: creat d1/db/d24/f9c x:0 0 0 2024-04-25T15:49:29.974 INFO:tasks.workunit.client.1.smithi193.stdout:2/127: rename d0/df/f2c to d0/f33 0 2024-04-25T15:49:29.975 INFO:tasks.workunit.client.1.smithi193.stdout:9/110: link fc df/f18 0 2024-04-25T15:49:29.976 INFO:tasks.workunit.client.1.smithi193.stdout:0/130: creat d0/d3/d5/d7/dd/d16/d18/f2a x:0 0 0 2024-04-25T15:49:29.977 INFO:tasks.workunit.client.1.smithi193.stdout:5/71: symlink d17/d18/l1a 0 2024-04-25T15:49:29.978 INFO:tasks.workunit.client.1.smithi193.stdout:7/69: mknod d12/d13/c16 0 2024-04-25T15:49:29.979 INFO:tasks.workunit.client.1.smithi193.stdout:7/70: write f3 [2886355,122288] 0 2024-04-25T15:49:29.979 INFO:tasks.workunit.client.1.smithi193.stdout:8/105: mkdir d2/d3/d23 0 2024-04-25T15:49:29.979 INFO:tasks.workunit.client.1.smithi193.stdout:6/114: mkdir dd/d12/d19/d21 0 2024-04-25T15:49:29.980 INFO:tasks.workunit.client.1.smithi193.stdout:8/106: write d2/d3/db/f19 [118731,80097] 0 2024-04-25T15:49:29.980 INFO:tasks.workunit.client.1.smithi193.stdout:4/108: unlink l5 0 2024-04-25T15:49:29.981 INFO:tasks.workunit.client.1.smithi193.stdout:6/115: read - dd/d12/d19/d1c/f20 zero size 2024-04-25T15:49:29.981 INFO:tasks.workunit.client.1.smithi193.stdout:2/128: fdatasync d0/f33 0 2024-04-25T15:49:29.982 INFO:tasks.workunit.client.1.smithi193.stdout:8/107: chown d2/d3/db 27668813 1 2024-04-25T15:49:29.982 INFO:tasks.workunit.client.0.smithi110.stdout:9/372: link d9/l10 d9/db/df/d25/l71 0 2024-04-25T15:49:29.983 INFO:tasks.workunit.client.1.smithi193.stdout:4/109: write db/d13/f1d [951200,80964] 0 2024-04-25T15:49:29.983 INFO:tasks.workunit.client.1.smithi193.stdout:6/116: write dd/f11 [948079,65456] 0 2024-04-25T15:49:29.983 INFO:tasks.workunit.client.1.smithi193.stdout:0/131: truncate d0/fc 2032091 0 2024-04-25T15:49:29.983 INFO:tasks.workunit.client.0.smithi110.stdout:3/538: write d0/d7/d1c/f22 [396832,81747] 0 2024-04-25T15:49:29.984 INFO:tasks.workunit.client.1.smithi193.stdout:6/117: dread - dd/f1b zero size 2024-04-25T15:49:29.984 INFO:tasks.workunit.client.1.smithi193.stdout:5/72: rename ld to d17/d18/l1b 0 2024-04-25T15:49:29.985 INFO:tasks.workunit.client.0.smithi110.stdout:5/474: write d4/d1d/d1f/d35/f76 [296352,9511] 0 2024-04-25T15:49:29.987 INFO:tasks.workunit.client.1.smithi193.stdout:1/84: creat dc/f14 x:0 0 0 2024-04-25T15:49:29.987 INFO:tasks.workunit.client.1.smithi193.stdout:4/110: dread db/fe [0,4194304] 0 2024-04-25T15:49:29.989 INFO:tasks.workunit.client.1.smithi193.stdout:7/71: creat d12/d13/f17 x:0 0 0 2024-04-25T15:49:29.989 INFO:tasks.workunit.client.0.smithi110.stdout:3/539: dread - d0/d7/d1c/d2f/d4c/fac zero size 2024-04-25T15:49:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:29 smithi193 ceph-mon[33747]: from='client.14722 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:49:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:29 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1054259809' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:49:29.990 INFO:tasks.workunit.client.1.smithi193.stdout:3/156: mkdir d8/d30/d31 0 2024-04-25T15:49:29.991 INFO:tasks.workunit.client.0.smithi110.stdout:0/472: dwrite d1/d47/f95 [0,4194304] 0 2024-04-25T15:49:29.991 INFO:tasks.workunit.client.1.smithi193.stdout:9/111: link df/d11/f17 df/d16/f19 0 2024-04-25T15:49:29.992 INFO:tasks.workunit.client.0.smithi110.stdout:0/473: write d1/d47/d54/f6d [716364,107215] 0 2024-04-25T15:49:29.993 INFO:tasks.workunit.client.1.smithi193.stdout:3/157: dread f2 [0,4194304] 0 2024-04-25T15:49:29.993 INFO:tasks.workunit.client.0.smithi110.stdout:5/475: creat d4/d1d/d1f/d35/f9a x:0 0 0 2024-04-25T15:49:29.996 INFO:tasks.workunit.client.0.smithi110.stdout:9/373: dwrite d9/db/df/f21 [0,4194304] 0 2024-04-25T15:49:29.996 INFO:tasks.workunit.client.0.smithi110.stdout:3/540: truncate d0/d7/d40/f61 917684 0 2024-04-25T15:49:29.998 INFO:tasks.workunit.client.1.smithi193.stdout:3/158: dwrite d8/fc [4194304,4194304] 0 2024-04-25T15:49:29.999 INFO:tasks.workunit.client.1.smithi193.stdout:5/73: symlink d17/l1c 0 2024-04-25T15:49:30.000 INFO:tasks.workunit.client.0.smithi110.stdout:9/374: write d9/d13/f3f [1330403,4586] 0 2024-04-25T15:49:30.002 INFO:tasks.workunit.client.0.smithi110.stdout:5/476: read d4/d1d/d1f/d35/d3d/d52/f82 [916589,124639] 0 2024-04-25T15:49:30.003 INFO:tasks.workunit.client.1.smithi193.stdout:4/111: creat db/d13/f21 x:0 0 0 2024-04-25T15:49:30.004 INFO:tasks.workunit.client.1.smithi193.stdout:8/108: mknod d2/c24 0 2024-04-25T15:49:30.004 INFO:tasks.workunit.client.1.smithi193.stdout:4/112: chown f2 1050 1 2024-04-25T15:49:30.006 INFO:tasks.workunit.client.0.smithi110.stdout:9/375: symlink d9/db/d58/l72 0 2024-04-25T15:49:30.008 INFO:tasks.workunit.client.1.smithi193.stdout:9/112: rename df/f18 to df/d16/f1a 0 2024-04-25T15:49:30.010 INFO:tasks.workunit.client.0.smithi110.stdout:3/541: rename d0/d7/d1c/d2f/lb1 to d0/d16/lb8 0 2024-04-25T15:49:30.013 INFO:tasks.workunit.client.0.smithi110.stdout:0/474: dread d1/f1c [0,4194304] 0 2024-04-25T15:49:30.014 INFO:tasks.workunit.client.0.smithi110.stdout:9/376: dread d9/d13/f27 [0,4194304] 0 2024-04-25T15:49:30.015 INFO:tasks.workunit.client.1.smithi193.stdout:0/132: creat d0/d3/d5/d7/dd/d16/d1a/f2b x:0 0 0 2024-04-25T15:49:30.015 INFO:tasks.workunit.client.0.smithi110.stdout:3/542: rmdir d0/d7/d1c/d2f/d4c 39 2024-04-25T15:49:30.016 INFO:tasks.workunit.client.0.smithi110.stdout:5/477: mkdir d4/d1d/d1f/d35/d7f/d92/d9b 0 2024-04-25T15:49:30.017 INFO:tasks.workunit.client.0.smithi110.stdout:5/478: dread - d4/d1d/d1f/f8a zero size 2024-04-25T15:49:30.018 INFO:tasks.workunit.client.1.smithi193.stdout:5/74: creat d17/f1d x:0 0 0 2024-04-25T15:49:30.020 INFO:tasks.workunit.client.0.smithi110.stdout:5/479: write d4/d15/d51/f78 [623812,57311] 0 2024-04-25T15:49:30.020 INFO:tasks.workunit.client.0.smithi110.stdout:0/475: rmdir d1/d47/d54 39 2024-04-25T15:49:30.024 INFO:tasks.workunit.client.0.smithi110.stdout:3/543: dwrite d0/d16/d8c/d90/d92/f93 [0,4194304] 0 2024-04-25T15:49:30.031 INFO:tasks.workunit.client.0.smithi110.stdout:5/480: unlink d4/d1d/d1f/d56/d6b/f89 0 2024-04-25T15:49:30.034 INFO:tasks.workunit.client.0.smithi110.stdout:3/544: symlink d0/d16/d8c/d90/d99/dad/lb9 0 2024-04-25T15:49:30.035 INFO:tasks.workunit.client.0.smithi110.stdout:3/545: stat d0/f1e 0 2024-04-25T15:49:30.039 INFO:tasks.workunit.client.1.smithi193.stdout:5/75: symlink d17/l1e 0 2024-04-25T15:49:30.040 INFO:tasks.workunit.client.0.smithi110.stdout:5/481: creat d4/d15/d51/f9c x:0 0 0 2024-04-25T15:49:30.041 INFO:tasks.workunit.client.0.smithi110.stdout:5/482: chown d4/d2c/d2a/l31 763 1 2024-04-25T15:49:30.043 INFO:tasks.workunit.client.0.smithi110.stdout:3/546: rename d0/d16/cae to d0/d16/d8c/cba 0 2024-04-25T15:49:30.045 INFO:tasks.workunit.client.0.smithi110.stdout:3/547: dread - d0/d7/d1c/d2f/d4c/fac zero size 2024-04-25T15:49:30.048 INFO:tasks.workunit.client.1.smithi193.stdout:3/159: creat d8/d13/f32 x:0 0 0 2024-04-25T15:49:30.049 INFO:tasks.workunit.client.1.smithi193.stdout:5/76: creat d17/f1f x:0 0 0 2024-04-25T15:49:30.052 INFO:tasks.workunit.client.1.smithi193.stdout:1/85: rename dc/f14 to dc/f15 0 2024-04-25T15:49:30.053 INFO:tasks.workunit.client.0.smithi110.stdout:5/483: getdents d4/d1d/d1f/d35/d3d/d52/d75 0 2024-04-25T15:49:30.054 INFO:tasks.workunit.client.1.smithi193.stdout:2/129: link d0/d9/l32 d0/d9/d14/d25/l34 0 2024-04-25T15:49:30.055 INFO:tasks.workunit.client.1.smithi193.stdout:3/160: symlink d8/d13/d14/l33 0 2024-04-25T15:49:30.058 INFO:tasks.workunit.client.1.smithi193.stdout:4/113: getdents db/d13/d1e 0 2024-04-25T15:49:30.059 INFO:tasks.workunit.client.0.smithi110.stdout:5/484: dwrite d4/d1d/f70 [0,4194304] 0 2024-04-25T15:49:30.059 INFO:tasks.workunit.client.1.smithi193.stdout:2/130: dwrite d0/f6 [0,4194304] 0 2024-04-25T15:49:30.061 INFO:tasks.workunit.client.1.smithi193.stdout:7/72: write fd [415675,89361] 0 2024-04-25T15:49:30.068 INFO:tasks.workunit.client.1.smithi193.stdout:3/161: unlink d8/d13/d14/f18 0 2024-04-25T15:49:30.070 INFO:tasks.workunit.client.0.smithi110.stdout:9/377: dwrite d9/db/df/d25/f4c [0,4194304] 0 2024-04-25T15:49:30.071 INFO:tasks.workunit.client.0.smithi110.stdout:5/485: dwrite d4/f22 [0,4194304] 0 2024-04-25T15:49:30.072 INFO:tasks.workunit.client.1.smithi193.stdout:7/73: truncate f8 3491665 0 2024-04-25T15:49:30.076 INFO:tasks.workunit.client.0.smithi110.stdout:0/476: dwrite d1/d23/d30/d33/d5b/d72/f82 [0,4194304] 0 2024-04-25T15:49:30.078 INFO:tasks.workunit.client.0.smithi110.stdout:0/477: dread - d1/d23/d30/d33/d5b/d78/fb1 zero size 2024-04-25T15:49:30.079 INFO:tasks.workunit.client.1.smithi193.stdout:4/114: truncate f2 2059760 0 2024-04-25T15:49:30.080 INFO:tasks.workunit.client.0.smithi110.stdout:9/378: dread d9/db/df/d25/f4c [0,4194304] 0 2024-04-25T15:49:30.081 INFO:tasks.workunit.client.1.smithi193.stdout:4/115: write db/d13/f21 [319647,10918] 0 2024-04-25T15:49:30.083 INFO:tasks.workunit.client.1.smithi193.stdout:6/118: sync 2024-04-25T15:49:30.083 INFO:tasks.workunit.client.1.smithi193.stdout:8/109: sync 2024-04-25T15:49:30.083 INFO:tasks.workunit.client.1.smithi193.stdout:0/133: sync 2024-04-25T15:49:30.084 INFO:tasks.workunit.client.0.smithi110.stdout:5/486: symlink d4/d1d/l9d 0 2024-04-25T15:49:30.088 INFO:tasks.workunit.client.1.smithi193.stdout:1/86: rmdir dc 39 2024-04-25T15:49:30.091 INFO:tasks.workunit.client.0.smithi110.stdout:5/487: dwrite d4/f22 [0,4194304] 0 2024-04-25T15:49:30.092 INFO:tasks.workunit.client.1.smithi193.stdout:7/74: rename fd to d12/f18 0 2024-04-25T15:49:30.093 INFO:tasks.workunit.client.0.smithi110.stdout:9/379: rename d9/db/df/d25/c52 to d9/d13/d5b/c73 0 2024-04-25T15:49:30.094 INFO:tasks.workunit.client.1.smithi193.stdout:2/131: dwrite d0/d9/f22 [0,4194304] 0 2024-04-25T15:49:30.095 INFO:tasks.workunit.client.1.smithi193.stdout:0/134: dwrite d0/f1e [0,4194304] 0 2024-04-25T15:49:30.100 INFO:tasks.workunit.client.1.smithi193.stdout:9/113: dwrite df/d16/f1a [0,4194304] 0 2024-04-25T15:49:30.102 INFO:tasks.workunit.client.1.smithi193.stdout:9/114: stat df 0 2024-04-25T15:49:30.102 INFO:tasks.workunit.client.1.smithi193.stdout:6/119: creat dd/d12/d19/d1c/f22 x:0 0 0 2024-04-25T15:49:30.104 INFO:tasks.workunit.client.1.smithi193.stdout:6/120: read dd/f15 [2308885,107753] 0 2024-04-25T15:49:30.104 INFO:tasks.workunit.client.0.smithi110.stdout:9/380: symlink d9/db/df/d55/l74 0 2024-04-25T15:49:30.108 INFO:tasks.workunit.client.1.smithi193.stdout:1/87: truncate dc/f15 31907 0 2024-04-25T15:49:30.108 INFO:tasks.workunit.client.1.smithi193.stdout:0/135: dwrite d0/f1e [0,4194304] 0 2024-04-25T15:49:30.116 INFO:tasks.workunit.client.1.smithi193.stdout:4/116: mkdir db/d13/d1e/d1f/d22 0 2024-04-25T15:49:30.117 INFO:tasks.workunit.client.1.smithi193.stdout:7/75: creat d12/d13/f19 x:0 0 0 2024-04-25T15:49:30.118 INFO:tasks.workunit.client.1.smithi193.stdout:1/88: dread f6 [0,4194304] 0 2024-04-25T15:49:30.118 INFO:tasks.workunit.client.0.smithi110.stdout:9/381: fdatasync d9/f34 0 2024-04-25T15:49:30.119 INFO:tasks.workunit.client.1.smithi193.stdout:3/162: write d8/d13/d14/f17 [1506989,127582] 0 2024-04-25T15:49:30.122 INFO:tasks.workunit.client.1.smithi193.stdout:2/132: creat d0/d9/d15/f35 x:0 0 0 2024-04-25T15:49:30.125 INFO:tasks.workunit.client.1.smithi193.stdout:4/117: dread db/d13/f1d [0,4194304] 0 2024-04-25T15:49:30.127 INFO:tasks.workunit.client.0.smithi110.stdout:9/382: mkdir d9/db/df/d75 0 2024-04-25T15:49:30.127 INFO:tasks.workunit.client.1.smithi193.stdout:3/163: dwrite d8/d13/d14/d27/f29 [0,4194304] 0 2024-04-25T15:49:30.129 INFO:tasks.workunit.client.1.smithi193.stdout:8/110: creat d2/d18/f25 x:0 0 0 2024-04-25T15:49:30.129 INFO:tasks.workunit.client.0.smithi110.stdout:0/478: write d1/d23/f42 [1394727,15800] 0 2024-04-25T15:49:30.133 INFO:tasks.workunit.client.0.smithi110.stdout:0/479: dread d1/d23/d30/d33/f4b [0,4194304] 0 2024-04-25T15:49:30.134 INFO:tasks.workunit.client.1.smithi193.stdout:3/164: dread d8/d13/d14/d27/f29 [0,4194304] 0 2024-04-25T15:49:30.137 INFO:tasks.workunit.client.1.smithi193.stdout:8/111: dwrite d2/d18/f25 [0,4194304] 0 2024-04-25T15:49:30.138 INFO:tasks.workunit.client.0.smithi110.stdout:5/488: dwrite d4/d1d/d1f/d35/d3d/d52/f82 [0,4194304] 0 2024-04-25T15:49:30.139 INFO:tasks.workunit.client.1.smithi193.stdout:8/112: fdatasync d2/f11 0 2024-04-25T15:49:30.140 INFO:tasks.workunit.client.1.smithi193.stdout:8/113: fsync d2/f11 0 2024-04-25T15:49:30.142 INFO:tasks.workunit.client.1.smithi193.stdout:3/165: dwrite d8/f10 [0,4194304] 0 2024-04-25T15:49:30.142 INFO:tasks.workunit.client.0.smithi110.stdout:0/480: creat d1/d20/d69/fb3 x:0 0 0 2024-04-25T15:49:30.146 INFO:tasks.workunit.client.0.smithi110.stdout:9/383: link d9/db/df/c14 d9/db/df/d75/c76 0 2024-04-25T15:49:30.147 INFO:tasks.workunit.client.1.smithi193.stdout:5/77: sync 2024-04-25T15:49:30.150 INFO:tasks.workunit.client.1.smithi193.stdout:6/121: creat dd/d12/d19/d21/f23 x:0 0 0 2024-04-25T15:49:30.155 INFO:tasks.workunit.client.0.smithi110.stdout:9/384: dread d9/db/df/d25/f51 [0,4194304] 0 2024-04-25T15:49:30.155 INFO:tasks.workunit.client.1.smithi193.stdout:8/114: mknod d2/c26 0 2024-04-25T15:49:30.156 INFO:tasks.workunit.client.1.smithi193.stdout:2/133: fsync d0/d9/d15/f35 0 2024-04-25T15:49:30.156 INFO:tasks.workunit.client.1.smithi193.stdout:2/134: chown d0/c1 8749 1 2024-04-25T15:49:30.158 INFO:tasks.workunit.client.1.smithi193.stdout:5/78: creat d17/d18/f20 x:0 0 0 2024-04-25T15:49:30.160 INFO:tasks.workunit.client.0.smithi110.stdout:9/385: rename d9/db/df/d25/c26 to d9/d13/d5b/d61/d67/c77 0 2024-04-25T15:49:30.162 INFO:tasks.workunit.client.1.smithi193.stdout:9/115: rename df/d11/f17 to df/f1b 0 2024-04-25T15:49:30.166 INFO:tasks.workunit.client.0.smithi110.stdout:9/386: stat d9/db/df/d75/c76 0 2024-04-25T15:49:30.167 INFO:tasks.workunit.client.1.smithi193.stdout:0/136: sync 2024-04-25T15:49:30.169 INFO:tasks.workunit.client.1.smithi193.stdout:0/137: fdatasync d0/d3/d5/d7/dd/d16/d1a/f2b 0 2024-04-25T15:49:30.169 INFO:tasks.workunit.client.1.smithi193.stdout:8/115: fdatasync d2/d18/f25 0 2024-04-25T15:49:30.172 INFO:tasks.workunit.client.0.smithi110.stdout:9/387: creat d9/db/df/d4d/f78 x:0 0 0 2024-04-25T15:49:30.174 INFO:tasks.workunit.client.0.smithi110.stdout:9/388: stat d9/d13/d5b/c73 0 2024-04-25T15:49:30.174 INFO:tasks.workunit.client.0.smithi110.stdout:9/389: chown d9/db/df/f33 503806 1 2024-04-25T15:49:30.175 INFO:tasks.workunit.client.1.smithi193.stdout:9/116: mkdir df/d16/d1c 0 2024-04-25T15:49:30.176 INFO:tasks.workunit.client.1.smithi193.stdout:7/76: getdents d12/d13 0 2024-04-25T15:49:30.178 INFO:tasks.workunit.client.1.smithi193.stdout:1/89: getdents dc 0 2024-04-25T15:49:30.179 INFO:tasks.workunit.client.1.smithi193.stdout:1/90: chown f1 873 1 2024-04-25T15:49:30.180 INFO:tasks.workunit.client.1.smithi193.stdout:5/79: dwrite f11 [0,4194304] 0 2024-04-25T15:49:30.181 INFO:tasks.workunit.client.1.smithi193.stdout:5/80: read - d17/f1f zero size 2024-04-25T15:49:30.181 INFO:tasks.workunit.client.1.smithi193.stdout:4/118: getdents db/d13/d1e/d1f 0 2024-04-25T15:49:30.182 INFO:tasks.workunit.client.1.smithi193.stdout:5/81: write fc [239268,111366] 0 2024-04-25T15:49:30.183 INFO:tasks.workunit.client.0.smithi110.stdout:9/390: dwrite d9/d2d/f30 [0,4194304] 0 2024-04-25T15:49:30.184 INFO:tasks.workunit.client.0.smithi110.stdout:9/391: chown d9/db/df/c14 185 1 2024-04-25T15:49:30.185 INFO:tasks.workunit.client.0.smithi110.stdout:9/392: fdatasync d9/db/df/d25/f37 0 2024-04-25T15:49:30.186 INFO:tasks.workunit.client.1.smithi193.stdout:7/77: rename d12/d13/f19 to d12/f1a 0 2024-04-25T15:49:30.187 INFO:tasks.workunit.client.0.smithi110.stdout:9/393: write d9/db/df/d55/f6b [671543,24388] 0 2024-04-25T15:49:30.192 INFO:tasks.workunit.client.1.smithi193.stdout:1/91: creat dc/f16 x:0 0 0 2024-04-25T15:49:30.192 INFO:tasks.workunit.client.1.smithi193.stdout:3/166: truncate d8/d13/d14/f1d 715079 0 2024-04-25T15:49:30.193 INFO:tasks.workunit.client.0.smithi110.stdout:9/394: rmdir d9/d13/d5b/d61/d67 39 2024-04-25T15:49:30.196 INFO:tasks.workunit.client.1.smithi193.stdout:4/119: mknod db/d13/c23 0 2024-04-25T15:49:30.198 INFO:tasks.workunit.client.0.smithi110.stdout:9/395: creat d9/db/df/d25/f79 x:0 0 0 2024-04-25T15:49:30.198 INFO:tasks.workunit.client.1.smithi193.stdout:8/116: write d2/d3/f16 [2119331,95187] 0 2024-04-25T15:49:30.200 INFO:tasks.workunit.client.1.smithi193.stdout:6/122: getdents dd 0 2024-04-25T15:49:30.201 INFO:tasks.workunit.client.1.smithi193.stdout:5/82: creat d17/d18/f21 x:0 0 0 2024-04-25T15:49:30.202 INFO:tasks.workunit.client.1.smithi193.stdout:9/117: mknod df/d16/d1c/c1d 0 2024-04-25T15:49:30.203 INFO:tasks.workunit.client.1.smithi193.stdout:7/78: fdatasync d12/f18 0 2024-04-25T15:49:30.203 INFO:tasks.workunit.client.1.smithi193.stdout:9/118: dread - df/f1b zero size 2024-04-25T15:49:30.204 INFO:tasks.workunit.client.0.smithi110.stdout:9/396: rename d9/d13/d5b/d61/l66 to d9/d13/d5b/d61/d67/l7a 0 2024-04-25T15:49:30.205 INFO:tasks.workunit.client.1.smithi193.stdout:0/138: creat d0/d3/f2c x:0 0 0 2024-04-25T15:49:30.206 INFO:tasks.workunit.client.0.smithi110.stdout:9/397: symlink d9/d44/d5e/l7b 0 2024-04-25T15:49:30.207 INFO:tasks.workunit.client.1.smithi193.stdout:3/167: dread d8/d13/d14/f21 [0,4194304] 0 2024-04-25T15:49:30.207 INFO:tasks.workunit.client.1.smithi193.stdout:4/120: mknod db/d13/d1e/d1f/c24 0 2024-04-25T15:49:30.208 INFO:tasks.workunit.client.1.smithi193.stdout:2/135: getdents d0/d9/d15/d31 0 2024-04-25T15:49:30.208 INFO:tasks.workunit.client.1.smithi193.stdout:4/121: stat db/d13/f21 0 2024-04-25T15:49:30.209 INFO:tasks.workunit.client.1.smithi193.stdout:0/139: dread d0/f17 [0,4194304] 0 2024-04-25T15:49:30.210 INFO:tasks.workunit.client.0.smithi110.stdout:9/398: mkdir d9/d13/d7c 0 2024-04-25T15:49:30.211 INFO:tasks.workunit.client.1.smithi193.stdout:8/117: mkdir d2/d3/d27 0 2024-04-25T15:49:30.212 INFO:tasks.workunit.client.1.smithi193.stdout:6/123: write dd/d12/d19/d1c/f20 [673394,26485] 0 2024-04-25T15:49:30.213 INFO:tasks.workunit.client.1.smithi193.stdout:6/124: dread - dd/d12/f1e zero size 2024-04-25T15:49:30.214 INFO:tasks.workunit.client.1.smithi193.stdout:5/83: mknod d17/d18/c22 0 2024-04-25T15:49:30.214 INFO:tasks.workunit.client.1.smithi193.stdout:6/125: write dd/d12/d19/d1c/f20 [786929,81783] 0 2024-04-25T15:49:30.215 INFO:tasks.workunit.client.1.smithi193.stdout:6/126: stat dd/d12/d19/f1d 0 2024-04-25T15:49:30.218 INFO:tasks.workunit.client.1.smithi193.stdout:9/119: creat df/d16/f1e x:0 0 0 2024-04-25T15:49:30.219 INFO:tasks.workunit.client.0.smithi110.stdout:9/399: rmdir d9/d13/d69 0 2024-04-25T15:49:30.221 INFO:tasks.workunit.client.1.smithi193.stdout:3/168: symlink d8/d13/d1f/l34 0 2024-04-25T15:49:30.222 INFO:tasks.workunit.client.1.smithi193.stdout:2/136: rmdir d0/d9/d14/d25 39 2024-04-25T15:49:30.223 INFO:tasks.workunit.client.1.smithi193.stdout:0/140: creat d0/d3/d5/f2d x:0 0 0 2024-04-25T15:49:30.225 INFO:tasks.workunit.client.1.smithi193.stdout:9/120: dwrite df/d16/f1a [4194304,4194304] 0 2024-04-25T15:49:30.225 INFO:tasks.workunit.client.1.smithi193.stdout:5/84: rename d17/d18/c22 to d17/d18/c23 0 2024-04-25T15:49:30.228 INFO:tasks.workunit.client.1.smithi193.stdout:4/122: fdatasync f2 0 2024-04-25T15:49:30.230 INFO:tasks.workunit.client.1.smithi193.stdout:4/123: chown f2 161 1 2024-04-25T15:49:30.231 INFO:tasks.workunit.client.1.smithi193.stdout:3/169: dwrite d8/f11 [0,4194304] 0 2024-04-25T15:49:30.232 INFO:tasks.workunit.client.1.smithi193.stdout:4/124: fsync db/d13/f16 0 2024-04-25T15:49:30.232 INFO:tasks.workunit.client.1.smithi193.stdout:6/127: link dd/f11 dd/f24 0 2024-04-25T15:49:30.233 INFO:tasks.workunit.client.1.smithi193.stdout:5/85: dread f15 [0,4194304] 0 2024-04-25T15:49:30.233 INFO:tasks.workunit.client.1.smithi193.stdout:6/128: stat dd/f15 0 2024-04-25T15:49:30.236 INFO:tasks.workunit.client.1.smithi193.stdout:9/121: truncate df/f1b 933555 0 2024-04-25T15:49:30.236 INFO:tasks.workunit.client.1.smithi193.stdout:9/122: chown cd 5936328 1 2024-04-25T15:49:30.238 INFO:tasks.workunit.client.1.smithi193.stdout:1/92: creat dc/f17 x:0 0 0 2024-04-25T15:49:30.240 INFO:tasks.workunit.client.1.smithi193.stdout:0/141: mkdir d0/d3/d5/d7/dd/d16/d26/d2e 0 2024-04-25T15:49:30.241 INFO:tasks.workunit.client.1.smithi193.stdout:8/118: truncate d2/d3/db/f1a 112914 0 2024-04-25T15:49:30.241 INFO:tasks.workunit.client.1.smithi193.stdout:0/142: readlink d0/d3/d5/d7/dd/l1b 0 2024-04-25T15:49:30.242 INFO:tasks.workunit.client.1.smithi193.stdout:3/170: rmdir d8/d13/d14 39 2024-04-25T15:49:30.242 INFO:tasks.workunit.client.1.smithi193.stdout:0/143: chown d0/l2 1374 1 2024-04-25T15:49:30.244 INFO:tasks.workunit.client.1.smithi193.stdout:5/86: unlink la 0 2024-04-25T15:49:30.245 INFO:tasks.workunit.client.1.smithi193.stdout:4/125: mknod db/d13/d1e/d1f/c25 0 2024-04-25T15:49:30.245 INFO:tasks.workunit.client.1.smithi193.stdout:1/93: dwrite dc/f16 [0,4194304] 0 2024-04-25T15:49:30.246 INFO:tasks.workunit.client.1.smithi193.stdout:4/126: stat db/fe 0 2024-04-25T15:49:30.247 INFO:tasks.workunit.client.1.smithi193.stdout:6/129: unlink dd/f15 0 2024-04-25T15:49:30.248 INFO:tasks.workunit.client.1.smithi193.stdout:9/123: rmdir df/d16/d1c 39 2024-04-25T15:49:30.248 INFO:tasks.workunit.client.1.smithi193.stdout:6/130: read dd/f24 [2677763,38558] 0 2024-04-25T15:49:30.251 INFO:tasks.workunit.client.1.smithi193.stdout:4/127: dread f2 [0,4194304] 0 2024-04-25T15:49:30.251 INFO:tasks.workunit.client.1.smithi193.stdout:6/131: write dd/d12/d19/d1c/f1f [941093,116942] 0 2024-04-25T15:49:30.253 INFO:tasks.workunit.client.1.smithi193.stdout:5/87: dwrite fc [0,4194304] 0 2024-04-25T15:49:30.255 INFO:tasks.workunit.client.1.smithi193.stdout:5/88: truncate f12 180333 0 2024-04-25T15:49:30.256 INFO:tasks.workunit.client.1.smithi193.stdout:0/144: creat d0/d3/d5/d7/dd/d16/f2f x:0 0 0 2024-04-25T15:49:30.259 INFO:tasks.workunit.client.1.smithi193.stdout:3/171: truncate d8/d13/d1f/f16 5166792 0 2024-04-25T15:49:30.262 INFO:tasks.workunit.client.1.smithi193.stdout:6/132: dwrite dd/d12/d19/d1c/f20 [0,4194304] 0 2024-04-25T15:49:30.263 INFO:tasks.workunit.client.1.smithi193.stdout:6/133: truncate dd/d12/f1e 563346 0 2024-04-25T15:49:30.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/134: chown dd/d12/d19/d21/f23 8 1 2024-04-25T15:49:30.290 INFO:tasks.workunit.client.1.smithi193.stdout:7/79: sync 2024-04-25T15:49:30.295 INFO:tasks.workunit.client.1.smithi193.stdout:7/80: dwrite f7 [0,4194304] 0 2024-04-25T15:49:30.296 INFO:tasks.workunit.client.1.smithi193.stdout:7/81: write fe [865489,91821] 0 2024-04-25T15:49:30.297 INFO:tasks.workunit.client.1.smithi193.stdout:9/124: chown df/l12 3 1 2024-04-25T15:49:30.307 INFO:tasks.workunit.client.1.smithi193.stdout:2/137: sync 2024-04-25T15:49:30.309 INFO:tasks.workunit.client.1.smithi193.stdout:4/128: mknod db/d13/c26 0 2024-04-25T15:49:30.310 INFO:tasks.workunit.client.0.smithi110.stdout:2/457: sync 2024-04-25T15:49:30.312 INFO:tasks.workunit.client.1.smithi193.stdout:5/89: read f13 [1192728,58655] 0 2024-04-25T15:49:30.313 INFO:tasks.workunit.client.1.smithi193.stdout:0/145: creat d0/d3/d5/d7/dd/d16/f30 x:0 0 0 2024-04-25T15:49:30.313 INFO:tasks.workunit.client.0.smithi110.stdout:2/458: truncate de/d40/d4f/f47 1140503 0 2024-04-25T15:49:30.314 INFO:tasks.workunit.client.1.smithi193.stdout:2/138: dwrite d0/f33 [0,4194304] 0 2024-04-25T15:49:30.314 INFO:tasks.workunit.client.0.smithi110.stdout:2/459: fsync de/d40/d5b/d5c/d5f/f73 0 2024-04-25T15:49:30.316 INFO:tasks.workunit.client.1.smithi193.stdout:6/135: creat dd/d12/d19/d21/f25 x:0 0 0 2024-04-25T15:49:30.319 INFO:tasks.workunit.client.1.smithi193.stdout:7/82: mknod d12/c1b 0 2024-04-25T15:49:30.320 INFO:tasks.workunit.client.1.smithi193.stdout:0/146: dwrite d0/d3/d5/d7/dd/d16/f30 [0,4194304] 0 2024-04-25T15:49:30.322 INFO:tasks.workunit.client.0.smithi110.stdout:2/460: dwrite de/f56 [4194304,4194304] 0 2024-04-25T15:49:30.326 INFO:tasks.workunit.client.1.smithi193.stdout:0/147: dread d0/d3/d5/d7/dd/d16/d18/f22 [0,4194304] 0 2024-04-25T15:49:30.326 INFO:tasks.workunit.client.1.smithi193.stdout:4/129: stat db/c17 0 2024-04-25T15:49:30.327 INFO:tasks.workunit.client.1.smithi193.stdout:0/148: dread - d0/d3/d5/d7/dd/f1d zero size 2024-04-25T15:49:30.327 INFO:tasks.workunit.client.1.smithi193.stdout:8/119: truncate d2/d3/f16 439238 0 2024-04-25T15:49:30.329 INFO:tasks.workunit.client.1.smithi193.stdout:0/149: dread - d0/d3/d5/d7/dd/d16/d1a/f2b zero size 2024-04-25T15:49:30.329 INFO:tasks.workunit.client.0.smithi110.stdout:2/461: write de/d40/d6e/f39 [285751,57199] 0 2024-04-25T15:49:30.334 INFO:tasks.workunit.client.1.smithi193.stdout:1/94: creat dc/f18 x:0 0 0 2024-04-25T15:49:30.336 INFO:tasks.workunit.client.1.smithi193.stdout:5/90: creat d17/d18/f24 x:0 0 0 2024-04-25T15:49:30.337 INFO:tasks.workunit.client.0.smithi110.stdout:2/462: mknod de/c89 0 2024-04-25T15:49:30.342 INFO:tasks.workunit.client.1.smithi193.stdout:5/91: dwrite d17/d18/f21 [0,4194304] 0 2024-04-25T15:49:30.343 INFO:tasks.workunit.client.0.smithi110.stdout:2/463: rename de/d40/d6e/c17 to de/d40/c8a 0 2024-04-25T15:49:30.344 INFO:tasks.workunit.client.1.smithi193.stdout:9/125: truncate df/d16/f19 1096552 0 2024-04-25T15:49:30.344 INFO:tasks.workunit.client.1.smithi193.stdout:9/126: fdatasync f6 0 2024-04-25T15:49:30.346 INFO:tasks.workunit.client.1.smithi193.stdout:4/130: fdatasync f2 0 2024-04-25T15:49:30.347 INFO:tasks.workunit.client.1.smithi193.stdout:4/131: readlink db/d13/l1b 0 2024-04-25T15:49:30.353 INFO:tasks.workunit.client.0.smithi110.stdout:2/464: link de/d40/f35 de/d40/d4f/f8b 0 2024-04-25T15:49:30.354 INFO:tasks.workunit.client.1.smithi193.stdout:2/139: mkdir d0/d9/d36 0 2024-04-25T15:49:30.356 INFO:tasks.workunit.client.0.smithi110.stdout:2/465: mkdir de/d40/d5b/d8c 0 2024-04-25T15:49:30.359 INFO:tasks.workunit.client.1.smithi193.stdout:6/136: stat dd/f24 0 2024-04-25T15:49:30.361 INFO:tasks.workunit.client.0.smithi110.stdout:2/466: creat de/d40/d5b/d66/f8d x:0 0 0 2024-04-25T15:49:30.362 INFO:tasks.workunit.client.0.smithi110.stdout:2/467: chown de/d40/d5b/d5c/l75 331576973 1 2024-04-25T15:49:30.363 INFO:tasks.workunit.client.1.smithi193.stdout:4/132: fdatasync f2 0 2024-04-25T15:49:30.366 INFO:tasks.workunit.client.1.smithi193.stdout:3/172: write d8/d13/d14/f1d [1294433,62414] 0 2024-04-25T15:49:30.366 INFO:tasks.workunit.client.0.smithi110.stdout:2/468: fsync de/d40/d6e/f3a 0 2024-04-25T15:49:30.368 INFO:tasks.workunit.client.1.smithi193.stdout:8/120: dwrite d2/d3/ff [0,4194304] 0 2024-04-25T15:49:30.368 INFO:tasks.workunit.client.1.smithi193.stdout:2/140: rename d0/d9/d14 to d0/d37 0 2024-04-25T15:49:30.370 INFO:tasks.workunit.client.1.smithi193.stdout:8/121: read d2/f11 [944232,105369] 0 2024-04-25T15:49:30.372 INFO:tasks.workunit.client.1.smithi193.stdout:8/122: write d2/f11 [2175765,129508] 0 2024-04-25T15:49:30.373 INFO:tasks.workunit.client.1.smithi193.stdout:9/127: creat df/d16/d1c/f1f x:0 0 0 2024-04-25T15:49:30.374 INFO:tasks.workunit.client.1.smithi193.stdout:2/141: dread d0/d9/d30/f23 [4194304,4194304] 0 2024-04-25T15:49:30.375 INFO:tasks.workunit.client.1.smithi193.stdout:4/133: write f2 [2876687,86212] 0 2024-04-25T15:49:30.376 INFO:tasks.workunit.client.0.smithi110.stdout:2/469: getdents de/d40/d6e 0 2024-04-25T15:49:30.377 INFO:tasks.workunit.client.1.smithi193.stdout:1/95: creat dc/f19 x:0 0 0 2024-04-25T15:49:30.378 INFO:tasks.workunit.client.1.smithi193.stdout:5/92: link d17/l1e d17/l25 0 2024-04-25T15:49:30.380 INFO:tasks.workunit.client.1.smithi193.stdout:6/137: rename dd/d12/d19/d1c/f1f to dd/d12/d19/d1c/f26 0 2024-04-25T15:49:30.382 INFO:tasks.workunit.client.1.smithi193.stdout:2/142: unlink d0/df/f2b 0 2024-04-25T15:49:30.382 INFO:tasks.workunit.client.1.smithi193.stdout:2/143: dread - d0/df/f19 zero size 2024-04-25T15:49:30.387 INFO:tasks.workunit.client.1.smithi193.stdout:5/93: rename f15 to d17/f26 0 2024-04-25T15:49:30.388 INFO:tasks.workunit.client.1.smithi193.stdout:9/128: symlink df/d16/l20 0 2024-04-25T15:49:30.389 INFO:tasks.workunit.client.1.smithi193.stdout:4/134: creat db/d13/d1e/d1f/d22/f27 x:0 0 0 2024-04-25T15:49:30.390 INFO:tasks.workunit.client.1.smithi193.stdout:4/135: write f2 [1918338,125937] 0 2024-04-25T15:49:30.391 INFO:tasks.workunit.client.1.smithi193.stdout:1/96: mknod dc/c1a 0 2024-04-25T15:49:30.395 INFO:tasks.workunit.client.1.smithi193.stdout:9/129: fdatasync f4 0 2024-04-25T15:49:30.397 INFO:tasks.workunit.client.1.smithi193.stdout:5/94: dwrite d17/f26 [0,4194304] 0 2024-04-25T15:49:30.397 INFO:tasks.workunit.client.1.smithi193.stdout:2/144: mkdir d0/d37/d25/d38 0 2024-04-25T15:49:30.398 INFO:tasks.workunit.client.1.smithi193.stdout:3/173: getdents d8/d13/d14/d27 0 2024-04-25T15:49:30.398 INFO:tasks.workunit.client.1.smithi193.stdout:2/145: fdatasync d0/df/f26 0 2024-04-25T15:49:30.399 INFO:tasks.workunit.client.1.smithi193.stdout:3/174: write d8/f10 [4839223,72041] 0 2024-04-25T15:49:30.400 INFO:tasks.workunit.client.1.smithi193.stdout:4/136: creat db/d13/d1e/d1f/f28 x:0 0 0 2024-04-25T15:49:30.400 INFO:tasks.workunit.client.1.smithi193.stdout:4/137: stat l7 0 2024-04-25T15:49:30.403 INFO:tasks.workunit.client.1.smithi193.stdout:9/130: creat df/d16/d1c/f21 x:0 0 0 2024-04-25T15:49:30.403 INFO:tasks.workunit.client.1.smithi193.stdout:6/138: write dd/f11 [1407842,101258] 0 2024-04-25T15:49:30.404 INFO:tasks.workunit.client.1.smithi193.stdout:5/95: mkdir d17/d27 0 2024-04-25T15:49:30.407 INFO:tasks.workunit.client.1.smithi193.stdout:5/96: fsync d17/d18/f21 0 2024-04-25T15:49:30.408 INFO:tasks.workunit.client.1.smithi193.stdout:4/138: symlink db/l29 0 2024-04-25T15:49:30.410 INFO:tasks.workunit.client.1.smithi193.stdout:3/175: dwrite d8/f15 [0,4194304] 0 2024-04-25T15:49:30.411 INFO:tasks.workunit.client.1.smithi193.stdout:1/97: creat dc/f1b x:0 0 0 2024-04-25T15:49:30.413 INFO:tasks.workunit.client.1.smithi193.stdout:5/97: mknod d17/c28 0 2024-04-25T15:49:30.413 INFO:tasks.workunit.client.1.smithi193.stdout:8/123: dwrite d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:30.414 INFO:tasks.workunit.client.1.smithi193.stdout:9/131: mknod df/d16/c22 0 2024-04-25T15:49:30.416 INFO:tasks.workunit.client.1.smithi193.stdout:8/124: chown d2/c26 14510188 1 2024-04-25T15:49:30.417 INFO:tasks.workunit.client.1.smithi193.stdout:2/146: creat d0/d9/f39 x:0 0 0 2024-04-25T15:49:30.418 INFO:tasks.workunit.client.1.smithi193.stdout:3/176: symlink d8/d30/l35 0 2024-04-25T15:49:30.418 INFO:tasks.workunit.client.1.smithi193.stdout:8/125: stat d2/d1f 0 2024-04-25T15:49:30.419 INFO:tasks.workunit.client.1.smithi193.stdout:4/139: creat db/d13/d1e/d1f/d22/f2a x:0 0 0 2024-04-25T15:49:30.421 INFO:tasks.workunit.client.1.smithi193.stdout:1/98: dread dc/f15 [0,4194304] 0 2024-04-25T15:49:30.421 INFO:tasks.workunit.client.1.smithi193.stdout:5/98: creat d17/d18/f29 x:0 0 0 2024-04-25T15:49:30.422 INFO:tasks.workunit.client.1.smithi193.stdout:6/139: dwrite f6 [4194304,4194304] 0 2024-04-25T15:49:30.425 INFO:tasks.workunit.client.1.smithi193.stdout:5/99: dread - d17/f1f zero size 2024-04-25T15:49:30.428 INFO:tasks.workunit.client.1.smithi193.stdout:2/147: mkdir d0/d9/d36/d3a 0 2024-04-25T15:49:30.428 INFO:tasks.workunit.client.1.smithi193.stdout:3/177: creat d8/d30/f36 x:0 0 0 2024-04-25T15:49:30.428 INFO:tasks.workunit.client.1.smithi193.stdout:4/140: mknod db/d13/c2b 0 2024-04-25T15:49:30.430 INFO:tasks.workunit.client.1.smithi193.stdout:5/100: write d17/d18/f24 [656813,50318] 0 2024-04-25T15:49:30.431 INFO:tasks.workunit.client.1.smithi193.stdout:6/140: rename dd/d12/l16 to dd/d12/l27 0 2024-04-25T15:49:30.431 INFO:tasks.workunit.client.1.smithi193.stdout:4/141: write f1 [5491228,64469] 0 2024-04-25T15:49:30.432 INFO:tasks.workunit.client.1.smithi193.stdout:3/178: mknod d8/c37 0 2024-04-25T15:49:30.434 INFO:tasks.workunit.client.1.smithi193.stdout:1/99: mknod dc/c1c 0 2024-04-25T15:49:30.435 INFO:tasks.workunit.client.1.smithi193.stdout:9/132: dwrite fc [0,4194304] 0 2024-04-25T15:49:30.435 INFO:tasks.workunit.client.1.smithi193.stdout:5/101: creat d17/f2a x:0 0 0 2024-04-25T15:49:30.436 INFO:tasks.workunit.client.1.smithi193.stdout:7/83: sync 2024-04-25T15:49:30.438 INFO:tasks.workunit.client.1.smithi193.stdout:6/141: dread - dd/d12/d19/d21/f23 zero size 2024-04-25T15:49:30.439 INFO:tasks.workunit.client.1.smithi193.stdout:9/133: readlink df/d16/l20 0 2024-04-25T15:49:30.442 INFO:tasks.workunit.client.1.smithi193.stdout:4/142: creat db/d13/d1e/d1f/d22/f2c x:0 0 0 2024-04-25T15:49:30.443 INFO:tasks.workunit.client.1.smithi193.stdout:8/126: dwrite d2/d3/d20/f22 [0,4194304] 0 2024-04-25T15:49:30.443 INFO:tasks.workunit.client.1.smithi193.stdout:5/102: write d17/f26 [1127196,96106] 0 2024-04-25T15:49:30.445 INFO:tasks.workunit.client.1.smithi193.stdout:7/84: write d12/f1a [974295,116729] 0 2024-04-25T15:49:30.445 INFO:tasks.workunit.client.1.smithi193.stdout:2/148: dread d0/f6 [0,4194304] 0 2024-04-25T15:49:30.446 INFO:tasks.workunit.client.1.smithi193.stdout:8/127: chown d2/d3/f16 556810544 1 2024-04-25T15:49:30.447 INFO:tasks.workunit.client.1.smithi193.stdout:8/128: fdatasync d2/d3/f16 0 2024-04-25T15:49:30.449 INFO:tasks.workunit.client.1.smithi193.stdout:1/100: dread dc/fe [0,4194304] 0 2024-04-25T15:49:30.449 INFO:tasks.workunit.client.1.smithi193.stdout:9/134: creat df/d16/d1c/f23 x:0 0 0 2024-04-25T15:49:30.450 INFO:tasks.workunit.client.1.smithi193.stdout:9/135: stat ca 0 2024-04-25T15:49:30.451 INFO:tasks.workunit.client.1.smithi193.stdout:7/85: chown f8 2 1 2024-04-25T15:49:30.453 INFO:tasks.workunit.client.1.smithi193.stdout:5/103: mknod d17/c2b 0 2024-04-25T15:49:30.456 INFO:tasks.workunit.client.1.smithi193.stdout:4/143: rename c8 to db/d13/d1e/d1f/d22/c2d 0 2024-04-25T15:49:30.460 INFO:tasks.workunit.client.1.smithi193.stdout:3/179: truncate d8/f15 551264 0 2024-04-25T15:49:30.462 INFO:tasks.workunit.client.1.smithi193.stdout:7/86: mkdir d12/d1c 0 2024-04-25T15:49:30.463 INFO:tasks.workunit.client.1.smithi193.stdout:9/136: chown f9 11 1 2024-04-25T15:49:30.464 INFO:tasks.workunit.client.1.smithi193.stdout:8/129: getdents d2/d1f 0 2024-04-25T15:49:30.465 INFO:tasks.workunit.client.1.smithi193.stdout:2/149: dwrite d0/f6 [0,4194304] 0 2024-04-25T15:49:30.465 INFO:tasks.workunit.client.1.smithi193.stdout:4/144: dwrite fa [0,4194304] 0 2024-04-25T15:49:30.466 INFO:tasks.workunit.client.1.smithi193.stdout:1/101: symlink dc/l1d 0 2024-04-25T15:49:30.466 INFO:tasks.workunit.client.1.smithi193.stdout:8/130: stat d2/d3/c17 0 2024-04-25T15:49:30.467 INFO:tasks.workunit.client.1.smithi193.stdout:0/150: sync 2024-04-25T15:49:30.470 INFO:tasks.workunit.client.1.smithi193.stdout:4/145: read - db/d13/d1e/d1f/d22/f2c zero size 2024-04-25T15:49:30.471 INFO:tasks.workunit.client.1.smithi193.stdout:0/151: chown d0/la 600759 1 2024-04-25T15:49:30.471 INFO:tasks.workunit.client.1.smithi193.stdout:6/142: link dd/d12/l18 dd/d12/d19/l28 0 2024-04-25T15:49:30.472 INFO:tasks.workunit.client.1.smithi193.stdout:7/87: creat d12/d1c/f1d x:0 0 0 2024-04-25T15:49:30.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/150: creat d0/d37/d25/d38/f3b x:0 0 0 2024-04-25T15:49:30.477 INFO:tasks.workunit.client.1.smithi193.stdout:5/104: dread fe [0,4194304] 0 2024-04-25T15:49:30.477 INFO:tasks.workunit.client.1.smithi193.stdout:6/143: write dd/d12/d19/f1d [129758,26673] 0 2024-04-25T15:49:30.478 INFO:tasks.workunit.client.1.smithi193.stdout:9/137: dwrite f6 [0,4194304] 0 2024-04-25T15:49:30.478 INFO:tasks.workunit.client.1.smithi193.stdout:6/144: readlink l9 0 2024-04-25T15:49:30.478 INFO:tasks.workunit.client.1.smithi193.stdout:7/88: mkdir d12/d1e 0 2024-04-25T15:49:30.479 INFO:tasks.workunit.client.1.smithi193.stdout:5/105: write fc [686464,11770] 0 2024-04-25T15:49:30.482 INFO:tasks.workunit.client.1.smithi193.stdout:3/180: getdents d8/d30 0 2024-04-25T15:49:30.482 INFO:tasks.workunit.client.1.smithi193.stdout:9/138: write df/d16/f1e [571401,127968] 0 2024-04-25T15:49:30.483 INFO:tasks.workunit.client.1.smithi193.stdout:8/131: truncate d2/d3/f16 272273 0 2024-04-25T15:49:30.486 INFO:tasks.workunit.client.1.smithi193.stdout:4/146: dread f4 [0,4194304] 0 2024-04-25T15:49:30.486 INFO:tasks.workunit.client.1.smithi193.stdout:9/139: write f6 [3925680,37695] 0 2024-04-25T15:49:30.487 INFO:tasks.workunit.client.1.smithi193.stdout:5/106: rename fc to d17/d18/f2c 0 2024-04-25T15:49:30.489 INFO:tasks.workunit.client.1.smithi193.stdout:2/151: mkdir d0/d9/d3c 0 2024-04-25T15:49:30.493 INFO:tasks.workunit.client.1.smithi193.stdout:3/181: mknod d8/d13/d14/d27/c38 0 2024-04-25T15:49:30.494 INFO:tasks.workunit.client.1.smithi193.stdout:0/152: link d0/c23 d0/d3/d5/c31 0 2024-04-25T15:49:30.494 INFO:tasks.workunit.client.1.smithi193.stdout:0/153: dread - d0/d3/d5/d7/dd/f1d zero size 2024-04-25T15:49:30.495 INFO:tasks.workunit.client.1.smithi193.stdout:1/102: truncate f1 2780585 0 2024-04-25T15:49:30.497 INFO:tasks.workunit.client.1.smithi193.stdout:9/140: stat l0 0 2024-04-25T15:49:30.498 INFO:tasks.workunit.client.1.smithi193.stdout:4/147: rmdir db 39 2024-04-25T15:49:30.498 INFO:tasks.workunit.client.1.smithi193.stdout:4/148: read f1 [4595996,126993] 0 2024-04-25T15:49:30.500 INFO:tasks.workunit.client.1.smithi193.stdout:5/107: truncate fe 192063 0 2024-04-25T15:49:30.500 INFO:tasks.workunit.client.1.smithi193.stdout:7/89: link c11 d12/d1e/c1f 0 2024-04-25T15:49:30.501 INFO:tasks.workunit.client.1.smithi193.stdout:8/132: getdents d2/d1f 0 2024-04-25T15:49:30.501 INFO:tasks.workunit.client.1.smithi193.stdout:7/90: write d12/f18 [498645,31719] 0 2024-04-25T15:49:30.502 INFO:tasks.workunit.client.1.smithi193.stdout:3/182: creat d8/f39 x:0 0 0 2024-04-25T15:49:30.503 INFO:tasks.workunit.client.1.smithi193.stdout:3/183: chown d8/l2c 376224026 1 2024-04-25T15:49:30.503 INFO:tasks.workunit.client.1.smithi193.stdout:9/141: unlink df/c15 0 2024-04-25T15:49:30.505 INFO:tasks.workunit.client.1.smithi193.stdout:5/108: dread d17/d18/f21 [0,4194304] 0 2024-04-25T15:49:30.505 INFO:tasks.workunit.client.1.smithi193.stdout:4/149: readlink db/lf 0 2024-04-25T15:49:30.505 INFO:tasks.workunit.client.1.smithi193.stdout:6/145: getdents dd 0 2024-04-25T15:49:30.506 INFO:tasks.workunit.client.1.smithi193.stdout:2/152: mkdir d0/d9/d3d 0 2024-04-25T15:49:30.506 INFO:tasks.workunit.client.1.smithi193.stdout:4/150: read - db/d13/d1e/d1f/d22/f27 zero size 2024-04-25T15:49:30.507 INFO:tasks.workunit.client.1.smithi193.stdout:2/153: fdatasync d0/d9/f22 0 2024-04-25T15:49:30.507 INFO:tasks.workunit.client.1.smithi193.stdout:7/91: mknod d12/c20 0 2024-04-25T15:49:30.510 INFO:tasks.workunit.client.1.smithi193.stdout:5/109: write d17/d18/f2c [3563993,105153] 0 2024-04-25T15:49:30.511 INFO:tasks.workunit.client.1.smithi193.stdout:6/146: creat dd/d12/d19/d1c/f29 x:0 0 0 2024-04-25T15:49:30.512 INFO:tasks.workunit.client.1.smithi193.stdout:5/110: chown c10 8381 1 2024-04-25T15:49:30.513 INFO:tasks.workunit.client.1.smithi193.stdout:8/133: fsync d2/d3/db/f1a 0 2024-04-25T15:49:30.513 INFO:tasks.workunit.client.1.smithi193.stdout:4/151: dwrite f2 [0,4194304] 0 2024-04-25T15:49:30.514 INFO:tasks.workunit.client.1.smithi193.stdout:8/134: chown d2/c26 2088901 1 2024-04-25T15:49:30.515 INFO:tasks.workunit.client.1.smithi193.stdout:6/147: truncate dd/d12/d19/d1c/f26 1989494 0 2024-04-25T15:49:30.517 INFO:tasks.workunit.client.1.smithi193.stdout:1/103: creat dc/f1e x:0 0 0 2024-04-25T15:49:30.518 INFO:tasks.workunit.client.1.smithi193.stdout:7/92: mknod d12/d1c/c21 0 2024-04-25T15:49:30.520 INFO:tasks.workunit.client.1.smithi193.stdout:5/111: dwrite d17/d18/f20 [0,4194304] 0 2024-04-25T15:49:30.522 INFO:tasks.workunit.client.1.smithi193.stdout:5/112: truncate d17/f2a 774673 0 2024-04-25T15:49:30.525 INFO:tasks.workunit.client.1.smithi193.stdout:4/152: fdatasync f4 0 2024-04-25T15:49:30.526 INFO:tasks.workunit.client.1.smithi193.stdout:8/135: symlink d2/l28 0 2024-04-25T15:49:30.527 INFO:tasks.workunit.client.1.smithi193.stdout:6/148: symlink dd/d12/d19/l2a 0 2024-04-25T15:49:30.528 INFO:tasks.workunit.client.1.smithi193.stdout:0/154: getdents d0/d3/d5/d7/dd/d16/d18 0 2024-04-25T15:49:30.530 INFO:tasks.workunit.client.1.smithi193.stdout:0/155: read - d0/d3/d5/d7/dd/f1d zero size 2024-04-25T15:49:30.530 INFO:tasks.workunit.client.1.smithi193.stdout:6/149: truncate dd/d12/d19/d1c/f29 807251 0 2024-04-25T15:49:30.531 INFO:tasks.workunit.client.1.smithi193.stdout:3/184: write d8/ff [110849,83798] 0 2024-04-25T15:49:30.532 INFO:tasks.workunit.client.1.smithi193.stdout:2/154: creat d0/d9/d3c/f3e x:0 0 0 2024-04-25T15:49:30.534 INFO:tasks.workunit.client.1.smithi193.stdout:7/93: rename f10 to d12/d1c/f22 0 2024-04-25T15:49:30.534 INFO:tasks.workunit.client.1.smithi193.stdout:4/153: dwrite f2 [0,4194304] 0 2024-04-25T15:49:30.538 INFO:tasks.workunit.client.1.smithi193.stdout:8/136: mkdir d2/d3/db/d29 0 2024-04-25T15:49:30.540 INFO:tasks.workunit.client.1.smithi193.stdout:1/104: rmdir dc 39 2024-04-25T15:49:30.541 INFO:tasks.workunit.client.1.smithi193.stdout:4/154: truncate db/d13/d1e/d1f/d22/f27 635764 0 2024-04-25T15:49:30.542 INFO:tasks.workunit.client.1.smithi193.stdout:6/150: dwrite fb [0,4194304] 0 2024-04-25T15:49:30.542 INFO:tasks.workunit.client.1.smithi193.stdout:3/185: dwrite d8/fc [4194304,4194304] 0 2024-04-25T15:49:30.543 INFO:tasks.workunit.client.1.smithi193.stdout:6/151: fdatasync f6 0 2024-04-25T15:49:30.544 INFO:tasks.workunit.client.1.smithi193.stdout:4/155: chown db/d13/f16 1 1 2024-04-25T15:49:30.546 INFO:tasks.workunit.client.1.smithi193.stdout:2/155: creat d0/d9/d36/f3f x:0 0 0 2024-04-25T15:49:30.547 INFO:tasks.workunit.client.1.smithi193.stdout:9/142: getdents df 0 2024-04-25T15:49:30.550 INFO:tasks.workunit.client.1.smithi193.stdout:9/143: readlink df/d16/l20 0 2024-04-25T15:49:30.551 INFO:tasks.workunit.client.1.smithi193.stdout:8/137: mknod d2/d3/d20/c2a 0 2024-04-25T15:49:30.554 INFO:tasks.workunit.client.1.smithi193.stdout:4/156: symlink db/d13/d1e/l2e 0 2024-04-25T15:49:30.556 INFO:tasks.workunit.client.1.smithi193.stdout:5/113: rmdir d17/d27 0 2024-04-25T15:49:30.559 INFO:tasks.workunit.client.1.smithi193.stdout:9/144: truncate df/d16/d1c/f21 874173 0 2024-04-25T15:49:30.559 INFO:tasks.workunit.client.1.smithi193.stdout:0/156: getdents d0/d3/d5/d7/dd/d16/d18/d28 0 2024-04-25T15:49:30.562 INFO:tasks.workunit.client.1.smithi193.stdout:3/186: dwrite d8/d13/f32 [0,4194304] 0 2024-04-25T15:49:30.565 INFO:tasks.workunit.client.1.smithi193.stdout:6/152: dwrite dd/f11 [0,4194304] 0 2024-04-25T15:49:30.565 INFO:tasks.workunit.client.1.smithi193.stdout:4/157: unlink db/d13/f16 0 2024-04-25T15:49:30.568 INFO:tasks.workunit.client.1.smithi193.stdout:1/105: mknod dc/c1f 0 2024-04-25T15:49:30.568 INFO:tasks.workunit.client.1.smithi193.stdout:9/145: rename ce to df/d11/c24 0 2024-04-25T15:49:30.569 INFO:tasks.workunit.client.1.smithi193.stdout:4/158: mknod db/d13/d1e/d1f/d22/c2f 0 2024-04-25T15:49:30.570 INFO:tasks.workunit.client.1.smithi193.stdout:2/156: dwrite d0/d9/d15/f2e [0,4194304] 0 2024-04-25T15:49:30.570 INFO:tasks.workunit.client.1.smithi193.stdout:3/187: chown d8/d13/d1f/l34 811 1 2024-04-25T15:49:30.574 INFO:tasks.workunit.client.1.smithi193.stdout:9/146: dread f6 [0,4194304] 0 2024-04-25T15:49:30.576 INFO:tasks.workunit.client.1.smithi193.stdout:0/157: symlink d0/d3/d5/d7/dd/d16/d18/d28/l32 0 2024-04-25T15:49:30.577 INFO:tasks.workunit.client.1.smithi193.stdout:2/157: rmdir d0/d37 39 2024-04-25T15:49:30.578 INFO:tasks.workunit.client.1.smithi193.stdout:2/158: write d0/d9/d15/f2e [3431654,96715] 0 2024-04-25T15:49:30.579 INFO:tasks.workunit.client.1.smithi193.stdout:9/147: dread fc [4194304,4194304] 0 2024-04-25T15:49:30.582 INFO:tasks.workunit.client.1.smithi193.stdout:6/153: link dd/f11 dd/f2b 0 2024-04-25T15:49:30.583 INFO:tasks.workunit.client.1.smithi193.stdout:6/154: write f6 [6545352,73840] 0 2024-04-25T15:49:30.584 INFO:tasks.workunit.client.1.smithi193.stdout:3/188: rename d8/d13/d14/f21 to d8/d30/d31/f3a 0 2024-04-25T15:49:30.585 INFO:tasks.workunit.client.1.smithi193.stdout:1/106: mknod dc/c20 0 2024-04-25T15:49:30.586 INFO:tasks.workunit.client.1.smithi193.stdout:3/189: dread d8/f15 [0,4194304] 0 2024-04-25T15:49:30.587 INFO:tasks.workunit.client.1.smithi193.stdout:5/114: getdents d17 0 2024-04-25T15:49:30.592 INFO:tasks.workunit.client.1.smithi193.stdout:2/159: rename d0/d9/d30/f23 to d0/d9/d30/f40 0 2024-04-25T15:49:30.593 INFO:tasks.workunit.client.1.smithi193.stdout:8/138: truncate d2/d3/ff 1761589 0 2024-04-25T15:49:30.595 INFO:tasks.workunit.client.1.smithi193.stdout:3/190: dwrite d8/f10 [0,4194304] 0 2024-04-25T15:49:30.596 INFO:tasks.workunit.client.1.smithi193.stdout:5/115: dwrite d17/d18/f24 [0,4194304] 0 2024-04-25T15:49:30.598 INFO:tasks.workunit.client.1.smithi193.stdout:0/158: rename d0/d3/d5/d7/dd/f1d to d0/d3/d5/d7/dd/d16/f33 0 2024-04-25T15:49:30.598 INFO:tasks.workunit.client.1.smithi193.stdout:1/107: stat dc/cd 0 2024-04-25T15:49:30.607 INFO:tasks.workunit.client.1.smithi193.stdout:3/191: mknod d8/d30/c3b 0 2024-04-25T15:49:30.608 INFO:tasks.workunit.client.1.smithi193.stdout:4/159: truncate fa 1069290 0 2024-04-25T15:49:30.610 INFO:tasks.workunit.client.1.smithi193.stdout:1/108: fdatasync dc/fe 0 2024-04-25T15:49:30.610 INFO:tasks.workunit.client.1.smithi193.stdout:1/109: chown dc/fe 834 1 2024-04-25T15:49:30.614 INFO:tasks.workunit.client.1.smithi193.stdout:6/155: chown dd/l14 1947 1 2024-04-25T15:49:30.614 INFO:tasks.workunit.client.1.smithi193.stdout:9/148: creat df/d16/f25 x:0 0 0 2024-04-25T15:49:30.615 INFO:tasks.workunit.client.1.smithi193.stdout:4/160: dwrite db/d13/d1e/d1f/d22/f2a [0,4194304] 0 2024-04-25T15:49:30.622 INFO:tasks.workunit.client.1.smithi193.stdout:0/159: creat d0/d3/d5/d7/dd/d16/d26/d2e/f34 x:0 0 0 2024-04-25T15:49:30.624 INFO:tasks.workunit.client.1.smithi193.stdout:8/139: write d2/fa [296750,1872] 0 2024-04-25T15:49:30.624 INFO:tasks.workunit.client.1.smithi193.stdout:4/161: symlink db/d13/d1e/l30 0 2024-04-25T15:49:30.626 INFO:tasks.workunit.client.1.smithi193.stdout:2/160: link d0/l12 d0/d9/d30/l41 0 2024-04-25T15:49:30.626 INFO:tasks.workunit.client.1.smithi193.stdout:2/161: chown d0/d9/d3d 6938 1 2024-04-25T15:49:30.626 INFO:tasks.workunit.client.1.smithi193.stdout:5/116: getdents d17 0 2024-04-25T15:49:30.627 INFO:tasks.workunit.client.1.smithi193.stdout:6/156: dread dd/fe [4194304,4194304] 0 2024-04-25T15:49:30.630 INFO:tasks.workunit.client.1.smithi193.stdout:3/192: rename d8/d13/d14/d27/f29 to d8/d13/f3c 0 2024-04-25T15:49:30.630 INFO:tasks.workunit.client.1.smithi193.stdout:4/162: dread db/d13/d1e/d1f/d22/f2a [0,4194304] 0 2024-04-25T15:49:30.632 INFO:tasks.workunit.client.1.smithi193.stdout:3/193: chown d8/d13/d1f/c25 0 1 2024-04-25T15:49:30.632 INFO:tasks.workunit.client.1.smithi193.stdout:1/110: creat dc/f21 x:0 0 0 2024-04-25T15:49:30.633 INFO:tasks.workunit.client.1.smithi193.stdout:6/157: dwrite f6 [0,4194304] 0 2024-04-25T15:49:30.633 INFO:tasks.workunit.client.1.smithi193.stdout:5/117: symlink d17/d18/l2d 0 2024-04-25T15:49:30.635 INFO:tasks.workunit.client.1.smithi193.stdout:0/160: link d0/d3/d5/d7/dd/d16/f30 d0/f35 0 2024-04-25T15:49:30.636 INFO:tasks.workunit.client.1.smithi193.stdout:0/161: chown d0/d3/d5/f24 8 1 2024-04-25T15:49:30.636 INFO:tasks.workunit.client.1.smithi193.stdout:8/140: creat d2/d1f/f2b x:0 0 0 2024-04-25T15:49:30.637 INFO:tasks.workunit.client.1.smithi193.stdout:2/162: mkdir d0/d42 0 2024-04-25T15:49:30.638 INFO:tasks.workunit.client.1.smithi193.stdout:2/163: dread - d0/f29 zero size 2024-04-25T15:49:30.641 INFO:tasks.workunit.client.1.smithi193.stdout:3/194: symlink d8/d13/d14/l3d 0 2024-04-25T15:49:30.641 INFO:tasks.workunit.client.1.smithi193.stdout:5/118: dwrite d17/d18/f21 [4194304,4194304] 0 2024-04-25T15:49:30.642 INFO:tasks.workunit.client.1.smithi193.stdout:3/195: readlink d8/d13/d1f/l24 0 2024-04-25T15:49:30.643 INFO:tasks.workunit.client.1.smithi193.stdout:7/94: sync 2024-04-25T15:49:30.644 INFO:tasks.workunit.client.1.smithi193.stdout:9/149: getdents df/d11 0 2024-04-25T15:49:30.646 INFO:tasks.workunit.client.1.smithi193.stdout:0/162: symlink d0/d3/d5/d7/dd/d16/l36 0 2024-04-25T15:49:30.646 INFO:tasks.workunit.client.1.smithi193.stdout:7/95: write f7 [2693119,106817] 0 2024-04-25T15:49:30.647 INFO:tasks.workunit.client.1.smithi193.stdout:4/163: dwrite db/d13/d1e/d1f/d22/f2a [4194304,4194304] 0 2024-04-25T15:49:30.649 INFO:tasks.workunit.client.1.smithi193.stdout:5/119: creat d17/d18/f2e x:0 0 0 2024-04-25T15:49:30.650 INFO:tasks.workunit.client.1.smithi193.stdout:5/120: stat d17/f1d 0 2024-04-25T15:49:30.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/158: creat dd/d12/d19/d1c/f2c x:0 0 0 2024-04-25T15:49:30.653 INFO:tasks.workunit.client.1.smithi193.stdout:9/150: mkdir df/d16/d1c/d26 0 2024-04-25T15:49:30.655 INFO:tasks.workunit.client.1.smithi193.stdout:8/141: mknod d2/d3/db/d29/c2c 0 2024-04-25T15:49:30.657 INFO:tasks.workunit.client.1.smithi193.stdout:0/163: stat d0/d3/d5/c31 0 2024-04-25T15:49:30.659 INFO:tasks.workunit.client.1.smithi193.stdout:2/164: chown d0/d37/d25/l34 0 1 2024-04-25T15:49:30.661 INFO:tasks.workunit.client.1.smithi193.stdout:4/164: symlink db/d13/d1e/d1f/d22/l31 0 2024-04-25T15:49:30.662 INFO:tasks.workunit.client.1.smithi193.stdout:5/121: mknod d17/c2f 0 2024-04-25T15:49:30.664 INFO:tasks.workunit.client.1.smithi193.stdout:3/196: rename d8/d13/f3c to d8/d13/d14/f3e 0 2024-04-25T15:49:30.664 INFO:tasks.workunit.client.1.smithi193.stdout:1/111: dwrite f1 [0,4194304] 0 2024-04-25T15:49:30.666 INFO:tasks.workunit.client.1.smithi193.stdout:1/112: fsync dc/f21 0 2024-04-25T15:49:30.666 INFO:tasks.workunit.client.1.smithi193.stdout:6/159: chown dd/f1b 62 1 2024-04-25T15:49:30.667 INFO:tasks.workunit.client.1.smithi193.stdout:4/165: dread f9 [4194304,4194304] 0 2024-04-25T15:49:30.668 INFO:tasks.workunit.client.1.smithi193.stdout:8/142: dread d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:30.669 INFO:tasks.workunit.client.1.smithi193.stdout:5/122: mkdir d17/d18/d30 0 2024-04-25T15:49:30.669 INFO:tasks.workunit.client.1.smithi193.stdout:6/160: truncate dd/d12/d19/d1c/f22 695937 0 2024-04-25T15:49:30.671 INFO:tasks.workunit.client.1.smithi193.stdout:7/96: rename d12/c20 to d12/d1c/c23 0 2024-04-25T15:49:30.672 INFO:tasks.workunit.client.1.smithi193.stdout:9/151: rename df/d16/d1c to df/d16/d1c/d26/d27 22 2024-04-25T15:49:30.673 INFO:tasks.workunit.client.1.smithi193.stdout:1/113: dread - dc/f12 zero size 2024-04-25T15:49:30.673 INFO:tasks.workunit.client.1.smithi193.stdout:6/161: dread dd/d12/d19/d1c/f29 [0,4194304] 0 2024-04-25T15:49:30.674 INFO:tasks.workunit.client.1.smithi193.stdout:4/166: mkdir db/d32 0 2024-04-25T15:49:30.675 INFO:tasks.workunit.client.1.smithi193.stdout:1/114: dread f6 [0,4194304] 0 2024-04-25T15:49:30.677 INFO:tasks.workunit.client.1.smithi193.stdout:8/143: dwrite d2/d18/f25 [0,4194304] 0 2024-04-25T15:49:30.677 INFO:tasks.workunit.client.1.smithi193.stdout:8/144: chown d2/d3/db 502 1 2024-04-25T15:49:30.679 INFO:tasks.workunit.client.1.smithi193.stdout:7/97: unlink fe 0 2024-04-25T15:49:30.679 INFO:tasks.workunit.client.1.smithi193.stdout:7/98: chown d12/d13/c16 116342166 1 2024-04-25T15:49:30.683 INFO:tasks.workunit.client.1.smithi193.stdout:2/165: creat d0/d37/f43 x:0 0 0 2024-04-25T15:49:30.683 INFO:tasks.workunit.client.1.smithi193.stdout:3/197: rename d8/d30/l35 to d8/d13/d14/d27/l3f 0 2024-04-25T15:49:30.684 INFO:tasks.workunit.client.1.smithi193.stdout:6/162: fdatasync dd/fe 0 2024-04-25T15:49:30.686 INFO:tasks.workunit.client.1.smithi193.stdout:0/164: truncate d0/d3/fe 1019496 0 2024-04-25T15:49:30.687 INFO:tasks.workunit.client.1.smithi193.stdout:4/167: symlink db/d13/d1e/d1f/l33 0 2024-04-25T15:49:30.692 INFO:tasks.workunit.client.1.smithi193.stdout:8/145: chown d2/d3/db/l12 7520987 1 2024-04-25T15:49:30.692 INFO:tasks.workunit.client.1.smithi193.stdout:8/146: write d2/fa [208212,118368] 0 2024-04-25T15:49:30.693 INFO:tasks.workunit.client.1.smithi193.stdout:7/99: stat d12/d1c/c23 0 2024-04-25T15:49:30.693 INFO:tasks.workunit.client.1.smithi193.stdout:8/147: dread - d2/d1f/f2b zero size 2024-04-25T15:49:30.695 INFO:tasks.workunit.client.1.smithi193.stdout:9/152: rename df to df/d16/d28 22 2024-04-25T15:49:30.697 INFO:tasks.workunit.client.1.smithi193.stdout:5/123: truncate d17/d18/f24 2649674 0 2024-04-25T15:49:30.698 INFO:tasks.workunit.client.1.smithi193.stdout:5/124: readlink d17/l1c 0 2024-04-25T15:49:30.698 INFO:tasks.workunit.client.1.smithi193.stdout:6/163: stat dd/d12/l18 0 2024-04-25T15:49:30.699 INFO:tasks.workunit.client.1.smithi193.stdout:0/165: symlink d0/d3/d5/d7/dd/d16/l37 0 2024-04-25T15:49:30.700 INFO:tasks.workunit.client.1.smithi193.stdout:1/115: mknod dc/c22 0 2024-04-25T15:49:30.702 INFO:tasks.workunit.client.1.smithi193.stdout:7/100: fsync d12/d1c/f22 0 2024-04-25T15:49:30.702 INFO:tasks.workunit.client.1.smithi193.stdout:0/166: chown d0/d3/d5/l11 1935459 1 2024-04-25T15:49:30.702 INFO:tasks.workunit.client.1.smithi193.stdout:8/148: dwrite d2/d3/db/f19 [0,4194304] 0 2024-04-25T15:49:30.703 INFO:tasks.workunit.client.1.smithi193.stdout:2/166: mkdir d0/d44 0 2024-04-25T15:49:30.704 INFO:tasks.workunit.client.1.smithi193.stdout:9/153: dwrite df/d16/f1e [0,4194304] 0 2024-04-25T15:49:30.705 INFO:tasks.workunit.client.1.smithi193.stdout:2/167: fsync d0/d9/d36/f3f 0 2024-04-25T15:49:30.708 INFO:tasks.workunit.client.1.smithi193.stdout:3/198: truncate d8/d30/d31/f3a 854503 0 2024-04-25T15:49:30.709 INFO:tasks.workunit.client.1.smithi193.stdout:5/125: mkdir d17/d31 0 2024-04-25T15:49:30.709 INFO:tasks.workunit.client.1.smithi193.stdout:9/154: chown fc 677 1 2024-04-25T15:49:30.713 INFO:tasks.workunit.client.1.smithi193.stdout:3/199: chown d8/d13/d14/d27/c38 152492 1 2024-04-25T15:49:30.714 INFO:tasks.workunit.client.1.smithi193.stdout:5/126: truncate d17/f1f 553030 0 2024-04-25T15:49:30.714 INFO:tasks.workunit.client.1.smithi193.stdout:8/149: mknod d2/d3/d27/c2d 0 2024-04-25T15:49:30.714 INFO:tasks.workunit.client.1.smithi193.stdout:7/101: dwrite f3 [0,4194304] 0 2024-04-25T15:49:30.716 INFO:tasks.workunit.client.1.smithi193.stdout:2/168: creat d0/d9/d3d/f45 x:0 0 0 2024-04-25T15:49:30.721 INFO:tasks.workunit.client.1.smithi193.stdout:6/164: dwrite dd/f11 [0,4194304] 0 2024-04-25T15:49:30.721 INFO:tasks.workunit.client.1.smithi193.stdout:9/155: dwrite f6 [0,4194304] 0 2024-04-25T15:49:30.723 INFO:tasks.workunit.client.1.smithi193.stdout:6/165: dread - dd/d12/d19/d21/f23 zero size 2024-04-25T15:49:30.731 INFO:tasks.workunit.client.1.smithi193.stdout:9/156: dwrite f6 [0,4194304] 0 2024-04-25T15:49:30.794 INFO:tasks.workunit.client.1.smithi193.stdout:3/200: creat d8/d30/f40 x:0 0 0 2024-04-25T15:49:30.796 INFO:tasks.workunit.client.1.smithi193.stdout:7/102: creat d12/d13/f24 x:0 0 0 2024-04-25T15:49:30.850 INFO:tasks.workunit.client.1.smithi193.stdout:6/166: mkdir dd/d12/d2d 0 2024-04-25T15:49:30.850 INFO:tasks.workunit.client.1.smithi193.stdout:6/167: readlink dd/d12/d19/l2a 0 2024-04-25T15:49:30.850 INFO:tasks.workunit.client.1.smithi193.stdout:2/169: dread d0/d9/d30/f40 [4194304,4194304] 0 2024-04-25T15:49:30.852 INFO:tasks.workunit.client.1.smithi193.stdout:1/116: link c4 dc/c23 0 2024-04-25T15:49:30.854 INFO:tasks.workunit.client.1.smithi193.stdout:6/168: dread f6 [4194304,4194304] 0 2024-04-25T15:49:30.854 INFO:tasks.workunit.client.1.smithi193.stdout:5/127: creat d17/d18/d30/f32 x:0 0 0 2024-04-25T15:49:30.855 INFO:tasks.workunit.client.1.smithi193.stdout:3/201: rmdir d8/d13/d1f 39 2024-04-25T15:49:30.856 INFO:tasks.workunit.client.1.smithi193.stdout:4/168: truncate f2 4146047 0 2024-04-25T15:49:30.856 INFO:tasks.workunit.client.1.smithi193.stdout:3/202: stat d8/d13/d14/d27 0 2024-04-25T15:49:30.857 INFO:tasks.workunit.client.1.smithi193.stdout:4/169: chown db/d13/c2b 1814091264 1 2024-04-25T15:49:30.858 INFO:tasks.workunit.client.1.smithi193.stdout:3/203: dread - d8/d30/f40 zero size 2024-04-25T15:49:30.859 INFO:tasks.workunit.client.1.smithi193.stdout:2/170: creat d0/d9/d30/f46 x:0 0 0 2024-04-25T15:49:30.861 INFO:tasks.workunit.client.1.smithi193.stdout:6/169: dwrite dd/d12/d19/d1c/f22 [0,4194304] 0 2024-04-25T15:49:30.862 INFO:tasks.workunit.client.1.smithi193.stdout:0/167: fsync d0/d3/fe 0 2024-04-25T15:49:30.863 INFO:tasks.workunit.client.1.smithi193.stdout:4/170: dread f4 [0,4194304] 0 2024-04-25T15:49:30.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:30 smithi110 ceph-mon[26344]: pgmap v32: 65 pgs: 65 active+clean; 1.3 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 35 MiB/s rd, 145 MiB/s wr, 309 op/s 2024-04-25T15:49:30.863 INFO:tasks.workunit.client.1.smithi193.stdout:5/128: mknod d17/d18/c33 0 2024-04-25T15:49:30.864 INFO:tasks.workunit.client.1.smithi193.stdout:9/157: write f4 [3321065,80237] 0 2024-04-25T15:49:30.864 INFO:tasks.workunit.client.1.smithi193.stdout:7/103: creat d12/d1e/f25 x:0 0 0 2024-04-25T15:49:30.866 INFO:tasks.workunit.client.1.smithi193.stdout:0/168: truncate d0/d3/d5/d7/dd/d16/d18/f2a 414624 0 2024-04-25T15:49:30.871 INFO:tasks.workunit.client.1.smithi193.stdout:1/117: symlink dc/l24 0 2024-04-25T15:49:30.873 INFO:tasks.workunit.client.1.smithi193.stdout:0/169: dread d0/f12 [0,4194304] 0 2024-04-25T15:49:30.873 INFO:tasks.workunit.client.1.smithi193.stdout:8/150: getdents d2 0 2024-04-25T15:49:30.873 INFO:tasks.workunit.client.1.smithi193.stdout:2/171: rename d0/d9/f22 to d0/d44/f47 0 2024-04-25T15:49:30.877 INFO:tasks.workunit.client.1.smithi193.stdout:4/171: unlink db/d13/d1e/l30 0 2024-04-25T15:49:30.878 INFO:tasks.workunit.client.1.smithi193.stdout:7/104: symlink d12/d1e/l26 0 2024-04-25T15:49:30.880 INFO:tasks.workunit.client.1.smithi193.stdout:8/151: dwrite d2/d18/f25 [4194304,4194304] 0 2024-04-25T15:49:30.884 INFO:tasks.workunit.client.1.smithi193.stdout:0/170: mkdir d0/d3/d5/d7/dd/d38 0 2024-04-25T15:49:30.886 INFO:tasks.workunit.client.1.smithi193.stdout:8/152: dread d2/d3/d20/f22 [0,4194304] 0 2024-04-25T15:49:30.888 INFO:tasks.workunit.client.1.smithi193.stdout:5/129: dwrite f13 [0,4194304] 0 2024-04-25T15:49:30.888 INFO:tasks.workunit.client.1.smithi193.stdout:4/172: rename db/c1c to db/c34 0 2024-04-25T15:49:30.889 INFO:tasks.workunit.client.1.smithi193.stdout:4/173: fdatasync db/fc 0 2024-04-25T15:49:30.889 INFO:tasks.workunit.client.1.smithi193.stdout:3/204: creat d8/d13/f41 x:0 0 0 2024-04-25T15:49:30.892 INFO:tasks.workunit.client.1.smithi193.stdout:7/105: rename c5 to d12/d13/c27 0 2024-04-25T15:49:30.893 INFO:tasks.workunit.client.1.smithi193.stdout:1/118: mknod dc/c25 0 2024-04-25T15:49:30.894 INFO:tasks.workunit.client.1.smithi193.stdout:1/119: write f1 [310215,81232] 0 2024-04-25T15:49:30.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/170: creat dd/f2e x:0 0 0 2024-04-25T15:49:30.901 INFO:tasks.workunit.client.1.smithi193.stdout:0/171: rmdir d0 39 2024-04-25T15:49:30.903 INFO:tasks.workunit.client.1.smithi193.stdout:9/158: write df/f1b [939920,5358] 0 2024-04-25T15:49:30.905 INFO:tasks.workunit.client.1.smithi193.stdout:5/130: mkdir d17/d18/d30/d34 0 2024-04-25T15:49:30.907 INFO:tasks.workunit.client.1.smithi193.stdout:3/205: write d8/d13/d1f/f22 [1724722,121367] 0 2024-04-25T15:49:30.907 INFO:tasks.workunit.client.1.smithi193.stdout:4/174: rename db/fe to db/f35 0 2024-04-25T15:49:30.909 INFO:tasks.workunit.client.1.smithi193.stdout:8/153: rename d2/d3/d20/l21 to d2/l2e 0 2024-04-25T15:49:30.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/159: dwrite df/d16/f1e [0,4194304] 0 2024-04-25T15:49:30.910 INFO:tasks.workunit.client.1.smithi193.stdout:7/106: creat d12/d1c/f28 x:0 0 0 2024-04-25T15:49:30.910 INFO:tasks.workunit.client.1.smithi193.stdout:8/154: readlink d2/d3/db/l12 0 2024-04-25T15:49:30.911 INFO:tasks.workunit.client.1.smithi193.stdout:8/155: read d2/d18/f25 [7275718,100008] 0 2024-04-25T15:49:30.914 INFO:tasks.workunit.client.1.smithi193.stdout:4/175: rename db/l29 to db/l36 0 2024-04-25T15:49:30.915 INFO:tasks.workunit.client.1.smithi193.stdout:1/120: mknod dc/c26 0 2024-04-25T15:49:30.916 INFO:tasks.workunit.client.1.smithi193.stdout:1/121: chown f1 710115 1 2024-04-25T15:49:30.916 INFO:tasks.workunit.client.1.smithi193.stdout:7/107: unlink d12/d1c/f22 0 2024-04-25T15:49:30.917 INFO:tasks.workunit.client.1.smithi193.stdout:6/171: dread dd/d12/d19/d1c/f22 [0,4194304] 0 2024-04-25T15:49:30.917 INFO:tasks.workunit.client.1.smithi193.stdout:9/160: mknod df/d11/c29 0 2024-04-25T15:49:30.919 INFO:tasks.workunit.client.1.smithi193.stdout:0/172: dread d0/d3/d5/d7/dd/d16/d18/f22 [0,4194304] 0 2024-04-25T15:49:30.920 INFO:tasks.workunit.client.1.smithi193.stdout:9/161: dread df/d16/d1c/f21 [0,4194304] 0 2024-04-25T15:49:30.920 INFO:tasks.workunit.client.1.smithi193.stdout:8/156: mkdir d2/d3/d2f 0 2024-04-25T15:49:30.921 INFO:tasks.workunit.client.1.smithi193.stdout:5/131: fdatasync fe 0 2024-04-25T15:49:30.921 INFO:tasks.workunit.client.1.smithi193.stdout:1/122: dread dc/f16 [0,4194304] 0 2024-04-25T15:49:30.923 INFO:tasks.workunit.client.1.smithi193.stdout:7/108: mknod d12/c29 0 2024-04-25T15:49:30.925 INFO:tasks.workunit.client.1.smithi193.stdout:6/172: truncate dd/f24 4948139 0 2024-04-25T15:49:30.926 INFO:tasks.workunit.client.1.smithi193.stdout:8/157: symlink d2/d1f/l30 0 2024-04-25T15:49:30.927 INFO:tasks.workunit.client.1.smithi193.stdout:0/173: dread d0/f12 [0,4194304] 0 2024-04-25T15:49:30.928 INFO:tasks.workunit.client.1.smithi193.stdout:9/162: rename f6 to df/d11/f2a 0 2024-04-25T15:49:30.929 INFO:tasks.workunit.client.1.smithi193.stdout:8/158: dread d2/d3/db/f1a [0,4194304] 0 2024-04-25T15:49:30.929 INFO:tasks.workunit.client.1.smithi193.stdout:2/172: truncate d0/d9/d15/f2e 77934 0 2024-04-25T15:49:30.930 INFO:tasks.workunit.client.1.smithi193.stdout:7/109: dwrite d12/d13/f14 [0,4194304] 0 2024-04-25T15:49:30.932 INFO:tasks.workunit.client.1.smithi193.stdout:7/110: read f3 [1442046,41405] 0 2024-04-25T15:49:30.935 INFO:tasks.workunit.client.1.smithi193.stdout:8/159: dread d2/d3/d20/f22 [0,4194304] 0 2024-04-25T15:49:30.937 INFO:tasks.workunit.client.1.smithi193.stdout:6/173: rmdir dd/d12/d19/d21 39 2024-04-25T15:49:30.937 INFO:tasks.workunit.client.1.smithi193.stdout:6/174: fdatasync dd/f24 0 2024-04-25T15:49:30.941 INFO:tasks.workunit.client.1.smithi193.stdout:5/132: rename f12 to d17/d18/d30/f35 0 2024-04-25T15:49:30.943 INFO:tasks.workunit.client.1.smithi193.stdout:9/163: symlink df/d16/d1c/l2b 0 2024-04-25T15:49:30.943 INFO:tasks.workunit.client.1.smithi193.stdout:9/164: readlink l7 0 2024-04-25T15:49:30.944 INFO:tasks.workunit.client.1.smithi193.stdout:0/174: symlink d0/d3/d5/d7/dd/l39 0 2024-04-25T15:49:30.948 INFO:tasks.workunit.client.1.smithi193.stdout:7/111: symlink d12/l2a 0 2024-04-25T15:49:30.954 INFO:tasks.workunit.client.1.smithi193.stdout:9/165: mkdir df/d2c 0 2024-04-25T15:49:30.955 INFO:tasks.workunit.client.1.smithi193.stdout:0/175: fsync d0/f35 0 2024-04-25T15:49:30.956 INFO:tasks.workunit.client.1.smithi193.stdout:0/176: chown d0/d3/d5/d7/l21 95 1 2024-04-25T15:49:30.961 INFO:tasks.workunit.client.1.smithi193.stdout:6/175: mkdir dd/d2f 0 2024-04-25T15:49:30.962 INFO:tasks.workunit.client.1.smithi193.stdout:4/176: truncate db/d13/d1e/d1f/d22/f2a 85693 0 2024-04-25T15:49:30.965 INFO:tasks.workunit.client.1.smithi193.stdout:1/123: rename dc/l10 to dc/l27 0 2024-04-25T15:49:30.966 INFO:tasks.workunit.client.1.smithi193.stdout:9/166: dread f4 [0,4194304] 0 2024-04-25T15:49:30.968 INFO:tasks.workunit.client.1.smithi193.stdout:8/160: rmdir d2/d3/d2f 0 2024-04-25T15:49:30.968 INFO:tasks.workunit.client.1.smithi193.stdout:8/161: chown d2/d3/db/l12 39448 1 2024-04-25T15:49:30.969 INFO:tasks.workunit.client.1.smithi193.stdout:4/177: readlink l3 0 2024-04-25T15:49:30.970 INFO:tasks.workunit.client.1.smithi193.stdout:5/133: rmdir d17/d31 0 2024-04-25T15:49:30.975 INFO:tasks.workunit.client.0.smithi110.stdout:8/571: sync 2024-04-25T15:49:30.975 INFO:tasks.workunit.client.0.smithi110.stdout:1/570: sync 2024-04-25T15:49:30.975 INFO:tasks.workunit.client.0.smithi110.stdout:7/443: sync 2024-04-25T15:49:30.978 INFO:tasks.workunit.client.0.smithi110.stdout:1/571: symlink d0/d25/lae 0 2024-04-25T15:49:30.980 INFO:tasks.workunit.client.0.smithi110.stdout:8/572: mknod d14/d21/d26/dab/dac/dbb/cc4 0 2024-04-25T15:49:30.981 INFO:tasks.workunit.client.0.smithi110.stdout:7/444: dread d1b/d29/d38/d8b/d89/f8c [0,4194304] 0 2024-04-25T15:49:30.983 INFO:tasks.workunit.client.0.smithi110.stdout:7/445: chown d1b/d29/d38/d55/f6b 63760 1 2024-04-25T15:49:30.984 INFO:tasks.workunit.client.0.smithi110.stdout:8/573: creat d14/d21/d26/d6a/d7e/da6/fc5 x:0 0 0 2024-04-25T15:49:30.985 INFO:tasks.workunit.client.0.smithi110.stdout:7/446: chown d1b/d29/d46/f5c 203958066 1 2024-04-25T15:49:30.985 INFO:tasks.workunit.client.0.smithi110.stdout:1/572: dwrite d0/d14/d81/f9d [0,4194304] 0 2024-04-25T15:49:30.986 INFO:tasks.workunit.client.0.smithi110.stdout:1/573: fdatasync d0/d14/d5f/d7d/f67 0 2024-04-25T15:49:30.988 INFO:tasks.workunit.client.0.smithi110.stdout:8/574: unlink d14/d32/d36/d48/f69 0 2024-04-25T15:49:30.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:30 smithi193 ceph-mon[33747]: pgmap v32: 65 pgs: 65 active+clean; 1.3 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 35 MiB/s rd, 145 MiB/s wr, 309 op/s 2024-04-25T15:49:30.990 INFO:tasks.workunit.client.0.smithi110.stdout:8/575: write d14/d4c/d71/d9d/fa1 [905777,107383] 0 2024-04-25T15:49:30.992 INFO:tasks.workunit.client.0.smithi110.stdout:7/447: dread f2 [0,4194304] 0 2024-04-25T15:49:30.994 INFO:tasks.workunit.client.0.smithi110.stdout:8/576: symlink d14/d4c/d71/d9d/lc6 0 2024-04-25T15:49:30.995 INFO:tasks.workunit.client.0.smithi110.stdout:8/577: dread - d14/f81 zero size 2024-04-25T15:49:30.997 INFO:tasks.workunit.client.0.smithi110.stdout:7/448: creat d1b/d29/d38/d8b/f92 x:0 0 0 2024-04-25T15:49:31.001 INFO:tasks.workunit.client.0.smithi110.stdout:7/449: rename l18 to d1b/d29/d87/l93 0 2024-04-25T15:49:31.004 INFO:tasks.workunit.client.0.smithi110.stdout:8/578: dwrite d14/d4c/d71/d9d/fb0 [0,4194304] 0 2024-04-25T15:49:31.005 INFO:tasks.workunit.client.0.smithi110.stdout:1/574: dwrite d0/d14/f53 [0,4194304] 0 2024-04-25T15:49:31.009 INFO:tasks.workunit.client.0.smithi110.stdout:1/575: creat d0/d51/d6b/faf x:0 0 0 2024-04-25T15:49:31.012 INFO:tasks.workunit.client.0.smithi110.stdout:7/450: dwrite d1b/d29/d38/d8b/d89/f8c [0,4194304] 0 2024-04-25T15:49:31.013 INFO:tasks.workunit.client.0.smithi110.stdout:7/451: dread - d1b/f6e zero size 2024-04-25T15:49:31.016 INFO:tasks.workunit.client.0.smithi110.stdout:7/452: creat d1b/d29/d87/d8e/f94 x:0 0 0 2024-04-25T15:49:31.018 INFO:tasks.workunit.client.0.smithi110.stdout:7/453: mkdir d1b/d29/d38/d55/d95 0 2024-04-25T15:49:31.020 INFO:tasks.workunit.client.0.smithi110.stdout:7/454: creat d1b/d29/d38/d8b/d89/f96 x:0 0 0 2024-04-25T15:49:31.020 INFO:tasks.workunit.client.0.smithi110.stdout:7/455: stat d1b/c49 0 2024-04-25T15:49:31.026 INFO:tasks.workunit.client.0.smithi110.stdout:7/456: dwrite d1b/d29/d38/d8b/d89/f8c [4194304,4194304] 0 2024-04-25T15:49:31.031 INFO:tasks.workunit.client.0.smithi110.stdout:7/457: creat d1b/f97 x:0 0 0 2024-04-25T15:49:31.034 INFO:tasks.workunit.client.0.smithi110.stdout:7/458: truncate f17 1241067 0 2024-04-25T15:49:31.034 INFO:tasks.workunit.client.1.smithi193.stdout:9/167: fsync f4 0 2024-04-25T15:49:31.035 INFO:tasks.workunit.client.1.smithi193.stdout:1/124: symlink dc/l28 0 2024-04-25T15:49:31.035 INFO:tasks.workunit.client.1.smithi193.stdout:0/177: write d0/d3/d5/d7/dd/d16/f30 [4977842,106430] 0 2024-04-25T15:49:31.037 INFO:tasks.workunit.client.1.smithi193.stdout:2/173: dwrite d0/d9/d30/f40 [0,4194304] 0 2024-04-25T15:49:31.037 INFO:tasks.workunit.client.1.smithi193.stdout:9/168: fdatasync df/d16/d1c/f23 0 2024-04-25T15:49:31.037 INFO:tasks.workunit.client.0.smithi110.stdout:7/459: creat d1b/d29/f98 x:0 0 0 2024-04-25T15:49:31.038 INFO:tasks.workunit.client.0.smithi110.stdout:8/579: dread d14/d21/d26/f30 [0,4194304] 0 2024-04-25T15:49:31.039 INFO:tasks.workunit.client.1.smithi193.stdout:2/174: fdatasync d0/d9/d36/f3f 0 2024-04-25T15:49:31.039 INFO:tasks.workunit.client.1.smithi193.stdout:4/178: creat db/d32/f37 x:0 0 0 2024-04-25T15:49:31.039 INFO:tasks.workunit.client.0.smithi110.stdout:8/580: chown d14/d21/d26/d6a 297774 1 2024-04-25T15:49:31.039 INFO:tasks.workunit.client.1.smithi193.stdout:5/134: mknod d17/d18/d30/d34/c36 0 2024-04-25T15:49:31.042 INFO:tasks.workunit.client.0.smithi110.stdout:7/460: mknod d1b/d29/c99 0 2024-04-25T15:49:31.044 INFO:tasks.workunit.client.0.smithi110.stdout:8/581: fdatasync d14/d21/d26/f98 0 2024-04-25T15:49:31.056 INFO:tasks.workunit.client.0.smithi110.stdout:7/461: write d1b/d29/d38/f5b [2578325,99794] 0 2024-04-25T15:49:31.056 INFO:tasks.workunit.client.0.smithi110.stdout:8/582: write d14/d21/d23/f24 [1145956,102059] 0 2024-04-25T15:49:31.057 INFO:tasks.workunit.client.0.smithi110.stdout:8/583: fsync fb 0 2024-04-25T15:49:31.058 INFO:tasks.workunit.client.0.smithi110.stdout:7/462: dread - d1b/d29/d38/d55/d7f/f64 zero size 2024-04-25T15:49:31.060 INFO:tasks.workunit.client.0.smithi110.stdout:7/463: symlink d1b/d29/d38/d55/d7f/l9a 0 2024-04-25T15:49:31.061 INFO:tasks.workunit.client.0.smithi110.stdout:7/464: write d1b/d29/d46/f47 [3628307,66578] 0 2024-04-25T15:49:31.062 INFO:tasks.workunit.client.0.smithi110.stdout:8/584: link d14/f80 d14/d21/d26/d6a/fc7 0 2024-04-25T15:49:31.063 INFO:tasks.workunit.client.0.smithi110.stdout:7/465: readlink d1b/d29/l30 0 2024-04-25T15:49:31.064 INFO:tasks.workunit.client.0.smithi110.stdout:8/585: write d14/d32/d36/d48/f5e [4160052,39513] 0 2024-04-25T15:49:31.065 INFO:tasks.workunit.client.0.smithi110.stdout:8/586: chown d14/d21/d26/c43 6 1 2024-04-25T15:49:31.068 INFO:tasks.workunit.client.0.smithi110.stdout:8/587: creat d14/d21/fc8 x:0 0 0 2024-04-25T15:49:31.071 INFO:tasks.workunit.client.1.smithi193.stdout:7/112: sync 2024-04-25T15:49:31.072 INFO:tasks.workunit.client.1.smithi193.stdout:9/169: rmdir df/d16 39 2024-04-25T15:49:31.072 INFO:tasks.workunit.client.0.smithi110.stdout:8/588: getdents d14/d21/d26/d6a/d7e 0 2024-04-25T15:49:31.072 INFO:tasks.workunit.client.1.smithi193.stdout:0/178: creat d0/d3/d5/d7/dd/d16/f3a x:0 0 0 2024-04-25T15:49:31.073 INFO:tasks.workunit.client.1.smithi193.stdout:1/125: mknod dc/c29 0 2024-04-25T15:49:31.073 INFO:tasks.workunit.client.1.smithi193.stdout:4/179: creat db/f38 x:0 0 0 2024-04-25T15:49:31.073 INFO:tasks.workunit.client.1.smithi193.stdout:7/113: chown d12/d1e/f25 1 1 2024-04-25T15:49:31.074 INFO:tasks.workunit.client.1.smithi193.stdout:1/126: write f1 [296678,110465] 0 2024-04-25T15:49:31.074 INFO:tasks.workunit.client.1.smithi193.stdout:5/135: rmdir d17 39 2024-04-25T15:49:31.075 INFO:tasks.workunit.client.1.smithi193.stdout:0/179: write d0/d3/d5/f24 [358138,102412] 0 2024-04-25T15:49:31.075 INFO:tasks.workunit.client.1.smithi193.stdout:7/114: write f7 [1817652,104088] 0 2024-04-25T15:49:31.075 INFO:tasks.workunit.client.0.smithi110.stdout:7/466: dwrite d1b/d29/f37 [0,4194304] 0 2024-04-25T15:49:31.077 INFO:tasks.workunit.client.1.smithi193.stdout:7/115: truncate d12/d13/f17 1011007 0 2024-04-25T15:49:31.077 INFO:tasks.workunit.client.0.smithi110.stdout:8/589: dread d14/d21/f62 [0,4194304] 0 2024-04-25T15:49:31.079 INFO:tasks.workunit.client.1.smithi193.stdout:1/127: dread dc/fe [0,4194304] 0 2024-04-25T15:49:31.080 INFO:tasks.workunit.client.0.smithi110.stdout:7/467: symlink d1b/d29/d46/l9b 0 2024-04-25T15:49:31.080 INFO:tasks.workunit.client.1.smithi193.stdout:4/180: fsync f4 0 2024-04-25T15:49:31.080 INFO:tasks.workunit.client.0.smithi110.stdout:7/468: fdatasync d1b/d29/d38/f53 0 2024-04-25T15:49:31.081 INFO:tasks.workunit.client.1.smithi193.stdout:4/181: fsync db/d32/f37 0 2024-04-25T15:49:31.083 INFO:tasks.workunit.client.1.smithi193.stdout:5/136: read d17/d18/f2c [1450505,25918] 0 2024-04-25T15:49:31.085 INFO:tasks.workunit.client.0.smithi110.stdout:7/469: creat d1b/d29/d38/d55/d95/f9c x:0 0 0 2024-04-25T15:49:31.086 INFO:tasks.workunit.client.1.smithi193.stdout:9/170: dread df/d16/d1c/f21 [0,4194304] 0 2024-04-25T15:49:31.086 INFO:tasks.workunit.client.1.smithi193.stdout:1/128: chown dc/l13 0 1 2024-04-25T15:49:31.086 INFO:tasks.workunit.client.1.smithi193.stdout:4/182: mknod db/d32/c39 0 2024-04-25T15:49:31.087 INFO:tasks.workunit.client.0.smithi110.stdout:7/470: symlink d1b/d29/d46/l9d 0 2024-04-25T15:49:31.088 INFO:tasks.workunit.client.0.smithi110.stdout:7/471: fdatasync d1b/d29/d38/f61 0 2024-04-25T15:49:31.089 INFO:tasks.workunit.client.1.smithi193.stdout:2/175: rename d0/f33 to d0/d37/f48 0 2024-04-25T15:49:31.090 INFO:tasks.workunit.client.1.smithi193.stdout:5/137: rmdir d17/d18/d30 39 2024-04-25T15:49:31.091 INFO:tasks.workunit.client.1.smithi193.stdout:2/176: read - d0/d9/d15/f35 zero size 2024-04-25T15:49:31.092 INFO:tasks.workunit.client.0.smithi110.stdout:7/472: chown d1b/d29/d38/d55/d7f/c63 150792 1 2024-04-25T15:49:31.093 INFO:tasks.workunit.client.1.smithi193.stdout:5/138: dread d17/f2a [0,4194304] 0 2024-04-25T15:49:31.093 INFO:tasks.workunit.client.0.smithi110.stdout:7/473: readlink d1b/d29/d38/d55/d7f/l9a 0 2024-04-25T15:49:31.093 INFO:tasks.workunit.client.1.smithi193.stdout:2/177: write d0/d9/d15/f35 [874946,13693] 0 2024-04-25T15:49:31.093 INFO:tasks.workunit.client.1.smithi193.stdout:1/129: dwrite dc/f18 [0,4194304] 0 2024-04-25T15:49:31.095 INFO:tasks.workunit.client.1.smithi193.stdout:2/178: dread - d0/d37/f43 zero size 2024-04-25T15:49:31.095 INFO:tasks.workunit.client.1.smithi193.stdout:4/183: read f1 [1094938,117815] 0 2024-04-25T15:49:31.095 INFO:tasks.workunit.client.1.smithi193.stdout:1/130: write dc/f1e [167646,14229] 0 2024-04-25T15:49:31.097 INFO:tasks.workunit.client.1.smithi193.stdout:1/131: write dc/f17 [180545,112419] 0 2024-04-25T15:49:31.098 INFO:tasks.workunit.client.1.smithi193.stdout:1/132: write dc/f19 [284137,29259] 0 2024-04-25T15:49:31.100 INFO:tasks.workunit.client.1.smithi193.stdout:0/180: rename d0/d3/d5/l11 to d0/d3/d5/d7/dd/l3b 0 2024-04-25T15:49:31.100 INFO:tasks.workunit.client.1.smithi193.stdout:1/133: write dc/f1b [254828,8560] 0 2024-04-25T15:49:31.101 INFO:tasks.workunit.client.0.smithi110.stdout:7/474: dwrite d1b/f88 [0,4194304] 0 2024-04-25T15:49:31.101 INFO:tasks.workunit.client.1.smithi193.stdout:1/134: write dc/f19 [1293278,39136] 0 2024-04-25T15:49:31.102 INFO:tasks.workunit.client.0.smithi110.stdout:8/590: dwrite d14/d32/f61 [0,4194304] 0 2024-04-25T15:49:31.106 INFO:tasks.workunit.client.1.smithi193.stdout:9/171: mkdir df/d2d 0 2024-04-25T15:49:31.106 INFO:tasks.workunit.client.0.smithi110.stdout:8/591: write d14/d21/d26/dab/fb1 [752037,127763] 0 2024-04-25T15:49:31.111 INFO:tasks.workunit.client.1.smithi193.stdout:4/184: dread db/d13/f1d [0,4194304] 0 2024-04-25T15:49:31.112 INFO:tasks.workunit.client.0.smithi110.stdout:8/592: write d14/d21/f62 [236901,120174] 0 2024-04-25T15:49:31.115 INFO:tasks.workunit.client.1.smithi193.stdout:7/116: rename d12/c1b to d12/d1e/c2b 0 2024-04-25T15:49:31.116 INFO:tasks.workunit.client.1.smithi193.stdout:7/117: stat cf 0 2024-04-25T15:49:31.117 INFO:tasks.workunit.client.0.smithi110.stdout:8/593: mkdir d14/d32/d36/db7/dc9 0 2024-04-25T15:49:31.117 INFO:tasks.workunit.client.1.smithi193.stdout:7/118: chown d12/d1c/f1d 24724461 1 2024-04-25T15:49:31.120 INFO:tasks.workunit.client.1.smithi193.stdout:5/139: mkdir d17/d18/d30/d34/d37 0 2024-04-25T15:49:31.124 INFO:tasks.workunit.client.1.smithi193.stdout:4/185: mknod db/d13/d1e/d1f/c3a 0 2024-04-25T15:49:31.124 INFO:tasks.workunit.client.1.smithi193.stdout:4/186: stat c6 0 2024-04-25T15:49:31.125 INFO:tasks.workunit.client.1.smithi193.stdout:4/187: write db/d13/f21 [524151,114265] 0 2024-04-25T15:49:31.125 INFO:tasks.workunit.client.1.smithi193.stdout:1/135: symlink dc/l2a 0 2024-04-25T15:49:31.128 INFO:tasks.workunit.client.1.smithi193.stdout:5/140: mkdir d17/d18/d30/d38 0 2024-04-25T15:49:31.128 INFO:tasks.workunit.client.1.smithi193.stdout:5/141: stat d17/f2a 0 2024-04-25T15:49:31.129 INFO:tasks.workunit.client.1.smithi193.stdout:5/142: fsync d17/d18/f29 0 2024-04-25T15:49:31.130 INFO:tasks.workunit.client.1.smithi193.stdout:9/172: write fc [7639166,52313] 0 2024-04-25T15:49:31.131 INFO:tasks.workunit.client.0.smithi110.stdout:7/475: dwrite f17 [0,4194304] 0 2024-04-25T15:49:31.133 INFO:tasks.workunit.client.1.smithi193.stdout:4/188: creat db/f3b x:0 0 0 2024-04-25T15:49:31.136 INFO:tasks.workunit.client.0.smithi110.stdout:8/594: dread d14/f1d [0,4194304] 0 2024-04-25T15:49:31.137 INFO:tasks.workunit.client.0.smithi110.stdout:7/476: dread d1b/f48 [0,4194304] 0 2024-04-25T15:49:31.142 INFO:tasks.workunit.client.0.smithi110.stdout:8/595: mknod d14/d4c/d71/d9d/cca 0 2024-04-25T15:49:31.143 INFO:tasks.workunit.client.0.smithi110.stdout:7/477: dwrite f19 [0,4194304] 0 2024-04-25T15:49:31.144 INFO:tasks.workunit.client.1.smithi193.stdout:5/143: unlink d17/d18/c23 0 2024-04-25T15:49:31.145 INFO:tasks.workunit.client.1.smithi193.stdout:0/181: dwrite d0/fc [0,4194304] 0 2024-04-25T15:49:31.146 INFO:tasks.workunit.client.1.smithi193.stdout:1/136: creat dc/f2b x:0 0 0 2024-04-25T15:49:31.147 INFO:tasks.workunit.client.0.smithi110.stdout:8/596: fsync d14/d32/d36/d48/fb3 0 2024-04-25T15:49:31.147 INFO:tasks.workunit.client.0.smithi110.stdout:4/539: sync 2024-04-25T15:49:31.147 INFO:tasks.workunit.client.0.smithi110.stdout:6/428: sync 2024-04-25T15:49:31.148 INFO:tasks.workunit.client.0.smithi110.stdout:4/540: chown d13/l28 925083661 1 2024-04-25T15:49:31.151 INFO:tasks.workunit.client.1.smithi193.stdout:9/173: link f4 df/d11/f2e 0 2024-04-25T15:49:31.151 INFO:tasks.workunit.client.0.smithi110.stdout:7/478: creat d1b/d29/d38/d55/d95/f9e x:0 0 0 2024-04-25T15:49:31.152 INFO:tasks.workunit.client.1.smithi193.stdout:9/174: chown df/f1b 111675 1 2024-04-25T15:49:31.153 INFO:tasks.workunit.client.1.smithi193.stdout:5/144: dwrite f13 [0,4194304] 0 2024-04-25T15:49:31.156 INFO:tasks.workunit.client.1.smithi193.stdout:7/119: getdents d12/d13 0 2024-04-25T15:49:31.158 INFO:tasks.workunit.client.0.smithi110.stdout:8/597: symlink d14/d32/d36/db7/lcb 0 2024-04-25T15:49:31.160 INFO:tasks.workunit.client.0.smithi110.stdout:8/598: write d14/d21/fc8 [765299,106755] 0 2024-04-25T15:49:31.160 INFO:tasks.workunit.client.1.smithi193.stdout:3/206: sync 2024-04-25T15:49:31.161 INFO:tasks.workunit.client.0.smithi110.stdout:8/599: stat d14/d21/d26/dab/dac/dbb 0 2024-04-25T15:49:31.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/207: write d8/d13/f41 [336609,69972] 0 2024-04-25T15:49:31.162 INFO:tasks.workunit.client.0.smithi110.stdout:8/600: write d14/f1b [5550647,65851] 0 2024-04-25T15:49:31.162 INFO:tasks.workunit.client.1.smithi193.stdout:3/208: write d8/d13/f41 [667742,61091] 0 2024-04-25T15:49:31.163 INFO:tasks.workunit.client.0.smithi110.stdout:6/429: creat d1/d12/d3d/f9d x:0 0 0 2024-04-25T15:49:31.165 INFO:tasks.workunit.client.0.smithi110.stdout:6/430: read - d1/db/d24/d2d/f96 zero size 2024-04-25T15:49:31.167 INFO:tasks.workunit.client.0.smithi110.stdout:4/541: creat d13/d24/d2d/d44/da4/fb3 x:0 0 0 2024-04-25T15:49:31.168 INFO:tasks.workunit.client.0.smithi110.stdout:7/479: write d1b/d29/d38/d55/d7f/f72 [3006094,26013] 0 2024-04-25T15:49:31.169 INFO:tasks.workunit.client.0.smithi110.stdout:7/480: readlink l3 0 2024-04-25T15:49:31.174 INFO:tasks.workunit.client.0.smithi110.stdout:8/601: symlink d14/d21/d26/d6a/db9/lcc 0 2024-04-25T15:49:31.176 INFO:tasks.workunit.client.1.smithi193.stdout:8/162: sync 2024-04-25T15:49:31.176 INFO:tasks.workunit.client.1.smithi193.stdout:6/176: sync 2024-04-25T15:49:31.176 INFO:tasks.workunit.client.1.smithi193.stdout:2/179: sync 2024-04-25T15:49:31.177 INFO:tasks.workunit.client.1.smithi193.stdout:2/180: stat d0/d9/d15/f35 0 2024-04-25T15:49:31.177 INFO:tasks.workunit.client.1.smithi193.stdout:2/181: chown d0/d9/d3c/f3e 798873790 1 2024-04-25T15:49:31.178 INFO:tasks.workunit.client.0.smithi110.stdout:6/431: rename d1/d12/d6d to d1/db/d22/d90/d9e 0 2024-04-25T15:49:31.178 INFO:tasks.workunit.client.1.smithi193.stdout:2/182: write d0/d9/d30/f40 [8981410,58681] 0 2024-04-25T15:49:31.179 INFO:tasks.workunit.client.1.smithi193.stdout:2/183: dread - d0/d9/d3c/f3e zero size 2024-04-25T15:49:31.180 INFO:tasks.workunit.client.1.smithi193.stdout:4/189: write f2 [4483702,88653] 0 2024-04-25T15:49:31.181 INFO:tasks.workunit.client.1.smithi193.stdout:4/190: write db/fc [4066736,3943] 0 2024-04-25T15:49:31.181 INFO:tasks.workunit.client.1.smithi193.stdout:7/120: creat d12/f2c x:0 0 0 2024-04-25T15:49:31.182 INFO:tasks.workunit.client.1.smithi193.stdout:4/191: chown db/d13/d1e/d1f/l33 9720457 1 2024-04-25T15:49:31.184 INFO:tasks.workunit.client.0.smithi110.stdout:7/481: rmdir d1b 39 2024-04-25T15:49:31.184 INFO:tasks.workunit.client.1.smithi193.stdout:7/121: dread d12/f1a [0,4194304] 0 2024-04-25T15:49:31.186 INFO:tasks.workunit.client.0.smithi110.stdout:6/432: dwrite d1/d12/d3d/f78 [0,4194304] 0 2024-04-25T15:49:31.186 INFO:tasks.workunit.client.0.smithi110.stdout:8/602: fdatasync d14/d21/d23/f25 0 2024-04-25T15:49:31.187 INFO:tasks.workunit.client.0.smithi110.stdout:4/542: dread d13/d24/d2d/f37 [0,4194304] 0 2024-04-25T15:49:31.196 INFO:tasks.workunit.client.0.smithi110.stdout:7/482: read d1b/d29/d38/d8b/d89/f8c [265129,66884] 0 2024-04-25T15:49:31.199 INFO:tasks.workunit.client.0.smithi110.stdout:8/603: mknod d14/d32/d36/d48/d8f/ccd 0 2024-04-25T15:49:31.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/184: fdatasync d0/d9/d15/f1b 0 2024-04-25T15:49:31.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/185: stat d0/d9/d3c 0 2024-04-25T15:49:31.205 INFO:tasks.workunit.client.1.smithi193.stdout:4/192: chown db/f35 433362 1 2024-04-25T15:49:31.205 INFO:tasks.workunit.client.0.smithi110.stdout:5/489: sync 2024-04-25T15:49:31.205 INFO:tasks.workunit.client.0.smithi110.stdout:4/543: dread d13/d24/d2d/d5e/f7b [0,4194304] 0 2024-04-25T15:49:31.205 INFO:tasks.workunit.client.0.smithi110.stdout:0/481: sync 2024-04-25T15:49:31.205 INFO:tasks.workunit.client.0.smithi110.stdout:9/400: sync 2024-04-25T15:49:31.206 INFO:tasks.workunit.client.0.smithi110.stdout:3/548: sync 2024-04-25T15:49:31.206 INFO:tasks.workunit.client.0.smithi110.stdout:2/470: sync 2024-04-25T15:49:31.206 INFO:tasks.workunit.client.0.smithi110.stdout:7/483: symlink d1b/d29/d87/l9f 0 2024-04-25T15:49:31.206 INFO:tasks.workunit.client.1.smithi193.stdout:9/175: dwrite df/d11/f2e [0,4194304] 0 2024-04-25T15:49:31.206 INFO:tasks.workunit.client.1.smithi193.stdout:4/193: read db/f35 [4398850,26399] 0 2024-04-25T15:49:31.207 INFO:tasks.workunit.client.1.smithi193.stdout:7/122: symlink d12/d1e/l2d 0 2024-04-25T15:49:31.209 INFO:tasks.workunit.client.0.smithi110.stdout:2/471: chown de/d40/f72 447708 1 2024-04-25T15:49:31.209 INFO:tasks.workunit.client.1.smithi193.stdout:9/176: dread f4 [0,4194304] 0 2024-04-25T15:49:31.209 INFO:tasks.workunit.client.0.smithi110.stdout:5/490: write d4/d2c/f27 [1597281,29963] 0 2024-04-25T15:49:31.211 INFO:tasks.workunit.client.0.smithi110.stdout:2/472: write de/f2d [3990933,106388] 0 2024-04-25T15:49:31.212 INFO:tasks.workunit.client.1.smithi193.stdout:1/137: rename f1 to dc/f2c 0 2024-04-25T15:49:31.212 INFO:tasks.workunit.client.1.smithi193.stdout:1/138: dread - dc/f12 zero size 2024-04-25T15:49:31.213 INFO:tasks.workunit.client.1.smithi193.stdout:1/139: write dc/f18 [5102145,34398] 0 2024-04-25T15:49:31.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/140: truncate dc/f12 392979 0 2024-04-25T15:49:31.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/141: chown dc 3757 1 2024-04-25T15:49:31.215 INFO:tasks.workunit.client.1.smithi193.stdout:1/142: readlink dc/l13 0 2024-04-25T15:49:31.215 INFO:tasks.workunit.client.0.smithi110.stdout:4/544: dread - d13/d24/d2d/d36/fb0 zero size 2024-04-25T15:49:31.219 INFO:tasks.workunit.client.1.smithi193.stdout:0/182: link d0/d3/fe d0/d3/d5/d7/dd/f3c 0 2024-04-25T15:49:31.220 INFO:tasks.workunit.client.0.smithi110.stdout:6/433: dread d1/db/d22/d33/d43/f53 [0,4194304] 0 2024-04-25T15:49:31.222 INFO:tasks.workunit.client.1.smithi193.stdout:8/163: truncate d2/d3/ff 1445305 0 2024-04-25T15:49:31.223 INFO:tasks.workunit.client.0.smithi110.stdout:5/491: dwrite d4/d1d/d1f/f8f [0,4194304] 0 2024-04-25T15:49:31.224 INFO:tasks.workunit.client.1.smithi193.stdout:6/177: creat dd/d2f/f30 x:0 0 0 2024-04-25T15:49:31.225 INFO:tasks.workunit.client.1.smithi193.stdout:6/178: write dd/d12/d19/d1c/f20 [2850742,38940] 0 2024-04-25T15:49:31.228 INFO:tasks.workunit.client.0.smithi110.stdout:8/604: getdents d14/d32/d36/db7/dc9 0 2024-04-25T15:49:31.229 INFO:tasks.workunit.client.0.smithi110.stdout:8/605: chown d14/d21/d26/d6a/f78 80 1 2024-04-25T15:49:31.230 INFO:tasks.workunit.client.0.smithi110.stdout:9/401: dread f8 [0,4194304] 0 2024-04-25T15:49:31.231 INFO:tasks.workunit.client.0.smithi110.stdout:8/606: write d14/f16 [778637,6135] 0 2024-04-25T15:49:31.233 INFO:tasks.workunit.client.0.smithi110.stdout:3/549: dread d0/f1e [0,4194304] 0 2024-04-25T15:49:31.236 INFO:tasks.workunit.client.1.smithi193.stdout:4/194: mknod db/d13/d1e/d1f/c3c 0 2024-04-25T15:49:31.239 INFO:tasks.workunit.client.0.smithi110.stdout:7/484: creat d1b/d29/d38/d55/fa0 x:0 0 0 2024-04-25T15:49:31.240 INFO:tasks.workunit.client.1.smithi193.stdout:7/123: creat d12/f2e x:0 0 0 2024-04-25T15:49:31.241 INFO:tasks.workunit.client.0.smithi110.stdout:7/485: chown d1b/d29/l30 0 1 2024-04-25T15:49:31.243 INFO:tasks.workunit.client.1.smithi193.stdout:9/177: unlink cd 0 2024-04-25T15:49:31.244 INFO:tasks.workunit.client.0.smithi110.stdout:0/482: dread d1/f8 [0,4194304] 0 2024-04-25T15:49:31.249 INFO:tasks.workunit.client.1.smithi193.stdout:3/209: rename d8/d13/d14/f1d to d8/d30/f42 0 2024-04-25T15:49:31.250 INFO:tasks.workunit.client.0.smithi110.stdout:4/545: dread - d13/d24/d2d/d36/d51/f6a zero size 2024-04-25T15:49:31.254 INFO:tasks.workunit.client.0.smithi110.stdout:6/434: creat d1/db/d22/d90/d9e/f9f x:0 0 0 2024-04-25T15:49:31.255 INFO:tasks.workunit.client.1.smithi193.stdout:0/183: stat d0/d3/d5/d7/dd/l3b 0 2024-04-25T15:49:31.257 INFO:tasks.workunit.client.0.smithi110.stdout:4/546: dwrite d13/f77 [0,4194304] 0 2024-04-25T15:49:31.257 INFO:tasks.workunit.client.1.smithi193.stdout:8/164: symlink d2/d3/l31 0 2024-04-25T15:49:31.259 INFO:tasks.workunit.client.1.smithi193.stdout:8/165: dread d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:31.264 INFO:tasks.workunit.client.1.smithi193.stdout:5/145: getdents d17 0 2024-04-25T15:49:31.265 INFO:tasks.workunit.client.1.smithi193.stdout:5/146: chown d17/d18/d30 2886090 1 2024-04-25T15:49:31.266 INFO:tasks.workunit.client.1.smithi193.stdout:4/195: symlink db/d13/l3d 0 2024-04-25T15:49:31.268 INFO:tasks.workunit.client.1.smithi193.stdout:7/124: symlink d12/d13/l2f 0 2024-04-25T15:49:31.268 INFO:tasks.workunit.client.1.smithi193.stdout:7/125: readlink la 0 2024-04-25T15:49:31.269 INFO:tasks.workunit.client.1.smithi193.stdout:7/126: chown d12/d1c/c21 0 1 2024-04-25T15:49:31.269 INFO:tasks.workunit.client.0.smithi110.stdout:8/607: dwrite d14/d21/d26/d6a/fc7 [0,4194304] 0 2024-04-25T15:49:31.281 INFO:tasks.workunit.client.1.smithi193.stdout:9/178: mkdir df/d2c/d2f 0 2024-04-25T15:49:31.282 INFO:tasks.workunit.client.1.smithi193.stdout:9/179: write df/d16/d1c/f23 [271651,129426] 0 2024-04-25T15:49:31.282 INFO:tasks.workunit.client.1.smithi193.stdout:7/127: creat d12/f30 x:0 0 0 2024-04-25T15:49:31.283 INFO:tasks.workunit.client.1.smithi193.stdout:7/128: fsync d12/d13/f17 0 2024-04-25T15:49:31.286 INFO:tasks.workunit.client.1.smithi193.stdout:1/143: creat dc/f2d x:0 0 0 2024-04-25T15:49:31.291 INFO:tasks.workunit.client.1.smithi193.stdout:6/179: creat dd/f31 x:0 0 0 2024-04-25T15:49:31.292 INFO:tasks.workunit.client.1.smithi193.stdout:5/147: write fe [421475,44987] 0 2024-04-25T15:49:31.294 INFO:tasks.workunit.client.1.smithi193.stdout:4/196: write db/d13/f1d [742180,51586] 0 2024-04-25T15:49:31.297 INFO:tasks.workunit.client.1.smithi193.stdout:9/180: mkdir df/d16/d1c/d30 0 2024-04-25T15:49:31.297 INFO:tasks.workunit.client.1.smithi193.stdout:9/181: write df/d16/f25 [347598,45829] 0 2024-04-25T15:49:31.303 INFO:tasks.workunit.client.1.smithi193.stdout:9/182: dwrite df/d16/f25 [0,4194304] 0 2024-04-25T15:49:31.303 INFO:tasks.workunit.client.1.smithi193.stdout:2/186: rename d0/c5 to d0/d9/c49 0 2024-04-25T15:49:31.304 INFO:tasks.workunit.client.1.smithi193.stdout:0/184: truncate d0/f1e 1069365 0 2024-04-25T15:49:31.305 INFO:tasks.workunit.client.1.smithi193.stdout:2/187: write d0/d9/d30/f46 [18951,10497] 0 2024-04-25T15:49:31.305 INFO:tasks.workunit.client.1.smithi193.stdout:1/144: fdatasync dc/f16 0 2024-04-25T15:49:31.312 INFO:tasks.workunit.client.1.smithi193.stdout:4/197: dread db/fc [0,4194304] 0 2024-04-25T15:49:31.314 INFO:tasks.workunit.client.1.smithi193.stdout:5/148: fdatasync d17/f2a 0 2024-04-25T15:49:31.320 INFO:tasks.workunit.client.1.smithi193.stdout:4/198: dwrite db/d32/f37 [0,4194304] 0 2024-04-25T15:49:31.323 INFO:tasks.workunit.client.0.smithi110.stdout:2/473: truncate de/d40/d6e/f42 1097457 0 2024-04-25T15:49:31.325 INFO:tasks.workunit.client.0.smithi110.stdout:0/483: write d1/d23/d30/d33/d5b/f98 [379070,60480] 0 2024-04-25T15:49:31.325 INFO:tasks.workunit.client.1.smithi193.stdout:7/129: dwrite f8 [0,4194304] 0 2024-04-25T15:49:31.325 INFO:tasks.workunit.client.1.smithi193.stdout:8/166: write d2/d3/ff [545651,63070] 0 2024-04-25T15:49:31.327 INFO:tasks.workunit.client.1.smithi193.stdout:8/167: chown d2/l28 0 1 2024-04-25T15:49:31.328 INFO:tasks.workunit.client.1.smithi193.stdout:7/130: write d12/f30 [784386,101048] 0 2024-04-25T15:49:31.328 INFO:tasks.workunit.client.1.smithi193.stdout:7/131: dread - fc zero size 2024-04-25T15:49:31.329 INFO:tasks.workunit.client.0.smithi110.stdout:3/550: dread d0/d7/d1c/d2f/d33/f44 [0,4194304] 0 2024-04-25T15:49:31.330 INFO:tasks.workunit.client.0.smithi110.stdout:3/551: chown d0/d7/d1c/d2f/d4c/c7b 2 1 2024-04-25T15:49:31.332 INFO:tasks.workunit.client.1.smithi193.stdout:0/185: mkdir d0/d3/d5/d7/dd/d16/d26/d2e/d3d 0 2024-04-25T15:49:31.333 INFO:tasks.workunit.client.0.smithi110.stdout:7/486: symlink d1b/d29/la1 0 2024-04-25T15:49:31.334 INFO:tasks.workunit.client.0.smithi110.stdout:5/492: truncate d4/d1d/d1f/f8f 426817 0 2024-04-25T15:49:31.335 INFO:tasks.workunit.client.1.smithi193.stdout:7/132: dwrite d12/d1c/f28 [0,4194304] 0 2024-04-25T15:49:31.335 INFO:tasks.workunit.client.0.smithi110.stdout:4/547: write d13/d24/d2d/d36/f40 [4768767,14104] 0 2024-04-25T15:49:31.338 INFO:tasks.workunit.client.0.smithi110.stdout:6/435: mkdir d1/db/da0 0 2024-04-25T15:49:31.339 INFO:tasks.workunit.client.0.smithi110.stdout:6/436: chown d1/dd/f8e 342487 1 2024-04-25T15:49:31.340 INFO:tasks.workunit.client.1.smithi193.stdout:5/149: creat d17/f39 x:0 0 0 2024-04-25T15:49:31.341 INFO:tasks.workunit.client.0.smithi110.stdout:2/474: creat de/d40/d6e/f8e x:0 0 0 2024-04-25T15:49:31.341 INFO:tasks.workunit.client.1.smithi193.stdout:5/150: dread - d17/d18/d30/f32 zero size 2024-04-25T15:49:31.341 INFO:tasks.workunit.client.1.smithi193.stdout:5/151: write fe [849921,65568] 0 2024-04-25T15:49:31.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/210: sync 2024-04-25T15:49:31.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/211: fdatasync d8/ff 0 2024-04-25T15:49:31.345 INFO:tasks.workunit.client.0.smithi110.stdout:0/484: rename d1/d23/d30/d33/d80/dad to d1/d23/d30/d33/d35/da1/db4 0 2024-04-25T15:49:31.348 INFO:tasks.workunit.client.1.smithi193.stdout:4/199: mknod db/d32/c3e 0 2024-04-25T15:49:31.349 INFO:tasks.workunit.client.1.smithi193.stdout:9/183: creat df/d2d/f31 x:0 0 0 2024-04-25T15:49:31.351 INFO:tasks.workunit.client.0.smithi110.stdout:2/475: dwrite de/d40/f72 [0,4194304] 0 2024-04-25T15:49:31.352 INFO:tasks.workunit.client.0.smithi110.stdout:7/487: creat d1b/d29/d87/fa2 x:0 0 0 2024-04-25T15:49:31.354 INFO:tasks.workunit.client.1.smithi193.stdout:1/145: mknod dc/c2e 0 2024-04-25T15:49:31.355 INFO:tasks.workunit.client.1.smithi193.stdout:1/146: fdatasync f6 0 2024-04-25T15:49:31.357 INFO:tasks.workunit.client.1.smithi193.stdout:8/168: unlink d2/d3/db/f19 0 2024-04-25T15:49:31.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/188: creat d0/d9/d36/d3a/f4a x:0 0 0 2024-04-25T15:49:31.360 INFO:tasks.workunit.client.0.smithi110.stdout:6/437: fdatasync d1/dd/f8e 0 2024-04-25T15:49:31.361 INFO:tasks.workunit.client.1.smithi193.stdout:8/169: dread d2/fa [0,4194304] 0 2024-04-25T15:49:31.363 INFO:tasks.workunit.client.1.smithi193.stdout:0/186: creat d0/d3/d5/d7/dd/d16/d18/d28/f3e x:0 0 0 2024-04-25T15:49:31.365 INFO:tasks.workunit.client.1.smithi193.stdout:6/180: dwrite dd/fe [4194304,4194304] 0 2024-04-25T15:49:31.366 INFO:tasks.workunit.client.1.smithi193.stdout:0/187: dread - d0/d3/d5/d7/dd/d16/d1a/f2b zero size 2024-04-25T15:49:31.366 INFO:tasks.workunit.client.1.smithi193.stdout:7/133: mkdir d12/d31 0 2024-04-25T15:49:31.367 INFO:tasks.workunit.client.0.smithi110.stdout:8/608: link d14/d21/d23/l65 d14/d32/d36/db7/dc9/lce 0 2024-04-25T15:49:31.368 INFO:tasks.workunit.client.1.smithi193.stdout:8/170: dread d2/d3/d20/f22 [0,4194304] 0 2024-04-25T15:49:31.368 INFO:tasks.workunit.client.0.smithi110.stdout:8/609: fdatasync d14/d21/d26/f29 0 2024-04-25T15:49:31.373 INFO:tasks.workunit.client.0.smithi110.stdout:7/488: write d1b/d29/f68 [602981,70850] 0 2024-04-25T15:49:31.373 INFO:tasks.workunit.client.1.smithi193.stdout:7/134: dwrite d12/d1e/f25 [0,4194304] 0 2024-04-25T15:49:31.374 INFO:tasks.workunit.client.0.smithi110.stdout:7/489: chown d1b/f48 14941 1 2024-04-25T15:49:31.374 INFO:tasks.workunit.client.1.smithi193.stdout:7/135: fdatasync d12/d1c/f1d 0 2024-04-25T15:49:31.375 INFO:tasks.workunit.client.1.smithi193.stdout:5/152: rename d17/d18/d30/d34/c36 to d17/c3a 0 2024-04-25T15:49:31.376 INFO:tasks.workunit.client.1.smithi193.stdout:5/153: chown d17/d18/d30 85 1 2024-04-25T15:49:31.377 INFO:tasks.workunit.client.0.smithi110.stdout:2/476: creat de/d40/f8f x:0 0 0 2024-04-25T15:49:31.381 INFO:tasks.workunit.client.0.smithi110.stdout:5/493: mkdir d4/d1d/d1f/d9e 0 2024-04-25T15:49:31.381 INFO:tasks.workunit.client.0.smithi110.stdout:5/494: stat d4/d2c/d2a/l31 0 2024-04-25T15:49:31.382 INFO:tasks.workunit.client.1.smithi193.stdout:3/212: mknod d8/c43 0 2024-04-25T15:49:31.382 INFO:tasks.workunit.client.0.smithi110.stdout:5/495: chown d4/d1d/d1f/d35/d7f/f8d 88940 1 2024-04-25T15:49:31.385 INFO:tasks.workunit.client.1.smithi193.stdout:4/200: rename db/d13/l3d to db/d13/l3f 0 2024-04-25T15:49:31.387 INFO:tasks.workunit.client.1.smithi193.stdout:9/184: mknod df/d11/c32 0 2024-04-25T15:49:31.388 INFO:tasks.workunit.client.0.smithi110.stdout:6/438: rmdir d1/dd 39 2024-04-25T15:49:31.388 INFO:tasks.workunit.client.0.smithi110.stdout:6/439: chown d1/d12/f1f 0 1 2024-04-25T15:49:31.393 INFO:tasks.workunit.client.1.smithi193.stdout:2/189: symlink d0/d9/d3d/l4b 0 2024-04-25T15:49:31.396 INFO:tasks.workunit.client.0.smithi110.stdout:3/552: link d0/d7/d1c/d2f/d33/f44 d0/d16/d8c/d90/d92/fbb 0 2024-04-25T15:49:31.396 INFO:tasks.workunit.client.0.smithi110.stdout:1/576: sync 2024-04-25T15:49:31.397 INFO:tasks.workunit.client.0.smithi110.stdout:3/553: readlink d0/d7/d1c/d2f/d33/l34 0 2024-04-25T15:49:31.401 INFO:tasks.workunit.client.1.smithi193.stdout:6/181: dwrite dd/d12/d19/d21/f25 [0,4194304] 0 2024-04-25T15:49:31.401 INFO:tasks.workunit.client.1.smithi193.stdout:0/188: dread d0/f17 [0,4194304] 0 2024-04-25T15:49:31.406 INFO:tasks.workunit.client.0.smithi110.stdout:4/548: creat d13/d24/d2d/d44/fb4 x:0 0 0 2024-04-25T15:49:31.407 INFO:tasks.workunit.client.1.smithi193.stdout:5/154: mkdir d17/d18/d30/d3b 0 2024-04-25T15:49:31.408 INFO:tasks.workunit.client.1.smithi193.stdout:5/155: chown d17/d18/d30/d38 1 1 2024-04-25T15:49:31.410 INFO:tasks.workunit.client.1.smithi193.stdout:3/213: mknod d8/d13/d14/c44 0 2024-04-25T15:49:31.411 INFO:tasks.workunit.client.0.smithi110.stdout:6/440: mkdir d1/d12/d3d/da1 0 2024-04-25T15:49:31.412 INFO:tasks.workunit.client.1.smithi193.stdout:4/201: fdatasync f1 0 2024-04-25T15:49:31.412 INFO:tasks.workunit.client.0.smithi110.stdout:0/485: link d1/d23/d30/d33/f2d d1/d6c/d7f/fb5 0 2024-04-25T15:49:31.413 INFO:tasks.workunit.client.0.smithi110.stdout:0/486: dread - d1/d20/d69/fb3 zero size 2024-04-25T15:49:31.413 INFO:tasks.workunit.client.1.smithi193.stdout:4/202: chown db/d13 52068 1 2024-04-25T15:49:31.414 INFO:tasks.workunit.client.0.smithi110.stdout:0/487: chown d1/d6c/d7f/d79 10624993 1 2024-04-25T15:49:31.414 INFO:tasks.workunit.client.0.smithi110.stdout:8/610: link d14/f16 d14/d4c/d71/dae/fcf 0 2024-04-25T15:49:31.414 INFO:tasks.workunit.client.1.smithi193.stdout:8/171: dwrite d2/d3/db/f1a [0,4194304] 0 2024-04-25T15:49:31.415 INFO:tasks.workunit.client.1.smithi193.stdout:8/172: chown d2/d18 205226 1 2024-04-25T15:49:31.415 INFO:tasks.workunit.client.0.smithi110.stdout:8/611: write d14/d21/d26/d6a/d7e/fa2 [102697,15937] 0 2024-04-25T15:49:31.416 INFO:tasks.workunit.client.1.smithi193.stdout:8/173: fsync d2/f11 0 2024-04-25T15:49:31.416 INFO:tasks.workunit.client.1.smithi193.stdout:8/174: stat d2/d3/c13 0 2024-04-25T15:49:31.417 INFO:tasks.workunit.client.1.smithi193.stdout:2/190: mknod d0/d9/d3c/c4c 0 2024-04-25T15:49:31.419 INFO:tasks.workunit.client.0.smithi110.stdout:7/490: rename d1b/d29/l4a to d1b/d29/d87/la3 0 2024-04-25T15:49:31.420 INFO:tasks.workunit.client.0.smithi110.stdout:7/491: write f17 [843132,38950] 0 2024-04-25T15:49:31.423 INFO:tasks.workunit.client.0.smithi110.stdout:2/477: truncate de/d40/d6e/d52/d6d/d7d/f85 778480 0 2024-04-25T15:49:31.427 INFO:tasks.workunit.client.0.smithi110.stdout:9/402: sync 2024-04-25T15:49:31.429 INFO:tasks.workunit.client.0.smithi110.stdout:7/492: dwrite d1b/d29/d38/d55/fa0 [0,4194304] 0 2024-04-25T15:49:31.430 INFO:tasks.workunit.client.0.smithi110.stdout:7/493: read f2 [517133,115392] 0 2024-04-25T15:49:31.433 INFO:tasks.workunit.client.0.smithi110.stdout:1/577: write d0/d25/d33/f45 [1147650,74362] 0 2024-04-25T15:49:31.438 INFO:tasks.workunit.client.0.smithi110.stdout:2/478: dread de/f28 [0,4194304] 0 2024-04-25T15:49:31.438 INFO:tasks.workunit.client.0.smithi110.stdout:2/479: dread - de/d40/d70/f84 zero size 2024-04-25T15:49:31.439 INFO:tasks.workunit.client.0.smithi110.stdout:2/480: stat de/d40/f35 0 2024-04-25T15:49:31.441 INFO:tasks.workunit.client.1.smithi193.stdout:7/136: truncate f7 2201876 0 2024-04-25T15:49:31.442 INFO:tasks.workunit.client.1.smithi193.stdout:6/182: dwrite dd/d12/d19/d1c/f22 [0,4194304] 0 2024-04-25T15:49:31.442 INFO:tasks.workunit.client.1.smithi193.stdout:7/137: fsync d12/f2c 0 2024-04-25T15:49:31.444 INFO:tasks.workunit.client.0.smithi110.stdout:4/549: mkdir d13/d24/d2d/db5 0 2024-04-25T15:49:31.445 INFO:tasks.workunit.client.1.smithi193.stdout:9/185: mkdir df/d2c/d2f/d33 0 2024-04-25T15:49:31.445 INFO:tasks.workunit.client.0.smithi110.stdout:5/496: link d4/d2c/d2a/l31 d4/d1d/d85/l9f 0 2024-04-25T15:49:31.446 INFO:tasks.workunit.client.0.smithi110.stdout:4/550: stat d13/d24/d2d/d44/d65/d99/daf/d56/d88/lae 0 2024-04-25T15:49:31.447 INFO:tasks.workunit.client.1.smithi193.stdout:4/203: rmdir db/d32 39 2024-04-25T15:49:31.449 INFO:tasks.workunit.client.0.smithi110.stdout:9/403: fsync d9/d2d/f5d 0 2024-04-25T15:49:31.452 INFO:tasks.workunit.client.0.smithi110.stdout:6/441: creat d1/d12/d3d/da1/fa2 x:0 0 0 2024-04-25T15:49:31.454 INFO:tasks.workunit.client.1.smithi193.stdout:3/214: write f2 [5113560,45687] 0 2024-04-25T15:49:31.454 INFO:tasks.workunit.client.1.smithi193.stdout:2/191: write d0/d44/f47 [1668824,87520] 0 2024-04-25T15:49:31.456 INFO:tasks.workunit.client.1.smithi193.stdout:8/175: symlink d2/d1f/l32 0 2024-04-25T15:49:31.458 INFO:tasks.workunit.client.0.smithi110.stdout:0/488: fdatasync d1/d23/f2f 0 2024-04-25T15:49:31.461 INFO:tasks.workunit.client.0.smithi110.stdout:1/578: rename d0/d14/d5f/d7d/d9c/ca5 to d0/d25/d33/da9/cb0 0 2024-04-25T15:49:31.462 INFO:tasks.workunit.client.1.smithi193.stdout:0/189: creat d0/d3/d5/d7/dd/d16/d26/d2e/d3d/f3f x:0 0 0 2024-04-25T15:49:31.462 INFO:tasks.workunit.client.0.smithi110.stdout:3/554: link d0/d7/d1c/d2f/d33/ca1 d0/d16/d8c/cbc 0 2024-04-25T15:49:31.465 INFO:tasks.workunit.client.1.smithi193.stdout:1/147: rename lb to dc/l2f 0 2024-04-25T15:49:31.465 INFO:tasks.workunit.client.1.smithi193.stdout:1/148: chown dc/c2e 154 1 2024-04-25T15:49:31.466 INFO:tasks.workunit.client.1.smithi193.stdout:5/156: getdents d17/d18/d30/d34/d37 0 2024-04-25T15:49:31.467 INFO:tasks.workunit.client.0.smithi110.stdout:9/404: dread d9/db/df/d25/f37 [4194304,4194304] 0 2024-04-25T15:49:31.468 INFO:tasks.workunit.client.0.smithi110.stdout:4/551: creat d13/d24/d2d/d36/d51/fb6 x:0 0 0 2024-04-25T15:49:31.470 INFO:tasks.workunit.client.0.smithi110.stdout:6/442: creat d1/db/d22/d90/fa3 x:0 0 0 2024-04-25T15:49:31.473 INFO:tasks.workunit.client.1.smithi193.stdout:3/215: rmdir d8/d13 39 2024-04-25T15:49:31.476 INFO:tasks.workunit.client.1.smithi193.stdout:9/186: dwrite df/d11/f2a [0,4194304] 0 2024-04-25T15:49:31.476 INFO:tasks.workunit.client.1.smithi193.stdout:8/176: dread d2/fa [0,4194304] 0 2024-04-25T15:49:31.478 INFO:tasks.workunit.client.1.smithi193.stdout:0/190: creat d0/d3/d5/d7/dd/f40 x:0 0 0 2024-04-25T15:49:31.479 INFO:tasks.workunit.client.0.smithi110.stdout:5/497: creat d4/d1d/d1f/d35/d3d/d52/d97/fa0 x:0 0 0 2024-04-25T15:49:31.480 INFO:tasks.workunit.client.0.smithi110.stdout:5/498: chown d4/d1d/l30 109153 1 2024-04-25T15:49:31.481 INFO:tasks.workunit.client.0.smithi110.stdout:5/499: chown d4/d1d/d1f/l5b 50497 1 2024-04-25T15:49:31.481 INFO:tasks.workunit.client.0.smithi110.stdout:9/405: rmdir d9/db/df/d2c/d3c 39 2024-04-25T15:49:31.484 INFO:tasks.workunit.client.0.smithi110.stdout:4/552: fdatasync fc 0 2024-04-25T15:49:31.484 INFO:tasks.workunit.client.0.smithi110.stdout:3/555: write d0/d7/d1c/d50/f6f [473032,19864] 0 2024-04-25T15:49:31.489 INFO:tasks.workunit.client.1.smithi193.stdout:6/183: symlink dd/d12/d19/l32 0 2024-04-25T15:49:31.491 INFO:tasks.workunit.client.0.smithi110.stdout:6/443: symlink d1/db/d22/d33/d63/la4 0 2024-04-25T15:49:31.492 INFO:tasks.workunit.client.1.smithi193.stdout:4/204: mkdir db/d32/d40 0 2024-04-25T15:49:31.493 INFO:tasks.workunit.client.1.smithi193.stdout:3/216: write d8/d30/f42 [533254,108315] 0 2024-04-25T15:49:31.494 INFO:tasks.workunit.client.1.smithi193.stdout:3/217: chown d8/cd 13547629 1 2024-04-25T15:49:31.495 INFO:tasks.workunit.client.0.smithi110.stdout:2/481: rename de/f80 to de/d40/d4f/f90 0 2024-04-25T15:49:31.496 INFO:tasks.workunit.client.1.smithi193.stdout:2/192: mkdir d0/d37/d4d 0 2024-04-25T15:49:31.497 INFO:tasks.workunit.client.0.smithi110.stdout:1/579: mkdir d0/d23/db1 0 2024-04-25T15:49:31.497 INFO:tasks.workunit.client.0.smithi110.stdout:1/580: chown d0/d51/d6b 84818 1 2024-04-25T15:49:31.498 INFO:tasks.workunit.client.1.smithi193.stdout:9/187: truncate df/d16/d1c/f21 988581 0 2024-04-25T15:49:31.499 INFO:tasks.workunit.client.0.smithi110.stdout:5/500: creat d4/d1d/d1f/d35/d3d/d52/d97/fa1 x:0 0 0 2024-04-25T15:49:31.501 INFO:tasks.workunit.client.0.smithi110.stdout:4/553: readlink d13/d24/l33 0 2024-04-25T15:49:31.505 INFO:tasks.workunit.client.0.smithi110.stdout:7/494: getdents d1b/d29/d87 0 2024-04-25T15:49:31.505 INFO:tasks.workunit.client.1.smithi193.stdout:7/138: link d12/d13/c15 d12/c32 0 2024-04-25T15:49:31.510 INFO:tasks.workunit.client.0.smithi110.stdout:0/489: rename d1/f4e to d1/d23/d30/d33/d5b/d83/d87/da8/fb6 0 2024-04-25T15:49:31.512 INFO:tasks.workunit.client.0.smithi110.stdout:0/490: truncate d1/d23/d30/d33/d5b/f9b 843579 0 2024-04-25T15:49:31.512 INFO:tasks.workunit.client.0.smithi110.stdout:2/482: dread - de/d40/d6e/d50/f7b zero size 2024-04-25T15:49:31.514 INFO:tasks.workunit.client.0.smithi110.stdout:9/406: mkdir d9/db/d7d 0 2024-04-25T15:49:31.514 INFO:tasks.workunit.client.1.smithi193.stdout:6/184: write dd/d12/d19/d1c/f29 [1683380,27747] 0 2024-04-25T15:49:31.516 INFO:tasks.workunit.client.0.smithi110.stdout:1/581: creat d0/d14/d5f/d7d/d72/d61/fb2 x:0 0 0 2024-04-25T15:49:31.516 INFO:tasks.workunit.client.0.smithi110.stdout:6/444: dread d1/db/d22/d33/f56 [0,4194304] 0 2024-04-25T15:49:31.516 INFO:tasks.workunit.client.1.smithi193.stdout:0/191: dwrite d0/d3/d5/d7/dd/f3c [0,4194304] 0 2024-04-25T15:49:31.516 INFO:tasks.workunit.client.1.smithi193.stdout:1/149: creat dc/f30 x:0 0 0 2024-04-25T15:49:31.517 INFO:tasks.workunit.client.0.smithi110.stdout:6/445: readlink d1/d12/l5f 0 2024-04-25T15:49:31.517 INFO:tasks.workunit.client.0.smithi110.stdout:5/501: readlink d4/d15/l29 0 2024-04-25T15:49:31.518 INFO:tasks.workunit.client.1.smithi193.stdout:5/157: link d17/l1c d17/d18/d30/d38/l3c 0 2024-04-25T15:49:31.519 INFO:tasks.workunit.client.0.smithi110.stdout:4/554: truncate d13/d24/d2d/f4f 3260162 0 2024-04-25T15:49:31.521 INFO:tasks.workunit.client.1.smithi193.stdout:7/139: fdatasync f3 0 2024-04-25T15:49:31.522 INFO:tasks.workunit.client.0.smithi110.stdout:7/495: readlink d1b/l1d 0 2024-04-25T15:49:31.523 INFO:tasks.workunit.client.1.smithi193.stdout:1/150: dread dc/f19 [0,4194304] 0 2024-04-25T15:49:31.524 INFO:tasks.workunit.client.1.smithi193.stdout:1/151: stat dc 0 2024-04-25T15:49:31.527 INFO:tasks.workunit.client.1.smithi193.stdout:3/218: unlink d8/d13/d1f/c25 0 2024-04-25T15:49:31.527 INFO:tasks.workunit.client.1.smithi193.stdout:3/219: write d8/d13/f2d [804013,40579] 0 2024-04-25T15:49:31.528 INFO:tasks.workunit.client.1.smithi193.stdout:3/220: readlink d8/d13/d14/l33 0 2024-04-25T15:49:31.529 INFO:tasks.workunit.client.1.smithi193.stdout:3/221: write d8/d30/f42 [226362,48062] 0 2024-04-25T15:49:31.529 INFO:tasks.workunit.client.1.smithi193.stdout:2/193: unlink d0/d9/d15/f2e 0 2024-04-25T15:49:31.532 INFO:tasks.workunit.client.0.smithi110.stdout:9/407: symlink d9/db/df/d55/l7e 0 2024-04-25T15:49:31.532 INFO:tasks.workunit.client.0.smithi110.stdout:3/556: write d0/ff [8478,102845] 0 2024-04-25T15:49:31.534 INFO:tasks.workunit.client.0.smithi110.stdout:9/408: truncate d9/db/df/d4d/f78 253562 0 2024-04-25T15:49:31.537 INFO:tasks.workunit.client.1.smithi193.stdout:6/185: symlink dd/d12/d19/d21/l33 0 2024-04-25T15:49:31.538 INFO:tasks.workunit.client.0.smithi110.stdout:5/502: symlink d4/d1d/la2 0 2024-04-25T15:49:31.539 INFO:tasks.workunit.client.0.smithi110.stdout:2/483: write f8 [2762521,80452] 0 2024-04-25T15:49:31.539 INFO:tasks.workunit.client.0.smithi110.stdout:3/557: dwrite d0/d16/f24 [4194304,4194304] 0 2024-04-25T15:49:31.542 INFO:tasks.workunit.client.1.smithi193.stdout:0/192: creat d0/d3/d5/d7/dd/d16/f41 x:0 0 0 2024-04-25T15:49:31.543 INFO:tasks.workunit.client.0.smithi110.stdout:2/484: dread de/d40/d6e/d52/d6d/d7d/f85 [0,4194304] 0 2024-04-25T15:49:31.546 INFO:tasks.workunit.client.0.smithi110.stdout:2/485: chown de/d40/l37 48 1 2024-04-25T15:49:31.546 INFO:tasks.workunit.client.0.smithi110.stdout:7/496: write d1b/d29/d38/d55/d7f/f6c [382308,86269] 0 2024-04-25T15:49:31.547 INFO:tasks.workunit.client.1.smithi193.stdout:7/140: rmdir d12 39 2024-04-25T15:49:31.548 INFO:tasks.workunit.client.0.smithi110.stdout:0/491: fdatasync d1/d47/f86 0 2024-04-25T15:49:31.549 INFO:tasks.workunit.client.1.smithi193.stdout:4/205: link f1 db/d13/d1e/f41 0 2024-04-25T15:49:31.550 INFO:tasks.workunit.client.0.smithi110.stdout:5/503: dwrite d4/d1d/d1f/d35/d3d/d52/f82 [0,4194304] 0 2024-04-25T15:49:31.550 INFO:tasks.workunit.client.0.smithi110.stdout:2/486: write de/d40/d6e/d52/f62 [546654,107468] 0 2024-04-25T15:49:31.550 INFO:tasks.workunit.client.1.smithi193.stdout:1/152: write dc/f2c [3784692,41724] 0 2024-04-25T15:49:31.551 INFO:tasks.workunit.client.1.smithi193.stdout:1/153: dread - dc/f21 zero size 2024-04-25T15:49:31.551 INFO:tasks.workunit.client.1.smithi193.stdout:1/154: fdatasync dc/f1e 0 2024-04-25T15:49:31.552 INFO:tasks.workunit.client.1.smithi193.stdout:4/206: dread db/f15 [4194304,4194304] 0 2024-04-25T15:49:31.553 INFO:tasks.workunit.client.1.smithi193.stdout:3/222: mkdir d8/d30/d45 0 2024-04-25T15:49:31.555 INFO:tasks.workunit.client.0.smithi110.stdout:9/409: creat d9/d44/f7f x:0 0 0 2024-04-25T15:49:31.556 INFO:tasks.workunit.client.1.smithi193.stdout:2/194: mknod d0/d44/c4e 0 2024-04-25T15:49:31.557 INFO:tasks.workunit.client.1.smithi193.stdout:2/195: truncate d0/d9/d36/f3f 636426 0 2024-04-25T15:49:31.558 INFO:tasks.workunit.client.0.smithi110.stdout:6/446: mkdir d1/db/da0/da5 0 2024-04-25T15:49:31.558 INFO:tasks.workunit.client.1.smithi193.stdout:8/177: getdents d2/d3/db/d29 0 2024-04-25T15:49:31.559 INFO:tasks.workunit.client.1.smithi193.stdout:6/186: readlink dd/l14 0 2024-04-25T15:49:31.562 INFO:tasks.workunit.client.0.smithi110.stdout:4/555: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/f57 [0,4194304] 0 2024-04-25T15:49:31.567 INFO:tasks.workunit.client.0.smithi110.stdout:7/497: fsync d1b/f45 0 2024-04-25T15:49:31.569 INFO:tasks.workunit.client.0.smithi110.stdout:0/492: rmdir d1/d20/d69 39 2024-04-25T15:49:31.571 INFO:tasks.workunit.client.0.smithi110.stdout:2/487: symlink de/d40/d5b/l91 0 2024-04-25T15:49:31.572 INFO:tasks.workunit.client.1.smithi193.stdout:3/223: symlink d8/d30/d31/l46 0 2024-04-25T15:49:31.574 INFO:tasks.workunit.client.1.smithi193.stdout:8/178: unlink d2/c15 0 2024-04-25T15:49:31.575 INFO:tasks.workunit.client.1.smithi193.stdout:6/187: mknod dd/d2f/c34 0 2024-04-25T15:49:31.575 INFO:tasks.workunit.client.0.smithi110.stdout:8/612: sync 2024-04-25T15:49:31.575 INFO:tasks.workunit.client.1.smithi193.stdout:6/188: write dd/f2b [3893359,61613] 0 2024-04-25T15:49:31.576 INFO:tasks.workunit.client.0.smithi110.stdout:9/410: read d9/db/f11 [1688123,3190] 0 2024-04-25T15:49:31.578 INFO:tasks.workunit.client.0.smithi110.stdout:6/447: rmdir d1/db/d22 39 2024-04-25T15:49:31.578 INFO:tasks.workunit.client.1.smithi193.stdout:4/207: write f4 [2829539,124779] 0 2024-04-25T15:49:31.579 INFO:tasks.workunit.client.1.smithi193.stdout:1/155: symlink dc/l31 0 2024-04-25T15:49:31.579 INFO:tasks.workunit.client.0.smithi110.stdout:3/558: symlink d0/d16/d8c/d90/da5/lbd 0 2024-04-25T15:49:31.581 INFO:tasks.workunit.client.1.smithi193.stdout:3/224: creat d8/d30/f47 x:0 0 0 2024-04-25T15:49:31.583 INFO:tasks.workunit.client.1.smithi193.stdout:1/156: dread dc/f2c [0,4194304] 0 2024-04-25T15:49:31.588 INFO:tasks.workunit.client.1.smithi193.stdout:4/208: rename db/d13/c26 to db/d13/d1e/d1f/d22/c42 0 2024-04-25T15:49:31.589 INFO:tasks.workunit.client.0.smithi110.stdout:0/493: dread d1/d47/d54/f6d [0,4194304] 0 2024-04-25T15:49:31.590 INFO:tasks.workunit.client.0.smithi110.stdout:1/582: getdents d0/d51 0 2024-04-25T15:49:31.590 INFO:tasks.workunit.client.0.smithi110.stdout:0/494: readlink d1/d6c/d7f/l6e 0 2024-04-25T15:49:31.590 INFO:tasks.workunit.client.1.smithi193.stdout:1/157: dwrite dc/f2d [0,4194304] 0 2024-04-25T15:49:31.591 INFO:tasks.workunit.client.0.smithi110.stdout:1/583: write d0/d14/d81/f9d [257160,117276] 0 2024-04-25T15:49:31.591 INFO:tasks.workunit.client.1.smithi193.stdout:7/141: dwrite d12/f1a [0,4194304] 0 2024-04-25T15:49:31.592 INFO:tasks.workunit.client.1.smithi193.stdout:8/179: link d2/d3/db/f1a d2/d3/d20/f33 0 2024-04-25T15:49:31.593 INFO:tasks.workunit.client.1.smithi193.stdout:7/142: read - d12/f2e zero size 2024-04-25T15:49:31.594 INFO:tasks.workunit.client.1.smithi193.stdout:7/143: rename d12 to d12/d1c/d33 22 2024-04-25T15:49:31.595 INFO:tasks.workunit.client.0.smithi110.stdout:9/411: mkdir d9/d44/d5e/d80 0 2024-04-25T15:49:31.595 INFO:tasks.workunit.client.1.smithi193.stdout:7/144: dread - fc zero size 2024-04-25T15:49:31.596 INFO:tasks.workunit.client.1.smithi193.stdout:7/145: write d12/d13/f14 [2175369,116382] 0 2024-04-25T15:49:31.597 INFO:tasks.workunit.client.0.smithi110.stdout:6/448: creat d1/db/d24/d69/d75/d82/fa6 x:0 0 0 2024-04-25T15:49:31.599 INFO:tasks.workunit.client.0.smithi110.stdout:6/449: truncate d1/d12/d3d/f4a 50991 0 2024-04-25T15:49:31.599 INFO:tasks.workunit.client.0.smithi110.stdout:6/450: stat d1/db/l3f 0 2024-04-25T15:49:31.600 INFO:tasks.workunit.client.0.smithi110.stdout:4/556: rename d13/c63 to d13/d24/d2d/d44/d90/cb7 0 2024-04-25T15:49:31.603 INFO:tasks.workunit.client.0.smithi110.stdout:3/559: truncate d0/d7/d1c/d2f/d33/f9e 620815 0 2024-04-25T15:49:31.603 INFO:tasks.workunit.client.1.smithi193.stdout:7/146: dwrite d12/d13/f14 [0,4194304] 0 2024-04-25T15:49:31.608 INFO:tasks.workunit.client.0.smithi110.stdout:8/613: dread f11 [0,4194304] 0 2024-04-25T15:49:31.610 INFO:tasks.workunit.client.0.smithi110.stdout:2/488: symlink de/l92 0 2024-04-25T15:49:31.611 INFO:tasks.workunit.client.1.smithi193.stdout:9/188: sync 2024-04-25T15:49:31.614 INFO:tasks.workunit.client.0.smithi110.stdout:2/489: truncate de/d40/d70/f84 262847 0 2024-04-25T15:49:31.615 INFO:tasks.workunit.client.1.smithi193.stdout:4/209: dread db/f35 [0,4194304] 0 2024-04-25T15:49:31.615 INFO:tasks.workunit.client.1.smithi193.stdout:7/147: creat d12/d1e/f34 x:0 0 0 2024-04-25T15:49:31.616 INFO:tasks.workunit.client.0.smithi110.stdout:8/614: dwrite d14/d32/f61 [0,4194304] 0 2024-04-25T15:49:31.617 INFO:tasks.workunit.client.1.smithi193.stdout:4/210: write db/d13/f21 [800340,48235] 0 2024-04-25T15:49:31.619 INFO:tasks.workunit.client.0.smithi110.stdout:0/495: creat d1/d47/d54/d99/fb7 x:0 0 0 2024-04-25T15:49:31.620 INFO:tasks.workunit.client.1.smithi193.stdout:6/189: getdents dd/d12/d19/d1c 0 2024-04-25T15:49:31.621 INFO:tasks.workunit.client.0.smithi110.stdout:1/584: symlink d0/d14/d5f/d7d/lb3 0 2024-04-25T15:49:31.623 INFO:tasks.workunit.client.0.smithi110.stdout:9/412: symlink d9/db/df/d55/l81 0 2024-04-25T15:49:31.623 INFO:tasks.workunit.client.1.smithi193.stdout:9/189: dwrite df/d16/f1a [4194304,4194304] 0 2024-04-25T15:49:31.623 INFO:tasks.workunit.client.0.smithi110.stdout:9/413: readlink d9/db/df/d55/l74 0 2024-04-25T15:49:31.625 INFO:tasks.workunit.client.0.smithi110.stdout:6/451: creat d1/db/da0/fa7 x:0 0 0 2024-04-25T15:49:31.626 INFO:tasks.workunit.client.1.smithi193.stdout:3/225: write d8/d13/d1f/f16 [1568402,115643] 0 2024-04-25T15:49:31.627 INFO:tasks.workunit.client.0.smithi110.stdout:6/452: dread - d1/db/d24/f9c zero size 2024-04-25T15:49:31.628 INFO:tasks.workunit.client.1.smithi193.stdout:7/148: rename d12/d1e/l2d to d12/l35 0 2024-04-25T15:49:31.629 INFO:tasks.workunit.client.0.smithi110.stdout:4/557: mkdir d13/d24/d2d/d44/d90/db8 0 2024-04-25T15:49:31.631 INFO:tasks.workunit.client.0.smithi110.stdout:9/414: dwrite d9/f5f [0,4194304] 0 2024-04-25T15:49:31.633 INFO:tasks.workunit.client.0.smithi110.stdout:3/560: creat d0/d16/d8c/d90/fbe x:0 0 0 2024-04-25T15:49:31.636 INFO:tasks.workunit.client.0.smithi110.stdout:7/498: creat d1b/d29/fa4 x:0 0 0 2024-04-25T15:49:31.638 INFO:tasks.workunit.client.1.smithi193.stdout:6/190: unlink fb 0 2024-04-25T15:49:31.641 INFO:tasks.workunit.client.1.smithi193.stdout:3/226: creat d8/d13/d14/d23/f48 x:0 0 0 2024-04-25T15:49:31.641 INFO:tasks.workunit.client.0.smithi110.stdout:5/504: getdents d4/d2c 0 2024-04-25T15:49:31.644 INFO:tasks.workunit.client.1.smithi193.stdout:1/158: getdents dc 0 2024-04-25T15:49:31.645 INFO:tasks.workunit.client.1.smithi193.stdout:6/191: creat dd/d12/f35 x:0 0 0 2024-04-25T15:49:31.645 INFO:tasks.workunit.client.1.smithi193.stdout:1/159: chown dc/l11 487002731 1 2024-04-25T15:49:31.651 INFO:tasks.workunit.client.1.smithi193.stdout:1/160: rmdir dc 39 2024-04-25T15:49:31.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/192: dread f6 [4194304,4194304] 0 2024-04-25T15:49:31.652 INFO:tasks.workunit.client.1.smithi193.stdout:4/211: dread f9 [0,4194304] 0 2024-04-25T15:49:31.658 INFO:tasks.workunit.client.1.smithi193.stdout:6/193: symlink dd/d2f/l36 0 2024-04-25T15:49:31.659 INFO:tasks.workunit.client.1.smithi193.stdout:1/161: dread dc/f2c [0,4194304] 0 2024-04-25T15:49:31.659 INFO:tasks.workunit.client.1.smithi193.stdout:9/190: getdents df/d2c/d2f 0 2024-04-25T15:49:31.660 INFO:tasks.workunit.client.1.smithi193.stdout:1/162: chown dc/l2a 1959551 1 2024-04-25T15:49:31.660 INFO:tasks.workunit.client.1.smithi193.stdout:9/191: chown df/d16/f19 10134 1 2024-04-25T15:49:31.660 INFO:tasks.workunit.client.1.smithi193.stdout:3/227: truncate d8/d13/f41 141715 0 2024-04-25T15:49:31.661 INFO:tasks.workunit.client.1.smithi193.stdout:9/192: stat df/d16/f25 0 2024-04-25T15:49:31.661 INFO:tasks.workunit.client.1.smithi193.stdout:3/228: write d8/d13/f32 [4340443,58410] 0 2024-04-25T15:49:31.661 INFO:tasks.workunit.client.1.smithi193.stdout:9/193: chown df/d11/f14 323 1 2024-04-25T15:49:31.662 INFO:tasks.workunit.client.0.smithi110.stdout:3/561: write d0/f70 [1078857,130186] 0 2024-04-25T15:49:31.662 INFO:tasks.workunit.client.1.smithi193.stdout:4/212: truncate db/d13/d1e/f41 78003 0 2024-04-25T15:49:31.663 INFO:tasks.workunit.client.0.smithi110.stdout:7/499: dwrite d1b/d29/f5d [0,4194304] 0 2024-04-25T15:49:31.663 INFO:tasks.workunit.client.1.smithi193.stdout:9/194: read df/f1b [49526,28137] 0 2024-04-25T15:49:31.663 INFO:tasks.workunit.client.1.smithi193.stdout:6/194: stat dd/f13 0 2024-04-25T15:49:31.665 INFO:tasks.workunit.client.0.smithi110.stdout:7/500: chown d1b/d29/d38/d55/d7f/l74 40540 1 2024-04-25T15:49:31.665 INFO:tasks.workunit.client.1.smithi193.stdout:1/163: symlink dc/l32 0 2024-04-25T15:49:31.667 INFO:tasks.workunit.client.1.smithi193.stdout:4/213: dread f9 [0,4194304] 0 2024-04-25T15:49:31.668 INFO:tasks.workunit.client.1.smithi193.stdout:4/214: dread - db/f38 zero size 2024-04-25T15:49:31.669 INFO:tasks.workunit.client.0.smithi110.stdout:6/453: truncate d1/db/d22/f4d 437055 0 2024-04-25T15:49:31.669 INFO:tasks.workunit.client.1.smithi193.stdout:4/215: write db/d13/f21 [1509548,11048] 0 2024-04-25T15:49:31.670 INFO:tasks.workunit.client.0.smithi110.stdout:6/454: fsync d1/d12/d3d/f4a 0 2024-04-25T15:49:31.670 INFO:tasks.workunit.client.0.smithi110.stdout:4/558: symlink d13/d24/d2d/d44/lb9 0 2024-04-25T15:49:31.670 INFO:tasks.workunit.client.1.smithi193.stdout:9/195: dwrite df/d11/f2a [0,4194304] 0 2024-04-25T15:49:31.671 INFO:tasks.workunit.client.0.smithi110.stdout:9/415: getdents d9/db/d7d 0 2024-04-25T15:49:31.671 INFO:tasks.workunit.client.0.smithi110.stdout:6/455: truncate d1/db/f8c 874028 0 2024-04-25T15:49:31.671 INFO:tasks.workunit.client.0.smithi110.stdout:4/559: fsync d13/d24/d2d/d44/d65/f72 0 2024-04-25T15:49:31.674 INFO:tasks.workunit.client.0.smithi110.stdout:9/416: dread d9/db/df/d25/f51 [0,4194304] 0 2024-04-25T15:49:31.674 INFO:tasks.workunit.client.1.smithi193.stdout:6/195: rename dd/d2f/f30 to dd/d12/d2d/f37 0 2024-04-25T15:49:31.675 INFO:tasks.workunit.client.0.smithi110.stdout:9/417: write d9/db/df/f6c [546832,24349] 0 2024-04-25T15:49:31.678 INFO:tasks.workunit.client.1.smithi193.stdout:4/216: dwrite db/d13/d1e/d1f/d22/f2c [0,4194304] 0 2024-04-25T15:49:31.681 INFO:tasks.workunit.client.0.smithi110.stdout:5/505: creat d4/d15/d51/d90/fa3 x:0 0 0 2024-04-25T15:49:31.683 INFO:tasks.workunit.client.0.smithi110.stdout:1/585: write d0/d25/d33/f24 [1045941,52508] 0 2024-04-25T15:49:31.684 INFO:tasks.workunit.client.0.smithi110.stdout:5/506: read d4/d2c/f19 [374245,115657] 0 2024-04-25T15:49:31.685 INFO:tasks.workunit.client.0.smithi110.stdout:2/490: creat de/d40/d6e/f93 x:0 0 0 2024-04-25T15:49:31.686 INFO:tasks.workunit.client.1.smithi193.stdout:2/196: sync 2024-04-25T15:49:31.686 INFO:tasks.workunit.client.1.smithi193.stdout:5/158: sync 2024-04-25T15:49:31.686 INFO:tasks.workunit.client.1.smithi193.stdout:0/193: sync 2024-04-25T15:49:31.687 INFO:tasks.workunit.client.1.smithi193.stdout:3/229: symlink d8/l49 0 2024-04-25T15:49:31.687 INFO:tasks.workunit.client.1.smithi193.stdout:5/159: chown d17/d18/d30/d3b 226 1 2024-04-25T15:49:31.687 INFO:tasks.workunit.client.1.smithi193.stdout:0/194: read - d0/d3/f2c zero size 2024-04-25T15:49:31.688 INFO:tasks.workunit.client.1.smithi193.stdout:2/197: dread - d0/d9/d3d/f45 zero size 2024-04-25T15:49:31.688 INFO:tasks.workunit.client.1.smithi193.stdout:0/195: chown d0/d3 0 1 2024-04-25T15:49:31.688 INFO:tasks.workunit.client.1.smithi193.stdout:5/160: stat d17/d18/f29 0 2024-04-25T15:49:31.689 INFO:tasks.workunit.client.1.smithi193.stdout:9/196: mkdir df/d2c/d34 0 2024-04-25T15:49:31.693 INFO:tasks.workunit.client.0.smithi110.stdout:3/562: rename d0/d16/d8c/d90 to d0/d2e/d88/dbf 0 2024-04-25T15:49:31.693 INFO:tasks.workunit.client.0.smithi110.stdout:3/563: readlink d0/d16/l51 0 2024-04-25T15:49:31.696 INFO:tasks.workunit.client.1.smithi193.stdout:3/230: dwrite d8/d13/d1f/f16 [0,4194304] 0 2024-04-25T15:49:31.696 INFO:tasks.workunit.client.1.smithi193.stdout:5/161: dwrite d17/f39 [0,4194304] 0 2024-04-25T15:49:31.700 INFO:tasks.workunit.client.0.smithi110.stdout:3/564: dwrite d0/d16/f3d [0,4194304] 0 2024-04-25T15:49:31.702 INFO:tasks.workunit.client.1.smithi193.stdout:7/149: sync 2024-04-25T15:49:31.703 INFO:tasks.workunit.client.1.smithi193.stdout:8/180: sync 2024-04-25T15:49:31.703 INFO:tasks.workunit.client.1.smithi193.stdout:6/196: creat dd/d12/d19/d1c/f38 x:0 0 0 2024-04-25T15:49:31.707 INFO:tasks.workunit.client.1.smithi193.stdout:4/217: symlink db/d13/d1e/d1f/l43 0 2024-04-25T15:49:31.707 INFO:tasks.workunit.client.0.smithi110.stdout:6/456: readlink d1/dd/l10 0 2024-04-25T15:49:31.708 INFO:tasks.workunit.client.1.smithi193.stdout:1/164: read dc/f15 [20003,95270] 0 2024-04-25T15:49:31.712 INFO:tasks.workunit.client.0.smithi110.stdout:9/418: creat d9/db/d58/f82 x:0 0 0 2024-04-25T15:49:31.714 INFO:tasks.workunit.client.0.smithi110.stdout:1/586: mkdir d0/d25/db4 0 2024-04-25T15:49:31.714 INFO:tasks.workunit.client.0.smithi110.stdout:2/491: dread de/d40/d4f/f4e [0,4194304] 0 2024-04-25T15:49:31.714 INFO:tasks.workunit.client.0.smithi110.stdout:6/457: read d1/db/d24/f29 [1519844,17459] 0 2024-04-25T15:49:31.714 INFO:tasks.workunit.client.1.smithi193.stdout:2/198: mkdir d0/d9/d30/d4f 0 2024-04-25T15:49:31.715 INFO:tasks.workunit.client.0.smithi110.stdout:1/587: fdatasync d0/d14/f1f 0 2024-04-25T15:49:31.716 INFO:tasks.workunit.client.0.smithi110.stdout:1/588: chown d0/d14/d5f/l87 3006798 1 2024-04-25T15:49:31.716 INFO:tasks.workunit.client.1.smithi193.stdout:0/196: mknod d0/d3/d5/d7/dd/c42 0 2024-04-25T15:49:31.717 INFO:tasks.workunit.client.0.smithi110.stdout:8/615: getdents d14/d4c 0 2024-04-25T15:49:31.719 INFO:tasks.workunit.client.0.smithi110.stdout:0/496: link d1/d23/d30/d33/c2e d1/d20/d69/cb8 0 2024-04-25T15:49:31.720 INFO:tasks.workunit.client.0.smithi110.stdout:0/497: fsync d1/d23/d30/fa3 0 2024-04-25T15:49:31.721 INFO:tasks.workunit.client.1.smithi193.stdout:3/231: symlink d8/d30/d31/l4a 0 2024-04-25T15:49:31.721 INFO:tasks.workunit.client.0.smithi110.stdout:0/498: chown d1/d23/d30/d33/d35/c97 970145 1 2024-04-25T15:49:31.721 INFO:tasks.workunit.client.0.smithi110.stdout:7/501: mknod d1b/ca5 0 2024-04-25T15:49:31.722 INFO:tasks.workunit.client.1.smithi193.stdout:0/197: dwrite d0/d3/d5/d7/dd/d16/d26/d2e/f34 [0,4194304] 0 2024-04-25T15:49:31.722 INFO:tasks.workunit.client.0.smithi110.stdout:0/499: fdatasync d1/d23/d30/d33/d35/f12 0 2024-04-25T15:49:31.723 INFO:tasks.workunit.client.0.smithi110.stdout:7/502: dread - d1b/d29/d38/d8b/f92 zero size 2024-04-25T15:49:31.724 INFO:tasks.workunit.client.1.smithi193.stdout:0/198: write d0/d3/fe [1455921,26784] 0 2024-04-25T15:49:31.725 INFO:tasks.workunit.client.1.smithi193.stdout:5/162: symlink d17/d18/d30/l3d 0 2024-04-25T15:49:31.727 INFO:tasks.workunit.client.1.smithi193.stdout:0/199: truncate d0/d3/d5/d7/dd/d16/d18/f2a 1444967 0 2024-04-25T15:49:31.727 INFO:tasks.workunit.client.1.smithi193.stdout:0/200: write d0/d3/d5/f24 [277546,35778] 0 2024-04-25T15:49:31.734 INFO:tasks.workunit.client.0.smithi110.stdout:4/560: write d13/d24/d2d/d36/d51/f8c [777783,20127] 0 2024-04-25T15:49:31.734 INFO:tasks.workunit.client.1.smithi193.stdout:8/181: rmdir d2/d3 39 2024-04-25T15:49:31.738 INFO:tasks.workunit.client.1.smithi193.stdout:4/218: symlink db/d13/d1e/d1f/d22/l44 0 2024-04-25T15:49:31.740 INFO:tasks.workunit.client.1.smithi193.stdout:2/199: mkdir d0/d9/d15/d31/d50 0 2024-04-25T15:49:31.741 INFO:tasks.workunit.client.0.smithi110.stdout:1/589: dread d0/d51/f79 [0,4194304] 0 2024-04-25T15:49:31.741 INFO:tasks.workunit.client.0.smithi110.stdout:1/590: readlink d0/d25/lae 0 2024-04-25T15:49:31.741 INFO:tasks.workunit.client.0.smithi110.stdout:2/492: chown de/d40/d6e/f3a 342018282 1 2024-04-25T15:49:31.742 INFO:tasks.workunit.client.1.smithi193.stdout:6/197: dwrite dd/f1b [0,4194304] 0 2024-04-25T15:49:31.743 INFO:tasks.workunit.client.0.smithi110.stdout:1/591: write d0/d25/d33/f48 [2101380,115407] 0 2024-04-25T15:49:31.743 INFO:tasks.workunit.client.0.smithi110.stdout:2/493: write de/d40/f72 [426347,95909] 0 2024-04-25T15:49:31.744 INFO:tasks.workunit.client.0.smithi110.stdout:6/458: creat d1/db/d24/d69/d75/fa8 x:0 0 0 2024-04-25T15:49:31.747 INFO:tasks.workunit.client.0.smithi110.stdout:8/616: rename d14/d4c/c4d to d14/d32/cd0 0 2024-04-25T15:49:31.750 INFO:tasks.workunit.client.1.smithi193.stdout:7/150: write f3 [2341658,62968] 0 2024-04-25T15:49:31.753 INFO:tasks.workunit.client.0.smithi110.stdout:7/503: symlink d1b/d29/d46/la6 0 2024-04-25T15:49:31.755 INFO:tasks.workunit.client.1.smithi193.stdout:9/197: write df/d16/d1c/f21 [1575818,92509] 0 2024-04-25T15:49:31.756 INFO:tasks.workunit.client.0.smithi110.stdout:0/500: mkdir d1/d23/d30/d33/d35/da1/db4/db9 0 2024-04-25T15:49:31.757 INFO:tasks.workunit.client.1.smithi193.stdout:0/201: mknod d0/d3/d5/d7/dd/d16/d26/d2e/d3d/c43 0 2024-04-25T15:49:31.760 INFO:tasks.workunit.client.1.smithi193.stdout:1/165: mknod dc/c33 0 2024-04-25T15:49:31.761 INFO:tasks.workunit.client.1.smithi193.stdout:4/219: dread db/f15 [0,4194304] 0 2024-04-25T15:49:31.762 INFO:tasks.workunit.client.1.smithi193.stdout:4/220: write db/d13/f21 [1628051,21495] 0 2024-04-25T15:49:31.763 INFO:tasks.workunit.client.1.smithi193.stdout:2/200: unlink d0/d9/d36/f3f 0 2024-04-25T15:49:31.764 INFO:tasks.workunit.client.1.smithi193.stdout:2/201: write d0/d37/d25/d38/f3b [1010565,7785] 0 2024-04-25T15:49:31.766 INFO:tasks.workunit.client.1.smithi193.stdout:3/232: mkdir d8/d13/d4b 0 2024-04-25T15:49:31.767 INFO:tasks.workunit.client.0.smithi110.stdout:6/459: write d1/db/d22/d90/f72 [1872050,89759] 0 2024-04-25T15:49:31.768 INFO:tasks.workunit.client.1.smithi193.stdout:7/151: creat d12/f36 x:0 0 0 2024-04-25T15:49:31.769 INFO:tasks.workunit.client.0.smithi110.stdout:8/617: creat d14/d32/d36/d48/d8f/fd1 x:0 0 0 2024-04-25T15:49:31.769 INFO:tasks.workunit.client.1.smithi193.stdout:7/152: chown d12/d1e/f25 1299 1 2024-04-25T15:49:31.770 INFO:tasks.workunit.client.1.smithi193.stdout:5/163: mkdir d17/d18/d3e 0 2024-04-25T15:49:31.771 INFO:tasks.workunit.client.1.smithi193.stdout:7/153: read d12/f18 [506437,123044] 0 2024-04-25T15:49:31.771 INFO:tasks.workunit.client.0.smithi110.stdout:7/504: mknod d1b/d29/d38/ca7 0 2024-04-25T15:49:31.773 INFO:tasks.workunit.client.1.smithi193.stdout:3/233: dwrite d8/d30/f42 [0,4194304] 0 2024-04-25T15:49:31.774 INFO:tasks.workunit.client.1.smithi193.stdout:0/202: symlink d0/d3/d5/d7/l44 0 2024-04-25T15:49:31.774 INFO:tasks.workunit.client.0.smithi110.stdout:3/565: link d0/d16/f79 d0/d7/d1c/d2f/d4c/d63/fc0 0 2024-04-25T15:49:31.775 INFO:tasks.workunit.client.0.smithi110.stdout:3/566: dread - d0/d2e/d88/dbf/d99/fb5 zero size 2024-04-25T15:49:31.776 INFO:tasks.workunit.client.1.smithi193.stdout:1/166: chown dc/f18 19509 1 2024-04-25T15:49:31.776 INFO:tasks.workunit.client.0.smithi110.stdout:3/567: chown d0/d7/d1c/d50/c62 207 1 2024-04-25T15:49:31.777 INFO:tasks.workunit.client.0.smithi110.stdout:4/561: mkdir d13/dba 0 2024-04-25T15:49:31.783 INFO:tasks.workunit.client.0.smithi110.stdout:5/507: getdents d4/d1d/d1f/d56 0 2024-04-25T15:49:31.784 INFO:tasks.workunit.client.1.smithi193.stdout:4/221: dread db/d13/d1e/d1f/d22/f2a [0,4194304] 0 2024-04-25T15:49:31.785 INFO:tasks.workunit.client.0.smithi110.stdout:5/508: chown d4/d1d/d85/l98 2546630 1 2024-04-25T15:49:31.785 INFO:tasks.workunit.client.1.smithi193.stdout:4/222: chown db/d13/d1e/d1f/c3a 126931080 1 2024-04-25T15:49:31.785 INFO:tasks.workunit.client.0.smithi110.stdout:2/494: dwrite de/d40/d6e/f3a [0,4194304] 0 2024-04-25T15:49:31.787 INFO:tasks.workunit.client.1.smithi193.stdout:2/202: rename d0/d9/d30 to d0/d42/d51 0 2024-04-25T15:49:31.788 INFO:tasks.workunit.client.1.smithi193.stdout:2/203: dread - d0/d37/f43 zero size 2024-04-25T15:49:31.792 INFO:tasks.workunit.client.0.smithi110.stdout:0/501: creat d1/d23/d30/d33/d5b/d78/da4/fba x:0 0 0 2024-04-25T15:49:31.796 INFO:tasks.workunit.client.1.smithi193.stdout:9/198: creat df/d16/d1c/d26/f35 x:0 0 0 2024-04-25T15:49:31.799 INFO:tasks.workunit.client.0.smithi110.stdout:3/568: readlink d0/l5d 0 2024-04-25T15:49:31.800 INFO:tasks.workunit.client.0.smithi110.stdout:9/419: link d9/c47 d9/db/df/c83 0 2024-04-25T15:49:31.801 INFO:tasks.workunit.client.1.smithi193.stdout:0/203: write d0/d3/d5/d7/dd/d16/d18/f22 [2193190,102074] 0 2024-04-25T15:49:31.802 INFO:tasks.workunit.client.1.smithi193.stdout:1/167: chown dc/l28 1900 1 2024-04-25T15:49:31.809 INFO:tasks.workunit.client.1.smithi193.stdout:5/164: creat d17/d18/d3e/f3f x:0 0 0 2024-04-25T15:49:31.809 INFO:tasks.workunit.client.1.smithi193.stdout:5/165: stat d17/d18/f2c 0 2024-04-25T15:49:31.810 INFO:tasks.workunit.client.1.smithi193.stdout:5/166: write f13 [2057169,43427] 0 2024-04-25T15:49:31.811 INFO:tasks.workunit.client.0.smithi110.stdout:7/505: link d1b/d29/d38/f5b d1b/d29/d38/d8b/fa8 0 2024-04-25T15:49:31.813 INFO:tasks.workunit.client.1.smithi193.stdout:9/199: symlink df/d16/d1c/d26/l36 0 2024-04-25T15:49:31.813 INFO:tasks.workunit.client.0.smithi110.stdout:7/506: read f14 [3375885,41080] 0 2024-04-25T15:49:31.815 INFO:tasks.workunit.client.0.smithi110.stdout:0/502: rename d1/d23/d30/d33/d5b/d83/d87 to d1/d23/d30/d33/d5b/dbb 0 2024-04-25T15:49:31.816 INFO:tasks.workunit.client.1.smithi193.stdout:7/154: mkdir d12/d31/d37 0 2024-04-25T15:49:31.817 INFO:tasks.workunit.client.0.smithi110.stdout:5/509: dread d4/d1d/d1f/f6e [0,4194304] 0 2024-04-25T15:49:31.817 INFO:tasks.workunit.client.0.smithi110.stdout:0/503: chown d1/d47/f86 21 1 2024-04-25T15:49:31.818 INFO:tasks.workunit.client.1.smithi193.stdout:9/200: dwrite df/d16/d1c/f21 [0,4194304] 0 2024-04-25T15:49:31.819 INFO:tasks.workunit.client.1.smithi193.stdout:3/234: mknod d8/d30/d45/c4c 0 2024-04-25T15:49:31.820 INFO:tasks.workunit.client.0.smithi110.stdout:6/460: truncate d1/db/f88 3346699 0 2024-04-25T15:49:31.823 INFO:tasks.workunit.client.1.smithi193.stdout:0/204: rmdir d0/d3/d5/d7 39 2024-04-25T15:49:31.824 INFO:tasks.workunit.client.0.smithi110.stdout:4/562: mkdir d13/d24/d2d/d44/d90/db8/dbb 0 2024-04-25T15:49:31.824 INFO:tasks.workunit.client.1.smithi193.stdout:3/235: dread d8/d13/f2d [0,4194304] 0 2024-04-25T15:49:31.825 INFO:tasks.workunit.client.0.smithi110.stdout:8/618: write d14/d21/f67 [1475435,3293] 0 2024-04-25T15:49:31.825 INFO:tasks.workunit.client.1.smithi193.stdout:6/198: getdents dd/d12/d2d 0 2024-04-25T15:49:31.829 INFO:tasks.workunit.client.1.smithi193.stdout:3/236: dread d8/f10 [0,4194304] 0 2024-04-25T15:49:31.829 INFO:tasks.workunit.client.1.smithi193.stdout:8/182: sync 2024-04-25T15:49:31.830 INFO:tasks.workunit.client.0.smithi110.stdout:2/495: dwrite de/d40/f35 [0,4194304] 0 2024-04-25T15:49:31.831 INFO:tasks.workunit.client.0.smithi110.stdout:7/507: dread d1b/d29/f68 [0,4194304] 0 2024-04-25T15:49:31.831 INFO:tasks.workunit.client.1.smithi193.stdout:6/199: dwrite dd/f2e [0,4194304] 0 2024-04-25T15:49:31.833 INFO:tasks.workunit.client.1.smithi193.stdout:7/155: mkdir d12/d38 0 2024-04-25T15:49:31.833 INFO:tasks.workunit.client.0.smithi110.stdout:6/461: read d1/dd/d3c/f51 [7097,82753] 0 2024-04-25T15:49:31.834 INFO:tasks.workunit.client.0.smithi110.stdout:6/462: dread - d1/dd/d3c/d91/f7f zero size 2024-04-25T15:49:31.838 INFO:tasks.workunit.client.0.smithi110.stdout:4/563: mkdir d13/d24/d2d/d44/d65/d99/dbc 0 2024-04-25T15:49:31.839 INFO:tasks.workunit.client.1.smithi193.stdout:9/201: dwrite f4 [0,4194304] 0 2024-04-25T15:49:31.840 INFO:tasks.workunit.client.0.smithi110.stdout:3/569: dread d0/d16/f1f [0,4194304] 0 2024-04-25T15:49:31.841 INFO:tasks.workunit.client.1.smithi193.stdout:9/202: write df/d16/f1e [3166915,60428] 0 2024-04-25T15:49:31.842 INFO:tasks.workunit.client.1.smithi193.stdout:1/168: symlink dc/l34 0 2024-04-25T15:49:31.842 INFO:tasks.workunit.client.1.smithi193.stdout:9/203: write df/d16/f1a [3113602,85202] 0 2024-04-25T15:49:31.844 INFO:tasks.workunit.client.1.smithi193.stdout:2/204: rename d0/df/c18 to d0/c52 0 2024-04-25T15:49:31.849 INFO:tasks.workunit.client.1.smithi193.stdout:5/167: mknod d17/d18/d30/d3b/c40 0 2024-04-25T15:49:31.851 INFO:tasks.workunit.client.0.smithi110.stdout:7/508: fsync d1b/d29/f57 0 2024-04-25T15:49:31.855 INFO:tasks.workunit.client.0.smithi110.stdout:4/564: dread d13/f22 [0,4194304] 0 2024-04-25T15:49:31.855 INFO:tasks.workunit.client.0.smithi110.stdout:5/510: truncate d4/d1d/f65 166122 0 2024-04-25T15:49:31.855 INFO:tasks.workunit.client.1.smithi193.stdout:5/168: dwrite d17/f1d [0,4194304] 0 2024-04-25T15:49:31.856 INFO:tasks.workunit.client.1.smithi193.stdout:4/223: sync 2024-04-25T15:49:31.856 INFO:tasks.workunit.client.0.smithi110.stdout:4/565: dread - d13/d24/d2d/d7d/d92/f96 zero size 2024-04-25T15:49:31.856 INFO:tasks.workunit.client.0.smithi110.stdout:5/511: chown d4/d1d/d1f/d35/d7f/f8d 1458 1 2024-04-25T15:49:31.857 INFO:tasks.workunit.client.1.smithi193.stdout:8/183: unlink d2/c26 0 2024-04-25T15:49:31.859 INFO:tasks.workunit.client.0.smithi110.stdout:3/570: mknod d0/d2e/cc1 0 2024-04-25T15:49:31.864 INFO:tasks.workunit.client.0.smithi110.stdout:9/420: truncate d9/db/df/f6c 422998 0 2024-04-25T15:49:31.866 INFO:tasks.workunit.client.0.smithi110.stdout:0/504: rename d1/d23/d30/fa3 to d1/fbc 0 2024-04-25T15:49:31.866 INFO:tasks.workunit.client.1.smithi193.stdout:0/205: mknod d0/d3/d5/d7/c45 0 2024-04-25T15:49:31.869 INFO:tasks.workunit.client.0.smithi110.stdout:7/509: creat d1b/d29/d38/d55/d95/fa9 x:0 0 0 2024-04-25T15:49:31.870 INFO:tasks.workunit.client.1.smithi193.stdout:7/156: dread f7 [0,4194304] 0 2024-04-25T15:49:31.871 INFO:tasks.workunit.client.0.smithi110.stdout:7/510: chown d1b/d29/d38/f54 1156 1 2024-04-25T15:49:31.872 INFO:tasks.workunit.client.0.smithi110.stdout:0/505: dwrite d1/d23/d30/d33/d5b/d83/f68 [0,4194304] 0 2024-04-25T15:49:31.873 INFO:tasks.workunit.client.0.smithi110.stdout:5/512: creat d4/d15/d51/fa4 x:0 0 0 2024-04-25T15:49:31.874 INFO:tasks.workunit.client.1.smithi193.stdout:1/169: rmdir dc 39 2024-04-25T15:49:31.876 INFO:tasks.workunit.client.1.smithi193.stdout:7/157: dwrite d12/f2e [0,4194304] 0 2024-04-25T15:49:31.881 INFO:tasks.workunit.client.0.smithi110.stdout:8/619: truncate d14/d4c/f75 205069 0 2024-04-25T15:49:31.882 INFO:tasks.workunit.client.0.smithi110.stdout:9/421: mkdir d9/d13/d84 0 2024-04-25T15:49:31.882 INFO:tasks.workunit.client.0.smithi110.stdout:8/620: write d14/f83 [7467293,21052] 0 2024-04-25T15:49:31.884 INFO:tasks.workunit.client.0.smithi110.stdout:8/621: write f5 [8727323,50496] 0 2024-04-25T15:49:31.887 INFO:tasks.workunit.client.0.smithi110.stdout:5/513: dread d4/d15/f1a [0,4194304] 0 2024-04-25T15:49:31.889 INFO:tasks.workunit.client.1.smithi193.stdout:3/237: fdatasync d8/d30/d31/f3a 0 2024-04-25T15:49:31.893 INFO:tasks.workunit.client.1.smithi193.stdout:4/224: chown fa 0 1 2024-04-25T15:49:31.895 INFO:tasks.workunit.client.0.smithi110.stdout:7/511: dread d1b/d29/f60 [0,4194304] 0 2024-04-25T15:49:31.896 INFO:tasks.workunit.client.1.smithi193.stdout:6/200: symlink dd/d12/d19/l39 0 2024-04-25T15:49:31.898 INFO:tasks.workunit.client.0.smithi110.stdout:3/571: creat d0/d7/d1c/db0/fc2 x:0 0 0 2024-04-25T15:49:31.899 INFO:tasks.workunit.client.1.smithi193.stdout:0/206: mkdir d0/d3/d5/d7/dd/d46 0 2024-04-25T15:49:31.900 INFO:tasks.workunit.client.1.smithi193.stdout:0/207: fsync d0/d3/d5/d7/dd/d16/f3a 0 2024-04-25T15:49:31.903 INFO:tasks.workunit.client.0.smithi110.stdout:7/512: dwrite d1b/f88 [4194304,4194304] 0 2024-04-25T15:49:31.906 INFO:tasks.workunit.client.0.smithi110.stdout:9/422: readlink d9/db/df/d25/l71 0 2024-04-25T15:49:31.907 INFO:tasks.workunit.client.0.smithi110.stdout:9/423: write d9/db/df/f68 [384489,43991] 0 2024-04-25T15:49:31.907 INFO:tasks.workunit.client.1.smithi193.stdout:7/158: mkdir d12/d31/d39 0 2024-04-25T15:49:31.908 INFO:tasks.workunit.client.0.smithi110.stdout:8/622: fdatasync d14/d21/d26/f7a 0 2024-04-25T15:49:31.910 INFO:tasks.workunit.client.1.smithi193.stdout:5/169: write d17/f2a [1036899,42521] 0 2024-04-25T15:49:31.910 INFO:tasks.workunit.client.0.smithi110.stdout:8/623: truncate d14/d32/fa5 453436 0 2024-04-25T15:49:31.911 INFO:tasks.workunit.client.0.smithi110.stdout:5/514: mknod d4/d1d/d1f/d35/d7f/ca5 0 2024-04-25T15:49:31.912 INFO:tasks.workunit.client.1.smithi193.stdout:2/205: mknod d0/d9/c53 0 2024-04-25T15:49:31.913 INFO:tasks.workunit.client.0.smithi110.stdout:4/566: mknod d13/d24/da6/cbd 0 2024-04-25T15:49:31.914 INFO:tasks.workunit.client.0.smithi110.stdout:0/506: symlink d1/d47/lbd 0 2024-04-25T15:49:31.917 INFO:tasks.workunit.client.1.smithi193.stdout:2/206: dwrite d0/d9/f39 [0,4194304] 0 2024-04-25T15:49:31.917 INFO:tasks.workunit.client.0.smithi110.stdout:4/567: dread d13/d24/d2d/d36/d51/f78 [0,4194304] 0 2024-04-25T15:49:31.919 INFO:tasks.workunit.client.1.smithi193.stdout:2/207: truncate d0/d42/d51/f46 141324 0 2024-04-25T15:49:31.919 INFO:tasks.workunit.client.1.smithi193.stdout:2/208: readlink d0/l2 0 2024-04-25T15:49:31.946 INFO:tasks.workunit.client.1.smithi193.stdout:6/201: sync 2024-04-25T15:49:31.947 INFO:tasks.workunit.client.1.smithi193.stdout:6/202: write dd/d12/d19/d1c/f22 [3840178,44046] 0 2024-04-25T15:49:31.983 INFO:tasks.workunit.client.1.smithi193.stdout:8/184: mkdir d2/d3/d27/d34 0 2024-04-25T15:49:31.985 INFO:tasks.workunit.client.1.smithi193.stdout:0/208: symlink d0/d3/d5/d7/dd/d16/d26/d2e/l47 0 2024-04-25T15:49:31.988 INFO:tasks.workunit.client.1.smithi193.stdout:8/185: dwrite d2/d3/ff [0,4194304] 0 2024-04-25T15:49:31.990 INFO:tasks.workunit.client.1.smithi193.stdout:9/204: creat df/d16/f37 x:0 0 0 2024-04-25T15:49:31.990 INFO:tasks.workunit.client.1.smithi193.stdout:9/205: fsync df/d16/d1c/f1f 0 2024-04-25T15:49:31.991 INFO:tasks.workunit.client.0.smithi110.stdout:3/572: dread d0/d7/fc [0,4194304] 0 2024-04-25T15:49:31.991 INFO:tasks.workunit.client.1.smithi193.stdout:9/206: read df/d16/d1c/f21 [1469488,61315] 0 2024-04-25T15:49:31.991 INFO:tasks.workunit.client.1.smithi193.stdout:5/170: mknod d17/d18/d30/d34/c41 0 2024-04-25T15:49:31.993 INFO:tasks.workunit.client.0.smithi110.stdout:8/624: mknod d14/d32/d36/db7/cd2 0 2024-04-25T15:49:31.993 INFO:tasks.workunit.client.0.smithi110.stdout:3/573: dread d0/d7/d1c/d2f/d33/f44 [0,4194304] 0 2024-04-25T15:49:31.996 INFO:tasks.workunit.client.0.smithi110.stdout:5/515: mknod d4/d1d/d1f/d35/d3d/ca6 0 2024-04-25T15:49:31.998 INFO:tasks.workunit.client.1.smithi193.stdout:5/171: dwrite f11 [0,4194304] 0 2024-04-25T15:49:31.998 INFO:tasks.workunit.client.1.smithi193.stdout:9/207: dwrite df/d16/d1c/d26/f35 [0,4194304] 0 2024-04-25T15:49:32.000 INFO:tasks.workunit.client.1.smithi193.stdout:3/238: mkdir d8/d4d 0 2024-04-25T15:49:32.001 INFO:tasks.workunit.client.0.smithi110.stdout:3/574: dwrite d0/f70 [0,4194304] 0 2024-04-25T15:49:32.003 INFO:tasks.workunit.client.1.smithi193.stdout:2/209: mknod d0/df/c54 0 2024-04-25T15:49:32.003 INFO:tasks.workunit.client.0.smithi110.stdout:7/513: write d1b/f4b [733396,124151] 0 2024-04-25T15:49:32.004 INFO:tasks.workunit.client.1.smithi193.stdout:2/210: chown d0/d44/c4e 0 1 2024-04-25T15:49:32.007 INFO:tasks.workunit.client.1.smithi193.stdout:6/203: fdatasync f6 0 2024-04-25T15:49:32.009 INFO:tasks.workunit.client.1.smithi193.stdout:1/170: dwrite dc/f16 [0,4194304] 0 2024-04-25T15:49:32.009 INFO:tasks.workunit.client.1.smithi193.stdout:6/204: dread - dd/d12/d19/d1c/f38 zero size 2024-04-25T15:49:32.010 INFO:tasks.workunit.client.1.smithi193.stdout:6/205: fdatasync dd/f1b 0 2024-04-25T15:49:32.015 INFO:tasks.workunit.client.1.smithi193.stdout:7/159: link d12/d13/f24 d12/d31/d39/f3a 0 2024-04-25T15:49:32.018 INFO:tasks.workunit.client.0.smithi110.stdout:5/516: rmdir d4/d1d 39 2024-04-25T15:49:32.022 INFO:tasks.workunit.client.0.smithi110.stdout:0/507: mknod d1/d20/cbe 0 2024-04-25T15:49:32.022 INFO:tasks.workunit.client.1.smithi193.stdout:8/186: dwrite d2/d3/d20/f22 [0,4194304] 0 2024-04-25T15:49:32.023 INFO:tasks.workunit.client.1.smithi193.stdout:8/187: chown d2/d3/l31 273 1 2024-04-25T15:49:32.023 INFO:tasks.workunit.client.0.smithi110.stdout:3/575: write d0/d2e/f76 [84992,98958] 0 2024-04-25T15:49:32.025 INFO:tasks.workunit.client.0.smithi110.stdout:4/568: symlink d13/dba/lbe 0 2024-04-25T15:49:32.025 INFO:tasks.workunit.client.1.smithi193.stdout:5/172: mknod d17/c42 0 2024-04-25T15:49:32.026 INFO:tasks.workunit.client.0.smithi110.stdout:4/569: read - d13/d24/d2d/d7d/d92/f9a zero size 2024-04-25T15:49:32.026 INFO:tasks.workunit.client.1.smithi193.stdout:5/173: write f11 [4008994,125615] 0 2024-04-25T15:49:32.027 INFO:tasks.workunit.client.0.smithi110.stdout:7/514: truncate d1b/f36 1726718 0 2024-04-25T15:49:32.028 INFO:tasks.workunit.client.0.smithi110.stdout:7/515: chown d1b/d29/d46/f6a 6210164 1 2024-04-25T15:49:32.029 INFO:tasks.workunit.client.1.smithi193.stdout:3/239: mknod d8/d13/d14/c4e 0 2024-04-25T15:49:32.033 INFO:tasks.workunit.client.0.smithi110.stdout:8/625: unlink d14/d4c/d55/c97 0 2024-04-25T15:49:32.034 INFO:tasks.workunit.client.0.smithi110.stdout:8/626: chown d14/d21/d26/f30 83 1 2024-04-25T15:49:32.035 INFO:tasks.workunit.client.0.smithi110.stdout:8/627: fdatasync d14/d4c/d71/d9d/fa1 0 2024-04-25T15:49:32.035 INFO:tasks.workunit.client.0.smithi110.stdout:5/517: rmdir d4/d2c 39 2024-04-25T15:49:32.036 INFO:tasks.workunit.client.1.smithi193.stdout:4/225: truncate db/d13/f21 1219377 0 2024-04-25T15:49:32.036 INFO:tasks.workunit.client.1.smithi193.stdout:4/226: stat db/d13/d1e/d1f/c25 0 2024-04-25T15:49:32.038 INFO:tasks.workunit.client.0.smithi110.stdout:3/576: fdatasync d0/d2e/f71 0 2024-04-25T15:49:32.039 INFO:tasks.workunit.client.0.smithi110.stdout:8/628: dread d14/d4c/d71/d9d/fa1 [0,4194304] 0 2024-04-25T15:49:32.041 INFO:tasks.workunit.client.1.smithi193.stdout:6/206: symlink dd/d12/d19/d21/l3a 0 2024-04-25T15:49:32.041 INFO:tasks.workunit.client.0.smithi110.stdout:8/629: truncate d14/d21/f62 505244 0 2024-04-25T15:49:32.042 INFO:tasks.workunit.client.0.smithi110.stdout:4/570: chown d13/d24/f48 3311 1 2024-04-25T15:49:32.042 INFO:tasks.workunit.client.0.smithi110.stdout:8/630: stat d14/d21/d26/dab/dac/dbb 0 2024-04-25T15:49:32.042 INFO:tasks.workunit.client.1.smithi193.stdout:0/209: mkdir d0/d3/d5/d7/dd/d46/d48 0 2024-04-25T15:49:32.043 INFO:tasks.workunit.client.0.smithi110.stdout:8/631: chown d14/d4c/d71/fb5 2 1 2024-04-25T15:49:32.043 INFO:tasks.workunit.client.0.smithi110.stdout:4/571: write d13/d24/d2d/d36/d51/f8c [839005,83614] 0 2024-04-25T15:49:32.049 INFO:tasks.workunit.client.1.smithi193.stdout:0/210: dwrite d0/d3/d5/d7/dd/f3c [0,4194304] 0 2024-04-25T15:49:32.050 INFO:tasks.workunit.client.0.smithi110.stdout:0/508: symlink d1/d23/d30/d33/d5b/d72/lbf 0 2024-04-25T15:49:32.053 INFO:tasks.workunit.client.1.smithi193.stdout:3/240: dread d8/f11 [0,4194304] 0 2024-04-25T15:49:32.055 INFO:tasks.workunit.client.1.smithi193.stdout:3/241: write d8/d13/d14/d23/f48 [551889,67245] 0 2024-04-25T15:49:32.055 INFO:tasks.workunit.client.1.smithi193.stdout:3/242: chown d8 109082790 1 2024-04-25T15:49:32.056 INFO:tasks.workunit.client.1.smithi193.stdout:3/243: chown d8/d13/d14/d27 10719123 1 2024-04-25T15:49:32.057 INFO:tasks.workunit.client.1.smithi193.stdout:2/211: symlink d0/l55 0 2024-04-25T15:49:32.060 INFO:tasks.workunit.client.1.smithi193.stdout:1/171: mkdir dc/d35 0 2024-04-25T15:49:32.061 INFO:tasks.workunit.client.1.smithi193.stdout:4/227: mkdir db/d32/d45 0 2024-04-25T15:49:32.063 INFO:tasks.workunit.client.1.smithi193.stdout:6/207: symlink dd/d12/d19/d21/l3b 0 2024-04-25T15:49:32.063 INFO:tasks.workunit.client.1.smithi193.stdout:6/208: chown dd/d12/f1e 1 1 2024-04-25T15:49:32.064 INFO:tasks.workunit.client.0.smithi110.stdout:9/424: getdents d9/db/df/d2c 0 2024-04-25T15:49:32.065 INFO:tasks.workunit.client.0.smithi110.stdout:9/425: fsync d9/db/df/d2c/f36 0 2024-04-25T15:49:32.065 INFO:tasks.workunit.client.1.smithi193.stdout:7/160: unlink d12/d13/c27 0 2024-04-25T15:49:32.067 INFO:tasks.workunit.client.1.smithi193.stdout:8/188: symlink d2/d3/d27/d34/l35 0 2024-04-25T15:49:32.069 INFO:tasks.workunit.client.0.smithi110.stdout:9/426: read d9/db/f1b [2711742,50590] 0 2024-04-25T15:49:32.072 INFO:tasks.workunit.client.1.smithi193.stdout:9/208: creat df/f38 x:0 0 0 2024-04-25T15:49:32.073 INFO:tasks.workunit.client.0.smithi110.stdout:7/516: truncate d1b/d29/f5d 3231797 0 2024-04-25T15:49:32.073 INFO:tasks.workunit.client.1.smithi193.stdout:9/209: chown df/d11/c29 61724678 1 2024-04-25T15:49:32.074 INFO:tasks.workunit.client.0.smithi110.stdout:7/517: write d1b/d29/f37 [2289548,3317] 0 2024-04-25T15:49:32.074 INFO:tasks.workunit.client.1.smithi193.stdout:8/189: dread d2/f11 [0,4194304] 0 2024-04-25T15:49:32.078 INFO:tasks.workunit.client.1.smithi193.stdout:9/210: dread df/d11/f2a [0,4194304] 0 2024-04-25T15:49:32.079 INFO:tasks.workunit.client.0.smithi110.stdout:4/572: getdents d13/d24/d2d/d7d/da2 0 2024-04-25T15:49:32.079 INFO:tasks.workunit.client.0.smithi110.stdout:5/518: dread d4/d1d/d1f/d35/d3d/d52/f55 [0,4194304] 0 2024-04-25T15:49:32.080 INFO:tasks.workunit.client.0.smithi110.stdout:5/519: fsync d4/d1d/d1f/f94 0 2024-04-25T15:49:32.082 INFO:tasks.workunit.client.0.smithi110.stdout:5/520: dread - d4/d1d/d1f/d35/d3d/d52/d97/fa1 zero size 2024-04-25T15:49:32.082 INFO:tasks.workunit.client.1.smithi193.stdout:2/212: rename d0/d37/d25/d38 to d0/d9/d36/d3a/d56 0 2024-04-25T15:49:32.083 INFO:tasks.workunit.client.0.smithi110.stdout:3/577: write d0/d7/d1c/d50/f6d [455050,106149] 0 2024-04-25T15:49:32.085 INFO:tasks.workunit.client.1.smithi193.stdout:1/172: write dc/f30 [98099,37184] 0 2024-04-25T15:49:32.088 INFO:tasks.workunit.client.0.smithi110.stdout:0/509: dwrite d1/d23/d30/d33/d35/f3a [0,4194304] 0 2024-04-25T15:49:32.088 INFO:tasks.workunit.client.1.smithi193.stdout:4/228: symlink db/d13/l46 0 2024-04-25T15:49:32.090 INFO:tasks.workunit.client.0.smithi110.stdout:8/632: truncate d14/f1b 3039614 0 2024-04-25T15:49:32.092 INFO:tasks.workunit.client.1.smithi193.stdout:6/209: chown dd/c10 0 1 2024-04-25T15:49:32.092 INFO:tasks.workunit.client.1.smithi193.stdout:5/174: dwrite d17/d18/f24 [0,4194304] 0 2024-04-25T15:49:32.093 INFO:tasks.workunit.client.0.smithi110.stdout:9/427: fsync d9/d13/f29 0 2024-04-25T15:49:32.096 INFO:tasks.workunit.client.0.smithi110.stdout:7/518: symlink d1b/d29/d38/d55/d95/laa 0 2024-04-25T15:49:32.098 INFO:tasks.workunit.client.0.smithi110.stdout:5/521: creat d4/d1d/d1f/d35/d7f/fa7 x:0 0 0 2024-04-25T15:49:32.105 INFO:tasks.workunit.client.1.smithi193.stdout:2/213: mknod d0/d9/d36/d3a/d56/c57 0 2024-04-25T15:49:32.109 INFO:tasks.workunit.client.0.smithi110.stdout:8/633: creat d14/d21/d23/fd3 x:0 0 0 2024-04-25T15:49:32.110 INFO:tasks.workunit.client.0.smithi110.stdout:8/634: dread - d14/d21/d23/fd3 zero size 2024-04-25T15:49:32.110 INFO:tasks.workunit.client.1.smithi193.stdout:4/229: fsync f1 0 2024-04-25T15:49:32.112 INFO:tasks.workunit.client.0.smithi110.stdout:9/428: unlink d9/db/df/f22 0 2024-04-25T15:49:32.113 INFO:tasks.workunit.client.1.smithi193.stdout:8/190: write d2/d18/f25 [6401465,37751] 0 2024-04-25T15:49:32.114 INFO:tasks.workunit.client.1.smithi193.stdout:8/191: chown d2/d3/db/d29 443012873 1 2024-04-25T15:49:32.115 INFO:tasks.workunit.client.1.smithi193.stdout:5/175: symlink d17/d18/d30/d3b/l43 0 2024-04-25T15:49:32.116 INFO:tasks.workunit.client.1.smithi193.stdout:5/176: write d17/f2a [403801,69031] 0 2024-04-25T15:49:32.117 INFO:tasks.workunit.client.0.smithi110.stdout:7/519: mkdir d1b/d29/d38/d55/dab 0 2024-04-25T15:49:32.119 INFO:tasks.workunit.client.1.smithi193.stdout:7/161: link d12/d13/f17 d12/d1c/f3b 0 2024-04-25T15:49:32.120 INFO:tasks.workunit.client.0.smithi110.stdout:5/522: mknod d4/d1d/d85/ca8 0 2024-04-25T15:49:32.120 INFO:tasks.workunit.client.0.smithi110.stdout:3/578: dwrite d0/d16/f79 [0,4194304] 0 2024-04-25T15:49:32.128 INFO:tasks.workunit.client.0.smithi110.stdout:8/635: creat d14/d21/d26/dab/dac/fd4 x:0 0 0 2024-04-25T15:49:32.130 INFO:tasks.workunit.client.0.smithi110.stdout:9/429: mknod d9/db/df/d2c/c85 0 2024-04-25T15:49:32.132 INFO:tasks.workunit.client.0.smithi110.stdout:0/510: truncate d1/d23/d30/d33/f88 8336261 0 2024-04-25T15:49:32.134 INFO:tasks.workunit.client.1.smithi193.stdout:4/230: mknod db/d32/c47 0 2024-04-25T15:49:32.135 INFO:tasks.workunit.client.0.smithi110.stdout:5/523: symlink d4/d1d/d1f/d56/d6b/la9 0 2024-04-25T15:49:32.137 INFO:tasks.workunit.client.0.smithi110.stdout:3/579: symlink d0/d2e/d88/dbf/lc3 0 2024-04-25T15:49:32.137 INFO:tasks.workunit.client.1.smithi193.stdout:9/211: sync 2024-04-25T15:49:32.141 INFO:tasks.workunit.client.0.smithi110.stdout:8/636: dread - d14/d4c/d55/f6d zero size 2024-04-25T15:49:32.141 INFO:tasks.workunit.client.0.smithi110.stdout:8/637: dread - d14/d21/d23/f7c zero size 2024-04-25T15:49:32.144 INFO:tasks.workunit.client.0.smithi110.stdout:9/430: symlink d9/db/d58/l86 0 2024-04-25T15:49:32.146 INFO:tasks.workunit.client.0.smithi110.stdout:7/520: mkdir d1b/dac 0 2024-04-25T15:49:32.147 INFO:tasks.workunit.client.1.smithi193.stdout:0/211: getdents d0/d3/d5/d7/dd/d16/d26/d2e/d3d 0 2024-04-25T15:49:32.147 INFO:tasks.workunit.client.1.smithi193.stdout:0/212: dread - d0/d3/f2c zero size 2024-04-25T15:49:32.149 INFO:tasks.workunit.client.0.smithi110.stdout:0/511: symlink d1/d23/d30/d33/d5b/dbb/lc0 0 2024-04-25T15:49:32.149 INFO:tasks.workunit.client.1.smithi193.stdout:3/244: getdents d8/d13 0 2024-04-25T15:49:32.150 INFO:tasks.workunit.client.1.smithi193.stdout:3/245: chown d8/fc 3981 1 2024-04-25T15:49:32.151 INFO:tasks.workunit.client.1.smithi193.stdout:2/214: link d0/d42/d51/f46 d0/d9/d36/d3a/f58 0 2024-04-25T15:49:32.152 INFO:tasks.workunit.client.0.smithi110.stdout:9/431: dwrite d9/db/df/d55/f60 [0,4194304] 0 2024-04-25T15:49:32.152 INFO:tasks.workunit.client.0.smithi110.stdout:5/524: write d4/d2c/d2a/f46 [632043,28957] 0 2024-04-25T15:49:32.154 INFO:tasks.workunit.client.0.smithi110.stdout:3/580: rename d0/d7/l5e to d0/d2e/d88/dbf/lc4 0 2024-04-25T15:49:32.155 INFO:tasks.workunit.client.1.smithi193.stdout:1/173: creat dc/f36 x:0 0 0 2024-04-25T15:49:32.157 INFO:tasks.workunit.client.0.smithi110.stdout:7/521: dwrite d1b/d29/d38/d55/f6b [4194304,4194304] 0 2024-04-25T15:49:32.159 INFO:tasks.workunit.client.0.smithi110.stdout:7/522: write d1b/d29/d38/d8b/f92 [822337,41547] 0 2024-04-25T15:49:32.159 INFO:tasks.workunit.client.0.smithi110.stdout:8/638: creat d14/d4c/d71/fd5 x:0 0 0 2024-04-25T15:49:32.160 INFO:tasks.workunit.client.1.smithi193.stdout:4/231: creat db/d13/d1e/f48 x:0 0 0 2024-04-25T15:49:32.166 INFO:tasks.workunit.client.1.smithi193.stdout:4/232: dwrite db/d32/f37 [0,4194304] 0 2024-04-25T15:49:32.168 INFO:tasks.workunit.client.0.smithi110.stdout:0/512: creat d1/d47/d54/d99/fc1 x:0 0 0 2024-04-25T15:49:32.168 INFO:tasks.workunit.client.0.smithi110.stdout:7/523: dwrite d1b/d29/d87/d8e/f94 [0,4194304] 0 2024-04-25T15:49:32.173 INFO:tasks.workunit.client.1.smithi193.stdout:5/177: creat d17/d18/d30/d38/f44 x:0 0 0 2024-04-25T15:49:32.173 INFO:tasks.workunit.client.1.smithi193.stdout:7/162: creat d12/d1e/f3c x:0 0 0 2024-04-25T15:49:32.174 INFO:tasks.workunit.client.1.smithi193.stdout:7/163: dread - d12/f2c zero size 2024-04-25T15:49:32.174 INFO:tasks.workunit.client.1.smithi193.stdout:5/178: write d17/d18/f2e [200629,53962] 0 2024-04-25T15:49:32.175 INFO:tasks.workunit.client.1.smithi193.stdout:5/179: chown fe 47459506 1 2024-04-25T15:49:32.176 INFO:tasks.workunit.client.1.smithi193.stdout:5/180: dread d17/d18/d30/f35 [0,4194304] 0 2024-04-25T15:49:32.176 INFO:tasks.workunit.client.1.smithi193.stdout:5/181: fdatasync f16 0 2024-04-25T15:49:32.177 INFO:tasks.workunit.client.1.smithi193.stdout:3/246: mkdir d8/d30/d31/d4f 0 2024-04-25T15:49:32.178 INFO:tasks.workunit.client.0.smithi110.stdout:9/432: dwrite f6 [0,4194304] 0 2024-04-25T15:49:32.184 INFO:tasks.workunit.client.1.smithi193.stdout:1/174: read dc/fe [1001192,52153] 0 2024-04-25T15:49:32.186 INFO:tasks.workunit.client.0.smithi110.stdout:7/524: chown d1b/l28 0 1 2024-04-25T15:49:32.190 INFO:tasks.workunit.client.0.smithi110.stdout:5/525: creat d4/d2c/d83/faa x:0 0 0 2024-04-25T15:49:32.191 INFO:tasks.workunit.client.0.smithi110.stdout:5/526: fsync d4/d1d/d1f/d35/f6a 0 2024-04-25T15:49:32.191 INFO:tasks.workunit.client.1.smithi193.stdout:6/210: getdents dd/d12/d19 0 2024-04-25T15:49:32.192 INFO:tasks.workunit.client.0.smithi110.stdout:5/527: chown d4/d15/c45 46512 1 2024-04-25T15:49:32.195 INFO:tasks.workunit.client.1.smithi193.stdout:7/164: stat f2 0 2024-04-25T15:49:32.196 INFO:tasks.workunit.client.1.smithi193.stdout:7/165: write d12/d1c/f28 [4020294,113886] 0 2024-04-25T15:49:32.197 INFO:tasks.workunit.client.1.smithi193.stdout:7/166: write d12/f1a [3272998,63070] 0 2024-04-25T15:49:32.198 INFO:tasks.workunit.client.1.smithi193.stdout:3/247: creat d8/d30/d31/f50 x:0 0 0 2024-04-25T15:49:32.201 INFO:tasks.workunit.client.0.smithi110.stdout:2/496: sync 2024-04-25T15:49:32.201 INFO:tasks.workunit.client.0.smithi110.stdout:6/463: sync 2024-04-25T15:49:32.201 INFO:tasks.workunit.client.0.smithi110.stdout:1/592: sync 2024-04-25T15:49:32.201 INFO:tasks.workunit.client.0.smithi110.stdout:6/464: stat f0 0 2024-04-25T15:49:32.202 INFO:tasks.workunit.client.0.smithi110.stdout:3/581: rmdir d0/d2e/d9f 0 2024-04-25T15:49:32.202 INFO:tasks.workunit.client.0.smithi110.stdout:2/497: write de/f2d [4348328,2434] 0 2024-04-25T15:49:32.203 INFO:tasks.workunit.client.1.smithi193.stdout:6/211: mkdir dd/d12/d19/d21/d3c 0 2024-04-25T15:49:32.206 INFO:tasks.workunit.client.1.smithi193.stdout:8/192: truncate d2/d18/f25 7565659 0 2024-04-25T15:49:32.208 INFO:tasks.workunit.client.1.smithi193.stdout:6/212: dwrite dd/f2e [0,4194304] 0 2024-04-25T15:49:32.208 INFO:tasks.workunit.client.0.smithi110.stdout:0/513: rename d1/d23/f2f to d1/d23/fc2 0 2024-04-25T15:49:32.209 INFO:tasks.workunit.client.1.smithi193.stdout:6/213: write dd/f24 [3082190,122195] 0 2024-04-25T15:49:32.209 INFO:tasks.workunit.client.1.smithi193.stdout:5/182: mkdir d17/d18/d45 0 2024-04-25T15:49:32.211 INFO:tasks.workunit.client.0.smithi110.stdout:5/528: mknod d4/d1d/d1f/d9e/cab 0 2024-04-25T15:49:32.215 INFO:tasks.workunit.client.1.smithi193.stdout:5/183: dwrite d17/d18/f20 [0,4194304] 0 2024-04-25T15:49:32.215 INFO:tasks.workunit.client.0.smithi110.stdout:8/639: write d14/d21/d26/d6a/f78 [874517,97898] 0 2024-04-25T15:49:32.215 INFO:tasks.workunit.client.0.smithi110.stdout:8/640: stat d14/l8b 0 2024-04-25T15:49:32.217 INFO:tasks.workunit.client.0.smithi110.stdout:8/641: stat d14/d21/d26/d6a/d7e/da6/fa8 0 2024-04-25T15:49:32.218 INFO:tasks.workunit.client.0.smithi110.stdout:8/642: write d14/d4c/d55/f7d [699671,109129] 0 2024-04-25T15:49:32.219 INFO:tasks.workunit.client.1.smithi193.stdout:9/212: sync 2024-04-25T15:49:32.219 INFO:tasks.workunit.client.0.smithi110.stdout:8/643: fsync d14/d32/f61 0 2024-04-25T15:49:32.221 INFO:tasks.workunit.client.0.smithi110.stdout:6/465: creat d1/db/d22/d33/d63/fa9 x:0 0 0 2024-04-25T15:49:32.222 INFO:tasks.workunit.client.1.smithi193.stdout:0/213: truncate d0/d3/d5/d7/dd/d16/d26/d2e/f34 1280506 0 2024-04-25T15:49:32.223 INFO:tasks.workunit.client.1.smithi193.stdout:4/233: dwrite db/f35 [0,4194304] 0 2024-04-25T15:49:32.225 INFO:tasks.workunit.client.0.smithi110.stdout:2/498: creat de/d40/d5b/d66/f94 x:0 0 0 2024-04-25T15:49:32.225 INFO:tasks.workunit.client.1.smithi193.stdout:1/175: getdents dc/d35 0 2024-04-25T15:49:32.226 INFO:tasks.workunit.client.0.smithi110.stdout:0/514: dread d1/f1c [0,4194304] 0 2024-04-25T15:49:32.227 INFO:tasks.workunit.client.0.smithi110.stdout:3/582: rename d0/d7/d1c/c9b to d0/d2e/d88/dbf/da5/cc5 0 2024-04-25T15:49:32.228 INFO:tasks.workunit.client.0.smithi110.stdout:2/499: dread - de/d40/d5b/d5c/d5f/f73 zero size 2024-04-25T15:49:32.228 INFO:tasks.workunit.client.0.smithi110.stdout:8/644: dwrite d14/d21/fba [0,4194304] 0 2024-04-25T15:49:32.229 INFO:tasks.workunit.client.0.smithi110.stdout:8/645: chown f11 18362678 1 2024-04-25T15:49:32.234 INFO:tasks.workunit.client.0.smithi110.stdout:7/525: creat d1b/fad x:0 0 0 2024-04-25T15:49:32.240 INFO:tasks.workunit.client.1.smithi193.stdout:7/167: write d12/f18 [252647,64174] 0 2024-04-25T15:49:32.245 INFO:tasks.workunit.client.0.smithi110.stdout:5/529: rename d4/d2c/d2a/f95 to d4/d1d/d1f/d35/d7f/fac 0 2024-04-25T15:49:32.246 INFO:tasks.workunit.client.0.smithi110.stdout:1/593: write d0/d14/d5f/d7d/d72/f7e [298976,75296] 0 2024-04-25T15:49:32.246 INFO:tasks.workunit.client.1.smithi193.stdout:2/215: link d0/d9/c49 d0/d9/d3c/c59 0 2024-04-25T15:49:32.248 INFO:tasks.workunit.client.0.smithi110.stdout:8/646: mkdir d14/d32/dd6 0 2024-04-25T15:49:32.252 INFO:tasks.workunit.client.0.smithi110.stdout:3/583: creat d0/d2e/fc6 x:0 0 0 2024-04-25T15:49:32.255 INFO:tasks.workunit.client.1.smithi193.stdout:0/214: rename d0/d3 to d0/d49 0 2024-04-25T15:49:32.257 INFO:tasks.workunit.client.0.smithi110.stdout:2/500: creat de/d40/d5b/d66/f95 x:0 0 0 2024-04-25T15:49:32.261 INFO:tasks.workunit.client.1.smithi193.stdout:8/193: mkdir d2/d36 0 2024-04-25T15:49:32.261 INFO:tasks.workunit.client.0.smithi110.stdout:7/526: creat d1b/d29/d38/d55/fae x:0 0 0 2024-04-25T15:49:32.265 INFO:tasks.workunit.client.1.smithi193.stdout:3/248: truncate d8/f15 288716 0 2024-04-25T15:49:32.273 INFO:tasks.workunit.client.1.smithi193.stdout:1/176: symlink dc/l37 0 2024-04-25T15:49:32.276 INFO:tasks.workunit.client.1.smithi193.stdout:8/194: creat d2/d3/f37 x:0 0 0 2024-04-25T15:49:32.278 INFO:tasks.workunit.client.0.smithi110.stdout:4/573: sync 2024-04-25T15:49:32.278 INFO:tasks.workunit.client.1.smithi193.stdout:6/214: creat dd/f3d x:0 0 0 2024-04-25T15:49:32.279 INFO:tasks.workunit.client.1.smithi193.stdout:7/168: dwrite d12/d13/f24 [0,4194304] 0 2024-04-25T15:49:32.279 INFO:tasks.workunit.client.1.smithi193.stdout:4/234: truncate db/d13/d1e/d1f/d22/f2c 2257373 0 2024-04-25T15:49:32.280 INFO:tasks.workunit.client.1.smithi193.stdout:7/169: chown d12/f36 150589179 1 2024-04-25T15:49:32.280 INFO:tasks.workunit.client.1.smithi193.stdout:4/235: chown db/d13/d1e/d1f/c3c 61259342 1 2024-04-25T15:49:32.280 INFO:tasks.workunit.client.1.smithi193.stdout:5/184: sync 2024-04-25T15:49:32.284 INFO:tasks.workunit.client.1.smithi193.stdout:3/249: mkdir d8/d13/d14/d51 0 2024-04-25T15:49:32.290 INFO:tasks.workunit.client.1.smithi193.stdout:1/177: chown dc/f2d 456600038 1 2024-04-25T15:49:32.292 INFO:tasks.workunit.client.1.smithi193.stdout:8/195: mkdir d2/d3/d20/d38 0 2024-04-25T15:49:32.293 INFO:tasks.workunit.client.1.smithi193.stdout:6/215: mknod dd/d12/d19/c3e 0 2024-04-25T15:49:32.295 INFO:tasks.workunit.client.1.smithi193.stdout:7/170: write f3 [2765856,92756] 0 2024-04-25T15:49:32.295 INFO:tasks.workunit.client.1.smithi193.stdout:9/213: getdents df/d2c 0 2024-04-25T15:49:32.298 INFO:tasks.workunit.client.1.smithi193.stdout:8/196: dwrite d2/d3/ff [0,4194304] 0 2024-04-25T15:49:32.299 INFO:tasks.workunit.client.1.smithi193.stdout:4/236: mknod db/d13/d1e/d1f/d22/c49 0 2024-04-25T15:49:32.300 INFO:tasks.workunit.client.1.smithi193.stdout:3/250: mknod d8/d13/d14/d23/c52 0 2024-04-25T15:49:32.302 INFO:tasks.workunit.client.1.smithi193.stdout:2/216: link d0/d9/d36/d3a/d56/c57 d0/d9/d36/d3a/c5a 0 2024-04-25T15:49:32.307 INFO:tasks.workunit.client.1.smithi193.stdout:0/215: rename d0/d49/d5/d7/dd/d16/l37 to d0/d49/l4a 0 2024-04-25T15:49:32.317 INFO:tasks.workunit.client.0.smithi110.stdout:5/530: unlink d4/d1d/d1f/d35/d7f/f8d 0 2024-04-25T15:49:32.318 INFO:tasks.workunit.client.1.smithi193.stdout:8/197: mknod d2/d18/c39 0 2024-04-25T15:49:32.319 INFO:tasks.workunit.client.1.smithi193.stdout:8/198: dread d2/fa [0,4194304] 0 2024-04-25T15:49:32.320 INFO:tasks.workunit.client.0.smithi110.stdout:8/647: rename d14/d4c/c79 to d14/d32/d36/db7/dc9/cd7 0 2024-04-25T15:49:32.323 INFO:tasks.workunit.client.0.smithi110.stdout:3/584: symlink d0/d7/d1c/d2f/d4c/lc7 0 2024-04-25T15:49:32.324 INFO:tasks.workunit.client.1.smithi193.stdout:3/251: creat d8/d30/d31/f53 x:0 0 0 2024-04-25T15:49:32.324 INFO:tasks.workunit.client.0.smithi110.stdout:1/594: dread d0/d14/f4b [0,4194304] 0 2024-04-25T15:49:32.325 INFO:tasks.workunit.client.0.smithi110.stdout:2/501: write de/d40/d4f/f4e [517673,72758] 0 2024-04-25T15:49:32.325 INFO:tasks.workunit.client.0.smithi110.stdout:1/595: read d0/d51/f79 [3104006,50373] 0 2024-04-25T15:49:32.325 INFO:tasks.workunit.client.0.smithi110.stdout:2/502: fdatasync f8 0 2024-04-25T15:49:32.326 INFO:tasks.workunit.client.1.smithi193.stdout:2/217: creat d0/d42/f5b x:0 0 0 2024-04-25T15:49:32.327 INFO:tasks.workunit.client.0.smithi110.stdout:9/433: link d9/db/f11 d9/db/df/d2c/d3c/f87 0 2024-04-25T15:49:32.328 INFO:tasks.workunit.client.1.smithi193.stdout:0/216: creat d0/d49/d5/d7/dd/d16/d26/d2e/f4b x:0 0 0 2024-04-25T15:49:32.330 INFO:tasks.workunit.client.1.smithi193.stdout:1/178: symlink dc/d35/l38 0 2024-04-25T15:49:32.333 INFO:tasks.workunit.client.0.smithi110.stdout:2/503: dread de/d40/f33 [0,4194304] 0 2024-04-25T15:49:32.333 INFO:tasks.workunit.client.0.smithi110.stdout:7/527: dread d1b/d29/d46/f6a [0,4194304] 0 2024-04-25T15:49:32.334 INFO:tasks.workunit.client.0.smithi110.stdout:7/528: fsync d1b/d29/f98 0 2024-04-25T15:49:32.335 INFO:tasks.workunit.client.0.smithi110.stdout:0/515: creat d1/d23/d81/fc3 x:0 0 0 2024-04-25T15:49:32.336 INFO:tasks.workunit.client.0.smithi110.stdout:0/516: chown d1/f8b 298 1 2024-04-25T15:49:32.337 INFO:tasks.workunit.client.0.smithi110.stdout:5/531: symlink d4/d1d/d1f/d35/d7f/d92/lad 0 2024-04-25T15:49:32.339 INFO:tasks.workunit.client.1.smithi193.stdout:8/199: rename d2/d3/d23 to d2/d3/d3a 0 2024-04-25T15:49:32.340 INFO:tasks.workunit.client.0.smithi110.stdout:8/648: rename d14/d21/d26/d6a/fc7 to d14/d21/d26/dab/dac/fd8 0 2024-04-25T15:49:32.341 INFO:tasks.workunit.client.1.smithi193.stdout:6/216: dwrite dd/d12/d19/d21/f23 [0,4194304] 0 2024-04-25T15:49:32.344 INFO:tasks.workunit.client.0.smithi110.stdout:1/596: creat d0/d14/d5f/d7d/d9c/fb5 x:0 0 0 2024-04-25T15:49:32.346 INFO:tasks.workunit.client.1.smithi193.stdout:3/252: mkdir d8/d13/d14/d27/d54 0 2024-04-25T15:49:32.347 INFO:tasks.workunit.client.1.smithi193.stdout:3/253: truncate d8/d30/f47 633724 0 2024-04-25T15:49:32.347 INFO:tasks.workunit.client.0.smithi110.stdout:3/585: dread d0/d7/f38 [0,4194304] 0 2024-04-25T15:49:32.348 INFO:tasks.workunit.client.0.smithi110.stdout:4/574: rmdir d13/d24/d26 39 2024-04-25T15:49:32.352 INFO:tasks.workunit.client.0.smithi110.stdout:6/466: getdents d1/db 0 2024-04-25T15:49:32.355 INFO:tasks.workunit.client.1.smithi193.stdout:5/185: sync 2024-04-25T15:49:32.355 INFO:tasks.workunit.client.0.smithi110.stdout:2/504: creat de/d40/f96 x:0 0 0 2024-04-25T15:49:32.357 INFO:tasks.workunit.client.0.smithi110.stdout:2/505: fdatasync de/d40/d6e/f93 0 2024-04-25T15:49:32.358 INFO:tasks.workunit.client.0.smithi110.stdout:7/529: creat d1b/d29/d38/d55/d7f/faf x:0 0 0 2024-04-25T15:49:32.359 INFO:tasks.workunit.client.0.smithi110.stdout:6/467: dwrite d1/db/d24/f9c [0,4194304] 0 2024-04-25T15:49:32.363 INFO:tasks.workunit.client.0.smithi110.stdout:0/517: read d1/d23/d30/d33/f2d [2924562,55057] 0 2024-04-25T15:49:32.363 INFO:tasks.workunit.client.0.smithi110.stdout:5/532: rmdir d4/d2c/d83 39 2024-04-25T15:49:32.365 INFO:tasks.workunit.client.1.smithi193.stdout:4/237: truncate db/d13/f1d 252588 0 2024-04-25T15:49:32.365 INFO:tasks.workunit.client.1.smithi193.stdout:4/238: chown db/d32/c47 11454 1 2024-04-25T15:49:32.367 INFO:tasks.workunit.client.1.smithi193.stdout:8/200: truncate d2/fa 1185318 0 2024-04-25T15:49:32.369 INFO:tasks.workunit.client.0.smithi110.stdout:9/434: rename d9/db/df/d25/l45 to d9/d13/d5b/l88 0 2024-04-25T15:49:32.369 INFO:tasks.workunit.client.0.smithi110.stdout:9/435: chown d9/db/df/f4f 791111 1 2024-04-25T15:49:32.370 INFO:tasks.workunit.client.0.smithi110.stdout:8/649: mknod d14/d21/d26/d6a/cd9 0 2024-04-25T15:49:32.370 INFO:tasks.workunit.client.0.smithi110.stdout:9/436: dread - d9/d44/f7f zero size 2024-04-25T15:49:32.371 INFO:tasks.workunit.client.0.smithi110.stdout:8/650: write d14/d21/fba [3019680,41338] 0 2024-04-25T15:49:32.371 INFO:tasks.workunit.client.0.smithi110.stdout:9/437: chown d9/d13/l1e 401 1 2024-04-25T15:49:32.373 INFO:tasks.workunit.client.1.smithi193.stdout:8/201: dwrite d2/d3/d20/f22 [0,4194304] 0 2024-04-25T15:49:32.375 INFO:tasks.workunit.client.1.smithi193.stdout:2/218: creat d0/d9/d15/d31/d50/f5c x:0 0 0 2024-04-25T15:49:32.375 INFO:tasks.workunit.client.1.smithi193.stdout:2/219: stat d0/d37/d25 0 2024-04-25T15:49:32.376 INFO:tasks.workunit.client.1.smithi193.stdout:0/217: creat d0/d49/d5/d7/dd/d38/f4c x:0 0 0 2024-04-25T15:49:32.376 INFO:tasks.workunit.client.1.smithi193.stdout:2/220: write d0/f11 [581071,19271] 0 2024-04-25T15:49:32.376 INFO:tasks.workunit.client.1.smithi193.stdout:6/217: dwrite dd/d12/d19/d1c/f20 [0,4194304] 0 2024-04-25T15:49:32.378 INFO:tasks.workunit.client.1.smithi193.stdout:0/218: dread - d0/d49/d5/d7/dd/d16/f41 zero size 2024-04-25T15:49:32.380 INFO:tasks.workunit.client.1.smithi193.stdout:1/179: truncate dc/f2c 161834 0 2024-04-25T15:49:32.383 INFO:tasks.workunit.client.0.smithi110.stdout:2/506: unlink de/d40/d5b/d66/f8d 0 2024-04-25T15:49:32.385 INFO:tasks.workunit.client.1.smithi193.stdout:5/186: stat d17/l25 0 2024-04-25T15:49:32.385 INFO:tasks.workunit.client.1.smithi193.stdout:6/218: dwrite dd/d12/d19/d1c/f38 [0,4194304] 0 2024-04-25T15:49:32.386 INFO:tasks.workunit.client.0.smithi110.stdout:7/530: fdatasync d1b/f45 0 2024-04-25T15:49:32.387 INFO:tasks.workunit.client.1.smithi193.stdout:7/171: link la d12/d13/l3d 0 2024-04-25T15:49:32.388 INFO:tasks.workunit.client.1.smithi193.stdout:7/172: fsync d12/f30 0 2024-04-25T15:49:32.388 INFO:tasks.workunit.client.1.smithi193.stdout:9/214: getdents df/d2c 0 2024-04-25T15:49:32.391 INFO:tasks.workunit.client.1.smithi193.stdout:6/219: dread dd/d12/d19/d21/f25 [0,4194304] 0 2024-04-25T15:49:32.396 INFO:tasks.workunit.client.0.smithi110.stdout:1/597: mkdir d0/d14/d5f/d7d/d72/d88/db6 0 2024-04-25T15:49:32.397 INFO:tasks.workunit.client.0.smithi110.stdout:8/651: dread f5 [4194304,4194304] 0 2024-04-25T15:49:32.398 INFO:tasks.workunit.client.0.smithi110.stdout:1/598: chown d0/d25/d33/f49 12056035 1 2024-04-25T15:49:32.399 INFO:tasks.workunit.client.1.smithi193.stdout:3/254: mkdir d8/d13/d14/d51/d55 0 2024-04-25T15:49:32.401 INFO:tasks.workunit.client.1.smithi193.stdout:8/202: symlink d2/d3/d3a/l3b 0 2024-04-25T15:49:32.404 INFO:tasks.workunit.client.1.smithi193.stdout:2/221: rename d0/d9/c53 to d0/d9/d36/d3a/d56/c5d 0 2024-04-25T15:49:32.405 INFO:tasks.workunit.client.1.smithi193.stdout:3/255: dwrite d8/d30/f47 [0,4194304] 0 2024-04-25T15:49:32.406 INFO:tasks.workunit.client.1.smithi193.stdout:1/180: chown dc/c26 4440 1 2024-04-25T15:49:32.408 INFO:tasks.workunit.client.1.smithi193.stdout:1/181: dread dc/f30 [0,4194304] 0 2024-04-25T15:49:32.409 INFO:tasks.workunit.client.1.smithi193.stdout:5/187: creat d17/d18/d30/d34/f46 x:0 0 0 2024-04-25T15:49:32.410 INFO:tasks.workunit.client.0.smithi110.stdout:7/531: rmdir d1b 39 2024-04-25T15:49:32.414 INFO:tasks.workunit.client.1.smithi193.stdout:5/188: write d17/d18/d30/d38/f44 [66665,53996] 0 2024-04-25T15:49:32.415 INFO:tasks.workunit.client.1.smithi193.stdout:3/256: dwrite d8/d30/f36 [0,4194304] 0 2024-04-25T15:49:32.415 INFO:tasks.workunit.client.1.smithi193.stdout:7/173: mknod d12/d13/c3e 0 2024-04-25T15:49:32.416 INFO:tasks.workunit.client.0.smithi110.stdout:0/518: mkdir d1/d23/d30/d33/d5b/d72/dc4 0 2024-04-25T15:49:32.419 INFO:tasks.workunit.client.1.smithi193.stdout:4/239: mknod db/d32/d40/c4a 0 2024-04-25T15:49:32.421 INFO:tasks.workunit.client.0.smithi110.stdout:1/599: dread d0/d25/f6 [0,4194304] 0 2024-04-25T15:49:32.426 INFO:tasks.workunit.client.0.smithi110.stdout:6/468: truncate d1/d12/d3d/f78 3693680 0 2024-04-25T15:49:32.430 INFO:tasks.workunit.client.1.smithi193.stdout:9/215: rename df/f1b to df/d2d/f39 0 2024-04-25T15:49:32.431 INFO:tasks.workunit.client.1.smithi193.stdout:9/216: fsync fc 0 2024-04-25T15:49:32.432 INFO:tasks.workunit.client.0.smithi110.stdout:3/586: creat d0/d7/d1c/fc8 x:0 0 0 2024-04-25T15:49:32.433 INFO:tasks.workunit.client.1.smithi193.stdout:2/222: fsync d0/d9/d36/d3a/f58 0 2024-04-25T15:49:32.436 INFO:tasks.workunit.client.1.smithi193.stdout:9/217: dwrite df/d16/f25 [0,4194304] 0 2024-04-25T15:49:32.439 INFO:tasks.workunit.client.1.smithi193.stdout:0/219: link d0/d49/d5/d7/dd/d16/f41 d0/d49/d5/d7/dd/d46/f4d 0 2024-04-25T15:49:32.439 INFO:tasks.workunit.client.0.smithi110.stdout:8/652: dwrite d14/f2d [0,4194304] 0 2024-04-25T15:49:32.440 INFO:tasks.workunit.client.1.smithi193.stdout:0/220: dread - d0/d49/d5/d7/dd/d38/f4c zero size 2024-04-25T15:49:32.441 INFO:tasks.workunit.client.0.smithi110.stdout:7/532: fsync f14 0 2024-04-25T15:49:32.442 INFO:tasks.workunit.client.1.smithi193.stdout:9/218: dwrite df/d16/d1c/f23 [0,4194304] 0 2024-04-25T15:49:32.448 INFO:tasks.workunit.client.0.smithi110.stdout:5/533: link d4/d1d/l9d d4/d1d/d1f/d35/lae 0 2024-04-25T15:49:32.453 INFO:tasks.workunit.client.1.smithi193.stdout:7/174: stat d12/d1e/c1f 0 2024-04-25T15:49:32.457 INFO:tasks.workunit.client.0.smithi110.stdout:9/438: rmdir d9/db/d58/d5a 0 2024-04-25T15:49:32.459 INFO:tasks.workunit.client.1.smithi193.stdout:6/220: mkdir dd/d12/d19/d21/d3c/d3f 0 2024-04-25T15:49:32.460 INFO:tasks.workunit.client.1.smithi193.stdout:6/221: chown dd/f2e 10962 1 2024-04-25T15:49:32.460 INFO:tasks.workunit.client.0.smithi110.stdout:3/587: write d0/d7/f69 [2618235,20327] 0 2024-04-25T15:49:32.463 INFO:tasks.workunit.client.0.smithi110.stdout:8/653: mknod d14/d21/d26/dab/dac/cda 0 2024-04-25T15:49:32.464 INFO:tasks.workunit.client.1.smithi193.stdout:2/223: dread d0/d42/d51/f40 [4194304,4194304] 0 2024-04-25T15:49:32.465 INFO:tasks.workunit.client.1.smithi193.stdout:2/224: chown d0/d9/d3c/c4c 0 1 2024-04-25T15:49:32.466 INFO:tasks.workunit.client.1.smithi193.stdout:3/257: write d8/d13/f2d [1086153,82023] 0 2024-04-25T15:49:32.470 INFO:tasks.workunit.client.1.smithi193.stdout:3/258: dread d8/d13/d14/d23/f48 [0,4194304] 0 2024-04-25T15:49:32.470 INFO:tasks.workunit.client.0.smithi110.stdout:0/519: truncate d1/d47/f86 1131930 0 2024-04-25T15:49:32.470 INFO:tasks.workunit.client.1.smithi193.stdout:3/259: fdatasync d8/d30/f42 0 2024-04-25T15:49:32.473 INFO:tasks.workunit.client.0.smithi110.stdout:1/600: symlink d0/d14/d5f/lb7 0 2024-04-25T15:49:32.474 INFO:tasks.workunit.client.0.smithi110.stdout:1/601: fsync d0/d51/f98 0 2024-04-25T15:49:32.474 INFO:tasks.workunit.client.1.smithi193.stdout:0/221: creat d0/d49/d5/d7/dd/d16/d18/f4e x:0 0 0 2024-04-25T15:49:32.475 INFO:tasks.workunit.client.1.smithi193.stdout:0/222: fdatasync d0/f12 0 2024-04-25T15:49:32.475 INFO:tasks.workunit.client.0.smithi110.stdout:6/469: mkdir d1/db/d24/daa 0 2024-04-25T15:49:32.477 INFO:tasks.workunit.client.0.smithi110.stdout:6/470: chown d1/dd/d3c/d91/f7f 120045 1 2024-04-25T15:49:32.479 INFO:tasks.workunit.client.0.smithi110.stdout:9/439: creat d9/db/df/d2c/d3c/f89 x:0 0 0 2024-04-25T15:49:32.479 INFO:tasks.workunit.client.0.smithi110.stdout:1/602: dread d0/d14/d5f/d7d/f64 [0,4194304] 0 2024-04-25T15:49:32.482 INFO:tasks.workunit.client.1.smithi193.stdout:5/189: link f13 d17/d18/d45/f47 0 2024-04-25T15:49:32.482 INFO:tasks.workunit.client.0.smithi110.stdout:5/534: write d4/d1d/d1f/f6e [953046,40584] 0 2024-04-25T15:49:32.483 INFO:tasks.workunit.client.0.smithi110.stdout:2/507: creat de/d40/d6e/d52/d6d/f97 x:0 0 0 2024-04-25T15:49:32.485 INFO:tasks.workunit.client.1.smithi193.stdout:4/240: mkdir db/d4b 0 2024-04-25T15:49:32.488 INFO:tasks.workunit.client.1.smithi193.stdout:8/203: rename d2/d18/f25 to d2/d3/d3a/f3c 0 2024-04-25T15:49:32.489 INFO:tasks.workunit.client.1.smithi193.stdout:8/204: read d2/d3/ff [2328440,8301] 0 2024-04-25T15:49:32.499 INFO:tasks.workunit.client.0.smithi110.stdout:9/440: dread d9/d2d/f30 [0,4194304] 0 2024-04-25T15:49:32.499 INFO:tasks.workunit.client.0.smithi110.stdout:1/603: dread d0/d23/f13 [0,4194304] 0 2024-04-25T15:49:32.502 INFO:tasks.workunit.client.0.smithi110.stdout:6/471: symlink d1/dd/d3c/d91/lab 0 2024-04-25T15:49:32.502 INFO:tasks.workunit.client.0.smithi110.stdout:8/654: dwrite d14/d4c/f91 [0,4194304] 0 2024-04-25T15:49:32.504 INFO:tasks.workunit.client.0.smithi110.stdout:8/655: dread - d14/d32/d36/d48/fb3 zero size 2024-04-25T15:49:32.504 INFO:tasks.workunit.client.0.smithi110.stdout:8/656: stat d14/d21/d26/dab 0 2024-04-25T15:49:32.505 INFO:tasks.workunit.client.1.smithi193.stdout:0/223: mknod d0/d49/d5/d7/dd/d16/d18/d28/c4f 0 2024-04-25T15:49:32.505 INFO:tasks.workunit.client.1.smithi193.stdout:0/224: read - d0/d49/f2c zero size 2024-04-25T15:49:32.507 INFO:tasks.workunit.client.1.smithi193.stdout:0/225: read d0/d49/d5/f24 [67244,66264] 0 2024-04-25T15:49:32.508 INFO:tasks.workunit.client.1.smithi193.stdout:0/226: chown d0/d49/d5/d7/dd/l25 36 1 2024-04-25T15:49:32.510 INFO:tasks.workunit.client.1.smithi193.stdout:5/190: read d17/d18/f2c [3242781,88227] 0 2024-04-25T15:49:32.511 INFO:tasks.workunit.client.1.smithi193.stdout:5/191: write d17/f2a [416989,37717] 0 2024-04-25T15:49:32.513 INFO:tasks.workunit.client.1.smithi193.stdout:4/241: mkdir db/d32/d4c 0 2024-04-25T15:49:32.515 INFO:tasks.workunit.client.1.smithi193.stdout:6/222: rename dd/d12/d19/d21 to dd/d12/d19/d1c/d40 0 2024-04-25T15:49:32.515 INFO:tasks.workunit.client.1.smithi193.stdout:0/227: dwrite d0/d49/d5/d7/dd/d16/f1c [0,4194304] 0 2024-04-25T15:49:32.517 INFO:tasks.workunit.client.1.smithi193.stdout:5/192: dread d17/d18/f20 [0,4194304] 0 2024-04-25T15:49:32.517 INFO:tasks.workunit.client.0.smithi110.stdout:7/533: creat d1b/d29/d38/d55/dab/fb0 x:0 0 0 2024-04-25T15:49:32.518 INFO:tasks.workunit.client.1.smithi193.stdout:5/193: write d17/d18/f29 [297826,79651] 0 2024-04-25T15:49:32.520 INFO:tasks.workunit.client.0.smithi110.stdout:0/520: mkdir d1/d23/dc5 0 2024-04-25T15:49:32.521 INFO:tasks.workunit.client.0.smithi110.stdout:5/535: dread d4/fd [0,4194304] 0 2024-04-25T15:49:32.522 INFO:tasks.workunit.client.1.smithi193.stdout:9/219: dwrite df/d2d/f39 [0,4194304] 0 2024-04-25T15:49:32.522 INFO:tasks.workunit.client.1.smithi193.stdout:9/220: chown l7 138733 1 2024-04-25T15:49:32.523 INFO:tasks.workunit.client.1.smithi193.stdout:7/175: truncate d12/f2e 2869997 0 2024-04-25T15:49:32.527 INFO:tasks.workunit.client.0.smithi110.stdout:1/604: symlink d0/d25/lb8 0 2024-04-25T15:49:32.528 INFO:tasks.workunit.client.0.smithi110.stdout:1/605: write d0/d14/d5f/d6f/f82 [1527916,43769] 0 2024-04-25T15:49:32.530 INFO:tasks.workunit.client.0.smithi110.stdout:1/606: truncate d0/d23/f3b 4425232 0 2024-04-25T15:49:32.531 INFO:tasks.workunit.client.1.smithi193.stdout:1/182: getdents dc 0 2024-04-25T15:49:32.532 INFO:tasks.workunit.client.0.smithi110.stdout:9/441: truncate d9/db/df/d2c/d3c/f87 4002614 0 2024-04-25T15:49:32.532 INFO:tasks.workunit.client.0.smithi110.stdout:8/657: symlink d14/d32/d36/db7/ldb 0 2024-04-25T15:49:32.541 INFO:tasks.workunit.client.1.smithi193.stdout:4/242: fdatasync db/d13/d1e/f41 0 2024-04-25T15:49:32.543 INFO:tasks.workunit.client.0.smithi110.stdout:3/588: creat d0/fc9 x:0 0 0 2024-04-25T15:49:32.545 INFO:tasks.workunit.client.0.smithi110.stdout:7/534: dread - d1b/d29/d38/f5e zero size 2024-04-25T15:49:32.546 INFO:tasks.workunit.client.0.smithi110.stdout:7/535: dread - d1b/d29/d38/d55/fae zero size 2024-04-25T15:49:32.547 INFO:tasks.workunit.client.1.smithi193.stdout:0/228: mknod d0/d49/d5/d7/dd/d16/c50 0 2024-04-25T15:49:32.547 INFO:tasks.workunit.client.0.smithi110.stdout:0/521: creat d1/d6c/fc6 x:0 0 0 2024-04-25T15:49:32.552 INFO:tasks.workunit.client.1.smithi193.stdout:5/194: mkdir d17/d18/d30/d38/d48 0 2024-04-25T15:49:32.553 INFO:tasks.workunit.client.0.smithi110.stdout:6/472: mknod d1/db/d24/daa/cac 0 2024-04-25T15:49:32.553 INFO:tasks.workunit.client.1.smithi193.stdout:5/195: dread d17/f1f [0,4194304] 0 2024-04-25T15:49:32.554 INFO:tasks.workunit.client.1.smithi193.stdout:5/196: chown d17/c2f 590 1 2024-04-25T15:49:32.554 INFO:tasks.workunit.client.1.smithi193.stdout:5/197: fsync d17/d18/f24 0 2024-04-25T15:49:32.556 INFO:tasks.workunit.client.1.smithi193.stdout:5/198: dread d17/d18/f2e [0,4194304] 0 2024-04-25T15:49:32.563 INFO:tasks.workunit.client.0.smithi110.stdout:2/508: dwrite de/f24 [4194304,4194304] 0 2024-04-25T15:49:32.565 INFO:tasks.workunit.client.0.smithi110.stdout:3/589: fsync d0/d2e/f87 0 2024-04-25T15:49:32.569 INFO:tasks.workunit.client.0.smithi110.stdout:7/536: creat d1b/d29/d38/d55/d7f/fb1 x:0 0 0 2024-04-25T15:49:32.570 INFO:tasks.workunit.client.0.smithi110.stdout:7/537: dread - d1b/f97 zero size 2024-04-25T15:49:32.572 INFO:tasks.workunit.client.1.smithi193.stdout:1/183: mknod dc/c39 0 2024-04-25T15:49:32.576 INFO:tasks.workunit.client.0.smithi110.stdout:6/473: unlink d1/dd/l40 0 2024-04-25T15:49:32.577 INFO:tasks.workunit.client.0.smithi110.stdout:5/536: dwrite d4/d1d/d1f/f49 [0,4194304] 0 2024-04-25T15:49:32.580 INFO:tasks.workunit.client.0.smithi110.stdout:5/537: stat d4/f6d 0 2024-04-25T15:49:32.580 INFO:tasks.workunit.client.1.smithi193.stdout:6/223: write dd/d12/d19/d1c/f20 [4545448,2206] 0 2024-04-25T15:49:32.581 INFO:tasks.workunit.client.1.smithi193.stdout:9/221: dwrite df/d11/f2a [0,4194304] 0 2024-04-25T15:49:32.582 INFO:tasks.workunit.client.1.smithi193.stdout:0/229: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f51 x:0 0 0 2024-04-25T15:49:32.582 INFO:tasks.workunit.client.1.smithi193.stdout:6/224: truncate dd/f3d 891034 0 2024-04-25T15:49:32.584 INFO:tasks.workunit.client.0.smithi110.stdout:0/522: dwrite d1/d23/f42 [0,4194304] 0 2024-04-25T15:49:32.585 INFO:tasks.workunit.client.0.smithi110.stdout:2/509: dread de/d40/f3e [0,4194304] 0 2024-04-25T15:49:32.586 INFO:tasks.workunit.client.1.smithi193.stdout:0/230: dread - d0/d49/d5/d7/dd/d38/f4c zero size 2024-04-25T15:49:32.586 INFO:tasks.workunit.client.0.smithi110.stdout:3/590: creat d0/d7/d1c/d50/fca x:0 0 0 2024-04-25T15:49:32.587 INFO:tasks.workunit.client.1.smithi193.stdout:8/205: link d2/c1b d2/d18/c3d 0 2024-04-25T15:49:32.589 INFO:tasks.workunit.client.0.smithi110.stdout:9/442: dwrite d9/d13/f1a [4194304,4194304] 0 2024-04-25T15:49:32.590 INFO:tasks.workunit.client.0.smithi110.stdout:3/591: write d0/d7/d1c/f56 [518373,21927] 0 2024-04-25T15:49:32.590 INFO:tasks.workunit.client.0.smithi110.stdout:8/658: dwrite d14/d21/f72 [0,4194304] 0 2024-04-25T15:49:32.591 INFO:tasks.workunit.client.0.smithi110.stdout:9/443: stat d9/db/df/l23 0 2024-04-25T15:49:32.593 INFO:tasks.workunit.client.1.smithi193.stdout:9/222: dwrite df/d16/d1c/d26/f35 [0,4194304] 0 2024-04-25T15:49:32.595 INFO:tasks.workunit.client.0.smithi110.stdout:2/510: dread de/d40/d4f/f4e [0,4194304] 0 2024-04-25T15:49:32.598 INFO:tasks.workunit.client.0.smithi110.stdout:7/538: mkdir d1b/d29/d38/d55/dab/db2 0 2024-04-25T15:49:32.603 INFO:tasks.workunit.client.0.smithi110.stdout:5/538: creat d4/d1d/d85/faf x:0 0 0 2024-04-25T15:49:32.603 INFO:tasks.workunit.client.1.smithi193.stdout:7/176: creat d12/d38/f3f x:0 0 0 2024-04-25T15:49:32.604 INFO:tasks.workunit.client.1.smithi193.stdout:2/225: getdents d0/d9/d15/d31 0 2024-04-25T15:49:32.605 INFO:tasks.workunit.client.0.smithi110.stdout:6/474: truncate d1/dd/f64 505968 0 2024-04-25T15:49:32.606 INFO:tasks.workunit.client.1.smithi193.stdout:2/226: write d0/d9/d36/d3a/f4a [287731,15765] 0 2024-04-25T15:49:32.607 INFO:tasks.workunit.client.1.smithi193.stdout:3/260: getdents d8/d13/d1f 0 2024-04-25T15:49:32.608 INFO:tasks.workunit.client.1.smithi193.stdout:3/261: dread - d8/d13/d1f/f1c zero size 2024-04-25T15:49:32.608 INFO:tasks.workunit.client.1.smithi193.stdout:2/227: write d0/d9/d3d/f45 [642432,35427] 0 2024-04-25T15:49:32.609 INFO:tasks.workunit.client.0.smithi110.stdout:5/539: dread d4/d15/f1a [0,4194304] 0 2024-04-25T15:49:32.610 INFO:tasks.workunit.client.0.smithi110.stdout:5/540: fdatasync d4/f22 0 2024-04-25T15:49:32.611 INFO:tasks.workunit.client.1.smithi193.stdout:4/243: symlink db/d4b/l4d 0 2024-04-25T15:49:32.616 INFO:tasks.workunit.client.0.smithi110.stdout:2/511: symlink de/d40/d5b/l98 0 2024-04-25T15:49:32.618 INFO:tasks.workunit.client.0.smithi110.stdout:2/512: write de/d40/d6e/d52/f62 [1637804,101668] 0 2024-04-25T15:49:32.619 INFO:tasks.workunit.client.0.smithi110.stdout:7/539: unlink d1b/d29/d38/d8b/fa8 0 2024-04-25T15:49:32.621 INFO:tasks.workunit.client.0.smithi110.stdout:6/475: mkdir d1/db/d22/d90/d76/dad 0 2024-04-25T15:49:32.622 INFO:tasks.workunit.client.0.smithi110.stdout:6/476: stat d1/db/d24/f9c 0 2024-04-25T15:49:32.623 INFO:tasks.workunit.client.0.smithi110.stdout:6/477: chown d1/db/c14 364821 1 2024-04-25T15:49:32.624 INFO:tasks.workunit.client.0.smithi110.stdout:3/592: rename d0/d7/d1c to d0/d16/dcb 0 2024-04-25T15:49:32.624 INFO:tasks.workunit.client.0.smithi110.stdout:6/478: write d1/db/d22/f57 [289550,71612] 0 2024-04-25T15:49:32.625 INFO:tasks.workunit.client.1.smithi193.stdout:0/231: rmdir d0/d49/d5/d7/dd/d46 39 2024-04-25T15:49:32.626 INFO:tasks.workunit.client.0.smithi110.stdout:3/593: truncate d0/d16/dcb/d2f/d4c/fac 157460 0 2024-04-25T15:49:32.627 INFO:tasks.workunit.client.0.smithi110.stdout:3/594: chown d0/d16/dcb/f35 7992346 1 2024-04-25T15:49:32.627 INFO:tasks.workunit.client.1.smithi193.stdout:0/232: truncate d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f51 246425 0 2024-04-25T15:49:32.628 INFO:tasks.workunit.client.1.smithi193.stdout:0/233: write d0/d49/d5/d7/dd/f40 [170029,39890] 0 2024-04-25T15:49:32.630 INFO:tasks.workunit.client.1.smithi193.stdout:5/199: mknod d17/d18/d45/c49 0 2024-04-25T15:49:32.631 INFO:tasks.workunit.client.0.smithi110.stdout:9/444: creat d9/d44/d5e/d80/f8a x:0 0 0 2024-04-25T15:49:32.632 INFO:tasks.workunit.client.0.smithi110.stdout:9/445: chown d9/db/df/d4d/f78 52735 1 2024-04-25T15:49:32.634 INFO:tasks.workunit.client.1.smithi193.stdout:2/228: chown d0/d9/lb 544829385 1 2024-04-25T15:49:32.636 INFO:tasks.workunit.client.1.smithi193.stdout:3/262: creat d8/d13/f56 x:0 0 0 2024-04-25T15:49:32.637 INFO:tasks.workunit.client.0.smithi110.stdout:0/523: dwrite d1/d20/f6a [0,4194304] 0 2024-04-25T15:49:32.638 INFO:tasks.workunit.client.0.smithi110.stdout:7/540: creat d1b/d29/d46/fb3 x:0 0 0 2024-04-25T15:49:32.638 INFO:tasks.workunit.client.1.smithi193.stdout:4/244: mknod db/d13/d1e/c4e 0 2024-04-25T15:49:32.642 INFO:tasks.workunit.client.0.smithi110.stdout:8/659: rename d14/d32/d36/db7/ldb to d14/d21/d26/dab/ldc 0 2024-04-25T15:49:32.643 INFO:tasks.workunit.client.0.smithi110.stdout:8/660: dread - d14/d21/d26/f4a zero size 2024-04-25T15:49:32.644 INFO:tasks.workunit.client.0.smithi110.stdout:8/661: write d14/d4c/d71/fd5 [576432,83096] 0 2024-04-25T15:49:32.644 INFO:tasks.workunit.client.0.smithi110.stdout:6/479: symlink d1/db/d22/d90/d9e/lae 0 2024-04-25T15:49:32.649 INFO:tasks.workunit.client.1.smithi193.stdout:0/234: rename d0/d49/d5/d7/dd/d16/d1a/f2b to d0/d49/d5/d7/dd/d16/d18/d28/f52 0 2024-04-25T15:49:32.650 INFO:tasks.workunit.client.0.smithi110.stdout:2/513: dread de/d40/f3d [4194304,4194304] 0 2024-04-25T15:49:32.651 INFO:tasks.workunit.client.1.smithi193.stdout:0/235: write d0/d49/d5/d7/dd/d16/d18/f2a [720044,84460] 0 2024-04-25T15:49:32.651 INFO:tasks.workunit.client.1.smithi193.stdout:0/236: readlink d0/d49/d5/l20 0 2024-04-25T15:49:32.653 INFO:tasks.workunit.client.1.smithi193.stdout:5/200: dread d17/d18/f2e [0,4194304] 0 2024-04-25T15:49:32.654 INFO:tasks.workunit.client.1.smithi193.stdout:0/237: truncate d0/d49/d5/d7/dd/d38/f4c 439884 0 2024-04-25T15:49:32.655 INFO:tasks.workunit.client.1.smithi193.stdout:1/184: creat dc/d35/f3a x:0 0 0 2024-04-25T15:49:32.656 INFO:tasks.workunit.client.0.smithi110.stdout:0/524: creat d1/d23/d30/d33/d5b/fc7 x:0 0 0 2024-04-25T15:49:32.656 INFO:tasks.workunit.client.0.smithi110.stdout:7/541: dread d1b/d29/d46/f51 [4194304,4194304] 0 2024-04-25T15:49:32.657 INFO:tasks.workunit.client.0.smithi110.stdout:0/525: chown d1/d23/d30/d33/d5b/d72/lbf 2004 1 2024-04-25T15:49:32.657 INFO:tasks.workunit.client.1.smithi193.stdout:2/229: symlink d0/d9/d15/d31/l5e 0 2024-04-25T15:49:32.658 INFO:tasks.workunit.client.0.smithi110.stdout:3/595: symlink d0/d2e/d88/dbf/d99/dad/daf/lcc 0 2024-04-25T15:49:32.659 INFO:tasks.workunit.client.1.smithi193.stdout:2/230: write d0/d9/d3d/f45 [279109,55204] 0 2024-04-25T15:49:32.659 INFO:tasks.workunit.client.0.smithi110.stdout:3/596: truncate d0/d2e/fc6 700166 0 2024-04-25T15:49:32.660 INFO:tasks.workunit.client.0.smithi110.stdout:3/597: chown d0/d2e 821394036 1 2024-04-25T15:49:32.661 INFO:tasks.workunit.client.1.smithi193.stdout:0/238: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/f4b [0,4194304] 0 2024-04-25T15:49:32.661 INFO:tasks.workunit.client.1.smithi193.stdout:2/231: write d0/d9/d15/f1b [4994672,91616] 0 2024-04-25T15:49:32.662 INFO:tasks.workunit.client.0.smithi110.stdout:6/480: mkdir d1/d12/daf 0 2024-04-25T15:49:32.663 INFO:tasks.workunit.client.0.smithi110.stdout:6/481: truncate d1/d12/d3d/f9d 1917 0 2024-04-25T15:49:32.664 INFO:tasks.workunit.client.1.smithi193.stdout:4/245: readlink db/d13/l1a 0 2024-04-25T15:49:32.664 INFO:tasks.workunit.client.0.smithi110.stdout:5/541: mknod d4/d2c/d83/cb0 0 2024-04-25T15:49:32.667 INFO:tasks.workunit.client.0.smithi110.stdout:9/446: creat d9/db/d7d/f8b x:0 0 0 2024-04-25T15:49:32.669 INFO:tasks.workunit.client.1.smithi193.stdout:9/223: truncate df/d11/f2e 804283 0 2024-04-25T15:49:32.674 INFO:tasks.workunit.client.0.smithi110.stdout:3/598: rename d0/d16/l74 to d0/d16/dcb/d2f/d4c/db2/lcd 0 2024-04-25T15:49:32.677 INFO:tasks.workunit.client.0.smithi110.stdout:0/526: dread d1/d23/f44 [0,4194304] 0 2024-04-25T15:49:32.678 INFO:tasks.workunit.client.1.smithi193.stdout:1/185: creat dc/d35/f3b x:0 0 0 2024-04-25T15:49:32.680 INFO:tasks.workunit.client.0.smithi110.stdout:5/542: rmdir d4/d15 39 2024-04-25T15:49:32.680 INFO:tasks.workunit.client.1.smithi193.stdout:3/263: dwrite d8/d13/f41 [0,4194304] 0 2024-04-25T15:49:32.681 INFO:tasks.workunit.client.1.smithi193.stdout:3/264: write d8/d13/d14/f20 [862366,124834] 0 2024-04-25T15:49:32.689 INFO:tasks.workunit.client.1.smithi193.stdout:3/265: dwrite d8/d30/f36 [0,4194304] 0 2024-04-25T15:49:32.689 INFO:tasks.workunit.client.0.smithi110.stdout:2/514: mkdir de/d40/d6e/d52/d6d/d99 0 2024-04-25T15:49:32.694 INFO:tasks.workunit.client.1.smithi193.stdout:5/201: dwrite d17/d18/f2e [0,4194304] 0 2024-04-25T15:49:32.697 INFO:tasks.workunit.client.0.smithi110.stdout:2/515: dwrite de/d40/f35 [0,4194304] 0 2024-04-25T15:49:32.699 INFO:tasks.workunit.client.1.smithi193.stdout:5/202: dread d17/d18/f24 [0,4194304] 0 2024-04-25T15:49:32.699 INFO:tasks.workunit.client.1.smithi193.stdout:6/225: getdents dd/d2f 0 2024-04-25T15:49:32.700 INFO:tasks.workunit.client.0.smithi110.stdout:5/543: creat d4/d1d/d1f/d35/d3d/d52/d97/fb1 x:0 0 0 2024-04-25T15:49:32.700 INFO:tasks.workunit.client.1.smithi193.stdout:8/206: getdents d2/d3/d3a 0 2024-04-25T15:49:32.704 INFO:tasks.workunit.client.1.smithi193.stdout:9/224: unlink df/d16/f1e 0 2024-04-25T15:49:32.707 INFO:tasks.workunit.client.1.smithi193.stdout:9/225: dread df/d16/f25 [0,4194304] 0 2024-04-25T15:49:32.708 INFO:tasks.workunit.client.1.smithi193.stdout:9/226: fsync df/d2d/f39 0 2024-04-25T15:49:32.708 INFO:tasks.workunit.client.0.smithi110.stdout:7/542: creat d1b/fb4 x:0 0 0 2024-04-25T15:49:32.709 INFO:tasks.workunit.client.1.smithi193.stdout:9/227: chown df/d16/d1c/f23 113 1 2024-04-25T15:49:32.709 INFO:tasks.workunit.client.0.smithi110.stdout:7/543: readlink d1b/d29/d46/la6 0 2024-04-25T15:49:32.710 INFO:tasks.workunit.client.1.smithi193.stdout:9/228: chown df/d11/f2a 3477 1 2024-04-25T15:49:32.710 INFO:tasks.workunit.client.1.smithi193.stdout:9/229: stat df/d16/f19 0 2024-04-25T15:49:32.711 INFO:tasks.workunit.client.0.smithi110.stdout:3/599: write d0/d16/dcb/d50/f9a [37856,39814] 0 2024-04-25T15:49:32.711 INFO:tasks.workunit.client.1.smithi193.stdout:1/186: chown dc/f18 0 1 2024-04-25T15:49:32.712 INFO:tasks.workunit.client.0.smithi110.stdout:7/544: dread d1b/d29/d38/d55/d7f/f78 [0,4194304] 0 2024-04-25T15:49:32.712 INFO:tasks.workunit.client.0.smithi110.stdout:6/482: creat d1/db/d24/d2d/fb0 x:0 0 0 2024-04-25T15:49:32.713 INFO:tasks.workunit.client.0.smithi110.stdout:0/527: symlink d1/d23/d30/d33/d5b/d72/dc4/lc8 0 2024-04-25T15:49:32.714 INFO:tasks.workunit.client.1.smithi193.stdout:7/177: sync 2024-04-25T15:49:32.714 INFO:tasks.workunit.client.1.smithi193.stdout:7/178: chown d12/d13/l2f 1835981694 1 2024-04-25T15:49:32.716 INFO:tasks.workunit.client.0.smithi110.stdout:9/447: creat d9/f8c x:0 0 0 2024-04-25T15:49:32.720 INFO:tasks.workunit.client.0.smithi110.stdout:3/600: mkdir d0/d16/d8c/dce 0 2024-04-25T15:49:32.721 INFO:tasks.workunit.client.1.smithi193.stdout:0/239: rmdir d0/d49/d5/d7/dd/d46 39 2024-04-25T15:49:32.724 INFO:tasks.workunit.client.0.smithi110.stdout:6/483: rename d1/l8 to d1/dd/d3c/lb1 0 2024-04-25T15:49:32.725 INFO:tasks.workunit.client.1.smithi193.stdout:4/246: creat db/d32/d45/f4f x:0 0 0 2024-04-25T15:49:32.727 INFO:tasks.workunit.client.1.smithi193.stdout:8/207: unlink d2/d18/c3d 0 2024-04-25T15:49:32.727 INFO:tasks.workunit.client.0.smithi110.stdout:3/601: creat d0/d16/d8c/fcf x:0 0 0 2024-04-25T15:49:32.729 INFO:tasks.workunit.client.0.smithi110.stdout:7/545: mkdir d1b/d29/db5 0 2024-04-25T15:49:32.730 INFO:tasks.workunit.client.0.smithi110.stdout:2/516: rename de/d40/d6e/d50/c77 to de/d40/d86/c9a 0 2024-04-25T15:49:32.732 INFO:tasks.workunit.client.0.smithi110.stdout:6/484: mkdir d1/d12/d3d/db2 0 2024-04-25T15:49:32.736 INFO:tasks.workunit.client.1.smithi193.stdout:1/187: truncate dc/f2b 922017 0 2024-04-25T15:49:32.736 INFO:tasks.workunit.client.1.smithi193.stdout:1/188: dread - dc/d35/f3b zero size 2024-04-25T15:49:32.737 INFO:tasks.workunit.client.1.smithi193.stdout:7/179: chown f8 153795 1 2024-04-25T15:49:32.738 INFO:tasks.workunit.client.1.smithi193.stdout:3/266: creat d8/d13/d4b/f57 x:0 0 0 2024-04-25T15:49:32.740 INFO:tasks.workunit.client.0.smithi110.stdout:9/448: rename d9/db/df/c48 to d9/db/d58/c8d 0 2024-04-25T15:49:32.744 INFO:tasks.workunit.client.0.smithi110.stdout:6/485: creat d1/db/d24/d69/fb3 x:0 0 0 2024-04-25T15:49:32.745 INFO:tasks.workunit.client.0.smithi110.stdout:6/486: readlink d1/db/d24/d2d/l54 0 2024-04-25T15:49:32.745 INFO:tasks.workunit.client.0.smithi110.stdout:0/528: creat d1/d23/d30/d33/d5b/d72/fc9 x:0 0 0 2024-04-25T15:49:32.746 INFO:tasks.workunit.client.1.smithi193.stdout:8/208: mknod d2/d3/db/c3e 0 2024-04-25T15:49:32.747 INFO:tasks.workunit.client.0.smithi110.stdout:5/544: link d4/d1d/d1f/d35/d3d/d52/f6f d4/fb2 0 2024-04-25T15:49:32.748 INFO:tasks.workunit.client.1.smithi193.stdout:6/226: creat dd/d12/d19/d1c/d40/d3c/d3f/f41 x:0 0 0 2024-04-25T15:49:32.748 INFO:tasks.workunit.client.0.smithi110.stdout:3/602: mkdir d0/d7/dd0 0 2024-04-25T15:49:32.749 INFO:tasks.workunit.client.0.smithi110.stdout:5/545: dread - d4/d1d/d1f/d35/d3d/d52/d97/fa1 zero size 2024-04-25T15:49:32.752 INFO:tasks.workunit.client.0.smithi110.stdout:3/603: write d0/d16/dcb/d50/f6d [979780,109147] 0 2024-04-25T15:49:32.753 INFO:tasks.workunit.client.0.smithi110.stdout:7/546: dread d1b/d29/d38/d8b/d89/f8c [0,4194304] 0 2024-04-25T15:49:32.754 INFO:tasks.workunit.client.1.smithi193.stdout:7/180: creat d12/d13/f40 x:0 0 0 2024-04-25T15:49:32.756 INFO:tasks.workunit.client.0.smithi110.stdout:0/529: dwrite d1/d20/f6a [0,4194304] 0 2024-04-25T15:49:32.760 INFO:tasks.workunit.client.0.smithi110.stdout:0/530: dread d1/d23/f44 [0,4194304] 0 2024-04-25T15:49:32.760 INFO:tasks.workunit.client.0.smithi110.stdout:5/546: truncate d4/d1d/d1f/d35/d3d/d52/d97/fa0 608197 0 2024-04-25T15:49:32.760 INFO:tasks.workunit.client.0.smithi110.stdout:9/449: link d9/db/df/f68 d9/d13/f8e 0 2024-04-25T15:49:32.760 INFO:tasks.workunit.client.0.smithi110.stdout:7/547: dread f2 [0,4194304] 0 2024-04-25T15:49:32.761 INFO:tasks.workunit.client.1.smithi193.stdout:2/232: getdents d0/d44 0 2024-04-25T15:49:32.764 INFO:tasks.workunit.client.1.smithi193.stdout:5/203: link c14 d17/d18/d45/c4a 0 2024-04-25T15:49:32.767 INFO:tasks.workunit.client.0.smithi110.stdout:6/487: dread d1/d12/f1b [0,4194304] 0 2024-04-25T15:49:32.767 INFO:tasks.workunit.client.0.smithi110.stdout:9/450: truncate d9/d13/d5b/d61/f63 269071 0 2024-04-25T15:49:32.767 INFO:tasks.workunit.client.1.smithi193.stdout:4/247: symlink db/l50 0 2024-04-25T15:49:32.768 INFO:tasks.workunit.client.1.smithi193.stdout:5/204: dread d17/f39 [0,4194304] 0 2024-04-25T15:49:32.768 INFO:tasks.workunit.client.0.smithi110.stdout:6/488: chown d1/d12/f5b 328140099 1 2024-04-25T15:49:32.769 INFO:tasks.workunit.client.1.smithi193.stdout:8/209: mkdir d2/d3/d20/d3f 0 2024-04-25T15:49:32.770 INFO:tasks.workunit.client.0.smithi110.stdout:2/517: write de/d40/d4f/f90 [3621182,1937] 0 2024-04-25T15:49:32.770 INFO:tasks.workunit.client.1.smithi193.stdout:9/230: creat df/d16/f3a x:0 0 0 2024-04-25T15:49:32.771 INFO:tasks.workunit.client.0.smithi110.stdout:0/531: dwrite d1/d23/d30/d33/d5b/fc7 [0,4194304] 0 2024-04-25T15:49:32.771 INFO:tasks.workunit.client.0.smithi110.stdout:6/489: dread - d1/db/d24/d69/d75/d82/fa6 zero size 2024-04-25T15:49:32.772 INFO:tasks.workunit.client.1.smithi193.stdout:5/205: dread d17/d18/f2e [0,4194304] 0 2024-04-25T15:49:32.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/547: creat d4/d1d/d1f/d35/fb3 x:0 0 0 2024-04-25T15:49:32.774 INFO:tasks.workunit.client.1.smithi193.stdout:1/189: unlink dc/c26 0 2024-04-25T15:49:32.778 INFO:tasks.workunit.client.1.smithi193.stdout:7/181: rename f3 to d12/d13/f41 0 2024-04-25T15:49:32.781 INFO:tasks.workunit.client.1.smithi193.stdout:1/190: dwrite dc/d35/f3a [0,4194304] 0 2024-04-25T15:49:32.782 INFO:tasks.workunit.client.0.smithi110.stdout:7/548: dread d1b/f45 [0,4194304] 0 2024-04-25T15:49:32.786 INFO:tasks.workunit.client.0.smithi110.stdout:5/548: creat d4/d1d/d1f/d35/d7f/d92/fb4 x:0 0 0 2024-04-25T15:49:32.789 INFO:tasks.workunit.client.1.smithi193.stdout:9/231: unlink df/d16/d1c/f1f 0 2024-04-25T15:49:32.789 INFO:tasks.workunit.client.1.smithi193.stdout:9/232: readlink l0 0 2024-04-25T15:49:32.790 INFO:tasks.workunit.client.0.smithi110.stdout:7/549: dwrite d1b/f6e [0,4194304] 0 2024-04-25T15:49:32.792 INFO:tasks.workunit.client.0.smithi110.stdout:7/550: fsync d1b/d29/d38/f61 0 2024-04-25T15:49:32.792 INFO:tasks.workunit.client.0.smithi110.stdout:9/451: dread d9/db/df/d25/f37 [0,4194304] 0 2024-04-25T15:49:32.793 INFO:tasks.workunit.client.1.smithi193.stdout:5/206: dread d17/d18/f20 [0,4194304] 0 2024-04-25T15:49:32.794 INFO:tasks.workunit.client.0.smithi110.stdout:3/604: creat d0/d7/d40/fd1 x:0 0 0 2024-04-25T15:49:32.798 INFO:tasks.workunit.client.1.smithi193.stdout:5/207: dwrite d17/f1d [0,4194304] 0 2024-04-25T15:49:32.799 INFO:tasks.workunit.client.0.smithi110.stdout:4/575: sync 2024-04-25T15:49:32.799 INFO:tasks.workunit.client.1.smithi193.stdout:5/208: dread - d17/d18/d30/f32 zero size 2024-04-25T15:49:32.801 INFO:tasks.workunit.client.1.smithi193.stdout:2/233: rename d0/d9/d36 to d0/d37/d25/d5f 0 2024-04-25T15:49:32.802 INFO:tasks.workunit.client.0.smithi110.stdout:5/549: dread - d4/d15/d51/fa4 zero size 2024-04-25T15:49:32.803 INFO:tasks.workunit.client.1.smithi193.stdout:0/240: getdents d0/d49/d5/d7 0 2024-04-25T15:49:32.805 INFO:tasks.workunit.client.1.smithi193.stdout:7/182: creat d12/d1e/f42 x:0 0 0 2024-04-25T15:49:32.807 INFO:tasks.workunit.client.1.smithi193.stdout:8/210: dwrite d2/d3/ff [0,4194304] 0 2024-04-25T15:49:32.813 INFO:tasks.workunit.client.0.smithi110.stdout:7/551: unlink d1b/d29/d38/ca7 0 2024-04-25T15:49:32.816 INFO:tasks.workunit.client.0.smithi110.stdout:0/532: creat d1/d23/d30/fca x:0 0 0 2024-04-25T15:49:32.818 INFO:tasks.workunit.client.0.smithi110.stdout:2/518: rename de/d40/d6e/f3a to de/d40/d6e/f9b 0 2024-04-25T15:49:32.820 INFO:tasks.workunit.client.1.smithi193.stdout:5/209: rmdir d17/d18/d45 39 2024-04-25T15:49:32.825 INFO:tasks.workunit.client.1.smithi193.stdout:2/234: dwrite d0/d37/f48 [0,4194304] 0 2024-04-25T15:49:32.833 INFO:tasks.workunit.client.0.smithi110.stdout:6/490: link d1/db/d22/d33/d43/f53 d1/d12/fb4 0 2024-04-25T15:49:32.834 INFO:tasks.workunit.client.0.smithi110.stdout:5/550: creat d4/d1d/d1f/d35/d3d/d52/d75/fb5 x:0 0 0 2024-04-25T15:49:32.835 INFO:tasks.workunit.client.0.smithi110.stdout:5/551: readlink d4/d1d/l50 0 2024-04-25T15:49:32.836 INFO:tasks.workunit.client.1.smithi193.stdout:0/241: fdatasync d0/d49/d5/d7/fb 0 2024-04-25T15:49:32.839 INFO:tasks.workunit.client.1.smithi193.stdout:1/191: mkdir dc/d3c 0 2024-04-25T15:49:32.840 INFO:tasks.workunit.client.0.smithi110.stdout:0/533: rmdir d1/d23/d30/d33/d35/da1/db4 39 2024-04-25T15:49:32.841 INFO:tasks.workunit.client.1.smithi193.stdout:9/233: mkdir df/d16/d1c/d30/d3b 0 2024-04-25T15:49:32.843 INFO:tasks.workunit.client.0.smithi110.stdout:4/576: mknod d13/d24/d26/cbf 0 2024-04-25T15:49:32.843 INFO:tasks.workunit.client.1.smithi193.stdout:5/210: dread d17/f1f [0,4194304] 0 2024-04-25T15:49:32.844 INFO:tasks.workunit.client.1.smithi193.stdout:5/211: chown d17/d18 1 1 2024-04-25T15:49:32.844 INFO:tasks.workunit.client.1.smithi193.stdout:2/235: rmdir d0/d37/d25 39 2024-04-25T15:49:32.846 INFO:tasks.workunit.client.0.smithi110.stdout:6/491: write d1/db/d24/f29 [3177799,48346] 0 2024-04-25T15:49:32.850 INFO:tasks.workunit.client.1.smithi193.stdout:0/242: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53 0 2024-04-25T15:49:32.852 INFO:tasks.workunit.client.0.smithi110.stdout:6/492: fsync d1/db/d24/f29 0 2024-04-25T15:49:32.854 INFO:tasks.workunit.client.0.smithi110.stdout:0/534: chown d1/d47/d54/lae 17 1 2024-04-25T15:49:32.854 INFO:tasks.workunit.client.1.smithi193.stdout:1/192: mkdir dc/d35/d3d 0 2024-04-25T15:49:32.855 INFO:tasks.workunit.client.1.smithi193.stdout:3/267: sync 2024-04-25T15:49:32.855 INFO:tasks.workunit.client.0.smithi110.stdout:3/605: dwrite d0/d7/f8b [0,4194304] 0 2024-04-25T15:49:32.856 INFO:tasks.workunit.client.0.smithi110.stdout:0/535: readlink d1/d20/d69/la5 0 2024-04-25T15:49:32.856 INFO:tasks.workunit.client.0.smithi110.stdout:5/552: dwrite d4/d1d/d1f/f32 [0,4194304] 0 2024-04-25T15:49:32.858 INFO:tasks.workunit.client.1.smithi193.stdout:3/268: dread d8/d30/f36 [0,4194304] 0 2024-04-25T15:49:32.858 INFO:tasks.workunit.client.0.smithi110.stdout:0/536: chown d1/d23/d30/d33/d5b/dbb 24 1 2024-04-25T15:49:32.858 INFO:tasks.workunit.client.0.smithi110.stdout:4/577: write d13/f22 [740554,81457] 0 2024-04-25T15:49:32.859 INFO:tasks.workunit.client.1.smithi193.stdout:3/269: fdatasync d8/d13/d1f/f1c 0 2024-04-25T15:49:32.859 INFO:tasks.workunit.client.1.smithi193.stdout:3/270: truncate d8/d13/d14/f20 1337426 0 2024-04-25T15:49:32.859 INFO:tasks.workunit.client.0.smithi110.stdout:4/578: chown d13/f77 11 1 2024-04-25T15:49:32.860 INFO:tasks.workunit.client.1.smithi193.stdout:2/236: creat d0/d9/d3c/f60 x:0 0 0 2024-04-25T15:49:32.861 INFO:tasks.workunit.client.0.smithi110.stdout:5/553: dread - d4/d1d/d1f/d35/d7f/fa7 zero size 2024-04-25T15:49:32.861 INFO:tasks.workunit.client.0.smithi110.stdout:1/607: sync 2024-04-25T15:49:32.861 INFO:tasks.workunit.client.0.smithi110.stdout:3/606: truncate d0/d16/d8c/fcf 628226 0 2024-04-25T15:49:32.862 INFO:tasks.workunit.client.0.smithi110.stdout:9/452: link d9/l6d d9/db/d7d/l8f 0 2024-04-25T15:49:32.869 INFO:tasks.workunit.client.1.smithi193.stdout:7/183: dwrite f7 [0,4194304] 0 2024-04-25T15:49:32.870 INFO:tasks.workunit.client.0.smithi110.stdout:6/493: mkdir d1/dd/db5 0 2024-04-25T15:49:32.870 INFO:tasks.workunit.client.0.smithi110.stdout:3/607: dwrite d0/fc9 [0,4194304] 0 2024-04-25T15:49:32.877 INFO:tasks.workunit.client.1.smithi193.stdout:9/234: creat df/d2c/d2f/d33/f3c x:0 0 0 2024-04-25T15:49:32.878 INFO:tasks.workunit.client.0.smithi110.stdout:5/554: rmdir d4/d1d/d1f/d56/d6b 39 2024-04-25T15:49:32.882 INFO:tasks.workunit.client.1.smithi193.stdout:4/248: sync 2024-04-25T15:49:32.882 INFO:tasks.workunit.client.1.smithi193.stdout:6/227: sync 2024-04-25T15:49:32.883 INFO:tasks.workunit.client.1.smithi193.stdout:1/193: dwrite dc/f36 [0,4194304] 0 2024-04-25T15:49:32.883 INFO:tasks.workunit.client.1.smithi193.stdout:2/237: creat d0/d42/f61 x:0 0 0 2024-04-25T15:49:32.884 INFO:tasks.workunit.client.0.smithi110.stdout:0/537: dread - d1/d23/d30/d33/d5b/d83/f53 zero size 2024-04-25T15:49:32.884 INFO:tasks.workunit.client.1.smithi193.stdout:6/228: chown dd/fe 4380576 1 2024-04-25T15:49:32.886 INFO:tasks.workunit.client.1.smithi193.stdout:5/212: symlink d17/d18/d30/d34/d37/l4b 0 2024-04-25T15:49:32.886 INFO:tasks.workunit.client.0.smithi110.stdout:7/552: rename d1b/d29/l40 to d1b/lb6 0 2024-04-25T15:49:32.887 INFO:tasks.workunit.client.1.smithi193.stdout:0/243: dwrite d0/d49/d5/f24 [0,4194304] 0 2024-04-25T15:49:32.887 INFO:tasks.workunit.client.1.smithi193.stdout:6/229: write dd/f24 [4973025,22617] 0 2024-04-25T15:49:32.888 INFO:tasks.workunit.client.1.smithi193.stdout:8/211: getdents d2/d3/d27/d34 0 2024-04-25T15:49:32.891 INFO:tasks.workunit.client.1.smithi193.stdout:0/244: dread d0/d49/d5/f24 [0,4194304] 0 2024-04-25T15:49:32.893 INFO:tasks.workunit.client.0.smithi110.stdout:1/608: dwrite d0/d23/f92 [0,4194304] 0 2024-04-25T15:49:32.893 INFO:tasks.workunit.client.0.smithi110.stdout:2/519: getdents de/d40/d6e/d52/d6d 0 2024-04-25T15:49:32.893 INFO:tasks.workunit.client.1.smithi193.stdout:7/184: creat d12/d13/f43 x:0 0 0 2024-04-25T15:49:32.896 INFO:tasks.workunit.client.0.smithi110.stdout:6/494: creat d1/db/da0/fb6 x:0 0 0 2024-04-25T15:49:32.896 INFO:tasks.workunit.client.0.smithi110.stdout:2/520: fdatasync de/d40/d4f/f4c 0 2024-04-25T15:49:32.897 INFO:tasks.workunit.client.0.smithi110.stdout:4/579: fdatasync d13/d24/d2d/f4f 0 2024-04-25T15:49:32.897 INFO:tasks.workunit.client.0.smithi110.stdout:2/521: dread - de/d40/d70/f7f zero size 2024-04-25T15:49:32.900 INFO:tasks.workunit.client.0.smithi110.stdout:5/555: symlink d4/d1d/d1f/d35/d3d/lb6 0 2024-04-25T15:49:32.904 INFO:tasks.workunit.client.1.smithi193.stdout:3/271: symlink d8/l58 0 2024-04-25T15:49:32.906 INFO:tasks.workunit.client.1.smithi193.stdout:4/249: dread f1 [0,4194304] 0 2024-04-25T15:49:32.908 INFO:tasks.workunit.client.1.smithi193.stdout:4/250: chown db/d13/d1e/d1f/d22/f27 1437172060 1 2024-04-25T15:49:32.911 INFO:tasks.workunit.client.0.smithi110.stdout:0/538: dread d1/d47/d54/f6d [0,4194304] 0 2024-04-25T15:49:32.912 INFO:tasks.workunit.client.0.smithi110.stdout:9/453: dread d9/d44/f53 [0,4194304] 0 2024-04-25T15:49:32.915 INFO:tasks.workunit.client.1.smithi193.stdout:9/235: dwrite df/d16/f25 [0,4194304] 0 2024-04-25T15:49:32.915 INFO:tasks.workunit.client.1.smithi193.stdout:9/236: stat df/d11/f14 0 2024-04-25T15:49:32.916 INFO:tasks.workunit.client.0.smithi110.stdout:1/609: rmdir d0 39 2024-04-25T15:49:32.917 INFO:tasks.workunit.client.1.smithi193.stdout:9/237: read f9 [905531,59494] 0 2024-04-25T15:49:32.921 INFO:tasks.workunit.client.1.smithi193.stdout:9/238: dread df/d16/d1c/f23 [0,4194304] 0 2024-04-25T15:49:32.921 INFO:tasks.workunit.client.1.smithi193.stdout:2/238: rename d0/d37/l1e to d0/d42/l62 0 2024-04-25T15:49:32.922 INFO:tasks.workunit.client.1.smithi193.stdout:9/239: truncate df/d16/f3a 605984 0 2024-04-25T15:49:32.922 INFO:tasks.workunit.client.1.smithi193.stdout:9/240: chown df/d2c/d2f/d33 1892178 1 2024-04-25T15:49:32.923 INFO:tasks.workunit.client.0.smithi110.stdout:7/553: write d1b/d29/d38/f54 [4192442,41097] 0 2024-04-25T15:49:32.930 INFO:tasks.workunit.client.0.smithi110.stdout:5/556: write d4/d2c/f27 [2123067,103457] 0 2024-04-25T15:49:32.930 INFO:tasks.workunit.client.0.smithi110.stdout:2/522: dread de/d40/f36 [0,4194304] 0 2024-04-25T15:49:32.930 INFO:tasks.workunit.client.1.smithi193.stdout:0/245: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/l54 0 2024-04-25T15:49:32.933 INFO:tasks.workunit.client.1.smithi193.stdout:8/212: write d2/d3/f16 [20103,69557] 0 2024-04-25T15:49:32.934 INFO:tasks.workunit.client.1.smithi193.stdout:3/272: creat d8/d13/d14/f59 x:0 0 0 2024-04-25T15:49:32.935 INFO:tasks.workunit.client.0.smithi110.stdout:9/454: mknod d9/db/df/c90 0 2024-04-25T15:49:32.936 INFO:tasks.workunit.client.1.smithi193.stdout:4/251: mkdir db/d4b/d51 0 2024-04-25T15:49:32.937 INFO:tasks.workunit.client.0.smithi110.stdout:1/610: readlink d0/d25/l46 0 2024-04-25T15:49:32.938 INFO:tasks.workunit.client.1.smithi193.stdout:1/194: creat dc/d3c/f3e x:0 0 0 2024-04-25T15:49:32.938 INFO:tasks.workunit.client.1.smithi193.stdout:0/246: dwrite d0/d49/d5/d7/dd/d16/d18/f4e [0,4194304] 0 2024-04-25T15:49:32.939 INFO:tasks.workunit.client.1.smithi193.stdout:4/252: truncate db/f38 465299 0 2024-04-25T15:49:32.941 INFO:tasks.workunit.client.1.smithi193.stdout:4/253: dread - db/d32/d45/f4f zero size 2024-04-25T15:49:32.941 INFO:tasks.workunit.client.1.smithi193.stdout:2/239: mknod d0/d9/d15/d31/c63 0 2024-04-25T15:49:32.942 INFO:tasks.workunit.client.1.smithi193.stdout:7/185: dwrite d12/d1c/f3b [0,4194304] 0 2024-04-25T15:49:32.946 INFO:tasks.workunit.client.1.smithi193.stdout:7/186: read f2 [1379473,67124] 0 2024-04-25T15:49:32.946 INFO:tasks.workunit.client.1.smithi193.stdout:5/213: rmdir d17/d18/d45 39 2024-04-25T15:49:32.948 INFO:tasks.workunit.client.1.smithi193.stdout:4/254: dwrite db/f35 [4194304,4194304] 0 2024-04-25T15:49:32.949 INFO:tasks.workunit.client.1.smithi193.stdout:4/255: stat db/d32/d45/f4f 0 2024-04-25T15:49:32.949 INFO:tasks.workunit.client.1.smithi193.stdout:4/256: fsync f4 0 2024-04-25T15:49:32.951 INFO:tasks.workunit.client.0.smithi110.stdout:2/523: rmdir de/d40/d6e/d50 39 2024-04-25T15:49:32.954 INFO:tasks.workunit.client.0.smithi110.stdout:6/495: dwrite d1/db/d24/d2d/f7b [0,4194304] 0 2024-04-25T15:49:32.954 INFO:tasks.workunit.client.0.smithi110.stdout:2/524: write de/d40/d5b/d66/f95 [1022228,109284] 0 2024-04-25T15:49:32.956 INFO:tasks.workunit.client.0.smithi110.stdout:9/455: unlink d9/d2d/f2f 0 2024-04-25T15:49:32.959 INFO:tasks.workunit.client.1.smithi193.stdout:3/273: rmdir d8/d13/d14/d23 39 2024-04-25T15:49:32.960 INFO:tasks.workunit.client.1.smithi193.stdout:3/274: write d8/d30/f40 [660825,108796] 0 2024-04-25T15:49:32.961 INFO:tasks.workunit.client.0.smithi110.stdout:7/554: creat d1b/d29/d38/d55/dab/db2/fb7 x:0 0 0 2024-04-25T15:49:32.963 INFO:tasks.workunit.client.0.smithi110.stdout:5/557: creat d4/d1d/d1f/d56/d6b/fb7 x:0 0 0 2024-04-25T15:49:32.965 INFO:tasks.workunit.client.1.smithi193.stdout:0/247: rmdir d0/d49/d5/d7/dd 39 2024-04-25T15:49:32.967 INFO:tasks.workunit.client.0.smithi110.stdout:0/539: creat d1/d23/d30/d33/d35/da1/fcb x:0 0 0 2024-04-25T15:49:32.969 INFO:tasks.workunit.client.0.smithi110.stdout:6/496: mknod d1/db/d22/d90/cb7 0 2024-04-25T15:49:32.970 INFO:tasks.workunit.client.1.smithi193.stdout:2/240: mknod d0/d9/d15/c64 0 2024-04-25T15:49:32.971 INFO:tasks.workunit.client.0.smithi110.stdout:6/497: dread d1/db/f8c [0,4194304] 0 2024-04-25T15:49:32.973 INFO:tasks.workunit.client.1.smithi193.stdout:9/241: truncate f4 893632 0 2024-04-25T15:49:32.974 INFO:tasks.workunit.client.1.smithi193.stdout:2/241: dread d0/d37/f48 [0,4194304] 0 2024-04-25T15:49:32.977 INFO:tasks.workunit.client.0.smithi110.stdout:4/580: getdents d13 0 2024-04-25T15:49:32.977 INFO:tasks.workunit.client.1.smithi193.stdout:6/230: creat dd/f42 x:0 0 0 2024-04-25T15:49:32.978 INFO:tasks.workunit.client.1.smithi193.stdout:9/242: dwrite df/d16/f1a [4194304,4194304] 0 2024-04-25T15:49:32.979 INFO:tasks.workunit.client.0.smithi110.stdout:5/558: creat d4/d1d/d1f/d9e/fb8 x:0 0 0 2024-04-25T15:49:32.983 INFO:tasks.workunit.client.1.smithi193.stdout:3/275: creat d8/d13/d14/d27/f5a x:0 0 0 2024-04-25T15:49:32.985 INFO:tasks.workunit.client.0.smithi110.stdout:7/555: dwrite d1b/d29/d38/d55/d7f/f78 [0,4194304] 0 2024-04-25T15:49:32.986 INFO:tasks.workunit.client.0.smithi110.stdout:6/498: mkdir d1/d12/db8 0 2024-04-25T15:49:32.989 INFO:tasks.workunit.client.0.smithi110.stdout:0/540: dread d1/d23/d30/d33/d5b/d72/f82 [0,4194304] 0 2024-04-25T15:49:32.989 INFO:tasks.workunit.client.1.smithi193.stdout:3/276: dwrite d8/d30/f40 [0,4194304] 0 2024-04-25T15:49:32.993 INFO:tasks.workunit.client.1.smithi193.stdout:7/187: symlink d12/d31/d39/l44 0 2024-04-25T15:49:32.996 INFO:tasks.workunit.client.0.smithi110.stdout:4/581: fdatasync d13/d24/d2d/d44/d65/d99/daf/d56/f82 0 2024-04-25T15:49:32.997 INFO:tasks.workunit.client.1.smithi193.stdout:9/243: fsync fc 0 2024-04-25T15:49:32.999 INFO:tasks.workunit.client.1.smithi193.stdout:5/214: link d17/f26 d17/d18/d3e/f4c 0 2024-04-25T15:49:32.999 INFO:tasks.workunit.client.1.smithi193.stdout:5/215: stat d17/c2b 0 2024-04-25T15:49:32.999 INFO:tasks.workunit.client.1.smithi193.stdout:5/216: readlink d17/d18/l2d 0 2024-04-25T15:49:33.000 INFO:tasks.workunit.client.0.smithi110.stdout:2/525: truncate de/d40/d5b/d5c/d5f/f7a 795848 0 2024-04-25T15:49:33.000 INFO:tasks.workunit.client.1.smithi193.stdout:5/217: write fe [507947,52030] 0 2024-04-25T15:49:33.003 INFO:tasks.workunit.client.1.smithi193.stdout:9/244: dwrite df/d16/f25 [4194304,4194304] 0 2024-04-25T15:49:33.005 INFO:tasks.workunit.client.1.smithi193.stdout:8/213: rmdir d2/d3/d20/d38 0 2024-04-25T15:49:33.006 INFO:tasks.workunit.client.0.smithi110.stdout:9/456: creat d9/db/f91 x:0 0 0 2024-04-25T15:49:33.009 INFO:tasks.workunit.client.0.smithi110.stdout:6/499: rmdir d1/db/d24/d69 39 2024-04-25T15:49:33.013 INFO:tasks.workunit.client.1.smithi193.stdout:1/195: creat dc/f3f x:0 0 0 2024-04-25T15:49:33.013 INFO:tasks.workunit.client.0.smithi110.stdout:7/556: mkdir d1b/d29/d38/d55/dab/db2/db8 0 2024-04-25T15:49:33.014 INFO:tasks.workunit.client.1.smithi193.stdout:1/196: write dc/f16 [5166305,74199] 0 2024-04-25T15:49:33.014 INFO:tasks.workunit.client.0.smithi110.stdout:7/557: write d1b/f88 [163646,107851] 0 2024-04-25T15:49:33.016 INFO:tasks.workunit.client.1.smithi193.stdout:4/257: sync 2024-04-25T15:49:33.016 INFO:tasks.workunit.client.1.smithi193.stdout:0/248: creat d0/d49/d5/d7/dd/d16/d26/f55 x:0 0 0 2024-04-25T15:49:33.017 INFO:tasks.workunit.client.0.smithi110.stdout:1/611: getdents d0 0 2024-04-25T15:49:33.019 INFO:tasks.workunit.client.0.smithi110.stdout:0/541: creat d1/d23/d30/d33/d5b/d72/d77/fcc x:0 0 0 2024-04-25T15:49:33.022 INFO:tasks.workunit.client.1.smithi193.stdout:2/242: mkdir d0/d42/d51/d4f/d65 0 2024-04-25T15:49:33.022 INFO:tasks.workunit.client.1.smithi193.stdout:2/243: chown d0/f6 1087097 1 2024-04-25T15:49:33.022 INFO:tasks.workunit.client.1.smithi193.stdout:2/244: read - d0/d9/d3c/f60 zero size 2024-04-25T15:49:33.023 INFO:tasks.workunit.client.1.smithi193.stdout:7/188: mkdir d12/d1c/d45 0 2024-04-25T15:49:33.023 INFO:tasks.workunit.client.1.smithi193.stdout:2/245: stat d0/l2 0 2024-04-25T15:49:33.023 INFO:tasks.workunit.client.1.smithi193.stdout:7/189: fdatasync d12/f2c 0 2024-04-25T15:49:33.027 INFO:tasks.workunit.client.0.smithi110.stdout:1/612: rename d0/d14/l4d to d0/d25/lb9 0 2024-04-25T15:49:33.028 INFO:tasks.workunit.client.1.smithi193.stdout:4/258: dread f4 [0,4194304] 0 2024-04-25T15:49:33.029 INFO:tasks.workunit.client.1.smithi193.stdout:9/245: dread df/d11/f2e [0,4194304] 0 2024-04-25T15:49:33.031 INFO:tasks.workunit.client.0.smithi110.stdout:0/542: chown d1/d23/d30/d33/d5b/d78/f7d 16057414 1 2024-04-25T15:49:33.033 INFO:tasks.workunit.client.1.smithi193.stdout:9/246: dread df/d16/d1c/d26/f35 [0,4194304] 0 2024-04-25T15:49:33.035 INFO:tasks.workunit.client.1.smithi193.stdout:0/249: mkdir d0/d49/d5/d7/dd/d16/d18/d28/d56 0 2024-04-25T15:49:33.036 INFO:tasks.workunit.client.1.smithi193.stdout:3/277: mknod d8/d13/d14/c5b 0 2024-04-25T15:49:33.039 INFO:tasks.workunit.client.0.smithi110.stdout:9/457: link f6 d9/d13/d84/f92 0 2024-04-25T15:49:33.041 INFO:tasks.workunit.client.1.smithi193.stdout:6/231: link dd/f3d dd/d12/d19/f43 0 2024-04-25T15:49:33.044 INFO:tasks.workunit.client.1.smithi193.stdout:2/246: rename d0/d42/d51/l41 to d0/d9/d3d/l66 0 2024-04-25T15:49:33.045 INFO:tasks.workunit.client.0.smithi110.stdout:7/558: symlink d1b/dac/lb9 0 2024-04-25T15:49:33.045 INFO:tasks.workunit.client.1.smithi193.stdout:5/218: symlink d17/d18/d45/l4d 0 2024-04-25T15:49:33.047 INFO:tasks.workunit.client.0.smithi110.stdout:0/543: symlink d1/d23/d30/d33/d5b/d72/d77/lcd 0 2024-04-25T15:49:33.049 INFO:tasks.workunit.client.0.smithi110.stdout:2/526: rmdir de/d40/d5b/d8c 0 2024-04-25T15:49:33.053 INFO:tasks.workunit.client.1.smithi193.stdout:9/247: creat df/d16/d1c/f3d x:0 0 0 2024-04-25T15:49:33.053 INFO:tasks.workunit.client.0.smithi110.stdout:2/527: dread de/d40/f35 [0,4194304] 0 2024-04-25T15:49:33.054 INFO:tasks.workunit.client.0.smithi110.stdout:6/500: creat d1/db/d24/fb9 x:0 0 0 2024-04-25T15:49:33.057 INFO:tasks.workunit.client.0.smithi110.stdout:7/559: rename d1b/d29/d46/f80 to d1b/d29/d87/fba 0 2024-04-25T15:49:33.058 INFO:tasks.workunit.client.0.smithi110.stdout:9/458: dwrite d9/db/df/d25/f51 [0,4194304] 0 2024-04-25T15:49:33.059 INFO:tasks.workunit.client.1.smithi193.stdout:0/250: unlink d0/d49/d5/d7/dd/d16/f2f 0 2024-04-25T15:49:33.059 INFO:tasks.workunit.client.1.smithi193.stdout:5/219: dread f13 [0,4194304] 0 2024-04-25T15:49:33.060 INFO:tasks.workunit.client.1.smithi193.stdout:0/251: chown d0/d49/d5/d7/dd/d16/d18/d28/c4f 31734265 1 2024-04-25T15:49:33.064 INFO:tasks.workunit.client.1.smithi193.stdout:6/232: rmdir dd/d12/d19/d1c 39 2024-04-25T15:49:33.065 INFO:tasks.workunit.client.1.smithi193.stdout:6/233: write dd/f1b [1272081,94500] 0 2024-04-25T15:49:33.068 INFO:tasks.workunit.client.1.smithi193.stdout:4/259: rename db/l10 to db/d13/d1e/d1f/l52 0 2024-04-25T15:49:33.069 INFO:tasks.workunit.client.0.smithi110.stdout:1/613: write d0/f3f [3293385,74131] 0 2024-04-25T15:49:33.070 INFO:tasks.workunit.client.0.smithi110.stdout:8/662: sync 2024-04-25T15:49:33.070 INFO:tasks.workunit.client.0.smithi110.stdout:2/528: creat de/d40/d5b/d66/f9c x:0 0 0 2024-04-25T15:49:33.072 INFO:tasks.workunit.client.0.smithi110.stdout:2/529: chown de/d40/d4f/c58 3077 1 2024-04-25T15:49:33.073 INFO:tasks.workunit.client.0.smithi110.stdout:2/530: stat de/d40/f33 0 2024-04-25T15:49:33.075 INFO:tasks.workunit.client.1.smithi193.stdout:8/214: rmdir d2/d3/d20/d3f 0 2024-04-25T15:49:33.076 INFO:tasks.workunit.client.0.smithi110.stdout:1/614: dwrite d0/d23/f5c [8388608,4194304] 0 2024-04-25T15:49:33.082 INFO:tasks.workunit.client.0.smithi110.stdout:1/615: dwrite d0/d14/d5f/d7d/d9c/fa0 [0,4194304] 0 2024-04-25T15:49:33.087 INFO:tasks.workunit.client.0.smithi110.stdout:9/459: rmdir d9/d44 39 2024-04-25T15:49:33.088 INFO:tasks.workunit.client.0.smithi110.stdout:9/460: chown d9/db/df 96547849 1 2024-04-25T15:49:33.091 INFO:tasks.workunit.client.1.smithi193.stdout:1/197: creat dc/f40 x:0 0 0 2024-04-25T15:49:33.093 INFO:tasks.workunit.client.0.smithi110.stdout:0/544: symlink d1/d47/lce 0 2024-04-25T15:49:33.095 INFO:tasks.workunit.client.1.smithi193.stdout:9/248: truncate df/d16/d1c/d26/f35 1926539 0 2024-04-25T15:49:33.097 INFO:tasks.workunit.client.1.smithi193.stdout:9/249: truncate df/d2c/d2f/d33/f3c 196110 0 2024-04-25T15:49:33.097 INFO:tasks.workunit.client.1.smithi193.stdout:5/220: symlink d17/d18/d30/d34/d37/l4e 0 2024-04-25T15:49:33.098 INFO:tasks.workunit.client.0.smithi110.stdout:8/663: chown d14/d4c/d55/c6c 2046778 1 2024-04-25T15:49:33.100 INFO:tasks.workunit.client.0.smithi110.stdout:6/501: dwrite d1/db/d24/d2d/f59 [0,4194304] 0 2024-04-25T15:49:33.101 INFO:tasks.workunit.client.0.smithi110.stdout:6/502: dread - d1/db/d22/d33/d63/fa9 zero size 2024-04-25T15:49:33.106 INFO:tasks.workunit.client.1.smithi193.stdout:7/190: sync 2024-04-25T15:49:33.106 INFO:tasks.workunit.client.1.smithi193.stdout:7/191: readlink d12/l2a 0 2024-04-25T15:49:33.106 INFO:tasks.workunit.client.1.smithi193.stdout:7/192: stat d12/d1e/f42 0 2024-04-25T15:49:33.110 INFO:tasks.workunit.client.0.smithi110.stdout:7/560: creat d1b/d29/d38/d55/dab/db2/db8/fbb x:0 0 0 2024-04-25T15:49:33.111 INFO:tasks.workunit.client.1.smithi193.stdout:7/193: dwrite d12/d13/f14 [0,4194304] 0 2024-04-25T15:49:33.111 INFO:tasks.workunit.client.1.smithi193.stdout:8/215: rename d2/d3/d3a/f3c to d2/d36/f40 0 2024-04-25T15:49:33.113 INFO:tasks.workunit.client.1.smithi193.stdout:7/194: truncate d12/d13/f40 107095 0 2024-04-25T15:49:33.113 INFO:tasks.workunit.client.1.smithi193.stdout:7/195: write d12/f30 [1563739,78363] 0 2024-04-25T15:49:33.115 INFO:tasks.workunit.client.1.smithi193.stdout:9/250: write df/d16/d1c/f23 [1930236,124153] 0 2024-04-25T15:49:33.116 INFO:tasks.workunit.client.1.smithi193.stdout:5/221: rmdir d17/d18/d3e 39 2024-04-25T15:49:33.120 INFO:tasks.workunit.client.1.smithi193.stdout:6/234: dwrite dd/f3d [0,4194304] 0 2024-04-25T15:49:33.120 INFO:tasks.workunit.client.0.smithi110.stdout:1/616: getdents d0/d75 0 2024-04-25T15:49:33.122 INFO:tasks.workunit.client.0.smithi110.stdout:9/461: mknod d9/d44/c93 0 2024-04-25T15:49:33.122 INFO:tasks.workunit.client.0.smithi110.stdout:9/462: readlink d9/db/df/l23 0 2024-04-25T15:49:33.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/196: dwrite d12/d1c/f28 [0,4194304] 0 2024-04-25T15:49:33.123 INFO:tasks.workunit.client.0.smithi110.stdout:9/463: chown d9/db/df/d75/c76 21111 1 2024-04-25T15:49:33.125 INFO:tasks.workunit.client.0.smithi110.stdout:2/531: creat de/d40/d6e/f9d x:0 0 0 2024-04-25T15:49:33.126 INFO:tasks.workunit.client.1.smithi193.stdout:2/247: creat d0/d37/f67 x:0 0 0 2024-04-25T15:49:33.129 INFO:tasks.workunit.client.0.smithi110.stdout:1/617: unlink d0/d14/d5f/d7d/cab 0 2024-04-25T15:49:33.132 INFO:tasks.workunit.client.1.smithi193.stdout:1/198: creat dc/d35/d3d/f41 x:0 0 0 2024-04-25T15:49:33.134 INFO:tasks.workunit.client.0.smithi110.stdout:0/545: rename d1/d20/d69/la5 to d1/lcf 0 2024-04-25T15:49:33.137 INFO:tasks.workunit.client.0.smithi110.stdout:9/464: mknod d9/db/df/d25/c94 0 2024-04-25T15:49:33.139 INFO:tasks.workunit.client.0.smithi110.stdout:8/664: truncate d14/f80 1052738 0 2024-04-25T15:49:33.139 INFO:tasks.workunit.client.0.smithi110.stdout:7/561: truncate d1b/d29/d38/f3d 607228 0 2024-04-25T15:49:33.140 INFO:tasks.workunit.client.0.smithi110.stdout:9/465: read d9/d13/f54 [2497694,26014] 0 2024-04-25T15:49:33.143 INFO:tasks.workunit.client.0.smithi110.stdout:6/503: creat d1/db/d24/d2d/fba x:0 0 0 2024-04-25T15:49:33.144 INFO:tasks.workunit.client.0.smithi110.stdout:2/532: dread de/d40/d4f/f8b [0,4194304] 0 2024-04-25T15:49:33.145 INFO:tasks.workunit.client.1.smithi193.stdout:0/252: creat d0/d49/d5/d7/dd/d46/d48/f57 x:0 0 0 2024-04-25T15:49:33.146 INFO:tasks.workunit.client.1.smithi193.stdout:2/248: dread d0/df/f26 [0,4194304] 0 2024-04-25T15:49:33.148 INFO:tasks.workunit.client.1.smithi193.stdout:8/216: write d2/d3/db/f1a [1052444,85671] 0 2024-04-25T15:49:33.149 INFO:tasks.workunit.client.1.smithi193.stdout:3/278: link d8/d13/d14/d23/c52 d8/d13/d14/d27/c5c 0 2024-04-25T15:49:33.154 INFO:tasks.workunit.client.1.smithi193.stdout:3/279: dwrite d8/d13/f32 [0,4194304] 0 2024-04-25T15:49:33.155 INFO:tasks.workunit.client.0.smithi110.stdout:8/665: mkdir d14/d4c/d71/d9d/ddd 0 2024-04-25T15:49:33.156 INFO:tasks.workunit.client.0.smithi110.stdout:8/666: truncate d14/d21/f62 885842 0 2024-04-25T15:49:33.156 INFO:tasks.workunit.client.1.smithi193.stdout:4/260: rmdir db/d32/d4c 0 2024-04-25T15:49:33.158 INFO:tasks.workunit.client.0.smithi110.stdout:7/562: rename d1b/f6e to d1b/d29/d38/fbc 0 2024-04-25T15:49:33.161 INFO:tasks.workunit.client.0.smithi110.stdout:9/466: unlink d9/d13/f3f 0 2024-04-25T15:49:33.162 INFO:tasks.workunit.client.0.smithi110.stdout:8/667: dwrite d14/d21/d26/d6a/d7e/fa3 [0,4194304] 0 2024-04-25T15:49:33.169 INFO:tasks.workunit.client.1.smithi193.stdout:1/199: dwrite dc/f1b [0,4194304] 0 2024-04-25T15:49:33.170 INFO:tasks.workunit.client.0.smithi110.stdout:8/668: dread d14/d21/d26/d6a/f78 [0,4194304] 0 2024-04-25T15:49:33.172 INFO:tasks.workunit.client.1.smithi193.stdout:9/251: mkdir df/d2c/d34/d3e 0 2024-04-25T15:49:33.173 INFO:tasks.workunit.client.1.smithi193.stdout:9/252: write df/d16/d1c/f3d [593259,130739] 0 2024-04-25T15:49:33.178 INFO:tasks.workunit.client.1.smithi193.stdout:9/253: dread fc [4194304,4194304] 0 2024-04-25T15:49:33.182 INFO:tasks.workunit.client.0.smithi110.stdout:7/563: rmdir d1b/d29/d87/d8e 39 2024-04-25T15:49:33.186 INFO:tasks.workunit.client.1.smithi193.stdout:4/261: unlink db/d13/l1a 0 2024-04-25T15:49:33.186 INFO:tasks.workunit.client.1.smithi193.stdout:6/235: dwrite dd/d12/d19/d1c/d40/f25 [0,4194304] 0 2024-04-25T15:49:33.187 INFO:tasks.workunit.client.0.smithi110.stdout:6/504: write d1/db/f8c [519847,12269] 0 2024-04-25T15:49:33.194 INFO:tasks.workunit.client.1.smithi193.stdout:1/200: creat dc/d35/d3d/f42 x:0 0 0 2024-04-25T15:49:33.197 INFO:tasks.workunit.client.0.smithi110.stdout:2/533: dwrite de/d40/d4f/f49 [0,4194304] 0 2024-04-25T15:49:33.199 INFO:tasks.workunit.client.1.smithi193.stdout:2/249: sync 2024-04-25T15:49:33.201 INFO:tasks.workunit.client.0.smithi110.stdout:1/618: creat d0/d14/d5f/fba x:0 0 0 2024-04-25T15:49:33.203 INFO:tasks.workunit.client.1.smithi193.stdout:9/254: mkdir df/d16/d1c/d26/d3f 0 2024-04-25T15:49:33.204 INFO:tasks.workunit.client.1.smithi193.stdout:9/255: write df/d16/f37 [986686,20781] 0 2024-04-25T15:49:33.205 INFO:tasks.workunit.client.0.smithi110.stdout:9/467: dwrite d9/d2d/f5d [0,4194304] 0 2024-04-25T15:49:33.209 INFO:tasks.workunit.client.1.smithi193.stdout:3/280: link d8/d30/f47 d8/d13/d4b/f5d 0 2024-04-25T15:49:33.210 INFO:tasks.workunit.client.0.smithi110.stdout:8/669: write d14/d21/d26/f7a [270746,56395] 0 2024-04-25T15:49:33.212 INFO:tasks.workunit.client.0.smithi110.stdout:8/670: write d14/d4c/d71/fb5 [553257,80618] 0 2024-04-25T15:49:33.213 INFO:tasks.workunit.client.0.smithi110.stdout:7/564: creat d1b/d29/d38/d55/fbd x:0 0 0 2024-04-25T15:49:33.213 INFO:tasks.workunit.client.0.smithi110.stdout:8/671: fdatasync d14/d4c/d71/d9d/fb0 0 2024-04-25T15:49:33.215 INFO:tasks.workunit.client.0.smithi110.stdout:7/565: write d1b/d29/d38/d55/dab/fb0 [670742,56374] 0 2024-04-25T15:49:33.215 INFO:tasks.workunit.client.1.smithi193.stdout:7/197: link d12/d1e/c1f d12/d1c/c46 0 2024-04-25T15:49:33.217 INFO:tasks.workunit.client.1.smithi193.stdout:1/201: creat dc/d35/d3d/f43 x:0 0 0 2024-04-25T15:49:33.219 INFO:tasks.workunit.client.1.smithi193.stdout:0/253: creat d0/d49/d5/d7/dd/f58 x:0 0 0 2024-04-25T15:49:33.220 INFO:tasks.workunit.client.1.smithi193.stdout:5/222: getdents d17/d18/d30/d34 0 2024-04-25T15:49:33.222 INFO:tasks.workunit.client.1.smithi193.stdout:0/254: fdatasync d0/d49/d5/d7/dd/d16/d18/f4e 0 2024-04-25T15:49:33.223 INFO:tasks.workunit.client.1.smithi193.stdout:0/255: chown d0/f12 5868 1 2024-04-25T15:49:33.223 INFO:tasks.workunit.client.1.smithi193.stdout:4/262: dwrite db/d13/d1e/d1f/d22/f2a [0,4194304] 0 2024-04-25T15:49:33.225 INFO:tasks.workunit.client.1.smithi193.stdout:0/256: truncate d0/d49/d5/d7/dd/d16/d18/f2a 1573687 0 2024-04-25T15:49:33.226 INFO:tasks.workunit.client.1.smithi193.stdout:0/257: chown d0/d49/d5/d7/dd 985 1 2024-04-25T15:49:33.226 INFO:tasks.workunit.client.0.smithi110.stdout:0/546: getdents d1/d23/d30/d33/d5b/d78 0 2024-04-25T15:49:33.229 INFO:tasks.workunit.client.0.smithi110.stdout:9/468: fdatasync d9/f34 0 2024-04-25T15:49:33.235 INFO:tasks.workunit.client.0.smithi110.stdout:8/672: unlink d14/d32/f61 0 2024-04-25T15:49:33.236 INFO:tasks.workunit.client.1.smithi193.stdout:9/256: dwrite df/d11/f2e [0,4194304] 0 2024-04-25T15:49:33.236 INFO:tasks.workunit.client.0.smithi110.stdout:7/566: mknod d1b/d29/d38/d55/dab/db2/cbe 0 2024-04-25T15:49:33.236 INFO:tasks.workunit.client.0.smithi110.stdout:1/619: dwrite d0/d25/d33/f47 [4194304,4194304] 0 2024-04-25T15:49:33.238 INFO:tasks.workunit.client.1.smithi193.stdout:1/202: mkdir dc/d3c/d44 0 2024-04-25T15:49:33.238 INFO:tasks.workunit.client.0.smithi110.stdout:1/620: chown d0/d14/d5f/d7d/d72 0 1 2024-04-25T15:49:33.240 INFO:tasks.workunit.client.0.smithi110.stdout:6/505: creat d1/d12/daf/fbb x:0 0 0 2024-04-25T15:49:33.242 INFO:tasks.workunit.client.0.smithi110.stdout:2/534: symlink de/d40/d6e/d50/l9e 0 2024-04-25T15:49:33.247 INFO:tasks.workunit.client.1.smithi193.stdout:8/217: link d2/d3/lc d2/l41 0 2024-04-25T15:49:33.247 INFO:tasks.workunit.client.1.smithi193.stdout:8/218: chown d2/c24 614344763 1 2024-04-25T15:49:33.249 INFO:tasks.workunit.client.0.smithi110.stdout:0/547: write d1/fbc [2358850,128867] 0 2024-04-25T15:49:33.250 INFO:tasks.workunit.client.1.smithi193.stdout:0/258: chown d0/d49/l4a 240032 1 2024-04-25T15:49:33.251 INFO:tasks.workunit.client.0.smithi110.stdout:8/673: write d14/f94 [318215,32223] 0 2024-04-25T15:49:33.253 INFO:tasks.workunit.client.0.smithi110.stdout:8/674: chown d14/d32/d36/d48/d8f/fd1 13958 1 2024-04-25T15:49:33.253 INFO:tasks.workunit.client.1.smithi193.stdout:8/219: dwrite d2/d3/db/f1a [0,4194304] 0 2024-04-25T15:49:33.254 INFO:tasks.workunit.client.1.smithi193.stdout:8/220: chown d2/d3/d3a 3681668 1 2024-04-25T15:49:33.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/506: chown d1/db/d24/d69/d75/d82/l8f 11 1 2024-04-25T15:49:33.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/507: readlink d1/db/l35 0 2024-04-25T15:49:33.256 INFO:tasks.workunit.client.0.smithi110.stdout:6/508: stat d1/d48 0 2024-04-25T15:49:33.256 INFO:tasks.workunit.client.0.smithi110.stdout:1/621: truncate d0/d25/d33/f50 723481 0 2024-04-25T15:49:33.261 INFO:tasks.workunit.client.0.smithi110.stdout:2/535: chown de/d40/d5b/d5c/l68 5853119 1 2024-04-25T15:49:33.265 INFO:tasks.workunit.client.1.smithi193.stdout:1/203: fdatasync dc/fe 0 2024-04-25T15:49:33.268 INFO:tasks.workunit.client.1.smithi193.stdout:2/250: link d0/le d0/df/l68 0 2024-04-25T15:49:33.268 INFO:tasks.workunit.client.0.smithi110.stdout:8/675: dread ff [0,4194304] 0 2024-04-25T15:49:33.268 INFO:tasks.workunit.client.1.smithi193.stdout:2/251: chown d0/d9/d15/d31/l5e 408971311 1 2024-04-25T15:49:33.270 INFO:tasks.workunit.client.1.smithi193.stdout:5/223: mkdir d17/d18/d30/d38/d48/d4f 0 2024-04-25T15:49:33.271 INFO:tasks.workunit.client.0.smithi110.stdout:9/469: dwrite d9/d2d/f30 [0,4194304] 0 2024-04-25T15:49:33.275 INFO:tasks.workunit.client.0.smithi110.stdout:7/567: dwrite d1b/f26 [0,4194304] 0 2024-04-25T15:49:33.275 INFO:tasks.workunit.client.1.smithi193.stdout:2/252: dwrite d0/d37/f67 [0,4194304] 0 2024-04-25T15:49:33.279 INFO:tasks.workunit.client.1.smithi193.stdout:3/281: link d8/d13/d14/d23/l28 d8/d30/d45/l5e 0 2024-04-25T15:49:33.286 INFO:tasks.workunit.client.1.smithi193.stdout:6/236: getdents dd/d12/d19 0 2024-04-25T15:49:33.288 INFO:tasks.workunit.client.1.smithi193.stdout:8/221: symlink d2/d3/l42 0 2024-04-25T15:49:33.288 INFO:tasks.workunit.client.1.smithi193.stdout:3/282: dread d8/d13/d14/f20 [0,4194304] 0 2024-04-25T15:49:33.291 INFO:tasks.workunit.client.1.smithi193.stdout:4/263: write db/d13/f1d [527049,48215] 0 2024-04-25T15:49:33.292 INFO:tasks.workunit.client.1.smithi193.stdout:9/257: creat df/d16/d1c/d26/d3f/f40 x:0 0 0 2024-04-25T15:49:33.293 INFO:tasks.workunit.client.1.smithi193.stdout:9/258: fsync df/d11/f14 0 2024-04-25T15:49:33.294 INFO:tasks.workunit.client.1.smithi193.stdout:6/237: dwrite dd/d12/d19/d1c/f20 [4194304,4194304] 0 2024-04-25T15:49:33.294 INFO:tasks.workunit.client.1.smithi193.stdout:9/259: stat df/d11/f14 0 2024-04-25T15:49:33.294 INFO:tasks.workunit.client.1.smithi193.stdout:6/238: chown dd/d12/d2d 1190467299 1 2024-04-25T15:49:33.295 INFO:tasks.workunit.client.1.smithi193.stdout:8/222: dread d2/d3/d20/f22 [0,4194304] 0 2024-04-25T15:49:33.298 INFO:tasks.workunit.client.1.smithi193.stdout:2/253: unlink d0/f6 0 2024-04-25T15:49:33.303 INFO:tasks.workunit.client.1.smithi193.stdout:6/239: read dd/d12/d19/f1d [120036,85274] 0 2024-04-25T15:49:33.304 INFO:tasks.workunit.client.1.smithi193.stdout:0/259: link d0/d49/d5/d7/dd/d16/f33 d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f59 0 2024-04-25T15:49:33.304 INFO:tasks.workunit.client.1.smithi193.stdout:0/260: chown d0/d49/fe 106304188 1 2024-04-25T15:49:33.308 INFO:tasks.workunit.client.1.smithi193.stdout:3/283: rmdir d8/d13 39 2024-04-25T15:49:33.312 INFO:tasks.workunit.client.1.smithi193.stdout:7/198: link c11 d12/d1e/c47 0 2024-04-25T15:49:33.315 INFO:tasks.workunit.client.1.smithi193.stdout:4/264: symlink db/d32/d45/l53 0 2024-04-25T15:49:33.317 INFO:tasks.workunit.client.1.smithi193.stdout:1/204: unlink c4 0 2024-04-25T15:49:33.319 INFO:tasks.workunit.client.1.smithi193.stdout:9/260: mkdir df/d16/d1c/d26/d3f/d41 0 2024-04-25T15:49:33.320 INFO:tasks.workunit.client.1.smithi193.stdout:9/261: write df/d11/f2e [2696828,118030] 0 2024-04-25T15:49:33.330 INFO:tasks.workunit.client.1.smithi193.stdout:6/240: symlink dd/d12/d19/d1c/d40/l44 0 2024-04-25T15:49:33.333 INFO:tasks.workunit.client.1.smithi193.stdout:3/284: truncate d8/d30/f36 270364 0 2024-04-25T15:49:33.335 INFO:tasks.workunit.client.1.smithi193.stdout:6/241: dwrite dd/d12/d19/d1c/d40/f25 [4194304,4194304] 0 2024-04-25T15:49:33.339 INFO:tasks.workunit.client.1.smithi193.stdout:5/224: truncate d17/d18/d45/f47 2816255 0 2024-04-25T15:49:33.341 INFO:tasks.workunit.client.0.smithi110.stdout:1/622: symlink d0/d14/d5f/d7d/d72/d88/lbb 0 2024-04-25T15:49:33.346 INFO:tasks.workunit.client.0.smithi110.stdout:8/676: mkdir d14/d32/d36/dde 0 2024-04-25T15:49:33.347 INFO:tasks.workunit.client.0.smithi110.stdout:8/677: chown d14/d32/d36/d48/d8f 16703496 1 2024-04-25T15:49:33.348 INFO:tasks.workunit.client.1.smithi193.stdout:9/262: symlink df/d2c/l42 0 2024-04-25T15:49:33.348 INFO:tasks.workunit.client.0.smithi110.stdout:9/470: rmdir d9/db/df/d2c 39 2024-04-25T15:49:33.352 INFO:tasks.workunit.client.1.smithi193.stdout:9/263: dread fc [4194304,4194304] 0 2024-04-25T15:49:33.353 INFO:tasks.workunit.client.1.smithi193.stdout:0/261: dwrite d0/d49/d5/d7/dd/d16/f41 [0,4194304] 0 2024-04-25T15:49:33.354 INFO:tasks.workunit.client.0.smithi110.stdout:0/548: symlink d1/d20/ld0 0 2024-04-25T15:49:33.355 INFO:tasks.workunit.client.1.smithi193.stdout:9/264: dread df/d2c/d2f/d33/f3c [0,4194304] 0 2024-04-25T15:49:33.355 INFO:tasks.workunit.client.1.smithi193.stdout:9/265: chown df/d16/d1c/f23 146340 1 2024-04-25T15:49:33.356 INFO:tasks.workunit.client.1.smithi193.stdout:9/266: stat l0 0 2024-04-25T15:49:33.356 INFO:tasks.workunit.client.0.smithi110.stdout:6/509: dread d1/fe [0,4194304] 0 2024-04-25T15:49:33.356 INFO:tasks.workunit.client.0.smithi110.stdout:6/510: chown d1/d48/f6c 18273 1 2024-04-25T15:49:33.366 INFO:tasks.workunit.client.1.smithi193.stdout:8/223: truncate d2/d3/ff 3176496 0 2024-04-25T15:49:33.368 INFO:tasks.workunit.client.0.smithi110.stdout:7/568: mknod d1b/d29/db5/cbf 0 2024-04-25T15:49:33.370 INFO:tasks.workunit.client.1.smithi193.stdout:6/242: dwrite dd/f11 [0,4194304] 0 2024-04-25T15:49:33.373 INFO:tasks.workunit.client.1.smithi193.stdout:4/265: mkdir db/d4b/d51/d54 0 2024-04-25T15:49:33.375 INFO:tasks.workunit.client.1.smithi193.stdout:4/266: chown db/lf 899578554 1 2024-04-25T15:49:33.378 INFO:tasks.workunit.client.1.smithi193.stdout:6/243: dwrite dd/f2e [0,4194304] 0 2024-04-25T15:49:33.379 INFO:tasks.workunit.client.1.smithi193.stdout:2/254: dwrite d0/d37/d25/d5f/d3a/f58 [0,4194304] 0 2024-04-25T15:49:33.380 INFO:tasks.workunit.client.1.smithi193.stdout:2/255: dread - d0/d9/d3c/f3e zero size 2024-04-25T15:49:33.381 INFO:tasks.workunit.client.1.smithi193.stdout:7/199: write d12/f2e [1358729,96448] 0 2024-04-25T15:49:33.384 INFO:tasks.workunit.client.0.smithi110.stdout:8/678: symlink d14/d32/d36/dde/ldf 0 2024-04-25T15:49:33.389 INFO:tasks.workunit.client.0.smithi110.stdout:9/471: dread - d9/db/df/d2c/f70 zero size 2024-04-25T15:49:33.391 INFO:tasks.workunit.client.1.smithi193.stdout:1/205: truncate dc/f2b 290695 0 2024-04-25T15:49:33.394 INFO:tasks.workunit.client.0.smithi110.stdout:3/608: sync 2024-04-25T15:49:33.394 INFO:tasks.workunit.client.0.smithi110.stdout:5/559: sync 2024-04-25T15:49:33.394 INFO:tasks.workunit.client.0.smithi110.stdout:4/582: sync 2024-04-25T15:49:33.394 INFO:tasks.workunit.client.0.smithi110.stdout:7/569: fdatasync d1b/d29/d87/d8e/f94 0 2024-04-25T15:49:33.395 INFO:tasks.workunit.client.1.smithi193.stdout:9/267: creat df/d16/d1c/d26/d3f/f43 x:0 0 0 2024-04-25T15:49:33.396 INFO:tasks.workunit.client.1.smithi193.stdout:0/262: read d0/d49/d5/d7/dd/d16/d26/d2e/f34 [557620,116598] 0 2024-04-25T15:49:33.396 INFO:tasks.workunit.client.0.smithi110.stdout:6/511: creat d1/db/d22/d90/d76/dad/fbc x:0 0 0 2024-04-25T15:49:33.398 INFO:tasks.workunit.client.0.smithi110.stdout:1/623: link d0/d51/ca4 d0/d25/cbc 0 2024-04-25T15:49:33.398 INFO:tasks.workunit.client.1.smithi193.stdout:3/285: rmdir d8/d13/d14/d51 39 2024-04-25T15:49:33.399 INFO:tasks.workunit.client.1.smithi193.stdout:0/263: dread d0/d49/d5/d7/dd/f40 [0,4194304] 0 2024-04-25T15:49:33.399 INFO:tasks.workunit.client.0.smithi110.stdout:2/536: getdents de/d40/d6e/d52 0 2024-04-25T15:49:33.400 INFO:tasks.workunit.client.0.smithi110.stdout:2/537: readlink de/l2e 0 2024-04-25T15:49:33.400 INFO:tasks.workunit.client.0.smithi110.stdout:8/679: mknod d14/d4c/d55/d8d/ce0 0 2024-04-25T15:49:33.401 INFO:tasks.workunit.client.1.smithi193.stdout:9/268: dwrite df/d2d/f39 [0,4194304] 0 2024-04-25T15:49:33.402 INFO:tasks.workunit.client.1.smithi193.stdout:8/224: creat d2/d3/d20/f43 x:0 0 0 2024-04-25T15:49:33.402 INFO:tasks.workunit.client.0.smithi110.stdout:8/680: read d14/d21/fc8 [583142,73798] 0 2024-04-25T15:49:33.405 INFO:tasks.workunit.client.0.smithi110.stdout:7/570: chown d1b/f36 53 1 2024-04-25T15:49:33.405 INFO:tasks.workunit.client.0.smithi110.stdout:7/571: chown d1b/d29/d87/fa2 28477868 1 2024-04-25T15:49:33.407 INFO:tasks.workunit.client.0.smithi110.stdout:0/549: creat d1/d23/d30/d33/d5b/d72/fd1 x:0 0 0 2024-04-25T15:49:33.411 INFO:tasks.workunit.client.0.smithi110.stdout:4/583: write d13/d24/f2a [3856369,76249] 0 2024-04-25T15:49:33.413 INFO:tasks.workunit.client.0.smithi110.stdout:5/560: symlink d4/d1d/lb9 0 2024-04-25T15:49:33.414 INFO:tasks.workunit.client.1.smithi193.stdout:4/267: creat db/d13/d1e/d1f/d22/f55 x:0 0 0 2024-04-25T15:49:33.417 INFO:tasks.workunit.client.1.smithi193.stdout:6/244: mkdir dd/d12/d19/d45 0 2024-04-25T15:49:33.419 INFO:tasks.workunit.client.0.smithi110.stdout:2/538: mkdir de/d40/d4f/d9f 0 2024-04-25T15:49:33.424 INFO:tasks.workunit.client.0.smithi110.stdout:8/681: mkdir d14/d32/d36/d48/de1 0 2024-04-25T15:49:33.424 INFO:tasks.workunit.client.0.smithi110.stdout:8/682: chown d14/d21/d26/f4a 804 1 2024-04-25T15:49:33.425 INFO:tasks.workunit.client.0.smithi110.stdout:8/683: truncate f3 4987090 0 2024-04-25T15:49:33.426 INFO:tasks.workunit.client.0.smithi110.stdout:3/609: unlink d0/d7/d40/f61 0 2024-04-25T15:49:33.427 INFO:tasks.workunit.client.0.smithi110.stdout:3/610: fdatasync d0/d16/dcb/d50/f6d 0 2024-04-25T15:49:33.432 INFO:tasks.workunit.client.0.smithi110.stdout:0/550: rename d1/d23/d30/d33/d5b/d72/dc4/lc8 to d1/d23/d30/d33/d35/ld2 0 2024-04-25T15:49:33.432 INFO:tasks.workunit.client.0.smithi110.stdout:1/624: dwrite d0/d51/f52 [4194304,4194304] 0 2024-04-25T15:49:33.433 INFO:tasks.workunit.client.1.smithi193.stdout:2/256: write d0/d42/d51/f40 [3976726,115764] 0 2024-04-25T15:49:33.434 INFO:tasks.workunit.client.0.smithi110.stdout:0/551: chown d1/d23/d30/d33/f2d 227 1 2024-04-25T15:49:33.434 INFO:tasks.workunit.client.1.smithi193.stdout:3/286: dread d8/d13/d14/d23/f48 [0,4194304] 0 2024-04-25T15:49:33.435 INFO:tasks.workunit.client.1.smithi193.stdout:3/287: stat d8/d13/d1f/f22 0 2024-04-25T15:49:33.436 INFO:tasks.workunit.client.1.smithi193.stdout:3/288: readlink d8/d13/l1e 0 2024-04-25T15:49:33.436 INFO:tasks.workunit.client.1.smithi193.stdout:0/264: symlink d0/d49/d5/d7/dd/d16/d18/d28/l5a 0 2024-04-25T15:49:33.438 INFO:tasks.workunit.client.1.smithi193.stdout:8/225: symlink d2/d3/d3a/l44 0 2024-04-25T15:49:33.439 INFO:tasks.workunit.client.0.smithi110.stdout:2/539: creat de/d40/d5b/d66/fa0 x:0 0 0 2024-04-25T15:49:33.441 INFO:tasks.workunit.client.0.smithi110.stdout:3/611: dread d0/d2e/f4f [0,4194304] 0 2024-04-25T15:49:33.443 INFO:tasks.workunit.client.0.smithi110.stdout:5/561: dread d4/d2c/d2a/f5c [0,4194304] 0 2024-04-25T15:49:33.444 INFO:tasks.workunit.client.0.smithi110.stdout:9/472: creat d9/f95 x:0 0 0 2024-04-25T15:49:33.450 INFO:tasks.workunit.client.1.smithi193.stdout:7/200: link f8 d12/d1c/d45/f48 0 2024-04-25T15:49:33.451 INFO:tasks.workunit.client.0.smithi110.stdout:4/584: dwrite fa [0,4194304] 0 2024-04-25T15:49:33.451 INFO:tasks.workunit.client.0.smithi110.stdout:1/625: creat d0/d14/d5f/d7d/fbd x:0 0 0 2024-04-25T15:49:33.456 INFO:tasks.workunit.client.1.smithi193.stdout:1/206: truncate dc/f12 674875 0 2024-04-25T15:49:33.458 INFO:tasks.workunit.client.1.smithi193.stdout:2/257: creat d0/d42/d51/d4f/f69 x:0 0 0 2024-04-25T15:49:33.460 INFO:tasks.workunit.client.0.smithi110.stdout:3/612: creat d0/d2e/d88/dbf/d92/fd2 x:0 0 0 2024-04-25T15:49:33.460 INFO:tasks.workunit.client.0.smithi110.stdout:7/572: write d1b/d29/f2b [299536,115863] 0 2024-04-25T15:49:33.461 INFO:tasks.workunit.client.0.smithi110.stdout:3/613: chown d0/d16/d8c 1372296 1 2024-04-25T15:49:33.462 INFO:tasks.workunit.client.0.smithi110.stdout:5/562: unlink d4/d1d/d1f/d56/d6b/c88 0 2024-04-25T15:49:33.462 INFO:tasks.workunit.client.0.smithi110.stdout:8/684: dwrite f11 [0,4194304] 0 2024-04-25T15:49:33.463 INFO:tasks.workunit.client.0.smithi110.stdout:7/573: write d1b/d29/d38/d55/fbd [72517,26883] 0 2024-04-25T15:49:33.463 INFO:tasks.workunit.client.1.smithi193.stdout:3/289: rmdir d8/d30 39 2024-04-25T15:49:33.465 INFO:tasks.workunit.client.1.smithi193.stdout:3/290: write d8/d13/d1f/f22 [3529402,109880] 0 2024-04-25T15:49:33.465 INFO:tasks.workunit.client.1.smithi193.stdout:9/269: mknod df/d16/c44 0 2024-04-25T15:49:33.466 INFO:tasks.workunit.client.0.smithi110.stdout:7/574: read d1b/d29/d38/d55/f6b [1306546,68798] 0 2024-04-25T15:49:33.467 INFO:tasks.workunit.client.1.smithi193.stdout:0/265: write d0/d49/d5/d7/dd/f40 [1131311,93911] 0 2024-04-25T15:49:33.467 INFO:tasks.workunit.client.1.smithi193.stdout:5/225: getdents d17/d18/d30/d38/d48 0 2024-04-25T15:49:33.469 INFO:tasks.workunit.client.1.smithi193.stdout:0/266: readlink d0/d49/d5/d7/l21 0 2024-04-25T15:49:33.470 INFO:tasks.workunit.client.1.smithi193.stdout:9/270: dread df/d16/f19 [0,4194304] 0 2024-04-25T15:49:33.471 INFO:tasks.workunit.client.1.smithi193.stdout:9/271: chown df/d16/c44 219 1 2024-04-25T15:49:33.471 INFO:tasks.workunit.client.1.smithi193.stdout:6/245: read dd/d12/d19/f1d [51793,84621] 0 2024-04-25T15:49:33.472 INFO:tasks.workunit.client.1.smithi193.stdout:9/272: fsync df/d11/f2e 0 2024-04-25T15:49:33.472 INFO:tasks.workunit.client.0.smithi110.stdout:2/540: unlink de/cf 0 2024-04-25T15:49:33.474 INFO:tasks.workunit.client.1.smithi193.stdout:1/207: write dc/f3f [299391,25238] 0 2024-04-25T15:49:33.474 INFO:tasks.workunit.client.1.smithi193.stdout:1/208: fsync dc/f12 0 2024-04-25T15:49:33.475 INFO:tasks.workunit.client.0.smithi110.stdout:3/614: stat d0/d7/d7d/f81 0 2024-04-25T15:49:33.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/258: rename d0/d37/d25/d5f/d3a/d56 to d0/d42/d51/d4f/d65/d6a 0 2024-04-25T15:49:33.477 INFO:tasks.workunit.client.0.smithi110.stdout:8/685: rmdir d14/d21/d26 39 2024-04-25T15:49:33.485 INFO:tasks.workunit.client.1.smithi193.stdout:5/226: symlink d17/d18/d30/d38/l50 0 2024-04-25T15:49:33.486 INFO:tasks.workunit.client.0.smithi110.stdout:7/575: symlink d1b/d29/d38/d8b/d89/lc0 0 2024-04-25T15:49:33.491 INFO:tasks.workunit.client.1.smithi193.stdout:0/267: creat d0/d49/d5/d7/dd/d46/d48/f5b x:0 0 0 2024-04-25T15:49:33.492 INFO:tasks.workunit.client.0.smithi110.stdout:0/552: creat d1/d23/d30/d33/d35/da1/db4/db9/fd3 x:0 0 0 2024-04-25T15:49:33.493 INFO:tasks.workunit.client.1.smithi193.stdout:7/201: mknod d12/d31/d37/c49 0 2024-04-25T15:49:33.493 INFO:tasks.workunit.client.0.smithi110.stdout:0/553: chown d1/d23/d30/d33/d5b/d83 1 1 2024-04-25T15:49:33.494 INFO:tasks.workunit.client.0.smithi110.stdout:0/554: fdatasync d1/f8 0 2024-04-25T15:49:33.495 INFO:tasks.workunit.client.0.smithi110.stdout:3/615: rmdir d0/d2e/d88/dbf/d92 39 2024-04-25T15:49:33.498 INFO:tasks.workunit.client.0.smithi110.stdout:8/686: mknod d14/d32/d36/db7/ce2 0 2024-04-25T15:49:33.499 INFO:tasks.workunit.client.0.smithi110.stdout:4/585: dwrite d13/d24/f58 [0,4194304] 0 2024-04-25T15:49:33.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/273: mknod df/d2d/c45 0 2024-04-25T15:49:33.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/274: chown df/d11/c32 0 1 2024-04-25T15:49:33.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/275: stat df/d2d/c45 0 2024-04-25T15:49:33.504 INFO:tasks.workunit.client.0.smithi110.stdout:5/563: symlink d4/d1d/d1f/lba 0 2024-04-25T15:49:33.504 INFO:tasks.workunit.client.0.smithi110.stdout:3/616: dwrite d0/d16/dcb/d2f/fa4 [4194304,4194304] 0 2024-04-25T15:49:33.506 INFO:tasks.workunit.client.0.smithi110.stdout:5/564: chown d4/d1d/d1f/d35/f9a 393 1 2024-04-25T15:49:33.506 INFO:tasks.workunit.client.0.smithi110.stdout:9/473: creat d9/f96 x:0 0 0 2024-04-25T15:49:33.510 INFO:tasks.workunit.client.1.smithi193.stdout:2/259: creat d0/d9/d15/d31/d50/f6b x:0 0 0 2024-04-25T15:49:33.510 INFO:tasks.workunit.client.0.smithi110.stdout:1/626: link d0/d23/f83 d0/d25/d33/fbe 0 2024-04-25T15:49:33.512 INFO:tasks.workunit.client.0.smithi110.stdout:2/541: mkdir de/d40/d6e/d52/d6d/d99/da1 0 2024-04-25T15:49:33.515 INFO:tasks.workunit.client.0.smithi110.stdout:0/555: creat d1/d23/d30/d33/d5b/d72/d77/fd4 x:0 0 0 2024-04-25T15:49:33.516 INFO:tasks.workunit.client.1.smithi193.stdout:4/268: getdents db/d32/d45 0 2024-04-25T15:49:33.516 INFO:tasks.workunit.client.1.smithi193.stdout:4/269: fsync f4 0 2024-04-25T15:49:33.518 INFO:tasks.workunit.client.0.smithi110.stdout:1/627: dwrite d0/d14/d5f/d6f/fa3 [0,4194304] 0 2024-04-25T15:49:33.520 INFO:tasks.workunit.client.0.smithi110.stdout:8/687: mknod d14/d32/d36/d48/ce3 0 2024-04-25T15:49:33.520 INFO:tasks.workunit.client.1.smithi193.stdout:7/202: creat d12/d38/f4a x:0 0 0 2024-04-25T15:49:33.522 INFO:tasks.workunit.client.0.smithi110.stdout:4/586: fsync d13/d24/d2d/f79 0 2024-04-25T15:49:33.525 INFO:tasks.workunit.client.0.smithi110.stdout:3/617: creat d0/d7/d7d/fd3 x:0 0 0 2024-04-25T15:49:33.526 INFO:tasks.workunit.client.1.smithi193.stdout:3/291: write d8/d13/d14/f3e [4265786,117873] 0 2024-04-25T15:49:33.526 INFO:tasks.workunit.client.1.smithi193.stdout:9/276: creat df/d2c/d34/f46 x:0 0 0 2024-04-25T15:49:33.528 INFO:tasks.workunit.client.0.smithi110.stdout:5/565: mknod d4/d15/d51/cbb 0 2024-04-25T15:49:33.529 INFO:tasks.workunit.client.0.smithi110.stdout:8/688: dwrite d14/d4c/d55/fb4 [0,4194304] 0 2024-04-25T15:49:33.531 INFO:tasks.workunit.client.0.smithi110.stdout:8/689: write d14/d32/d36/d48/f5e [3975469,70798] 0 2024-04-25T15:49:33.532 INFO:tasks.workunit.client.0.smithi110.stdout:9/474: fsync d9/db/f1b 0 2024-04-25T15:49:33.533 INFO:tasks.workunit.client.1.smithi193.stdout:2/260: unlink d0/d9/d15/l21 0 2024-04-25T15:49:33.533 INFO:tasks.workunit.client.0.smithi110.stdout:9/475: dread - d9/db/df/d25/f79 zero size 2024-04-25T15:49:33.536 INFO:tasks.workunit.client.0.smithi110.stdout:7/576: link d1b/d29/d38/d55/f6b d1b/d29/d38/d55/dab/db2/fc1 0 2024-04-25T15:49:33.539 INFO:tasks.workunit.client.1.smithi193.stdout:2/261: dwrite d0/d37/d25/d5f/d3a/f4a [0,4194304] 0 2024-04-25T15:49:33.541 INFO:tasks.workunit.client.1.smithi193.stdout:4/270: mkdir db/d13/d1e/d1f/d56 0 2024-04-25T15:49:33.542 INFO:tasks.workunit.client.0.smithi110.stdout:1/628: mkdir d0/d14/d5f/d7d/d72/d88/dbf 0 2024-04-25T15:49:33.543 INFO:tasks.workunit.client.0.smithi110.stdout:1/629: stat d0/d51/f79 0 2024-04-25T15:49:33.544 INFO:tasks.workunit.client.1.smithi193.stdout:7/203: symlink d12/l4b 0 2024-04-25T15:49:33.545 INFO:tasks.workunit.client.0.smithi110.stdout:4/587: symlink d13/d24/d2d/d44/da4/lc0 0 2024-04-25T15:49:33.546 INFO:tasks.workunit.client.1.smithi193.stdout:7/204: stat d12/d38/f4a 0 2024-04-25T15:49:33.548 INFO:tasks.workunit.client.1.smithi193.stdout:3/292: fsync d8/d30/d31/f50 0 2024-04-25T15:49:33.548 INFO:tasks.workunit.client.1.smithi193.stdout:7/205: chown d12/d13/f41 25 1 2024-04-25T15:49:33.549 INFO:tasks.workunit.client.1.smithi193.stdout:2/262: dwrite d0/fc [0,4194304] 0 2024-04-25T15:49:33.550 INFO:tasks.workunit.client.0.smithi110.stdout:5/566: symlink d4/d15/d51/d90/lbc 0 2024-04-25T15:49:33.550 INFO:tasks.workunit.client.0.smithi110.stdout:3/618: truncate d0/d2e/d88/dbf/d92/fd2 1005646 0 2024-04-25T15:49:33.550 INFO:tasks.workunit.client.1.smithi193.stdout:1/209: dwrite dc/f30 [0,4194304] 0 2024-04-25T15:49:33.552 INFO:tasks.workunit.client.1.smithi193.stdout:8/226: sync 2024-04-25T15:49:33.555 INFO:tasks.workunit.client.1.smithi193.stdout:5/227: link d17/d18/d30/l3d d17/d18/d45/l51 0 2024-04-25T15:49:33.559 INFO:tasks.workunit.client.1.smithi193.stdout:0/268: creat d0/d49/d5/d7/dd/f5c x:0 0 0 2024-04-25T15:49:33.562 INFO:tasks.workunit.client.1.smithi193.stdout:7/206: dwrite d12/f18 [0,4194304] 0 2024-04-25T15:49:33.563 INFO:tasks.workunit.client.1.smithi193.stdout:3/293: dwrite d8/f39 [0,4194304] 0 2024-04-25T15:49:33.564 INFO:tasks.workunit.client.0.smithi110.stdout:0/556: unlink d1/d47/f7b 0 2024-04-25T15:49:33.565 INFO:tasks.workunit.client.1.smithi193.stdout:9/277: creat df/d16/d1c/d30/d3b/f47 x:0 0 0 2024-04-25T15:49:33.566 INFO:tasks.workunit.client.0.smithi110.stdout:1/630: creat d0/d14/fc0 x:0 0 0 2024-04-25T15:49:33.566 INFO:tasks.workunit.client.1.smithi193.stdout:9/278: chown df/d16/d1c/d26/d3f/d41 107166713 1 2024-04-25T15:49:33.568 INFO:tasks.workunit.client.0.smithi110.stdout:1/631: chown d0/d14/d5f/d7d/d72/c96 11458 1 2024-04-25T15:49:33.587 INFO:tasks.workunit.client.0.smithi110.stdout:6/512: sync 2024-04-25T15:49:33.594 INFO:tasks.workunit.client.0.smithi110.stdout:6/513: dwrite d1/db/da0/fb6 [0,4194304] 0 2024-04-25T15:49:33.619 INFO:tasks.workunit.client.0.smithi110.stdout:0/557: dread d1/d23/d30/d33/d5b/d78/f7d [4194304,4194304] 0 2024-04-25T15:49:33.626 INFO:tasks.workunit.client.0.smithi110.stdout:0/558: dwrite d1/d47/f95 [4194304,4194304] 0 2024-04-25T15:49:33.630 INFO:tasks.workunit.client.0.smithi110.stdout:0/559: write d1/d23/d30/d33/d35/da1/db4/db9/fd3 [810008,96474] 0 2024-04-25T15:49:33.633 INFO:tasks.workunit.client.1.smithi193.stdout:1/210: chown dc/l27 2905 1 2024-04-25T15:49:33.636 INFO:tasks.workunit.client.1.smithi193.stdout:2/263: symlink d0/d37/d25/d5f/d3a/l6c 0 2024-04-25T15:49:33.637 INFO:tasks.workunit.client.1.smithi193.stdout:2/264: dread - d0/d9/d15/d31/d50/f6b zero size 2024-04-25T15:49:33.638 INFO:tasks.workunit.client.0.smithi110.stdout:5/567: creat d4/d1d/d1f/d9e/fbd x:0 0 0 2024-04-25T15:49:33.640 INFO:tasks.workunit.client.0.smithi110.stdout:5/568: write d4/d1d/d1f/d35/d3d/d52/f82 [1864792,17749] 0 2024-04-25T15:49:33.642 INFO:tasks.workunit.client.0.smithi110.stdout:5/569: write d4/d1d/d1f/d56/f7e [779708,54192] 0 2024-04-25T15:49:33.648 INFO:tasks.workunit.client.1.smithi193.stdout:5/228: mkdir d17/d18/d30/d34/d52 0 2024-04-25T15:49:33.649 INFO:tasks.workunit.client.1.smithi193.stdout:5/229: chown d17/d18/d30/d38 0 1 2024-04-25T15:49:33.649 INFO:tasks.workunit.client.1.smithi193.stdout:5/230: write d17/d18/f24 [624326,103966] 0 2024-04-25T15:49:33.653 INFO:tasks.workunit.client.0.smithi110.stdout:9/476: mkdir d9/d97 0 2024-04-25T15:49:33.653 INFO:tasks.workunit.client.1.smithi193.stdout:5/231: dread d17/d18/f2e [0,4194304] 0 2024-04-25T15:49:33.655 INFO:tasks.workunit.client.1.smithi193.stdout:4/271: mkdir db/d13/d1e/d1f/d56/d57 0 2024-04-25T15:49:33.659 INFO:tasks.workunit.client.0.smithi110.stdout:7/577: unlink d1b/d29/l81 0 2024-04-25T15:49:33.666 INFO:tasks.workunit.client.1.smithi193.stdout:6/246: getdents dd/d12/d19/d1c 0 2024-04-25T15:49:33.666 INFO:tasks.workunit.client.1.smithi193.stdout:7/207: mknod d12/d13/c4c 0 2024-04-25T15:49:33.669 INFO:tasks.workunit.client.1.smithi193.stdout:3/294: fsync d8/f11 0 2024-04-25T15:49:33.671 INFO:tasks.workunit.client.0.smithi110.stdout:6/514: read - d1/dd/d3c/f7c zero size 2024-04-25T15:49:33.672 INFO:tasks.workunit.client.1.smithi193.stdout:6/247: dwrite dd/d12/d19/d1c/f2c [0,4194304] 0 2024-04-25T15:49:33.675 INFO:tasks.workunit.client.1.smithi193.stdout:9/279: rmdir df/d16/d1c 39 2024-04-25T15:49:33.676 INFO:tasks.workunit.client.1.smithi193.stdout:9/280: write df/d11/f14 [839804,59489] 0 2024-04-25T15:49:33.676 INFO:tasks.workunit.client.1.smithi193.stdout:3/295: dwrite d8/d13/d14/f3e [4194304,4194304] 0 2024-04-25T15:49:33.690 INFO:tasks.workunit.client.1.smithi193.stdout:5/232: mkdir d17/d53 0 2024-04-25T15:49:33.690 INFO:tasks.workunit.client.1.smithi193.stdout:3/296: dwrite d8/d13/f56 [0,4194304] 0 2024-04-25T15:49:33.690 INFO:tasks.workunit.client.1.smithi193.stdout:7/208: dread f2 [0,4194304] 0 2024-04-25T15:49:33.692 INFO:tasks.workunit.client.1.smithi193.stdout:3/297: read - d8/d13/d14/f59 zero size 2024-04-25T15:49:33.692 INFO:tasks.workunit.client.1.smithi193.stdout:1/211: dwrite dc/f36 [0,4194304] 0 2024-04-25T15:49:33.695 INFO:tasks.workunit.client.1.smithi193.stdout:0/269: fdatasync d0/f1e 0 2024-04-25T15:49:33.697 INFO:tasks.workunit.client.1.smithi193.stdout:0/270: write d0/d49/fe [4739985,37185] 0 2024-04-25T15:49:33.698 INFO:tasks.workunit.client.0.smithi110.stdout:7/578: creat d1b/d29/d46/fc2 x:0 0 0 2024-04-25T15:49:33.699 INFO:tasks.workunit.client.0.smithi110.stdout:7/579: chown d1b/d29/d87/fba 16034 1 2024-04-25T15:49:33.700 INFO:tasks.workunit.client.0.smithi110.stdout:6/515: truncate d1/db/d24/f66 492236 0 2024-04-25T15:49:33.702 INFO:tasks.workunit.client.1.smithi193.stdout:0/271: dwrite d0/d49/d5/d7/dd/f5c [0,4194304] 0 2024-04-25T15:49:33.703 INFO:tasks.workunit.client.0.smithi110.stdout:0/560: truncate d1/d47/f8e 1023474 0 2024-04-25T15:49:33.706 INFO:tasks.workunit.client.1.smithi193.stdout:6/248: creat dd/d12/d19/d1c/f46 x:0 0 0 2024-04-25T15:49:33.707 INFO:tasks.workunit.client.1.smithi193.stdout:6/249: chown dd/d12/d19/d1c/f46 1930641 1 2024-04-25T15:49:33.708 INFO:tasks.workunit.client.1.smithi193.stdout:9/281: write df/d16/d1c/f23 [744062,21032] 0 2024-04-25T15:49:33.709 INFO:tasks.workunit.client.1.smithi193.stdout:9/282: stat l2 0 2024-04-25T15:49:33.710 INFO:tasks.workunit.client.1.smithi193.stdout:6/250: write dd/d12/d19/d1c/d40/d3c/d3f/f41 [269832,50800] 0 2024-04-25T15:49:33.713 INFO:tasks.workunit.client.0.smithi110.stdout:5/570: rename d4/d2c/l14 to d4/d1d/d1f/d35/d3d/d93/lbe 0 2024-04-25T15:49:33.715 INFO:tasks.workunit.client.1.smithi193.stdout:9/283: dwrite df/d2d/f31 [0,4194304] 0 2024-04-25T15:49:33.715 INFO:tasks.workunit.client.0.smithi110.stdout:5/571: chown d4/d1d/d1f/f32 26560405 1 2024-04-25T15:49:33.723 INFO:tasks.workunit.client.0.smithi110.stdout:0/561: truncate d1/d6c/d7f/fb5 2398777 0 2024-04-25T15:49:33.724 INFO:tasks.workunit.client.1.smithi193.stdout:7/209: symlink d12/l4d 0 2024-04-25T15:49:33.725 INFO:tasks.workunit.client.0.smithi110.stdout:4/588: getdents d13/d24/d2d/d44/d65/d99/daf/d56 0 2024-04-25T15:49:33.727 INFO:tasks.workunit.client.1.smithi193.stdout:5/233: symlink d17/d18/d30/d34/d37/l54 0 2024-04-25T15:49:33.729 INFO:tasks.workunit.client.0.smithi110.stdout:3/619: getdents d0/d2e/d88/dbf/d99/dad/daf 0 2024-04-25T15:49:33.731 INFO:tasks.workunit.client.1.smithi193.stdout:4/272: mknod db/d4b/d51/d54/c58 0 2024-04-25T15:49:33.731 INFO:tasks.workunit.client.0.smithi110.stdout:0/562: dwrite d1/d47/d54/d99/fb7 [0,4194304] 0 2024-04-25T15:49:33.737 INFO:tasks.workunit.client.0.smithi110.stdout:9/477: rename d9/db/df/d75 to d9/d13/d84/d98 0 2024-04-25T15:49:33.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:33 smithi193 ceph-mon[33747]: pgmap v33: 65 pgs: 65 active+clean; 1.3 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 35 MiB/s rd, 136 MiB/s wr, 283 op/s 2024-04-25T15:49:33.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:49:33.740 INFO:tasks.workunit.client.0.smithi110.stdout:0/563: dwrite d1/d47/d54/f9c [0,4194304] 0 2024-04-25T15:49:33.740 INFO:tasks.workunit.client.1.smithi193.stdout:1/212: dwrite dc/d35/d3d/f43 [0,4194304] 0 2024-04-25T15:49:33.741 INFO:tasks.workunit.client.1.smithi193.stdout:3/298: dread d8/d30/f36 [0,4194304] 0 2024-04-25T15:49:33.741 INFO:tasks.workunit.client.1.smithi193.stdout:1/213: truncate dc/d35/f3b 366677 0 2024-04-25T15:49:33.741 INFO:tasks.workunit.client.0.smithi110.stdout:5/572: mknod d4/d2c/d2a/cbf 0 2024-04-25T15:49:33.742 INFO:tasks.workunit.client.1.smithi193.stdout:3/299: fdatasync d8/d13/f56 0 2024-04-25T15:49:33.752 INFO:tasks.workunit.client.1.smithi193.stdout:6/251: symlink dd/d12/d19/d1c/d40/l47 0 2024-04-25T15:49:33.753 INFO:tasks.workunit.client.1.smithi193.stdout:6/252: write dd/d12/d19/d1c/d40/f25 [2735889,127875] 0 2024-04-25T15:49:33.753 INFO:tasks.workunit.client.1.smithi193.stdout:6/253: chown f6 63 1 2024-04-25T15:49:33.755 INFO:tasks.workunit.client.1.smithi193.stdout:8/227: getdents d2/d36 0 2024-04-25T15:49:33.761 INFO:tasks.workunit.client.0.smithi110.stdout:9/478: mknod d9/d44/d5e/d80/c99 0 2024-04-25T15:49:33.761 INFO:tasks.workunit.client.1.smithi193.stdout:5/234: fdatasync d17/d18/f20 0 2024-04-25T15:49:33.768 INFO:tasks.workunit.client.0.smithi110.stdout:5/573: dread d4/d2c/d2a/f46 [0,4194304] 0 2024-04-25T15:49:33.770 INFO:tasks.workunit.client.1.smithi193.stdout:1/214: read dc/f2d [2679042,102781] 0 2024-04-25T15:49:33.783 INFO:tasks.workunit.client.0.smithi110.stdout:5/574: creat d4/d1d/d1f/d35/d3d/d52/d75/fc0 x:0 0 0 2024-04-25T15:49:33.785 INFO:tasks.workunit.client.0.smithi110.stdout:8/690: write d14/d21/d26/d6a/f78 [973388,34263] 0 2024-04-25T15:49:33.786 INFO:tasks.workunit.client.0.smithi110.stdout:9/479: dread d9/db/df/f6c [0,4194304] 0 2024-04-25T15:49:33.787 INFO:tasks.workunit.client.1.smithi193.stdout:8/228: fsync d2/d36/f40 0 2024-04-25T15:49:33.790 INFO:tasks.workunit.client.0.smithi110.stdout:0/564: rmdir d1/d23/d30/d33/d5b/d72/dc4 0 2024-04-25T15:49:33.794 INFO:tasks.workunit.client.0.smithi110.stdout:5/575: symlink d4/d1d/d1f/d35/d7f/d92/lc1 0 2024-04-25T15:49:33.796 INFO:tasks.workunit.client.0.smithi110.stdout:5/576: read d4/f22 [3167747,45510] 0 2024-04-25T15:49:33.796 INFO:tasks.workunit.client.0.smithi110.stdout:5/577: stat d4/d2c/f27 0 2024-04-25T15:49:33.797 INFO:tasks.workunit.client.0.smithi110.stdout:3/620: getdents d0/d16/dcb/d2f/d4c/d63 0 2024-04-25T15:49:33.797 INFO:tasks.workunit.client.0.smithi110.stdout:5/578: dread - d4/d1d/d1f/f94 zero size 2024-04-25T15:49:33.798 INFO:tasks.workunit.client.0.smithi110.stdout:5/579: readlink d4/d1d/d85/l9f 0 2024-04-25T15:49:33.800 INFO:tasks.workunit.client.1.smithi193.stdout:5/235: dwrite d17/d18/d30/f35 [0,4194304] 0 2024-04-25T15:49:33.801 INFO:tasks.workunit.client.1.smithi193.stdout:8/229: dread d2/d36/f40 [0,4194304] 0 2024-04-25T15:49:33.802 INFO:tasks.workunit.client.1.smithi193.stdout:8/230: read d2/d36/f40 [1848775,101852] 0 2024-04-25T15:49:33.802 INFO:tasks.workunit.client.1.smithi193.stdout:1/215: readlink dc/l2f 0 2024-04-25T15:49:33.804 INFO:tasks.workunit.client.1.smithi193.stdout:3/300: creat d8/d13/d14/d51/f5f x:0 0 0 2024-04-25T15:49:33.805 INFO:tasks.workunit.client.0.smithi110.stdout:0/565: creat d1/d6c/fd5 x:0 0 0 2024-04-25T15:49:33.805 INFO:tasks.workunit.client.1.smithi193.stdout:3/301: write d8/d13/d14/f59 [64476,66624] 0 2024-04-25T15:49:33.806 INFO:tasks.workunit.client.0.smithi110.stdout:0/566: chown d1/f2a 1 1 2024-04-25T15:49:33.806 INFO:tasks.workunit.client.1.smithi193.stdout:0/272: link d0/l2 d0/d49/d5/d7/dd/d46/d48/l5d 0 2024-04-25T15:49:33.809 INFO:tasks.workunit.client.1.smithi193.stdout:9/284: sync 2024-04-25T15:49:33.809 INFO:tasks.workunit.client.0.smithi110.stdout:5/580: rmdir d4/d1d/d1f/d35/d3d 39 2024-04-25T15:49:33.810 INFO:tasks.workunit.client.1.smithi193.stdout:9/285: write df/d16/d1c/f3d [1603087,6906] 0 2024-04-25T15:49:33.813 INFO:tasks.workunit.client.1.smithi193.stdout:6/254: mknod dd/d12/d19/c48 0 2024-04-25T15:49:33.813 INFO:tasks.workunit.client.1.smithi193.stdout:0/273: dwrite d0/d49/d5/d7/dd/d16/f41 [0,4194304] 0 2024-04-25T15:49:33.814 INFO:tasks.workunit.client.0.smithi110.stdout:9/480: mknod d9/d13/d7c/c9a 0 2024-04-25T15:49:33.814 INFO:tasks.workunit.client.1.smithi193.stdout:0/274: chown d0/f35 517864 1 2024-04-25T15:49:33.814 INFO:tasks.workunit.client.1.smithi193.stdout:6/255: chown dd/d12/d19/d1c/d40/l33 707 1 2024-04-25T15:49:33.815 INFO:tasks.workunit.client.1.smithi193.stdout:6/256: stat dd/d12/d19/d1c/d40/d3c 0 2024-04-25T15:49:33.816 INFO:tasks.workunit.client.1.smithi193.stdout:6/257: write dd/f42 [676104,107973] 0 2024-04-25T15:49:33.819 INFO:tasks.workunit.client.1.smithi193.stdout:6/258: dread dd/d12/d19/d1c/d40/f25 [4194304,4194304] 0 2024-04-25T15:49:33.824 INFO:tasks.workunit.client.0.smithi110.stdout:0/567: truncate d1/d23/d30/d33/d35/f36 313845 0 2024-04-25T15:49:33.826 INFO:tasks.workunit.client.0.smithi110.stdout:5/581: dwrite d4/d1d/d1f/d35/d3d/f53 [4194304,4194304] 0 2024-04-25T15:49:33.826 INFO:tasks.workunit.client.0.smithi110.stdout:5/582: chown d4/d15/d51/c80 204335 1 2024-04-25T15:49:33.827 INFO:tasks.workunit.client.0.smithi110.stdout:5/583: chown d4/d1d/d1f/d35/d7f/d92/d9b 2 1 2024-04-25T15:49:33.834 INFO:tasks.workunit.client.1.smithi193.stdout:8/231: dread d2/fa [0,4194304] 0 2024-04-25T15:49:33.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:33 smithi110 ceph-mon[26344]: pgmap v33: 65 pgs: 65 active+clean; 1.3 GiB data, 5.5 GiB used, 531 GiB / 536 GiB avail; 35 MiB/s rd, 136 MiB/s wr, 283 op/s 2024-04-25T15:49:33.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:49:33.892 INFO:tasks.workunit.client.0.smithi110.stdout:3/621: link d0/d16/dcb/d2f/d4c/fac d0/d16/dcb/d2f/fd4 0 2024-04-25T15:49:33.893 INFO:tasks.workunit.client.1.smithi193.stdout:1/216: stat dc/c2e 0 2024-04-25T15:49:33.894 INFO:tasks.workunit.client.0.smithi110.stdout:3/622: write d0/d16/dcb/f35 [1200204,7952] 0 2024-04-25T15:49:33.898 INFO:tasks.workunit.client.1.smithi193.stdout:3/302: creat d8/d13/f60 x:0 0 0 2024-04-25T15:49:33.900 INFO:tasks.workunit.client.0.smithi110.stdout:0/568: fdatasync d1/f8d 0 2024-04-25T15:49:33.902 INFO:tasks.workunit.client.1.smithi193.stdout:2/265: mkdir d0/d37/d25/d5f/d3a/d6d 0 2024-04-25T15:49:33.903 INFO:tasks.workunit.client.0.smithi110.stdout:7/580: write d1b/d29/f60 [453139,93160] 0 2024-04-25T15:49:33.904 INFO:tasks.workunit.client.0.smithi110.stdout:7/581: truncate d1b/f26 5112729 0 2024-04-25T15:49:33.906 INFO:tasks.workunit.client.0.smithi110.stdout:5/584: rename d4/d2c/d2a/l31 to d4/d1d/d1f/d35/d3d/d52/lc2 0 2024-04-25T15:49:33.906 INFO:tasks.workunit.client.0.smithi110.stdout:1/632: creat d0/d14/d5f/d7d/fc1 x:0 0 0 2024-04-25T15:49:33.907 INFO:tasks.workunit.client.0.smithi110.stdout:1/633: fdatasync d0/d14/fad 0 2024-04-25T15:49:33.908 INFO:tasks.workunit.client.1.smithi193.stdout:9/286: rename df/d2c/l42 to df/d16/d1c/d26/l48 0 2024-04-25T15:49:33.909 INFO:tasks.workunit.client.0.smithi110.stdout:1/634: truncate d0/d14/d5f/d7d/d72/d61/fb2 887223 0 2024-04-25T15:49:33.911 INFO:tasks.workunit.client.1.smithi193.stdout:0/275: mkdir d0/d49/d5/d7/dd/d16/d26/d5e 0 2024-04-25T15:49:33.912 INFO:tasks.workunit.client.0.smithi110.stdout:3/623: symlink d0/d16/dcb/d50/ld5 0 2024-04-25T15:49:33.913 INFO:tasks.workunit.client.0.smithi110.stdout:2/542: creat de/fa2 x:0 0 0 2024-04-25T15:49:33.914 INFO:tasks.workunit.client.1.smithi193.stdout:4/273: getdents db/d13 0 2024-04-25T15:49:33.914 INFO:tasks.workunit.client.1.smithi193.stdout:4/274: read db/d13/d1e/d1f/d22/f27 [328533,72412] 0 2024-04-25T15:49:33.921 INFO:tasks.workunit.client.0.smithi110.stdout:1/635: chown d0/d14/d81/c90 6652 1 2024-04-25T15:49:33.921 INFO:tasks.workunit.client.0.smithi110.stdout:1/636: chown d0/d23/f3b 1459 1 2024-04-25T15:49:33.922 INFO:tasks.workunit.client.1.smithi193.stdout:8/232: truncate d2/d3/d20/f22 2640925 0 2024-04-25T15:49:33.923 INFO:tasks.workunit.client.1.smithi193.stdout:8/233: write d2/d3/d20/f43 [1030121,59555] 0 2024-04-25T15:49:33.923 INFO:tasks.workunit.client.1.smithi193.stdout:8/234: readlink d2/l41 0 2024-04-25T15:49:33.923 INFO:tasks.workunit.client.0.smithi110.stdout:3/624: creat d0/d16/d8c/fd6 x:0 0 0 2024-04-25T15:49:33.924 INFO:tasks.workunit.client.0.smithi110.stdout:1/637: truncate d0/d14/d5f/d7d/d72/d61/fb2 1487591 0 2024-04-25T15:49:33.925 INFO:tasks.workunit.client.0.smithi110.stdout:2/543: rename de/f4b to de/d40/d4f/d9f/fa3 0 2024-04-25T15:49:33.926 INFO:tasks.workunit.client.1.smithi193.stdout:3/303: truncate d8/d30/f36 998666 0 2024-04-25T15:49:33.926 INFO:tasks.workunit.client.1.smithi193.stdout:8/235: dread d2/d3/db/f1a [0,4194304] 0 2024-04-25T15:49:33.927 INFO:tasks.workunit.client.0.smithi110.stdout:0/569: fsync d1/d6c/f94 0 2024-04-25T15:49:33.928 INFO:tasks.workunit.client.1.smithi193.stdout:3/304: read d8/fc [3244350,69133] 0 2024-04-25T15:49:33.929 INFO:tasks.workunit.client.0.smithi110.stdout:6/516: write d1/db/d22/d33/d43/f53 [4287672,10467] 0 2024-04-25T15:49:33.929 INFO:tasks.workunit.client.1.smithi193.stdout:2/266: read d0/df/f10 [1663002,31726] 0 2024-04-25T15:49:33.930 INFO:tasks.workunit.client.0.smithi110.stdout:3/625: dwrite d0/d2e/d88/dbf/d92/f93 [0,4194304] 0 2024-04-25T15:49:33.931 INFO:tasks.workunit.client.1.smithi193.stdout:9/287: creat df/d2c/d2f/f49 x:0 0 0 2024-04-25T15:49:33.932 INFO:tasks.workunit.client.0.smithi110.stdout:5/585: symlink d4/d1d/d1f/d56/lc3 0 2024-04-25T15:49:33.934 INFO:tasks.workunit.client.1.smithi193.stdout:0/276: rename d0/d49/d5/d7/dd/d16/l36 to d0/d49/d5/d7/dd/d16/d18/d28/d56/l5f 0 2024-04-25T15:49:33.940 INFO:tasks.workunit.client.0.smithi110.stdout:1/638: symlink d0/d25/d33/lc2 0 2024-04-25T15:49:33.941 INFO:tasks.workunit.client.0.smithi110.stdout:1/639: dread - d0/d51/d6b/faf zero size 2024-04-25T15:49:33.943 INFO:tasks.workunit.client.1.smithi193.stdout:7/210: write f8 [5223660,90833] 0 2024-04-25T15:49:33.946 INFO:tasks.workunit.client.1.smithi193.stdout:1/217: symlink dc/d3c/l45 0 2024-04-25T15:49:33.950 INFO:tasks.workunit.client.0.smithi110.stdout:5/586: dread d4/d1d/d1f/f64 [0,4194304] 0 2024-04-25T15:49:33.950 INFO:tasks.workunit.client.0.smithi110.stdout:6/517: dread d1/db/d22/d33/f3b [0,4194304] 0 2024-04-25T15:49:33.951 INFO:tasks.workunit.client.0.smithi110.stdout:1/640: mknod d0/d14/d5f/d6f/cc3 0 2024-04-25T15:49:33.952 INFO:tasks.workunit.client.0.smithi110.stdout:5/587: chown d4/d1d/d1f/d35/d3d/d52/d97/fb1 6029 1 2024-04-25T15:49:33.952 INFO:tasks.workunit.client.0.smithi110.stdout:1/641: chown d0/d14/f57 1226 1 2024-04-25T15:49:33.954 INFO:tasks.workunit.client.0.smithi110.stdout:3/626: mkdir d0/d16/dcb/dd7 0 2024-04-25T15:49:33.955 INFO:tasks.workunit.client.0.smithi110.stdout:3/627: write d0/d2e/fc6 [1212436,34495] 0 2024-04-25T15:49:33.956 INFO:tasks.workunit.client.0.smithi110.stdout:3/628: chown d0/d7/d40/f7e 14 1 2024-04-25T15:49:33.957 INFO:tasks.workunit.client.0.smithi110.stdout:3/629: dread d0/d16/dcb/d2f/d4c/fac [0,4194304] 0 2024-04-25T15:49:33.959 INFO:tasks.workunit.client.1.smithi193.stdout:8/236: rename d2/d3/db to d2/d3/d20/d45 0 2024-04-25T15:49:33.960 INFO:tasks.workunit.client.1.smithi193.stdout:0/277: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60 0 2024-04-25T15:49:33.961 INFO:tasks.workunit.client.1.smithi193.stdout:0/278: fsync d0/d49/d5/d7/dd/d16/f41 0 2024-04-25T15:49:33.962 INFO:tasks.workunit.client.1.smithi193.stdout:0/279: write d0/d49/d5/d7/dd/d46/d48/f57 [62532,92046] 0 2024-04-25T15:49:33.964 INFO:tasks.workunit.client.0.smithi110.stdout:3/630: dwrite d0/d16/dcb/d2f/d4c/fa0 [0,4194304] 0 2024-04-25T15:49:33.965 INFO:tasks.workunit.client.1.smithi193.stdout:5/236: truncate d17/d18/f20 4895320 0 2024-04-25T15:49:33.965 INFO:tasks.workunit.client.0.smithi110.stdout:3/631: readlink d0/d7/l60 0 2024-04-25T15:49:33.967 INFO:tasks.workunit.client.0.smithi110.stdout:5/588: mkdir d4/d1d/d1f/d35/d3d/d93/dc4 0 2024-04-25T15:49:33.967 INFO:tasks.workunit.client.0.smithi110.stdout:3/632: stat d0/d16/dcb/d2f/d33/c37 0 2024-04-25T15:49:33.967 INFO:tasks.workunit.client.0.smithi110.stdout:3/633: stat d0/d7/l4e 0 2024-04-25T15:49:33.970 INFO:tasks.workunit.client.0.smithi110.stdout:1/642: mkdir d0/d14/d5f/d7d/d72/d61/dc4 0 2024-04-25T15:49:33.971 INFO:tasks.workunit.client.0.smithi110.stdout:1/643: write d0/d14/fc0 [318104,27192] 0 2024-04-25T15:49:33.978 INFO:tasks.workunit.client.1.smithi193.stdout:1/218: fsync dc/f17 0 2024-04-25T15:49:33.979 INFO:tasks.workunit.client.1.smithi193.stdout:1/219: write f6 [1286313,35353] 0 2024-04-25T15:49:33.979 INFO:tasks.workunit.client.0.smithi110.stdout:8/691: dwrite d14/d4c/d71/dae/fcf [4194304,4194304] 0 2024-04-25T15:49:33.979 INFO:tasks.workunit.client.1.smithi193.stdout:1/220: chown dc/d3c/f3e 70562 1 2024-04-25T15:49:33.980 INFO:tasks.workunit.client.0.smithi110.stdout:3/634: dread d0/d16/dcb/f56 [0,4194304] 0 2024-04-25T15:49:33.980 INFO:tasks.workunit.client.1.smithi193.stdout:1/221: read - dc/d35/d3d/f42 zero size 2024-04-25T15:49:33.981 INFO:tasks.workunit.client.1.smithi193.stdout:1/222: write dc/f3f [1334278,80713] 0 2024-04-25T15:49:33.985 INFO:tasks.workunit.client.0.smithi110.stdout:9/481: write d9/d13/f27 [663989,102810] 0 2024-04-25T15:49:33.988 INFO:tasks.workunit.client.0.smithi110.stdout:5/589: rename d4/d1d/d1f/d35/d3d/d52/d97/fa0 to d4/d1d/d1f/d35/d3d/fc5 0 2024-04-25T15:49:33.990 INFO:tasks.workunit.client.1.smithi193.stdout:9/288: rename df/d11/f14 to df/d2c/d34/d3e/f4a 0 2024-04-25T15:49:33.991 INFO:tasks.workunit.client.0.smithi110.stdout:1/644: fdatasync d0/d51/d6b/f7c 0 2024-04-25T15:49:33.995 INFO:tasks.workunit.client.0.smithi110.stdout:8/692: creat d14/d32/fe4 x:0 0 0 2024-04-25T15:49:33.998 INFO:tasks.workunit.client.1.smithi193.stdout:5/237: symlink d17/d18/d30/d34/d37/l55 0 2024-04-25T15:49:33.999 INFO:tasks.workunit.client.0.smithi110.stdout:8/693: dread f6 [4194304,4194304] 0 2024-04-25T15:49:34.001 INFO:tasks.workunit.client.1.smithi193.stdout:6/259: getdents dd/d12 0 2024-04-25T15:49:34.002 INFO:tasks.workunit.client.0.smithi110.stdout:9/482: rmdir d9/d13 39 2024-04-25T15:49:34.004 INFO:tasks.workunit.client.1.smithi193.stdout:7/211: link d12/d1e/f34 d12/d38/f4e 0 2024-04-25T15:49:34.004 INFO:tasks.workunit.client.1.smithi193.stdout:7/212: chown d12/d13 1088647718 1 2024-04-25T15:49:34.009 INFO:tasks.workunit.client.0.smithi110.stdout:9/483: dwrite d9/db/df/d55/f6b [0,4194304] 0 2024-04-25T15:49:34.009 INFO:tasks.workunit.client.0.smithi110.stdout:5/590: fsync d4/d1d/d1f/f42 0 2024-04-25T15:49:34.011 INFO:tasks.workunit.client.0.smithi110.stdout:5/591: stat d4/d1d/d1f/f42 0 2024-04-25T15:49:34.011 INFO:tasks.workunit.client.0.smithi110.stdout:7/582: write d1b/d29/d38/d8b/d89/f8c [7435453,36450] 0 2024-04-25T15:49:34.012 INFO:tasks.workunit.client.1.smithi193.stdout:1/223: dread dc/f2b [0,4194304] 0 2024-04-25T15:49:34.015 INFO:tasks.workunit.client.1.smithi193.stdout:3/305: truncate d8/d13/f56 1467567 0 2024-04-25T15:49:34.017 INFO:tasks.workunit.client.1.smithi193.stdout:6/260: dread dd/fe [4194304,4194304] 0 2024-04-25T15:49:34.022 INFO:tasks.workunit.client.0.smithi110.stdout:3/635: symlink d0/d7/dd0/ld8 0 2024-04-25T15:49:34.023 INFO:tasks.workunit.client.0.smithi110.stdout:3/636: dread - d0/d16/dcb/fc8 zero size 2024-04-25T15:49:34.025 INFO:tasks.workunit.client.1.smithi193.stdout:2/267: rename d0/d9/d3c to d0/d37/d25/d5f/d3a/d6d/d6e 0 2024-04-25T15:49:34.026 INFO:tasks.workunit.client.0.smithi110.stdout:8/694: chown d14/l33 371 1 2024-04-25T15:49:34.029 INFO:tasks.workunit.client.1.smithi193.stdout:0/280: mkdir d0/d49/d5/d61 0 2024-04-25T15:49:34.030 INFO:tasks.workunit.client.1.smithi193.stdout:5/238: rmdir d17/d18/d30 39 2024-04-25T15:49:34.030 INFO:tasks.workunit.client.1.smithi193.stdout:2/268: dwrite d0/d42/d51/d4f/f69 [0,4194304] 0 2024-04-25T15:49:34.031 INFO:tasks.workunit.client.1.smithi193.stdout:2/269: chown d0/d44 1903207 1 2024-04-25T15:49:34.035 INFO:tasks.workunit.client.1.smithi193.stdout:1/224: chown dc/l24 57845 1 2024-04-25T15:49:34.038 INFO:tasks.workunit.client.1.smithi193.stdout:6/261: fdatasync dd/d12/d19/f1d 0 2024-04-25T15:49:34.038 INFO:tasks.workunit.client.1.smithi193.stdout:1/225: dread dc/d35/d3d/f43 [0,4194304] 0 2024-04-25T15:49:34.038 INFO:tasks.workunit.client.1.smithi193.stdout:6/262: dread - dd/d12/f35 zero size 2024-04-25T15:49:34.042 INFO:tasks.workunit.client.0.smithi110.stdout:6/518: write d1/db/d22/d33/d63/f9a [684919,118750] 0 2024-04-25T15:49:34.046 INFO:tasks.workunit.client.1.smithi193.stdout:8/237: rename d2/d3/l31 to d2/d3/d20/d45/l46 0 2024-04-25T15:49:34.049 INFO:tasks.workunit.client.1.smithi193.stdout:9/289: creat df/d16/d1c/d26/d3f/d41/f4b x:0 0 0 2024-04-25T15:49:34.050 INFO:tasks.workunit.client.0.smithi110.stdout:5/592: dread d4/f6d [0,4194304] 0 2024-04-25T15:49:34.050 INFO:tasks.workunit.client.0.smithi110.stdout:8/695: mkdir d14/d4c/d55/de5 0 2024-04-25T15:49:34.051 INFO:tasks.workunit.client.1.smithi193.stdout:4/275: dwrite db/d13/d1e/d1f/d22/f2c [0,4194304] 0 2024-04-25T15:49:34.053 INFO:tasks.workunit.client.0.smithi110.stdout:9/484: creat d9/d13/d84/f9b x:0 0 0 2024-04-25T15:49:34.057 INFO:tasks.workunit.client.0.smithi110.stdout:7/583: mknod d1b/cc3 0 2024-04-25T15:49:34.059 INFO:tasks.workunit.client.0.smithi110.stdout:5/593: dwrite d4/d2c/d83/faa [0,4194304] 0 2024-04-25T15:49:34.061 INFO:tasks.workunit.client.0.smithi110.stdout:5/594: write d4/d1d/d1f/d35/d3d/d52/d75/fb5 [19412,75964] 0 2024-04-25T15:49:34.066 INFO:tasks.workunit.client.0.smithi110.stdout:3/637: dread d0/d7/f17 [4194304,4194304] 0 2024-04-25T15:49:34.069 INFO:tasks.workunit.client.0.smithi110.stdout:8/696: symlink d14/d4c/d71/le6 0 2024-04-25T15:49:34.070 INFO:tasks.workunit.client.1.smithi193.stdout:7/213: mknod d12/d1c/d45/c4f 0 2024-04-25T15:49:34.071 INFO:tasks.workunit.client.0.smithi110.stdout:9/485: creat d9/db/df/d25/f9c x:0 0 0 2024-04-25T15:49:34.080 INFO:tasks.workunit.client.1.smithi193.stdout:6/263: symlink dd/d12/d19/d1c/l49 0 2024-04-25T15:49:34.080 INFO:tasks.workunit.client.1.smithi193.stdout:6/264: truncate dd/f2e 4864969 0 2024-04-25T15:49:34.083 INFO:tasks.workunit.client.1.smithi193.stdout:8/238: rmdir d2/d18 39 2024-04-25T15:49:34.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/290: creat df/d16/d1c/d26/d3f/f4c x:0 0 0 2024-04-25T15:49:34.091 INFO:tasks.workunit.client.1.smithi193.stdout:5/239: fsync f13 0 2024-04-25T15:49:34.092 INFO:tasks.workunit.client.0.smithi110.stdout:1/645: dwrite d0/d23/f5a [0,4194304] 0 2024-04-25T15:49:34.095 INFO:tasks.workunit.client.1.smithi193.stdout:6/265: dread f6 [0,4194304] 0 2024-04-25T15:49:34.095 INFO:tasks.workunit.client.0.smithi110.stdout:3/638: dread d0/d2e/f76 [0,4194304] 0 2024-04-25T15:49:34.096 INFO:tasks.workunit.client.1.smithi193.stdout:5/240: dwrite f11 [0,4194304] 0 2024-04-25T15:49:34.099 INFO:tasks.workunit.client.1.smithi193.stdout:0/281: truncate d0/d49/d5/d7/dd/d46/f4d 1641463 0 2024-04-25T15:49:34.099 INFO:tasks.workunit.client.1.smithi193.stdout:0/282: fsync d0/d49/d5/d7/dd/d38/f4c 0 2024-04-25T15:49:34.105 INFO:tasks.workunit.client.0.smithi110.stdout:9/486: truncate d9/db/df/d2c/d3c/f49 1159557 0 2024-04-25T15:49:34.106 INFO:tasks.workunit.client.1.smithi193.stdout:7/214: symlink d12/d1c/d45/l50 0 2024-04-25T15:49:34.107 INFO:tasks.workunit.client.1.smithi193.stdout:7/215: fdatasync d12/d13/f40 0 2024-04-25T15:49:34.107 INFO:tasks.workunit.client.0.smithi110.stdout:5/595: creat d4/d1d/d1f/d35/fc6 x:0 0 0 2024-04-25T15:49:34.109 INFO:tasks.workunit.client.1.smithi193.stdout:3/306: sync 2024-04-25T15:49:34.110 INFO:tasks.workunit.client.0.smithi110.stdout:1/646: rename d0/d25/c63 to d0/d14/d5f/d7d/cc5 0 2024-04-25T15:49:34.112 INFO:tasks.workunit.client.0.smithi110.stdout:1/647: fsync d0/d23/f5c 0 2024-04-25T15:49:34.115 INFO:tasks.workunit.client.0.smithi110.stdout:2/544: dwrite de/d40/d4f/d9f/fa3 [0,4194304] 0 2024-04-25T15:49:34.116 INFO:tasks.workunit.client.1.smithi193.stdout:8/239: mknod d2/d1f/c47 0 2024-04-25T15:49:34.117 INFO:tasks.workunit.client.0.smithi110.stdout:8/697: link d14/d4c/d55/f6d d14/d4c/d55/fe7 0 2024-04-25T15:49:34.118 INFO:tasks.workunit.client.0.smithi110.stdout:6/519: write d1/db/d22/d90/f72 [634708,101795] 0 2024-04-25T15:49:34.119 INFO:tasks.workunit.client.0.smithi110.stdout:6/520: chown d1/db/d22/d90/d9e 10 1 2024-04-25T15:49:34.120 INFO:tasks.workunit.client.1.smithi193.stdout:9/291: mkdir df/d2c/d34/d3e/d4d 0 2024-04-25T15:49:34.120 INFO:tasks.workunit.client.0.smithi110.stdout:5/596: symlink d4/d1d/d1f/d35/d3d/d93/lc7 0 2024-04-25T15:49:34.127 INFO:tasks.workunit.client.0.smithi110.stdout:7/584: write d1b/f36 [1336264,109068] 0 2024-04-25T15:49:34.132 INFO:tasks.workunit.client.1.smithi193.stdout:6/266: rename dd/d12/d19/d1c/d40/l47 to dd/d2f/l4a 0 2024-04-25T15:49:34.133 INFO:tasks.workunit.client.0.smithi110.stdout:5/597: mkdir d4/d1d/d1f/d35/d3d/dc8 0 2024-04-25T15:49:34.135 INFO:tasks.workunit.client.1.smithi193.stdout:6/267: write dd/d12/d19/d1c/d40/d3c/d3f/f41 [417930,97104] 0 2024-04-25T15:49:34.135 INFO:tasks.workunit.client.0.smithi110.stdout:8/698: rename d14/d4c to d14/d21/d26/d6a/d7e/da6/de8 0 2024-04-25T15:49:34.135 INFO:tasks.workunit.client.1.smithi193.stdout:5/241: dwrite d17/d18/d30/f32 [0,4194304] 0 2024-04-25T15:49:34.136 INFO:tasks.workunit.client.1.smithi193.stdout:6/268: readlink dd/d12/d19/d1c/d40/l3b 0 2024-04-25T15:49:34.141 INFO:tasks.workunit.client.0.smithi110.stdout:9/487: creat d9/d13/d5b/d61/f9d x:0 0 0 2024-04-25T15:49:34.143 INFO:tasks.workunit.client.1.smithi193.stdout:1/226: write dc/f2c [360516,51872] 0 2024-04-25T15:49:34.143 INFO:tasks.workunit.client.1.smithi193.stdout:7/216: chown d12/d1c/c46 631903 1 2024-04-25T15:49:34.143 INFO:tasks.workunit.client.0.smithi110.stdout:5/598: creat d4/d1d/d1f/d35/d3d/d52/fc9 x:0 0 0 2024-04-25T15:49:34.145 INFO:tasks.workunit.client.1.smithi193.stdout:0/283: dwrite d0/d49/d5/f24 [0,4194304] 0 2024-04-25T15:49:34.145 INFO:tasks.workunit.client.1.smithi193.stdout:3/307: creat d8/d13/f61 x:0 0 0 2024-04-25T15:49:34.148 INFO:tasks.workunit.client.0.smithi110.stdout:6/521: rename d1/d12/d3d to d1/db/d22/d33/d63/dbd 0 2024-04-25T15:49:34.149 INFO:tasks.workunit.client.0.smithi110.stdout:3/639: dwrite d0/d16/dcb/f94 [0,4194304] 0 2024-04-25T15:49:34.154 INFO:tasks.workunit.client.0.smithi110.stdout:8/699: mkdir d14/d21/d26/d6a/de9 0 2024-04-25T15:49:34.158 INFO:tasks.workunit.client.1.smithi193.stdout:4/276: creat db/f59 x:0 0 0 2024-04-25T15:49:34.160 INFO:tasks.workunit.client.1.smithi193.stdout:2/270: link d0/d9/d15/d31/l2f d0/d9/d15/l6f 0 2024-04-25T15:49:34.166 INFO:tasks.workunit.client.1.smithi193.stdout:2/271: dread d0/d42/d51/d4f/d65/d6a/f3b [0,4194304] 0 2024-04-25T15:49:34.168 INFO:tasks.workunit.client.1.smithi193.stdout:9/292: rename df/d16 to df/d2c/d2f/d4e 0 2024-04-25T15:49:34.171 INFO:tasks.workunit.client.0.smithi110.stdout:6/522: mkdir d1/db/d24/d69/d75/d82/dbe 0 2024-04-25T15:49:34.171 INFO:tasks.workunit.client.1.smithi193.stdout:2/272: dwrite d0/f11 [0,4194304] 0 2024-04-25T15:49:34.173 INFO:tasks.workunit.client.1.smithi193.stdout:2/273: fsync d0/d9/d15/d31/d50/f6b 0 2024-04-25T15:49:34.176 INFO:tasks.workunit.client.0.smithi110.stdout:8/700: creat d14/d21/d23/fea x:0 0 0 2024-04-25T15:49:34.177 INFO:tasks.workunit.client.1.smithi193.stdout:6/269: fdatasync dd/d12/d19/d1c/f26 0 2024-04-25T15:49:34.178 INFO:tasks.workunit.client.0.smithi110.stdout:1/648: getdents d0/d23 0 2024-04-25T15:49:34.179 INFO:tasks.workunit.client.1.smithi193.stdout:7/217: symlink d12/d1c/l51 0 2024-04-25T15:49:34.180 INFO:tasks.workunit.client.1.smithi193.stdout:7/218: stat d12/f30 0 2024-04-25T15:49:34.180 INFO:tasks.workunit.client.0.smithi110.stdout:7/585: write d1b/f31 [951465,126097] 0 2024-04-25T15:49:34.181 INFO:tasks.workunit.client.0.smithi110.stdout:9/488: symlink d9/db/l9e 0 2024-04-25T15:49:34.184 INFO:tasks.workunit.client.0.smithi110.stdout:5/599: unlink d4/d2c/l28 0 2024-04-25T15:49:34.185 INFO:tasks.workunit.client.1.smithi193.stdout:0/284: mkdir d0/d49/d5/d7/dd/d46/d48/d62 0 2024-04-25T15:49:34.186 INFO:tasks.workunit.client.1.smithi193.stdout:0/285: write d0/d49/d5/d7/dd/d16/f30 [5403236,31764] 0 2024-04-25T15:49:34.187 INFO:tasks.workunit.client.1.smithi193.stdout:0/286: fsync d0/d49/d5/d7/dd/d16/f1c 0 2024-04-25T15:49:34.188 INFO:tasks.workunit.client.0.smithi110.stdout:6/523: unlink d1/db/d24/daa/cac 0 2024-04-25T15:49:34.189 INFO:tasks.workunit.client.1.smithi193.stdout:0/287: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60 4555169 1 2024-04-25T15:49:34.191 INFO:tasks.workunit.client.1.smithi193.stdout:4/277: creat db/d13/d1e/d1f/d22/f5a x:0 0 0 2024-04-25T15:49:34.192 INFO:tasks.workunit.client.1.smithi193.stdout:5/242: write d17/d18/d45/f47 [1493692,5726] 0 2024-04-25T15:49:34.192 INFO:tasks.workunit.client.0.smithi110.stdout:8/701: creat d14/d21/d26/d6a/d7e/da6/de8/d55/d8d/feb x:0 0 0 2024-04-25T15:49:34.196 INFO:tasks.workunit.client.0.smithi110.stdout:7/586: creat d1b/d29/d38/d55/d95/fc4 x:0 0 0 2024-04-25T15:49:34.196 INFO:tasks.workunit.client.1.smithi193.stdout:9/293: creat df/d2c/d2f/d4e/d1c/d26/f4f x:0 0 0 2024-04-25T15:49:34.197 INFO:tasks.workunit.client.0.smithi110.stdout:9/489: fdatasync d9/d13/f54 0 2024-04-25T15:49:34.198 INFO:tasks.workunit.client.0.smithi110.stdout:9/490: fsync d9/db/df/d2c/f70 0 2024-04-25T15:49:34.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/274: mknod d0/d9/d15/c70 0 2024-04-25T15:49:34.201 INFO:tasks.workunit.client.0.smithi110.stdout:6/524: rename d1/db/d22/d33/d63/la4 to d1/db/d24/d69/lbf 0 2024-04-25T15:49:34.202 INFO:tasks.workunit.client.0.smithi110.stdout:6/525: write d1/db/d22/d90/d76/dad/fbc [679684,48879] 0 2024-04-25T15:49:34.204 INFO:tasks.workunit.client.0.smithi110.stdout:3/640: link d0/d7/fe d0/d2e/d88/dbf/d99/dad/daf/fd9 0 2024-04-25T15:49:34.210 INFO:tasks.workunit.client.0.smithi110.stdout:7/587: mknod d1b/d29/d38/d8b/cc5 0 2024-04-25T15:49:34.211 INFO:tasks.workunit.client.1.smithi193.stdout:3/308: creat d8/d4d/f62 x:0 0 0 2024-04-25T15:49:34.212 INFO:tasks.workunit.client.0.smithi110.stdout:1/649: dwrite d0/d14/d5f/d7d/d72/f5e [0,4194304] 0 2024-04-25T15:49:34.214 INFO:tasks.workunit.client.0.smithi110.stdout:7/588: read d1b/d29/d38/d55/f6b [6001260,118471] 0 2024-04-25T15:49:34.217 INFO:tasks.workunit.client.0.smithi110.stdout:9/491: mkdir d9/d44/d5e/d80/d9f 0 2024-04-25T15:49:34.217 INFO:tasks.workunit.client.0.smithi110.stdout:8/702: write d14/d21/d23/f7c [605794,41440] 0 2024-04-25T15:49:34.219 INFO:tasks.workunit.client.0.smithi110.stdout:6/526: symlink d1/d48/lc0 0 2024-04-25T15:49:34.222 INFO:tasks.workunit.client.1.smithi193.stdout:8/240: link d2/f11 d2/f48 0 2024-04-25T15:49:34.225 INFO:tasks.workunit.client.1.smithi193.stdout:7/219: dwrite d12/d1e/f34 [0,4194304] 0 2024-04-25T15:49:34.225 INFO:tasks.workunit.client.0.smithi110.stdout:3/641: unlink d0/d16/dcb/f56 0 2024-04-25T15:49:34.225 INFO:tasks.workunit.client.0.smithi110.stdout:9/492: dwrite d9/db/df/d25/f79 [0,4194304] 0 2024-04-25T15:49:34.226 INFO:tasks.workunit.client.0.smithi110.stdout:9/493: chown d9/db/df/d25/f37 39715 1 2024-04-25T15:49:34.227 INFO:tasks.workunit.client.1.smithi193.stdout:4/278: mkdir db/d4b/d51/d54/d5b 0 2024-04-25T15:49:34.228 INFO:tasks.workunit.client.0.smithi110.stdout:9/494: readlink d9/db/d58/l72 0 2024-04-25T15:49:34.230 INFO:tasks.workunit.client.1.smithi193.stdout:8/241: dwrite d2/d1f/f2b [0,4194304] 0 2024-04-25T15:49:34.231 INFO:tasks.workunit.client.1.smithi193.stdout:4/279: write db/d13/d1e/d1f/d22/f2a [4859139,113740] 0 2024-04-25T15:49:34.231 INFO:tasks.workunit.client.0.smithi110.stdout:1/650: mkdir d0/d25/dc6 0 2024-04-25T15:49:34.232 INFO:tasks.workunit.client.0.smithi110.stdout:5/600: dwrite d4/d1d/f65 [0,4194304] 0 2024-04-25T15:49:34.234 INFO:tasks.workunit.client.1.smithi193.stdout:0/288: dwrite d0/d49/d5/d7/dd/d16/d18/d28/f52 [0,4194304] 0 2024-04-25T15:49:34.234 INFO:tasks.workunit.client.1.smithi193.stdout:5/243: creat d17/d18/d45/f56 x:0 0 0 2024-04-25T15:49:34.240 INFO:tasks.workunit.client.0.smithi110.stdout:8/703: rename d14/d21/d26/d6a/d7e/da6/de8/c54 to d14/d21/d26/dab/dac/cec 0 2024-04-25T15:49:34.242 INFO:tasks.workunit.client.0.smithi110.stdout:6/527: write d1/db/d24/d2d/f34 [1629072,83021] 0 2024-04-25T15:49:34.242 INFO:tasks.workunit.client.1.smithi193.stdout:1/227: link dc/d3c/l45 dc/d3c/l46 0 2024-04-25T15:49:34.243 INFO:tasks.workunit.client.1.smithi193.stdout:1/228: dread - dc/d3c/f3e zero size 2024-04-25T15:49:34.246 INFO:tasks.workunit.client.1.smithi193.stdout:3/309: truncate d8/d13/d14/d23/f48 755099 0 2024-04-25T15:49:34.248 INFO:tasks.workunit.client.0.smithi110.stdout:3/642: fdatasync d0/d16/dcb/d2f/d33/f36 0 2024-04-25T15:49:34.253 INFO:tasks.workunit.client.1.smithi193.stdout:7/220: creat d12/f52 x:0 0 0 2024-04-25T15:49:34.255 INFO:tasks.workunit.client.0.smithi110.stdout:9/495: mkdir d9/db/df/d2c/d3c/d56/da0 0 2024-04-25T15:49:34.256 INFO:tasks.workunit.client.0.smithi110.stdout:6/528: dwrite f0 [4194304,4194304] 0 2024-04-25T15:49:34.257 INFO:tasks.workunit.client.0.smithi110.stdout:8/704: dwrite d14/d21/d26/d6a/d7e/da6/de8/d55/d8d/feb [0,4194304] 0 2024-04-25T15:49:34.258 INFO:tasks.workunit.client.0.smithi110.stdout:1/651: mknod d0/d23/cc7 0 2024-04-25T15:49:34.261 INFO:tasks.workunit.client.0.smithi110.stdout:1/652: chown d0/d14/d5f/d7d/d72/c95 2956236 1 2024-04-25T15:49:34.262 INFO:tasks.workunit.client.0.smithi110.stdout:5/601: fdatasync d4/fd 0 2024-04-25T15:49:34.266 INFO:tasks.workunit.client.1.smithi193.stdout:4/280: creat db/d4b/d51/d54/f5c x:0 0 0 2024-04-25T15:49:34.267 INFO:tasks.workunit.client.1.smithi193.stdout:4/281: dread - db/f59 zero size 2024-04-25T15:49:34.270 INFO:tasks.workunit.client.1.smithi193.stdout:8/242: creat d2/d3/d20/d45/f49 x:0 0 0 2024-04-25T15:49:34.271 INFO:tasks.workunit.client.0.smithi110.stdout:7/589: dwrite fb [0,4194304] 0 2024-04-25T15:49:34.273 INFO:tasks.workunit.client.0.smithi110.stdout:5/602: read d4/f6 [1556351,74281] 0 2024-04-25T15:49:34.276 INFO:tasks.workunit.client.1.smithi193.stdout:6/270: rename dd/d12/d19/d1c/d40/l3a to dd/l4b 0 2024-04-25T15:49:34.279 INFO:tasks.workunit.client.1.smithi193.stdout:2/275: mknod d0/d37/c71 0 2024-04-25T15:49:34.284 INFO:tasks.workunit.client.1.smithi193.stdout:2/276: dwrite d0/f11 [0,4194304] 0 2024-04-25T15:49:34.286 INFO:tasks.workunit.client.1.smithi193.stdout:7/221: creat d12/d31/d39/f53 x:0 0 0 2024-04-25T15:49:34.291 INFO:tasks.workunit.client.1.smithi193.stdout:3/310: write d8/d30/f47 [3024307,34892] 0 2024-04-25T15:49:34.293 INFO:tasks.workunit.client.1.smithi193.stdout:3/311: write d8/d13/d4b/f5d [3958795,21924] 0 2024-04-25T15:49:34.294 INFO:tasks.workunit.client.1.smithi193.stdout:5/244: rename f13 to d17/f57 0 2024-04-25T15:49:34.296 INFO:tasks.workunit.client.1.smithi193.stdout:8/243: write d2/d36/f40 [5581892,127086] 0 2024-04-25T15:49:34.298 INFO:tasks.workunit.client.1.smithi193.stdout:0/289: symlink d0/d49/d5/d7/dd/d16/d26/d5e/l63 0 2024-04-25T15:49:34.300 INFO:tasks.workunit.client.1.smithi193.stdout:7/222: symlink d12/d31/l54 0 2024-04-25T15:49:34.302 INFO:tasks.workunit.client.0.smithi110.stdout:6/529: creat d1/db/d22/d33/d63/dbd/da1/fc1 x:0 0 0 2024-04-25T15:49:34.303 INFO:tasks.workunit.client.1.smithi193.stdout:9/294: getdents df/d2c/d2f/d4e 0 2024-04-25T15:49:34.303 INFO:tasks.workunit.client.1.smithi193.stdout:9/295: chown l2 27608 1 2024-04-25T15:49:34.304 INFO:tasks.workunit.client.0.smithi110.stdout:8/705: creat d14/d21/d26/d6a/d7e/da6/fed x:0 0 0 2024-04-25T15:49:34.305 INFO:tasks.workunit.client.1.smithi193.stdout:6/271: symlink dd/l4c 0 2024-04-25T15:49:34.305 INFO:tasks.workunit.client.1.smithi193.stdout:0/290: dwrite d0/d49/d5/d7/dd/d16/f3a [0,4194304] 0 2024-04-25T15:49:34.308 INFO:tasks.workunit.client.0.smithi110.stdout:6/530: dwrite d1/db/d24/fb9 [0,4194304] 0 2024-04-25T15:49:34.311 INFO:tasks.workunit.client.1.smithi193.stdout:5/245: dread d17/d18/f24 [0,4194304] 0 2024-04-25T15:49:34.312 INFO:tasks.workunit.client.1.smithi193.stdout:0/291: dwrite d0/f12 [0,4194304] 0 2024-04-25T15:49:34.314 INFO:tasks.workunit.client.1.smithi193.stdout:0/292: stat d0/d49/d5/d7/dd/d46/d48/f5b 0 2024-04-25T15:49:34.315 INFO:tasks.workunit.client.1.smithi193.stdout:4/282: rename db/f35 to db/d13/d1e/d1f/d22/f5d 0 2024-04-25T15:49:34.316 INFO:tasks.workunit.client.1.smithi193.stdout:4/283: readlink db/d13/l1b 0 2024-04-25T15:49:34.316 INFO:tasks.workunit.client.0.smithi110.stdout:7/590: mkdir d1b/d29/d38/d8b/d89/dc6 0 2024-04-25T15:49:34.318 INFO:tasks.workunit.client.0.smithi110.stdout:7/591: dread - d1b/d29/d38/d55/d7f/f64 zero size 2024-04-25T15:49:34.320 INFO:tasks.workunit.client.0.smithi110.stdout:7/592: truncate d1b/d29/d38/d8b/d89/f96 47875 0 2024-04-25T15:49:34.321 INFO:tasks.workunit.client.1.smithi193.stdout:8/244: mkdir d2/d3/d20/d45/d4a 0 2024-04-25T15:49:34.323 INFO:tasks.workunit.client.1.smithi193.stdout:1/229: creat dc/f47 x:0 0 0 2024-04-25T15:49:34.326 INFO:tasks.workunit.client.0.smithi110.stdout:3/643: rename d0/f75 to d0/d7/d7d/fda 0 2024-04-25T15:49:34.327 INFO:tasks.workunit.client.0.smithi110.stdout:3/644: chown d0/d2e/d88/dbf/d92/fbb 125 1 2024-04-25T15:49:34.328 INFO:tasks.workunit.client.1.smithi193.stdout:2/277: mknod d0/c72 0 2024-04-25T15:49:34.329 INFO:tasks.workunit.client.1.smithi193.stdout:1/230: dwrite dc/f12 [0,4194304] 0 2024-04-25T15:49:34.329 INFO:tasks.workunit.client.0.smithi110.stdout:8/706: creat d14/d21/d26/dab/fee x:0 0 0 2024-04-25T15:49:34.331 INFO:tasks.workunit.client.1.smithi193.stdout:7/223: write d12/d13/f41 [2688269,95537] 0 2024-04-25T15:49:34.333 INFO:tasks.workunit.client.1.smithi193.stdout:7/224: dread - d12/d1e/f3c zero size 2024-04-25T15:49:34.336 INFO:tasks.workunit.client.1.smithi193.stdout:9/296: mknod df/d2d/c50 0 2024-04-25T15:49:34.337 INFO:tasks.workunit.client.1.smithi193.stdout:3/312: write d8/f10 [2670322,84812] 0 2024-04-25T15:49:34.337 INFO:tasks.workunit.client.1.smithi193.stdout:2/278: dwrite d0/d42/d51/d4f/f69 [0,4194304] 0 2024-04-25T15:49:34.340 INFO:tasks.workunit.client.1.smithi193.stdout:2/279: dread - d0/d9/d15/d31/d50/f5c zero size 2024-04-25T15:49:34.342 INFO:tasks.workunit.client.1.smithi193.stdout:7/225: dwrite d12/f36 [0,4194304] 0 2024-04-25T15:49:34.343 INFO:tasks.workunit.client.1.smithi193.stdout:2/280: chown d0/d37/c71 14587537 1 2024-04-25T15:49:34.344 INFO:tasks.workunit.client.1.smithi193.stdout:7/226: read d12/d38/f4e [1432627,70139] 0 2024-04-25T15:49:34.344 INFO:tasks.workunit.client.1.smithi193.stdout:2/281: chown d0/d37/d25/d5f/d3a/l6c 1677860 1 2024-04-25T15:49:34.346 INFO:tasks.workunit.client.1.smithi193.stdout:6/272: dread dd/d12/d19/d1c/d40/f25 [4194304,4194304] 0 2024-04-25T15:49:34.346 INFO:tasks.workunit.client.1.smithi193.stdout:2/282: write d0/d9/d15/f35 [1218874,49849] 0 2024-04-25T15:49:34.347 INFO:tasks.workunit.client.0.smithi110.stdout:5/603: dwrite d4/d1d/d1f/f42 [0,4194304] 0 2024-04-25T15:49:34.353 INFO:tasks.workunit.client.0.smithi110.stdout:1/653: rename d0/d25/dc6 to d0/d51/dc8 0 2024-04-25T15:49:34.358 INFO:tasks.workunit.client.0.smithi110.stdout:3/645: unlink d0/d16/dcb/d50/f6f 0 2024-04-25T15:49:34.360 INFO:tasks.workunit.client.1.smithi193.stdout:4/284: symlink db/d32/d40/l5e 0 2024-04-25T15:49:34.361 INFO:tasks.workunit.client.0.smithi110.stdout:9/496: truncate d9/db/df/d25/f79 3106741 0 2024-04-25T15:49:34.361 INFO:tasks.workunit.client.0.smithi110.stdout:1/654: dwrite d0/d51/d6b/faf [0,4194304] 0 2024-04-25T15:49:34.366 INFO:tasks.workunit.client.0.smithi110.stdout:1/655: read d0/d14/f1d [1421963,92034] 0 2024-04-25T15:49:34.366 INFO:tasks.workunit.client.0.smithi110.stdout:1/656: chown d0/d51/f98 374595 1 2024-04-25T15:49:34.370 INFO:tasks.workunit.client.1.smithi193.stdout:2/283: dread d0/d42/d51/f40 [4194304,4194304] 0 2024-04-25T15:49:34.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/231: rmdir dc/d35 39 2024-04-25T15:49:34.375 INFO:tasks.workunit.client.1.smithi193.stdout:0/293: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f59 [0,4194304] 0 2024-04-25T15:49:34.380 INFO:tasks.workunit.client.1.smithi193.stdout:0/294: dread d0/d49/d5/d7/dd/d16/d18/f2a [0,4194304] 0 2024-04-25T15:49:34.383 INFO:tasks.workunit.client.0.smithi110.stdout:8/707: creat d14/d32/dd6/fef x:0 0 0 2024-04-25T15:49:34.385 INFO:tasks.workunit.client.0.smithi110.stdout:5/604: write d4/d1d/d1f/d35/f39 [2794008,57253] 0 2024-04-25T15:49:34.387 INFO:tasks.workunit.client.0.smithi110.stdout:6/531: rmdir d1/db/d22/d33/d63/dbd/db2 0 2024-04-25T15:49:34.388 INFO:tasks.workunit.client.0.smithi110.stdout:6/532: truncate d1/fe 3409741 0 2024-04-25T15:49:34.389 INFO:tasks.workunit.client.1.smithi193.stdout:7/227: rename d12/d1c/d45/c4f to d12/d31/d39/c55 0 2024-04-25T15:49:34.390 INFO:tasks.workunit.client.1.smithi193.stdout:7/228: fsync d12/d13/f40 0 2024-04-25T15:49:34.393 INFO:tasks.workunit.client.1.smithi193.stdout:5/246: truncate d17/f39 3833390 0 2024-04-25T15:49:34.394 INFO:tasks.workunit.client.1.smithi193.stdout:4/285: creat db/d32/f5f x:0 0 0 2024-04-25T15:49:34.395 INFO:tasks.workunit.client.0.smithi110.stdout:6/533: dwrite d1/fe [0,4194304] 0 2024-04-25T15:49:34.396 INFO:tasks.workunit.client.0.smithi110.stdout:9/497: symlink d9/la1 0 2024-04-25T15:49:34.399 INFO:tasks.workunit.client.1.smithi193.stdout:9/297: mknod df/c51 0 2024-04-25T15:49:34.401 INFO:tasks.workunit.client.1.smithi193.stdout:9/298: write df/d2c/d2f/d4e/d1c/d26/d3f/f4c [732336,104746] 0 2024-04-25T15:49:34.403 INFO:tasks.workunit.client.0.smithi110.stdout:5/605: dwrite d4/f25 [0,4194304] 0 2024-04-25T15:49:34.406 INFO:tasks.workunit.client.1.smithi193.stdout:3/313: fdatasync d8/d30/f36 0 2024-04-25T15:49:34.409 INFO:tasks.workunit.client.0.smithi110.stdout:3/646: rename d0/d16/dcb/d2f/d4c/c55 to d0/cdb 0 2024-04-25T15:49:34.411 INFO:tasks.workunit.client.0.smithi110.stdout:3/647: readlink d0/d7/dd0/ld8 0 2024-04-25T15:49:34.413 INFO:tasks.workunit.client.0.smithi110.stdout:5/606: dwrite d4/d1d/d1f/d35/f76 [0,4194304] 0 2024-04-25T15:49:34.415 INFO:tasks.workunit.client.0.smithi110.stdout:8/708: truncate d14/d21/d26/f98 600803 0 2024-04-25T15:49:34.417 INFO:tasks.workunit.client.1.smithi193.stdout:6/273: rename f6 to dd/d12/f4d 0 2024-04-25T15:49:34.418 INFO:tasks.workunit.client.0.smithi110.stdout:6/534: mkdir d1/db/d22/d33/d63/dc2 0 2024-04-25T15:49:34.418 INFO:tasks.workunit.client.1.smithi193.stdout:6/274: readlink dd/d12/l18 0 2024-04-25T15:49:34.419 INFO:tasks.workunit.client.1.smithi193.stdout:6/275: fdatasync dd/f1b 0 2024-04-25T15:49:34.419 INFO:tasks.workunit.client.1.smithi193.stdout:7/229: mknod d12/d13/c56 0 2024-04-25T15:49:34.426 INFO:tasks.workunit.client.0.smithi110.stdout:6/535: dwrite d1/d12/daf/fbb [0,4194304] 0 2024-04-25T15:49:34.432 INFO:tasks.workunit.client.0.smithi110.stdout:3/648: symlink d0/d16/ldc 0 2024-04-25T15:49:34.433 INFO:tasks.workunit.client.1.smithi193.stdout:1/232: mknod dc/d35/d3d/c48 0 2024-04-25T15:49:34.435 INFO:tasks.workunit.client.1.smithi193.stdout:9/299: rmdir df/d2c/d2f/d33 39 2024-04-25T15:49:34.437 INFO:tasks.workunit.client.0.smithi110.stdout:1/657: write d0/d23/f30 [1606565,4283] 0 2024-04-25T15:49:34.438 INFO:tasks.workunit.client.0.smithi110.stdout:9/498: mknod d9/db/df/ca2 0 2024-04-25T15:49:34.439 INFO:tasks.workunit.client.1.smithi193.stdout:1/233: dwrite dc/d35/d3d/f42 [0,4194304] 0 2024-04-25T15:49:34.441 INFO:tasks.workunit.client.1.smithi193.stdout:3/314: symlink d8/d13/d14/d23/l63 0 2024-04-25T15:49:34.442 INFO:tasks.workunit.client.1.smithi193.stdout:0/295: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/l64 0 2024-04-25T15:49:34.443 INFO:tasks.workunit.client.0.smithi110.stdout:6/536: dread - d1/d79/f7a zero size 2024-04-25T15:49:34.445 INFO:tasks.workunit.client.1.smithi193.stdout:8/245: rename d2/d3/d20/d45/l12 to d2/d18/l4b 0 2024-04-25T15:49:34.445 INFO:tasks.workunit.client.0.smithi110.stdout:5/607: dread d4/d2c/f19 [0,4194304] 0 2024-04-25T15:49:34.449 INFO:tasks.workunit.client.1.smithi193.stdout:6/276: readlink dd/d12/d19/l39 0 2024-04-25T15:49:34.450 INFO:tasks.workunit.client.0.smithi110.stdout:1/658: chown d0/d23/f13 502 1 2024-04-25T15:49:34.451 INFO:tasks.workunit.client.0.smithi110.stdout:6/537: dwrite d1/db/d22/d33/d43/f60 [0,4194304] 0 2024-04-25T15:49:34.451 INFO:tasks.workunit.client.1.smithi193.stdout:7/230: symlink d12/d1c/l57 0 2024-04-25T15:49:34.452 INFO:tasks.workunit.client.1.smithi193.stdout:6/277: dread dd/f11 [0,4194304] 0 2024-04-25T15:49:34.452 INFO:tasks.workunit.client.1.smithi193.stdout:7/231: fdatasync d12/d31/d39/f53 0 2024-04-25T15:49:34.452 INFO:tasks.workunit.client.1.smithi193.stdout:6/278: read dd/f2b [1063759,87213] 0 2024-04-25T15:49:34.454 INFO:tasks.workunit.client.1.smithi193.stdout:6/279: rename dd/d12 to dd/d12/d19/d1c/d40/d4e 22 2024-04-25T15:49:34.462 INFO:tasks.workunit.client.0.smithi110.stdout:3/649: dwrite d0/d2e/f87 [0,4194304] 0 2024-04-25T15:49:34.462 INFO:tasks.workunit.client.0.smithi110.stdout:5/608: dread d4/d1d/d1f/d35/d3d/f53 [0,4194304] 0 2024-04-25T15:49:34.466 INFO:tasks.workunit.client.1.smithi193.stdout:1/234: rmdir dc/d35/d3d 39 2024-04-25T15:49:34.467 INFO:tasks.workunit.client.1.smithi193.stdout:1/235: write dc/f30 [4668962,55228] 0 2024-04-25T15:49:34.468 INFO:tasks.workunit.client.1.smithi193.stdout:3/315: rmdir d8/d30/d31 39 2024-04-25T15:49:34.468 INFO:tasks.workunit.client.1.smithi193.stdout:3/316: truncate f2 5159686 0 2024-04-25T15:49:34.469 INFO:tasks.workunit.client.1.smithi193.stdout:3/317: fdatasync d8/d13/d14/d27/f5a 0 2024-04-25T15:49:34.472 INFO:tasks.workunit.client.0.smithi110.stdout:8/709: getdents d14/d21/d26/d6a/d7e/da6/de8/d55/d8d 0 2024-04-25T15:49:34.472 INFO:tasks.workunit.client.1.smithi193.stdout:8/246: read d2/d3/ff [2755531,116328] 0 2024-04-25T15:49:34.473 INFO:tasks.workunit.client.0.smithi110.stdout:8/710: write d14/d32/d36/d48/fb3 [1499577,72594] 0 2024-04-25T15:49:34.474 INFO:tasks.workunit.client.0.smithi110.stdout:6/538: mknod d1/dd/db5/cc3 0 2024-04-25T15:49:34.475 INFO:tasks.workunit.client.1.smithi193.stdout:7/232: dread - fc zero size 2024-04-25T15:49:34.477 INFO:tasks.workunit.client.0.smithi110.stdout:3/650: write d0/d2e/f4f [2706330,83087] 0 2024-04-25T15:49:34.478 INFO:tasks.workunit.client.1.smithi193.stdout:6/280: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f 0 2024-04-25T15:49:34.479 INFO:tasks.workunit.client.0.smithi110.stdout:9/499: rmdir d9/d44/d5e/d80/d9f 0 2024-04-25T15:49:34.485 INFO:tasks.workunit.client.0.smithi110.stdout:3/651: dwrite d0/d2e/d88/dbf/fbe [0,4194304] 0 2024-04-25T15:49:34.489 INFO:tasks.workunit.client.0.smithi110.stdout:3/652: read d0/d7/f6c [156846,126353] 0 2024-04-25T15:49:34.489 INFO:tasks.workunit.client.0.smithi110.stdout:8/711: chown d14/c1a 196 1 2024-04-25T15:49:34.489 INFO:tasks.workunit.client.0.smithi110.stdout:9/500: dwrite d9/db/df/d55/f6b [0,4194304] 0 2024-04-25T15:49:34.492 INFO:tasks.workunit.client.0.smithi110.stdout:8/712: dread - d14/d21/d26/d6a/d7e/da6/de8/d55/d8d/fad zero size 2024-04-25T15:49:34.502 INFO:tasks.workunit.client.0.smithi110.stdout:6/539: mknod d1/db/d24/cc4 0 2024-04-25T15:49:34.503 INFO:tasks.workunit.client.0.smithi110.stdout:6/540: read - d1/db/d24/d2d/fb0 zero size 2024-04-25T15:49:34.503 INFO:tasks.workunit.client.0.smithi110.stdout:1/659: truncate d0/d14/d5f/d6f/fa3 2531159 0 2024-04-25T15:49:34.504 INFO:tasks.workunit.client.1.smithi193.stdout:4/286: rmdir db/d4b/d51/d54/d5b 0 2024-04-25T15:49:34.504 INFO:tasks.workunit.client.1.smithi193.stdout:0/296: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/f34 [0,4194304] 0 2024-04-25T15:49:34.505 INFO:tasks.workunit.client.0.smithi110.stdout:8/713: creat d14/d32/d36/db7/dc9/ff0 x:0 0 0 2024-04-25T15:49:34.505 INFO:tasks.workunit.client.1.smithi193.stdout:2/284: getdents d0/d37/d25/d5f 0 2024-04-25T15:49:34.506 INFO:tasks.workunit.client.1.smithi193.stdout:9/300: fdatasync df/d2c/d2f/d33/f3c 0 2024-04-25T15:49:34.509 INFO:tasks.workunit.client.0.smithi110.stdout:9/501: symlink d9/la3 0 2024-04-25T15:49:34.510 INFO:tasks.workunit.client.1.smithi193.stdout:7/233: dwrite d12/d1e/f42 [0,4194304] 0 2024-04-25T15:49:34.510 INFO:tasks.workunit.client.1.smithi193.stdout:8/247: write d2/f48 [2347427,48222] 0 2024-04-25T15:49:34.511 INFO:tasks.workunit.client.0.smithi110.stdout:6/541: dread - d1/f5e zero size 2024-04-25T15:49:34.512 INFO:tasks.workunit.client.1.smithi193.stdout:5/247: link d17/d18/d45/l51 d17/l58 0 2024-04-25T15:49:34.515 INFO:tasks.workunit.client.1.smithi193.stdout:0/297: chown d0/d49/d5/d7/dd/d16/d18/f22 415250010 1 2024-04-25T15:49:34.516 INFO:tasks.workunit.client.1.smithi193.stdout:8/248: write d2/f48 [1033214,54249] 0 2024-04-25T15:49:34.516 INFO:tasks.workunit.client.0.smithi110.stdout:1/660: symlink d0/d25/d33/d8e/lc9 0 2024-04-25T15:49:34.518 INFO:tasks.workunit.client.0.smithi110.stdout:1/661: readlink d0/d14/d5f/d7d/d72/l9f 0 2024-04-25T15:49:34.518 INFO:tasks.workunit.client.0.smithi110.stdout:5/609: getdents d4/d1d/d1f/d35/d7f/d92 0 2024-04-25T15:49:34.518 INFO:tasks.workunit.client.1.smithi193.stdout:6/281: dwrite dd/d12/d19/d1c/f20 [8388608,4194304] 0 2024-04-25T15:49:34.519 INFO:tasks.workunit.client.0.smithi110.stdout:5/610: write d4/f22 [1696366,130631] 0 2024-04-25T15:49:34.520 INFO:tasks.workunit.client.1.smithi193.stdout:9/301: dread fc [4194304,4194304] 0 2024-04-25T15:49:34.522 INFO:tasks.workunit.client.1.smithi193.stdout:9/302: read - df/d2c/d2f/f49 zero size 2024-04-25T15:49:34.523 INFO:tasks.workunit.client.0.smithi110.stdout:5/611: dread d4/d1d/d1f/d35/d3d/d52/d75/fb5 [0,4194304] 0 2024-04-25T15:49:34.523 INFO:tasks.workunit.client.1.smithi193.stdout:6/282: read dd/d12/d19/d1c/d40/d3c/d3f/f41 [79490,42550] 0 2024-04-25T15:49:34.525 INFO:tasks.workunit.client.1.smithi193.stdout:4/287: rename db/d13/d1e/d1f/d22/f55 to db/d32/d45/f60 0 2024-04-25T15:49:34.530 INFO:tasks.workunit.client.1.smithi193.stdout:7/234: creat d12/d31/f58 x:0 0 0 2024-04-25T15:49:34.531 INFO:tasks.workunit.client.1.smithi193.stdout:1/236: creat dc/f49 x:0 0 0 2024-04-25T15:49:34.532 INFO:tasks.workunit.client.1.smithi193.stdout:1/237: write dc/f49 [994804,59128] 0 2024-04-25T15:49:34.533 INFO:tasks.workunit.client.1.smithi193.stdout:5/248: write d17/d18/d45/f47 [2660351,1550] 0 2024-04-25T15:49:34.535 INFO:tasks.workunit.client.0.smithi110.stdout:9/502: rename d9/db/f11 to d9/db/df/d2c/d3c/fa4 0 2024-04-25T15:49:34.535 INFO:tasks.workunit.client.0.smithi110.stdout:8/714: dwrite d14/d21/d26/d6a/d7e/da6/de8/f50 [4194304,4194304] 0 2024-04-25T15:49:34.537 INFO:tasks.workunit.client.0.smithi110.stdout:5/612: creat d4/d1d/d1f/d35/d7f/d92/fca x:0 0 0 2024-04-25T15:49:34.538 INFO:tasks.workunit.client.1.smithi193.stdout:0/298: rmdir d0/d49/d5/d7/dd/d16/d18/d28 39 2024-04-25T15:49:34.539 INFO:tasks.workunit.client.0.smithi110.stdout:8/715: write d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/fa1 [1906051,99900] 0 2024-04-25T15:49:34.540 INFO:tasks.workunit.client.0.smithi110.stdout:5/613: write d4/d1d/d1f/d35/d3d/d52/d97/fb1 [187735,95787] 0 2024-04-25T15:49:34.541 INFO:tasks.workunit.client.0.smithi110.stdout:6/542: mknod d1/d12/db8/cc5 0 2024-04-25T15:49:34.542 INFO:tasks.workunit.client.1.smithi193.stdout:9/303: unlink fc 0 2024-04-25T15:49:34.542 INFO:tasks.workunit.client.0.smithi110.stdout:8/716: dread - d14/d21/d23/fd3 zero size 2024-04-25T15:49:34.542 INFO:tasks.workunit.client.0.smithi110.stdout:3/653: link d0/d16/dcb/d2f/d33/c37 d0/d2e/d88/dbf/cdd 0 2024-04-25T15:49:34.543 INFO:tasks.workunit.client.1.smithi193.stdout:6/283: dread dd/f3d [0,4194304] 0 2024-04-25T15:49:34.543 INFO:tasks.workunit.client.0.smithi110.stdout:6/543: stat d1/db/d22/d33/d63/dbd 0 2024-04-25T15:49:34.548 INFO:tasks.workunit.client.1.smithi193.stdout:0/299: dwrite d0/d49/d5/d7/dd/d16/d18/f22 [0,4194304] 0 2024-04-25T15:49:34.551 INFO:tasks.workunit.client.1.smithi193.stdout:7/235: mknod d12/d1e/c59 0 2024-04-25T15:49:34.552 INFO:tasks.workunit.client.1.smithi193.stdout:9/304: dwrite df/d11/f2e [4194304,4194304] 0 2024-04-25T15:49:34.554 INFO:tasks.workunit.client.1.smithi193.stdout:1/238: dread dc/d35/f3a [0,4194304] 0 2024-04-25T15:49:34.555 INFO:tasks.workunit.client.0.smithi110.stdout:8/717: dread d14/d21/f72 [0,4194304] 0 2024-04-25T15:49:34.558 INFO:tasks.workunit.client.0.smithi110.stdout:8/718: readlink d14/d21/d26/d6a/d7e/da6/de8/d71/le6 0 2024-04-25T15:49:34.558 INFO:tasks.workunit.client.1.smithi193.stdout:5/249: creat d17/f59 x:0 0 0 2024-04-25T15:49:34.560 INFO:tasks.workunit.client.0.smithi110.stdout:3/654: creat d0/d2e/d88/fde x:0 0 0 2024-04-25T15:49:34.562 INFO:tasks.workunit.client.1.smithi193.stdout:2/285: creat d0/d9/f73 x:0 0 0 2024-04-25T15:49:34.568 INFO:tasks.workunit.client.0.smithi110.stdout:5/614: mkdir d4/d1d/d1f/d35/d7f/d92/d9b/dcb 0 2024-04-25T15:49:34.572 INFO:tasks.workunit.client.0.smithi110.stdout:9/503: dwrite d9/db/df/f68 [0,4194304] 0 2024-04-25T15:49:34.573 INFO:tasks.workunit.client.1.smithi193.stdout:6/284: dread dd/f2b [4194304,4194304] 0 2024-04-25T15:49:34.576 INFO:tasks.workunit.client.1.smithi193.stdout:8/249: truncate d2/f11 1092831 0 2024-04-25T15:49:34.577 INFO:tasks.workunit.client.1.smithi193.stdout:3/318: sync 2024-04-25T15:49:34.579 INFO:tasks.workunit.client.1.smithi193.stdout:0/300: creat d0/d49/d5/d7/dd/d16/d26/d5e/f65 x:0 0 0 2024-04-25T15:49:34.579 INFO:tasks.workunit.client.1.smithi193.stdout:7/236: creat d12/d1c/f5a x:0 0 0 2024-04-25T15:49:34.580 INFO:tasks.workunit.client.1.smithi193.stdout:7/237: read f2 [1053347,120500] 0 2024-04-25T15:49:34.582 INFO:tasks.workunit.client.1.smithi193.stdout:5/250: rename d17/d18/c33 to d17/d53/c5a 0 2024-04-25T15:49:34.584 INFO:tasks.workunit.client.1.smithi193.stdout:8/250: creat d2/d3/d3a/f4c x:0 0 0 2024-04-25T15:49:34.585 INFO:tasks.workunit.client.1.smithi193.stdout:4/288: creat db/f61 x:0 0 0 2024-04-25T15:49:34.586 INFO:tasks.workunit.client.1.smithi193.stdout:0/301: symlink d0/d49/d5/d7/dd/d16/d26/l66 0 2024-04-25T15:49:34.588 INFO:tasks.workunit.client.0.smithi110.stdout:1/662: getdents d0/d23 0 2024-04-25T15:49:34.589 INFO:tasks.workunit.client.0.smithi110.stdout:5/615: creat d4/d1d/d1f/d35/d3d/d93/fcc x:0 0 0 2024-04-25T15:49:34.589 INFO:tasks.workunit.client.1.smithi193.stdout:8/251: write d2/d3/d20/d45/f49 [961880,10662] 0 2024-04-25T15:49:34.589 INFO:tasks.workunit.client.1.smithi193.stdout:1/239: mkdir dc/d4a 0 2024-04-25T15:49:34.591 INFO:tasks.workunit.client.1.smithi193.stdout:4/289: truncate db/d32/d45/f60 511926 0 2024-04-25T15:49:34.591 INFO:tasks.workunit.client.1.smithi193.stdout:6/285: dwrite dd/d12/d2d/f37 [0,4194304] 0 2024-04-25T15:49:34.592 INFO:tasks.workunit.client.1.smithi193.stdout:2/286: symlink d0/d37/l74 0 2024-04-25T15:49:34.593 INFO:tasks.workunit.client.1.smithi193.stdout:6/286: truncate dd/f42 1268633 0 2024-04-25T15:49:34.593 INFO:tasks.workunit.client.1.smithi193.stdout:5/251: chown d17/d18/d3e/f3f 47599 1 2024-04-25T15:49:34.595 INFO:tasks.workunit.client.0.smithi110.stdout:9/504: dwrite d9/d44/d5e/d80/f8a [0,4194304] 0 2024-04-25T15:49:34.595 INFO:tasks.workunit.client.0.smithi110.stdout:1/663: symlink d0/d51/lca 0 2024-04-25T15:49:34.595 INFO:tasks.workunit.client.0.smithi110.stdout:9/505: stat d9/db 0 2024-04-25T15:49:34.597 INFO:tasks.workunit.client.1.smithi193.stdout:3/319: truncate d8/d13/d14/d23/f48 1742811 0 2024-04-25T15:49:34.601 INFO:tasks.workunit.client.0.smithi110.stdout:5/616: dread d4/d1d/d1f/d35/d3d/d52/d75/fb5 [0,4194304] 0 2024-04-25T15:49:34.601 INFO:tasks.workunit.client.1.smithi193.stdout:1/240: rmdir dc/d3c 39 2024-04-25T15:49:34.604 INFO:tasks.workunit.client.0.smithi110.stdout:8/719: dread d14/d21/d26/d6a/d7e/da6/fa8 [0,4194304] 0 2024-04-25T15:49:34.605 INFO:tasks.workunit.client.1.smithi193.stdout:7/238: dread d12/d1e/f25 [0,4194304] 0 2024-04-25T15:49:34.606 INFO:tasks.workunit.client.1.smithi193.stdout:7/239: chown d12/d13/c16 53396 1 2024-04-25T15:49:34.606 INFO:tasks.workunit.client.1.smithi193.stdout:7/240: dread - d12/d31/d39/f53 zero size 2024-04-25T15:49:34.607 INFO:tasks.workunit.client.0.smithi110.stdout:8/720: stat d14/d21/d23/f24 0 2024-04-25T15:49:34.607 INFO:tasks.workunit.client.0.smithi110.stdout:9/506: dwrite d9/d13/f1c [0,4194304] 0 2024-04-25T15:49:34.609 INFO:tasks.workunit.client.1.smithi193.stdout:4/290: mknod db/d13/d1e/d1f/d22/c62 0 2024-04-25T15:49:34.609 INFO:tasks.workunit.client.0.smithi110.stdout:9/507: write d9/db/df/d55/f60 [944484,35276] 0 2024-04-25T15:49:34.611 INFO:tasks.workunit.client.0.smithi110.stdout:8/721: write d14/d21/f62 [532486,69153] 0 2024-04-25T15:49:34.615 INFO:tasks.workunit.client.0.smithi110.stdout:9/508: read d9/f5f [2660869,3948] 0 2024-04-25T15:49:34.615 INFO:tasks.workunit.client.1.smithi193.stdout:6/287: creat dd/d2f/f50 x:0 0 0 2024-04-25T15:49:34.616 INFO:tasks.workunit.client.1.smithi193.stdout:6/288: write dd/d12/d19/d1c/f20 [71819,18976] 0 2024-04-25T15:49:34.617 INFO:tasks.workunit.client.1.smithi193.stdout:6/289: fsync dd/d12/d19/d1c/d40/f23 0 2024-04-25T15:49:34.618 INFO:tasks.workunit.client.1.smithi193.stdout:6/290: write dd/d12/f35 [130301,76334] 0 2024-04-25T15:49:34.618 INFO:tasks.workunit.client.0.smithi110.stdout:5/617: rename d4/d2c/f19 to d4/d1d/d1f/d35/d7f/d92/fcd 0 2024-04-25T15:49:34.619 INFO:tasks.workunit.client.1.smithi193.stdout:6/291: readlink dd/d12/l27 0 2024-04-25T15:49:34.619 INFO:tasks.workunit.client.1.smithi193.stdout:9/305: getdents df/d2c/d2f/d4e/d1c/d30 0 2024-04-25T15:49:34.619 INFO:tasks.workunit.client.0.smithi110.stdout:6/544: dwrite d1/db/d24/f66 [0,4194304] 0 2024-04-25T15:49:34.622 INFO:tasks.workunit.client.1.smithi193.stdout:5/252: unlink d17/l1e 0 2024-04-25T15:49:34.623 INFO:tasks.workunit.client.1.smithi193.stdout:5/253: stat d17/d18/l19 0 2024-04-25T15:49:34.623 INFO:tasks.workunit.client.1.smithi193.stdout:5/254: write d17/f59 [32359,67773] 0 2024-04-25T15:49:34.626 INFO:tasks.workunit.client.1.smithi193.stdout:3/320: creat d8/d13/f64 x:0 0 0 2024-04-25T15:49:34.627 INFO:tasks.workunit.client.1.smithi193.stdout:3/321: readlink d8/l58 0 2024-04-25T15:49:34.633 INFO:tasks.workunit.client.1.smithi193.stdout:3/322: dwrite d8/d30/f47 [0,4194304] 0 2024-04-25T15:49:34.637 INFO:tasks.workunit.client.1.smithi193.stdout:3/323: dread d8/f39 [0,4194304] 0 2024-04-25T15:49:34.637 INFO:tasks.workunit.client.0.smithi110.stdout:9/509: symlink d9/d13/d7c/la5 0 2024-04-25T15:49:34.638 INFO:tasks.workunit.client.1.smithi193.stdout:3/324: read - d8/d13/f64 zero size 2024-04-25T15:49:34.638 INFO:tasks.workunit.client.1.smithi193.stdout:7/241: unlink d12/l4d 0 2024-04-25T15:49:34.640 INFO:tasks.workunit.client.1.smithi193.stdout:3/325: fdatasync d8/d13/d14/f3e 0 2024-04-25T15:49:34.640 INFO:tasks.workunit.client.1.smithi193.stdout:7/242: read f8 [3639047,29068] 0 2024-04-25T15:49:34.641 INFO:tasks.workunit.client.1.smithi193.stdout:4/291: fsync db/d13/f21 0 2024-04-25T15:49:34.641 INFO:tasks.workunit.client.1.smithi193.stdout:3/326: write d8/d13/f64 [297469,113354] 0 2024-04-25T15:49:34.641 INFO:tasks.workunit.client.1.smithi193.stdout:7/243: write d12/f2c [185783,38041] 0 2024-04-25T15:49:34.642 INFO:tasks.workunit.client.0.smithi110.stdout:6/545: symlink d1/db/d24/d69/d75/lc6 0 2024-04-25T15:49:34.643 INFO:tasks.workunit.client.0.smithi110.stdout:6/546: write d1/dd/d3c/d91/f7f [725047,122288] 0 2024-04-25T15:49:34.651 INFO:tasks.workunit.client.0.smithi110.stdout:9/510: chown d9/c46 123437290 1 2024-04-25T15:49:34.652 INFO:tasks.workunit.client.0.smithi110.stdout:8/722: write d14/d21/f2a [1644584,8707] 0 2024-04-25T15:49:34.653 INFO:tasks.workunit.client.0.smithi110.stdout:5/618: creat d4/d1d/d1f/d35/d7f/d92/d9b/dcb/fce x:0 0 0 2024-04-25T15:49:34.654 INFO:tasks.workunit.client.1.smithi193.stdout:0/302: truncate d0/d49/d5/d7/dd/f3c 512266 0 2024-04-25T15:49:34.657 INFO:tasks.workunit.client.0.smithi110.stdout:1/664: link d0/d14/c59 d0/d14/d5f/ccb 0 2024-04-25T15:49:34.658 INFO:tasks.workunit.client.1.smithi193.stdout:8/252: link d2/d3/c17 d2/d3/d3a/c4d 0 2024-04-25T15:49:34.658 INFO:tasks.workunit.client.0.smithi110.stdout:1/665: write d0/d14/d5f/d7d/d72/d61/fb2 [895806,81065] 0 2024-04-25T15:49:34.662 INFO:tasks.workunit.client.0.smithi110.stdout:8/723: fdatasync d14/d21/fc8 0 2024-04-25T15:49:34.664 INFO:tasks.workunit.client.0.smithi110.stdout:5/619: fsync d4/f8 0 2024-04-25T15:49:34.666 INFO:tasks.workunit.client.0.smithi110.stdout:5/620: write d4/d2c/d83/faa [1726600,121889] 0 2024-04-25T15:49:34.667 INFO:tasks.workunit.client.0.smithi110.stdout:5/621: readlink d4/d1d/d1f/l5b 0 2024-04-25T15:49:34.668 INFO:tasks.workunit.client.0.smithi110.stdout:6/547: mkdir d1/db/d22/d33/d63/dc2/dc7 0 2024-04-25T15:49:34.669 INFO:tasks.workunit.client.0.smithi110.stdout:6/548: write d1/db/d24/d2d/fb0 [580979,24707] 0 2024-04-25T15:49:34.676 INFO:tasks.workunit.client.0.smithi110.stdout:5/622: dwrite d4/d1d/d1f/f49 [4194304,4194304] 0 2024-04-25T15:49:34.677 INFO:tasks.workunit.client.0.smithi110.stdout:9/511: dread d9/f24 [0,4194304] 0 2024-04-25T15:49:34.678 INFO:tasks.workunit.client.1.smithi193.stdout:4/292: unlink db/d13/d1e/d1f/c25 0 2024-04-25T15:49:34.679 INFO:tasks.workunit.client.0.smithi110.stdout:9/512: write d9/db/df/d2c/f70 [549600,78720] 0 2024-04-25T15:49:34.679 INFO:tasks.workunit.client.1.smithi193.stdout:7/244: mknod d12/d31/d39/c5b 0 2024-04-25T15:49:34.686 INFO:tasks.workunit.client.0.smithi110.stdout:1/666: dwrite d0/d25/f41 [0,4194304] 0 2024-04-25T15:49:34.693 INFO:tasks.workunit.client.1.smithi193.stdout:2/287: sync 2024-04-25T15:49:34.693 INFO:tasks.workunit.client.1.smithi193.stdout:5/255: symlink d17/d18/d30/d38/d48/d4f/l5b 0 2024-04-25T15:49:34.693 INFO:tasks.workunit.client.1.smithi193.stdout:0/303: creat d0/d49/d5/d7/dd/d16/d26/f67 x:0 0 0 2024-04-25T15:49:34.696 INFO:tasks.workunit.client.1.smithi193.stdout:1/241: link dc/f3f dc/f4b 0 2024-04-25T15:49:34.700 INFO:tasks.workunit.client.0.smithi110.stdout:8/724: mknod d14/d21/cf1 0 2024-04-25T15:49:34.701 INFO:tasks.workunit.client.0.smithi110.stdout:8/725: write d14/d32/d36/d48/f66 [5818518,21803] 0 2024-04-25T15:49:34.705 INFO:tasks.workunit.client.0.smithi110.stdout:1/667: unlink d0/d25/d33/f45 0 2024-04-25T15:49:34.705 INFO:tasks.workunit.client.0.smithi110.stdout:1/668: stat d0/f27 0 2024-04-25T15:49:34.706 INFO:tasks.workunit.client.0.smithi110.stdout:1/669: chown d0/d25/f41 419058 1 2024-04-25T15:49:34.706 INFO:tasks.workunit.client.1.smithi193.stdout:4/293: creat db/d13/d1e/d1f/f63 x:0 0 0 2024-04-25T15:49:34.707 INFO:tasks.workunit.client.1.smithi193.stdout:4/294: chown db/d13/l19 899643 1 2024-04-25T15:49:34.708 INFO:tasks.workunit.client.0.smithi110.stdout:6/549: link d1/db/d24/d2d/fb0 d1/db/da0/fc8 0 2024-04-25T15:49:34.712 INFO:tasks.workunit.client.1.smithi193.stdout:9/306: creat df/d2c/d2f/d4e/d1c/d26/f52 x:0 0 0 2024-04-25T15:49:34.713 INFO:tasks.workunit.client.1.smithi193.stdout:9/307: write df/d2c/d2f/d4e/d1c/f23 [1148563,40803] 0 2024-04-25T15:49:34.714 INFO:tasks.workunit.client.1.smithi193.stdout:9/308: fsync df/d2c/d2f/d4e/d1c/d26/d3f/f43 0 2024-04-25T15:49:34.714 INFO:tasks.workunit.client.0.smithi110.stdout:9/513: unlink d9/d13/d5b/d61/d67/l7a 0 2024-04-25T15:49:34.719 INFO:tasks.workunit.client.1.smithi193.stdout:2/288: unlink d0/d9/d3d/f45 0 2024-04-25T15:49:34.720 INFO:tasks.workunit.client.0.smithi110.stdout:1/670: mkdir d0/d14/d5f/d7d/d9c/dcc 0 2024-04-25T15:49:34.721 INFO:tasks.workunit.client.0.smithi110.stdout:0/570: sync 2024-04-25T15:49:34.721 INFO:tasks.workunit.client.0.smithi110.stdout:7/593: sync 2024-04-25T15:49:34.721 INFO:tasks.workunit.client.0.smithi110.stdout:2/545: sync 2024-04-25T15:49:34.721 INFO:tasks.workunit.client.0.smithi110.stdout:4/589: sync 2024-04-25T15:49:34.722 INFO:tasks.workunit.client.1.smithi193.stdout:8/253: getdents d2/d3/d27/d34 0 2024-04-25T15:49:34.723 INFO:tasks.workunit.client.0.smithi110.stdout:7/594: write d1b/d29/d38/d55/d7f/fb1 [163436,5090] 0 2024-04-25T15:49:34.723 INFO:tasks.workunit.client.1.smithi193.stdout:8/254: write d2/d1f/f2b [820984,29668] 0 2024-04-25T15:49:34.724 INFO:tasks.workunit.client.0.smithi110.stdout:9/514: rmdir d9/d2d 39 2024-04-25T15:49:34.724 INFO:tasks.workunit.client.1.smithi193.stdout:8/255: write d2/d3/d3a/f4c [79495,104508] 0 2024-04-25T15:49:34.725 INFO:tasks.workunit.client.0.smithi110.stdout:8/726: truncate d14/f80 1861619 0 2024-04-25T15:49:34.725 INFO:tasks.workunit.client.1.smithi193.stdout:8/256: chown d2/d3/d20 299086769 1 2024-04-25T15:49:34.726 INFO:tasks.workunit.client.0.smithi110.stdout:6/550: symlink d1/db/d22/d33/d63/dc2/dc7/lc9 0 2024-04-25T15:49:34.726 INFO:tasks.workunit.client.0.smithi110.stdout:9/515: read d9/d44/f53 [1161804,108899] 0 2024-04-25T15:49:34.728 INFO:tasks.workunit.client.0.smithi110.stdout:9/516: chown d9/db/df/d25/l71 110618446 1 2024-04-25T15:49:34.728 INFO:tasks.workunit.client.1.smithi193.stdout:7/245: rename cf to d12/d38/c5c 0 2024-04-25T15:49:34.730 INFO:tasks.workunit.client.1.smithi193.stdout:6/292: getdents dd/d12/d19 0 2024-04-25T15:49:34.731 INFO:tasks.workunit.client.0.smithi110.stdout:4/590: fdatasync d13/d24/d2d/f4f 0 2024-04-25T15:49:34.731 INFO:tasks.workunit.client.0.smithi110.stdout:4/591: stat d13/d24/f48 0 2024-04-25T15:49:34.734 INFO:tasks.workunit.client.0.smithi110.stdout:2/546: creat de/d40/d6e/d50/fa4 x:0 0 0 2024-04-25T15:49:34.737 INFO:tasks.workunit.client.0.smithi110.stdout:5/623: getdents d4/d1d 0 2024-04-25T15:49:34.738 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:34 smithi193 ceph-mon[33747]: pgmap v34: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 62 MiB/s rd, 239 MiB/s wr, 464 op/s 2024-04-25T15:49:34.739 INFO:tasks.workunit.client.0.smithi110.stdout:7/595: creat d1b/d29/d38/d8b/fc7 x:0 0 0 2024-04-25T15:49:34.740 INFO:tasks.workunit.client.0.smithi110.stdout:2/547: dwrite de/d40/f72 [0,4194304] 0 2024-04-25T15:49:34.744 INFO:tasks.workunit.client.0.smithi110.stdout:8/727: symlink d14/d21/d26/d6a/d7e/da6/de8/d71/lf2 0 2024-04-25T15:49:34.746 INFO:tasks.workunit.client.1.smithi193.stdout:0/304: symlink d0/d49/d5/l68 0 2024-04-25T15:49:34.748 INFO:tasks.workunit.client.1.smithi193.stdout:0/305: dread - d0/d49/d5/d7/dd/d16/d26/f67 zero size 2024-04-25T15:49:34.748 INFO:tasks.workunit.client.0.smithi110.stdout:9/517: symlink d9/d44/d5e/la6 0 2024-04-25T15:49:34.749 INFO:tasks.workunit.client.1.smithi193.stdout:1/242: symlink dc/d35/l4c 0 2024-04-25T15:49:34.749 INFO:tasks.workunit.client.1.smithi193.stdout:1/243: truncate dc/f4b 1431574 0 2024-04-25T15:49:34.750 INFO:tasks.workunit.client.0.smithi110.stdout:1/671: mkdir d0/d25/db4/dcd 0 2024-04-25T15:49:34.750 INFO:tasks.workunit.client.0.smithi110.stdout:1/672: chown d0/d51/d6b/faf 56 1 2024-04-25T15:49:34.751 INFO:tasks.workunit.client.0.smithi110.stdout:1/673: fsync d0/d14/fad 0 2024-04-25T15:49:34.753 INFO:tasks.workunit.client.1.smithi193.stdout:3/327: rmdir d8/d30/d31/d4f 0 2024-04-25T15:49:34.758 INFO:tasks.workunit.client.0.smithi110.stdout:6/551: dwrite d1/db/d24/d2d/fb0 [0,4194304] 0 2024-04-25T15:49:34.760 INFO:tasks.workunit.client.1.smithi193.stdout:4/295: rename db/d32/f37 to db/d13/d1e/d1f/f64 0 2024-04-25T15:49:34.764 INFO:tasks.workunit.client.0.smithi110.stdout:5/624: chown d4/d1d/d1f/d35/d3d/d93/lbe 67038919 1 2024-04-25T15:49:34.764 INFO:tasks.workunit.client.1.smithi193.stdout:7/246: creat d12/d1c/d45/f5d x:0 0 0 2024-04-25T15:49:34.765 INFO:tasks.workunit.client.0.smithi110.stdout:7/596: truncate d1b/d29/d46/f6a 2729047 0 2024-04-25T15:49:34.766 INFO:tasks.workunit.client.0.smithi110.stdout:6/552: dwrite d1/db/d24/fb9 [0,4194304] 0 2024-04-25T15:49:34.766 INFO:tasks.workunit.client.0.smithi110.stdout:7/597: dread - d1b/d29/d46/fb3 zero size 2024-04-25T15:49:34.768 INFO:tasks.workunit.client.1.smithi193.stdout:6/293: creat dd/d12/d19/f51 x:0 0 0 2024-04-25T15:49:34.770 INFO:tasks.workunit.client.0.smithi110.stdout:7/598: dread d1b/d29/d38/d55/fbd [0,4194304] 0 2024-04-25T15:49:34.770 INFO:tasks.workunit.client.1.smithi193.stdout:9/309: symlink df/l53 0 2024-04-25T15:49:34.771 INFO:tasks.workunit.client.0.smithi110.stdout:2/548: creat de/d40/d5b/d5c/fa5 x:0 0 0 2024-04-25T15:49:34.773 INFO:tasks.workunit.client.1.smithi193.stdout:5/256: unlink d17/c3a 0 2024-04-25T15:49:34.773 INFO:tasks.workunit.client.1.smithi193.stdout:2/289: symlink d0/d37/d4d/l75 0 2024-04-25T15:49:34.774 INFO:tasks.workunit.client.0.smithi110.stdout:4/592: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/d88/f20 [0,4194304] 0 2024-04-25T15:49:34.774 INFO:tasks.workunit.client.0.smithi110.stdout:8/728: symlink d14/d32/dd6/lf3 0 2024-04-25T15:49:34.775 INFO:tasks.workunit.client.1.smithi193.stdout:0/306: creat d0/d49/d5/d7/dd/d16/d26/d5e/f69 x:0 0 0 2024-04-25T15:49:34.777 INFO:tasks.workunit.client.0.smithi110.stdout:9/518: creat d9/d44/d5e/fa7 x:0 0 0 2024-04-25T15:49:34.779 INFO:tasks.workunit.client.0.smithi110.stdout:7/599: dread d1b/f4b [0,4194304] 0 2024-04-25T15:49:34.781 INFO:tasks.workunit.client.1.smithi193.stdout:8/257: creat d2/d3/d20/d45/d4a/f4e x:0 0 0 2024-04-25T15:49:34.783 INFO:tasks.workunit.client.1.smithi193.stdout:4/296: fdatasync db/d13/d1e/d1f/f64 0 2024-04-25T15:49:34.785 INFO:tasks.workunit.client.1.smithi193.stdout:7/247: creat d12/d31/d37/f5e x:0 0 0 2024-04-25T15:49:34.786 INFO:tasks.workunit.client.1.smithi193.stdout:4/297: read f2 [86493,100270] 0 2024-04-25T15:49:34.788 INFO:tasks.workunit.client.0.smithi110.stdout:8/729: dread d14/d32/d36/d48/f5e [4194304,4194304] 0 2024-04-25T15:49:34.788 INFO:tasks.workunit.client.0.smithi110.stdout:6/553: creat d1/db/d22/d90/d76/fca x:0 0 0 2024-04-25T15:49:34.789 INFO:tasks.workunit.client.1.smithi193.stdout:9/310: symlink df/d11/l54 0 2024-04-25T15:49:34.797 INFO:tasks.workunit.client.1.smithi193.stdout:5/257: dwrite d17/d18/d3e/f3f [0,4194304] 0 2024-04-25T15:49:34.797 INFO:tasks.workunit.client.0.smithi110.stdout:4/593: creat d13/d24/d2d/d44/d90/fc1 x:0 0 0 2024-04-25T15:49:34.799 INFO:tasks.workunit.client.0.smithi110.stdout:2/549: dread f8 [0,4194304] 0 2024-04-25T15:49:34.799 INFO:tasks.workunit.client.0.smithi110.stdout:9/519: mknod d9/d44/ca8 0 2024-04-25T15:49:34.801 INFO:tasks.workunit.client.1.smithi193.stdout:3/328: creat d8/d13/d14/d27/d54/f65 x:0 0 0 2024-04-25T15:49:34.805 INFO:tasks.workunit.client.0.smithi110.stdout:7/600: mkdir d1b/d29/d38/d55/d7f/dc8 0 2024-04-25T15:49:34.806 INFO:tasks.workunit.client.1.smithi193.stdout:7/248: mknod d12/d1c/d45/c5f 0 2024-04-25T15:49:34.809 INFO:tasks.workunit.client.0.smithi110.stdout:8/730: creat d14/d21/d26/dab/dac/ff4 x:0 0 0 2024-04-25T15:49:34.811 INFO:tasks.workunit.client.1.smithi193.stdout:7/249: dwrite d12/d13/f14 [0,4194304] 0 2024-04-25T15:49:34.811 INFO:tasks.workunit.client.1.smithi193.stdout:2/290: write d0/d42/d51/d4f/d65/d6a/f3b [70603,27815] 0 2024-04-25T15:49:34.815 INFO:tasks.workunit.client.1.smithi193.stdout:0/307: mkdir d0/d49/d6a 0 2024-04-25T15:49:34.818 INFO:tasks.workunit.client.1.smithi193.stdout:5/258: mknod d17/d18/d30/d38/c5c 0 2024-04-25T15:49:34.818 INFO:tasks.workunit.client.1.smithi193.stdout:9/311: write df/d2c/d2f/d33/f3c [806008,108865] 0 2024-04-25T15:49:34.819 INFO:tasks.workunit.client.0.smithi110.stdout:4/594: rmdir d13/d24/d2d/d44/d65/d99 39 2024-04-25T15:49:34.822 INFO:tasks.workunit.client.0.smithi110.stdout:2/550: chown de/d40/d6e/c51 1299 1 2024-04-25T15:49:34.822 INFO:tasks.workunit.client.1.smithi193.stdout:3/329: creat d8/d13/f66 x:0 0 0 2024-04-25T15:49:34.824 INFO:tasks.workunit.client.0.smithi110.stdout:9/520: creat d9/d13/fa9 x:0 0 0 2024-04-25T15:49:34.826 INFO:tasks.workunit.client.0.smithi110.stdout:4/595: dwrite d13/d24/d2d/d44/d90/fc1 [0,4194304] 0 2024-04-25T15:49:34.827 INFO:tasks.workunit.client.0.smithi110.stdout:1/674: rmdir d0/d75 0 2024-04-25T15:49:34.828 INFO:tasks.workunit.client.0.smithi110.stdout:7/601: mknod d1b/d29/d87/d8e/cc9 0 2024-04-25T15:49:34.830 INFO:tasks.workunit.client.0.smithi110.stdout:5/625: creat d4/d1d/d1f/fcf x:0 0 0 2024-04-25T15:49:34.831 INFO:tasks.workunit.client.1.smithi193.stdout:7/250: mkdir d12/d31/d39/d60 0 2024-04-25T15:49:34.832 INFO:tasks.workunit.client.1.smithi193.stdout:7/251: write d12/d31/f58 [39523,15141] 0 2024-04-25T15:49:34.837 INFO:tasks.workunit.client.1.smithi193.stdout:0/308: creat d0/d49/d5/d7/dd/d46/f6b x:0 0 0 2024-04-25T15:49:34.837 INFO:tasks.workunit.client.0.smithi110.stdout:5/626: dwrite d4/d1d/d1f/d35/d7f/d92/fca [0,4194304] 0 2024-04-25T15:49:34.841 INFO:tasks.workunit.client.0.smithi110.stdout:5/627: write d4/d1d/d1f/f6e [2355438,26299] 0 2024-04-25T15:49:34.842 INFO:tasks.workunit.client.0.smithi110.stdout:1/675: rename d0/d51/f79 to d0/d25/db4/dcd/fce 0 2024-04-25T15:49:34.845 INFO:tasks.workunit.client.0.smithi110.stdout:4/596: creat d13/d24/d2d/d44/d90/db8/fc2 x:0 0 0 2024-04-25T15:49:34.846 INFO:tasks.workunit.client.0.smithi110.stdout:5/628: dwrite d4/d15/d51/f78 [0,4194304] 0 2024-04-25T15:49:34.847 INFO:tasks.workunit.client.0.smithi110.stdout:5/629: chown d4/f7c 12 1 2024-04-25T15:49:34.848 INFO:tasks.workunit.client.0.smithi110.stdout:7/602: mkdir d1b/d29/db5/dca 0 2024-04-25T15:49:34.853 INFO:tasks.workunit.client.1.smithi193.stdout:1/244: rmdir dc/d3c/d44 0 2024-04-25T15:49:34.853 INFO:tasks.workunit.client.0.smithi110.stdout:5/630: dread d4/d2c/d83/faa [0,4194304] 0 2024-04-25T15:49:34.853 INFO:tasks.workunit.client.1.smithi193.stdout:3/330: rmdir d8/d13/d14 39 2024-04-25T15:49:34.854 INFO:tasks.workunit.client.1.smithi193.stdout:8/258: link d2/c24 d2/d3/d3a/c4f 0 2024-04-25T15:49:34.854 INFO:tasks.workunit.client.0.smithi110.stdout:5/631: chown d4/d1d/d1f/f64 973 1 2024-04-25T15:49:34.856 INFO:tasks.workunit.client.1.smithi193.stdout:5/259: dwrite d17/f2a [0,4194304] 0 2024-04-25T15:49:34.856 INFO:tasks.workunit.client.1.smithi193.stdout:8/259: read d2/d36/f40 [6678816,44763] 0 2024-04-25T15:49:34.859 INFO:tasks.workunit.client.1.smithi193.stdout:0/309: creat d0/d49/d5/d7/dd/d16/f6c x:0 0 0 2024-04-25T15:49:34.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:34 smithi110 ceph-mon[26344]: pgmap v34: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 62 MiB/s rd, 239 MiB/s wr, 464 op/s 2024-04-25T15:49:34.864 INFO:tasks.workunit.client.0.smithi110.stdout:3/655: sync 2024-04-25T15:49:34.864 INFO:tasks.workunit.client.0.smithi110.stdout:8/731: dwrite d14/d21/d26/d6a/d7e/da6/de8/d71/dae/fcf [4194304,4194304] 0 2024-04-25T15:49:34.871 INFO:tasks.workunit.client.0.smithi110.stdout:2/551: truncate de/d40/d70/f84 158461 0 2024-04-25T15:49:34.875 INFO:tasks.workunit.client.1.smithi193.stdout:6/294: link dd/d12/d19/d1c/d40/l3b dd/d12/d19/d1c/d40/d3c/d3f/l52 0 2024-04-25T15:49:34.876 INFO:tasks.workunit.client.1.smithi193.stdout:6/295: write dd/f1b [2576914,112052] 0 2024-04-25T15:49:34.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/260: mkdir d2/d3/d3a/d50 0 2024-04-25T15:49:34.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/261: write d2/d1f/f2b [3476917,128121] 0 2024-04-25T15:49:34.880 INFO:tasks.workunit.client.0.smithi110.stdout:9/521: dwrite d9/db/df/d2c/d3c/f49 [0,4194304] 0 2024-04-25T15:49:34.880 INFO:tasks.workunit.client.1.smithi193.stdout:5/260: mkdir d17/d18/d30/d34/d37/d5d 0 2024-04-25T15:49:34.881 INFO:tasks.workunit.client.1.smithi193.stdout:4/298: sync 2024-04-25T15:49:34.882 INFO:tasks.workunit.client.1.smithi193.stdout:5/261: write d17/d18/d3e/f3f [1652314,94495] 0 2024-04-25T15:49:34.883 INFO:tasks.workunit.client.0.smithi110.stdout:7/603: mkdir d1b/d29/d38/d55/d95/dcb 0 2024-04-25T15:49:34.883 INFO:tasks.workunit.client.1.smithi193.stdout:1/245: mkdir dc/d4a/d4d 0 2024-04-25T15:49:34.884 INFO:tasks.workunit.client.0.smithi110.stdout:7/604: chown d1b/d29/d38/d8b/d89/f96 10827 1 2024-04-25T15:49:34.885 INFO:tasks.workunit.client.0.smithi110.stdout:6/554: getdents d1/db/d24 0 2024-04-25T15:49:34.885 INFO:tasks.workunit.client.1.smithi193.stdout:8/262: dwrite d2/d1f/f2b [0,4194304] 0 2024-04-25T15:49:34.886 INFO:tasks.workunit.client.0.smithi110.stdout:6/555: chown d1/db/d22/d33/d63/dc2 6806 1 2024-04-25T15:49:34.886 INFO:tasks.workunit.client.0.smithi110.stdout:7/605: dread - d1b/d29/d38/d55/f8d zero size 2024-04-25T15:49:34.887 INFO:tasks.workunit.client.1.smithi193.stdout:2/291: link d0/d9/l32 d0/d37/d25/d5f/d3a/d6d/l76 0 2024-04-25T15:49:34.888 INFO:tasks.workunit.client.0.smithi110.stdout:7/606: write d1b/d29/d38/d55/fae [547910,606] 0 2024-04-25T15:49:34.889 INFO:tasks.workunit.client.1.smithi193.stdout:3/331: rmdir d8/d13/d14/d23 39 2024-04-25T15:49:34.894 INFO:tasks.workunit.client.0.smithi110.stdout:3/656: mkdir d0/d7/dd0/ddf 0 2024-04-25T15:49:34.894 INFO:tasks.workunit.client.1.smithi193.stdout:6/296: rmdir dd 39 2024-04-25T15:49:34.895 INFO:tasks.workunit.client.0.smithi110.stdout:1/676: write d0/d51/d6b/f7c [3104907,25986] 0 2024-04-25T15:49:34.895 INFO:tasks.workunit.client.0.smithi110.stdout:1/677: readlink d0/d14/l3c 0 2024-04-25T15:49:34.896 INFO:tasks.workunit.client.0.smithi110.stdout:1/678: stat d0/d14/d5f/d7d/d72/d61 0 2024-04-25T15:49:34.897 INFO:tasks.workunit.client.0.smithi110.stdout:2/552: mknod de/d6f/ca6 0 2024-04-25T15:49:34.905 INFO:tasks.workunit.client.1.smithi193.stdout:1/246: truncate dc/d35/d3d/f42 4577575 0 2024-04-25T15:49:34.906 INFO:tasks.workunit.client.0.smithi110.stdout:6/556: creat d1/db/d22/d90/fcb x:0 0 0 2024-04-25T15:49:34.907 INFO:tasks.workunit.client.0.smithi110.stdout:7/607: rename d1b/d29/d38/fbc to d1b/d29/d38/d8b/d89/fcc 0 2024-04-25T15:49:34.908 INFO:tasks.workunit.client.0.smithi110.stdout:7/608: stat d1b/d29/d38/d55/f8d 0 2024-04-25T15:49:34.908 INFO:tasks.workunit.client.1.smithi193.stdout:3/332: creat d8/d4d/f67 x:0 0 0 2024-04-25T15:49:34.909 INFO:tasks.workunit.client.0.smithi110.stdout:0/571: sync 2024-04-25T15:49:34.911 INFO:tasks.workunit.client.0.smithi110.stdout:3/657: chown d0/d7/d40/l42 61 1 2024-04-25T15:49:34.912 INFO:tasks.workunit.client.0.smithi110.stdout:3/658: chown d0/d2e/d88/dbf/d99/dad/daf/lcc 19 1 2024-04-25T15:49:34.912 INFO:tasks.workunit.client.1.smithi193.stdout:0/310: creat d0/d49/d5/f6d x:0 0 0 2024-04-25T15:49:34.913 INFO:tasks.workunit.client.1.smithi193.stdout:0/311: fdatasync d0/d49/f2c 0 2024-04-25T15:49:34.913 INFO:tasks.workunit.client.0.smithi110.stdout:2/553: mknod de/d40/d4f/d9f/ca7 0 2024-04-25T15:49:34.915 INFO:tasks.workunit.client.1.smithi193.stdout:4/299: unlink db/l20 0 2024-04-25T15:49:34.915 INFO:tasks.workunit.client.1.smithi193.stdout:4/300: write db/d13/d1e/f48 [358170,59748] 0 2024-04-25T15:49:34.916 INFO:tasks.workunit.client.1.smithi193.stdout:5/262: mknod d17/d18/d30/d34/d37/d5d/c5e 0 2024-04-25T15:49:34.917 INFO:tasks.workunit.client.1.smithi193.stdout:4/301: dread - db/d4b/d51/d54/f5c zero size 2024-04-25T15:49:34.917 INFO:tasks.workunit.client.1.smithi193.stdout:4/302: fdatasync db/d13/d1e/d1f/f64 0 2024-04-25T15:49:34.918 INFO:tasks.workunit.client.1.smithi193.stdout:4/303: truncate db/d32/f5f 95979 0 2024-04-25T15:49:34.919 INFO:tasks.workunit.client.1.smithi193.stdout:1/247: rename dc/l2f to dc/d4a/d4d/l4e 0 2024-04-25T15:49:34.920 INFO:tasks.workunit.client.0.smithi110.stdout:5/632: creat d4/d2c/fd0 x:0 0 0 2024-04-25T15:49:34.920 INFO:tasks.workunit.client.0.smithi110.stdout:8/732: dwrite d14/f1b [0,4194304] 0 2024-04-25T15:49:34.922 INFO:tasks.workunit.client.1.smithi193.stdout:3/333: creat d8/d30/d45/f68 x:0 0 0 2024-04-25T15:49:34.923 INFO:tasks.workunit.client.1.smithi193.stdout:1/248: dread dc/d35/d3d/f42 [0,4194304] 0 2024-04-25T15:49:34.923 INFO:tasks.workunit.client.1.smithi193.stdout:1/249: fsync dc/f47 0 2024-04-25T15:49:34.925 INFO:tasks.workunit.client.0.smithi110.stdout:7/609: mkdir d1b/d29/d38/d55/dab/db2/dcd 0 2024-04-25T15:49:34.926 INFO:tasks.workunit.client.1.smithi193.stdout:3/334: dread d8/ff [0,4194304] 0 2024-04-25T15:49:34.929 INFO:tasks.workunit.client.0.smithi110.stdout:2/554: creat de/d40/d5b/d5c/d5f/fa8 x:0 0 0 2024-04-25T15:49:34.930 INFO:tasks.workunit.client.0.smithi110.stdout:9/522: creat d9/faa x:0 0 0 2024-04-25T15:49:34.934 INFO:tasks.workunit.client.1.smithi193.stdout:2/292: link d0/d9/d15/d31/d50/f5c d0/d37/d25/f77 0 2024-04-25T15:49:34.934 INFO:tasks.workunit.client.1.smithi193.stdout:3/335: dwrite d8/d30/d31/f53 [0,4194304] 0 2024-04-25T15:49:34.935 INFO:tasks.workunit.client.1.smithi193.stdout:1/250: symlink dc/d3c/l4f 0 2024-04-25T15:49:34.940 INFO:tasks.workunit.client.1.smithi193.stdout:1/251: dread dc/f12 [0,4194304] 0 2024-04-25T15:49:34.941 INFO:tasks.workunit.client.0.smithi110.stdout:1/679: rename d0/d25/lb9 to d0/d14/d5f/d7d/d9c/lcf 0 2024-04-25T15:49:34.941 INFO:tasks.workunit.client.1.smithi193.stdout:1/252: fsync dc/d35/d3d/f41 0 2024-04-25T15:49:34.942 INFO:tasks.workunit.client.1.smithi193.stdout:5/263: unlink d17/f39 0 2024-04-25T15:49:34.943 INFO:tasks.workunit.client.1.smithi193.stdout:8/263: getdents d2/d3/d20 0 2024-04-25T15:49:34.944 INFO:tasks.workunit.client.0.smithi110.stdout:7/610: symlink d1b/d29/d38/lce 0 2024-04-25T15:49:34.945 INFO:tasks.workunit.client.1.smithi193.stdout:8/264: chown d2/d36 14 1 2024-04-25T15:49:34.945 INFO:tasks.workunit.client.1.smithi193.stdout:7/252: sync 2024-04-25T15:49:34.945 INFO:tasks.workunit.client.1.smithi193.stdout:9/312: sync 2024-04-25T15:49:34.946 INFO:tasks.workunit.client.0.smithi110.stdout:3/659: creat d0/d7/dd0/ddf/fe0 x:0 0 0 2024-04-25T15:49:34.946 INFO:tasks.workunit.client.1.smithi193.stdout:4/304: dread db/fc [0,4194304] 0 2024-04-25T15:49:34.949 INFO:tasks.workunit.client.1.smithi193.stdout:3/336: fdatasync d8/d13/d14/f20 0 2024-04-25T15:49:34.950 INFO:tasks.workunit.client.0.smithi110.stdout:4/597: getdents d13/d24/d2d/d44/d65/d99 0 2024-04-25T15:49:34.951 INFO:tasks.workunit.client.1.smithi193.stdout:6/297: link dd/d12/l27 dd/d12/d2d/l53 0 2024-04-25T15:49:34.953 INFO:tasks.workunit.client.0.smithi110.stdout:6/557: rmdir d1/db/d24/d69/d75/d82/dbe 0 2024-04-25T15:49:34.954 INFO:tasks.workunit.client.0.smithi110.stdout:7/611: dread d1b/d29/d38/d55/d7f/f6c [0,4194304] 0 2024-04-25T15:49:34.954 INFO:tasks.workunit.client.1.smithi193.stdout:0/312: creat d0/d49/d5/d7/dd/f6e x:0 0 0 2024-04-25T15:49:34.955 INFO:tasks.workunit.client.1.smithi193.stdout:0/313: readlink d0/d49/d5/d7/dd/d16/d26/l66 0 2024-04-25T15:49:34.956 INFO:tasks.workunit.client.1.smithi193.stdout:1/253: mkdir dc/d4a/d50 0 2024-04-25T15:49:34.958 INFO:tasks.workunit.client.0.smithi110.stdout:8/733: rename d14/d21/d23/l7b to d14/d32/d36/d48/lf5 0 2024-04-25T15:49:34.959 INFO:tasks.workunit.client.1.smithi193.stdout:5/264: creat d17/d18/d30/d38/d48/d4f/f5f x:0 0 0 2024-04-25T15:49:34.960 INFO:tasks.workunit.client.0.smithi110.stdout:1/680: rmdir d0/d25/db4 39 2024-04-25T15:49:34.961 INFO:tasks.workunit.client.0.smithi110.stdout:2/555: write de/d40/f35 [2051866,106867] 0 2024-04-25T15:49:34.962 INFO:tasks.workunit.client.0.smithi110.stdout:2/556: chown de/d40/d6e/d50/l9e 1 1 2024-04-25T15:49:34.963 INFO:tasks.workunit.client.0.smithi110.stdout:2/557: write de/f56 [3401400,55053] 0 2024-04-25T15:49:34.964 INFO:tasks.workunit.client.1.smithi193.stdout:8/265: symlink d2/d1f/l51 0 2024-04-25T15:49:34.966 INFO:tasks.workunit.client.1.smithi193.stdout:5/265: dwrite f16 [0,4194304] 0 2024-04-25T15:49:34.966 INFO:tasks.workunit.client.0.smithi110.stdout:3/660: mkdir d0/d2e/d88/dbf/d99/de1 0 2024-04-25T15:49:34.967 INFO:tasks.workunit.client.0.smithi110.stdout:3/661: chown d0/d16/dcb/d50/fca 65089095 1 2024-04-25T15:49:34.968 INFO:tasks.workunit.client.1.smithi193.stdout:9/313: mknod df/d2c/d2f/d4e/d1c/d30/c55 0 2024-04-25T15:49:34.973 INFO:tasks.workunit.client.0.smithi110.stdout:4/598: symlink d13/d24/d2d/d5e/lc3 0 2024-04-25T15:49:34.974 INFO:tasks.workunit.client.0.smithi110.stdout:6/558: mkdir d1/db/d24/d69/d75/dcc 0 2024-04-25T15:49:34.976 INFO:tasks.workunit.client.0.smithi110.stdout:7/612: creat d1b/d29/d46/fcf x:0 0 0 2024-04-25T15:49:34.977 INFO:tasks.workunit.client.0.smithi110.stdout:5/633: link d4/d1d/l86 d4/d1d/d1f/d35/d3d/d93/ld1 0 2024-04-25T15:49:34.977 INFO:tasks.workunit.client.1.smithi193.stdout:6/298: fdatasync dd/f2b 0 2024-04-25T15:49:34.979 INFO:tasks.workunit.client.1.smithi193.stdout:0/314: rename d0/d49/d5/d7/dd/d16/f6c to d0/d49/d5/d7/dd/d16/f6f 0 2024-04-25T15:49:34.979 INFO:tasks.workunit.client.1.smithi193.stdout:3/337: dread d8/f15 [0,4194304] 0 2024-04-25T15:49:34.980 INFO:tasks.workunit.client.1.smithi193.stdout:0/315: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60 0 1 2024-04-25T15:49:34.980 INFO:tasks.workunit.client.0.smithi110.stdout:8/734: rename d14/d32/d36/f3c to d14/d21/d26/d6a/d7e/da6/ff6 0 2024-04-25T15:49:34.982 INFO:tasks.workunit.client.0.smithi110.stdout:1/681: mknod d0/d14/d5f/d7d/cd0 0 2024-04-25T15:49:34.984 INFO:tasks.workunit.client.1.smithi193.stdout:1/254: dread dc/f1e [0,4194304] 0 2024-04-25T15:49:34.985 INFO:tasks.workunit.client.0.smithi110.stdout:0/572: getdents d1/d23/d30/d33/d5b/d78/da4 0 2024-04-25T15:49:34.985 INFO:tasks.workunit.client.1.smithi193.stdout:0/316: dwrite d0/d49/d5/d7/dd/d16/d26/f67 [0,4194304] 0 2024-04-25T15:49:34.986 INFO:tasks.workunit.client.1.smithi193.stdout:0/317: dread - d0/d49/d5/d7/dd/f6e zero size 2024-04-25T15:49:34.987 INFO:tasks.workunit.client.1.smithi193.stdout:8/266: rename d2/d1f/l32 to d2/d36/l52 0 2024-04-25T15:49:34.988 INFO:tasks.workunit.client.0.smithi110.stdout:4/599: dread d13/d24/d2d/d5e/f7b [0,4194304] 0 2024-04-25T15:49:34.990 INFO:tasks.workunit.client.0.smithi110.stdout:4/600: read d13/d24/d2d/d44/d65/d99/daf/d56/f8e [78092,89421] 0 2024-04-25T15:49:34.994 INFO:tasks.workunit.client.1.smithi193.stdout:5/266: creat d17/d18/d30/d3b/f60 x:0 0 0 2024-04-25T15:49:34.996 INFO:tasks.workunit.client.0.smithi110.stdout:9/523: link f6 d9/d44/fab 0 2024-04-25T15:49:34.998 INFO:tasks.workunit.client.1.smithi193.stdout:4/305: mkdir db/d13/d1e/d1f/d56/d57/d65 0 2024-04-25T15:49:34.999 INFO:tasks.workunit.client.0.smithi110.stdout:4/601: dread d13/d24/d2d/d36/d51/f8c [0,4194304] 0 2024-04-25T15:49:34.999 INFO:tasks.workunit.client.1.smithi193.stdout:7/253: truncate d12/f2e 2411532 0 2024-04-25T15:49:35.000 INFO:tasks.workunit.client.0.smithi110.stdout:4/602: write d13/d24/d2d/d7d/d92/f9a [12816,82607] 0 2024-04-25T15:49:35.003 INFO:tasks.workunit.client.0.smithi110.stdout:8/735: rmdir d14/d21/d26/dab/dac/dbb 39 2024-04-25T15:49:35.003 INFO:tasks.workunit.client.0.smithi110.stdout:8/736: fdatasync d14/f16 0 2024-04-25T15:49:35.004 INFO:tasks.workunit.client.0.smithi110.stdout:1/682: fsync d0/d14/d5f/d7d/f64 0 2024-04-25T15:49:35.008 INFO:tasks.workunit.client.0.smithi110.stdout:5/634: dwrite d4/d1d/d1f/d35/d3d/d52/f55 [0,4194304] 0 2024-04-25T15:49:35.010 INFO:tasks.workunit.client.1.smithi193.stdout:9/314: dread df/d2c/d2f/d4e/f25 [0,4194304] 0 2024-04-25T15:49:35.010 INFO:tasks.workunit.client.0.smithi110.stdout:5/635: chown d4/d15/c66 237 1 2024-04-25T15:49:35.011 INFO:tasks.workunit.client.1.smithi193.stdout:7/254: dread d12/f1a [0,4194304] 0 2024-04-25T15:49:35.012 INFO:tasks.workunit.client.1.smithi193.stdout:3/338: rename d8/d13/d14/d51/d55 to d8/d13/d1f/d69 0 2024-04-25T15:49:35.013 INFO:tasks.workunit.client.0.smithi110.stdout:3/662: symlink d0/d2e/d88/dbf/le2 0 2024-04-25T15:49:35.016 INFO:tasks.workunit.client.0.smithi110.stdout:9/524: creat d9/db/df/d2c/d3c/fac x:0 0 0 2024-04-25T15:49:35.016 INFO:tasks.workunit.client.0.smithi110.stdout:5/636: read d4/d1d/d1f/f64 [346047,78418] 0 2024-04-25T15:49:35.017 INFO:tasks.workunit.client.0.smithi110.stdout:9/525: write d9/d44/f53 [1824841,80212] 0 2024-04-25T15:49:35.019 INFO:tasks.workunit.client.1.smithi193.stdout:8/267: mkdir d2/d3/d20/d45/d4a/d53 0 2024-04-25T15:49:35.019 INFO:tasks.workunit.client.0.smithi110.stdout:9/526: write d9/db/df/d2c/d3c/f89 [911439,11068] 0 2024-04-25T15:49:35.019 INFO:tasks.workunit.client.1.smithi193.stdout:8/268: chown d2/d3/d20/d45/f49 38 1 2024-04-25T15:49:35.020 INFO:tasks.workunit.client.0.smithi110.stdout:9/527: chown d9/d13/d84/d98/c76 481 1 2024-04-25T15:49:35.020 INFO:tasks.workunit.client.0.smithi110.stdout:6/559: mknod d1/db/da0/da5/ccd 0 2024-04-25T15:49:35.021 INFO:tasks.workunit.client.0.smithi110.stdout:9/528: dread - d9/f8c zero size 2024-04-25T15:49:35.026 INFO:tasks.workunit.client.1.smithi193.stdout:2/293: sync 2024-04-25T15:49:35.029 INFO:tasks.workunit.client.0.smithi110.stdout:2/558: dwrite de/d40/d6e/f42 [0,4194304] 0 2024-04-25T15:49:35.033 INFO:tasks.workunit.client.1.smithi193.stdout:6/299: link dd/d12/d19/d1c/f29 dd/d12/d19/d1c/d40/d3c/d3f/f54 0 2024-04-25T15:49:35.035 INFO:tasks.workunit.client.1.smithi193.stdout:6/300: read dd/f2e [2622126,118209] 0 2024-04-25T15:49:35.035 INFO:tasks.workunit.client.0.smithi110.stdout:1/683: unlink d0/d14/f7a 0 2024-04-25T15:49:35.036 INFO:tasks.workunit.client.0.smithi110.stdout:2/559: dwrite de/d40/d4f/f49 [0,4194304] 0 2024-04-25T15:49:35.040 INFO:tasks.workunit.client.1.smithi193.stdout:7/255: stat c6 0 2024-04-25T15:49:35.041 INFO:tasks.workunit.client.1.smithi193.stdout:3/339: fdatasync d8/f39 0 2024-04-25T15:49:35.041 INFO:tasks.workunit.client.0.smithi110.stdout:0/573: write d1/d23/f24 [10286,9630] 0 2024-04-25T15:49:35.042 INFO:tasks.workunit.client.1.smithi193.stdout:3/340: write d8/d13/f60 [830955,44285] 0 2024-04-25T15:49:35.042 INFO:tasks.workunit.client.0.smithi110.stdout:0/574: chown d1/d23/d30/d33/d5b/d72/f82 340 1 2024-04-25T15:49:35.043 INFO:tasks.workunit.client.0.smithi110.stdout:5/637: write d4/d15/f1a [4288991,80570] 0 2024-04-25T15:49:35.044 INFO:tasks.workunit.client.0.smithi110.stdout:0/575: chown d1/d23/d30/d33/d5b/d78/f7d 347666 1 2024-04-25T15:49:35.046 INFO:tasks.workunit.client.1.smithi193.stdout:0/318: rename d0/d49/d5/d7/dd/d16/d26/d5e/f65 to d0/d49/d5/d7/dd/d16/d18/d28/d56/f70 0 2024-04-25T15:49:35.047 INFO:tasks.workunit.client.0.smithi110.stdout:1/684: dread d0/d14/fc0 [0,4194304] 0 2024-04-25T15:49:35.047 INFO:tasks.workunit.client.0.smithi110.stdout:6/560: truncate d1/f26 3472271 0 2024-04-25T15:49:35.048 INFO:tasks.workunit.client.1.smithi193.stdout:0/319: dread d0/d49/d5/d7/dd/d38/f4c [0,4194304] 0 2024-04-25T15:49:35.051 INFO:tasks.workunit.client.0.smithi110.stdout:7/613: creat d1b/d29/fd0 x:0 0 0 2024-04-25T15:49:35.052 INFO:tasks.workunit.client.0.smithi110.stdout:6/561: write d1/db/d22/d33/d63/dbd/f9d [45167,115560] 0 2024-04-25T15:49:35.052 INFO:tasks.workunit.client.0.smithi110.stdout:4/603: creat d13/d24/d2d/d44/d65/d99/dbc/fc4 x:0 0 0 2024-04-25T15:49:35.054 INFO:tasks.workunit.client.1.smithi193.stdout:2/294: fdatasync d0/df/f10 0 2024-04-25T15:49:35.055 INFO:tasks.workunit.client.0.smithi110.stdout:5/638: dwrite d4/d1d/d1f/d35/d3d/d52/d97/fa1 [0,4194304] 0 2024-04-25T15:49:35.057 INFO:tasks.workunit.client.0.smithi110.stdout:5/639: chown d4/d1d/d1f/d35/d3d/d52/d97 893630 1 2024-04-25T15:49:35.058 INFO:tasks.workunit.client.0.smithi110.stdout:2/560: mkdir de/d40/d5b/d66/da9 0 2024-04-25T15:49:35.060 INFO:tasks.workunit.client.0.smithi110.stdout:8/737: rename d14/d21/d26/d6a/d7e/da6/de8/d55/f6d to d14/d32/d36/db7/ff7 0 2024-04-25T15:49:35.061 INFO:tasks.workunit.client.0.smithi110.stdout:8/738: chown d14/d21/f92 109 1 2024-04-25T15:49:35.062 INFO:tasks.workunit.client.0.smithi110.stdout:8/739: stat d14/d32/d36/d48/ca0 0 2024-04-25T15:49:35.062 INFO:tasks.workunit.client.0.smithi110.stdout:8/740: fsync d14/f16 0 2024-04-25T15:49:35.063 INFO:tasks.workunit.client.1.smithi193.stdout:1/255: creat dc/d35/f51 x:0 0 0 2024-04-25T15:49:35.063 INFO:tasks.workunit.client.1.smithi193.stdout:1/256: stat dc/d35/l38 0 2024-04-25T15:49:35.064 INFO:tasks.workunit.client.0.smithi110.stdout:9/529: mkdir d9/dad 0 2024-04-25T15:49:35.064 INFO:tasks.workunit.client.0.smithi110.stdout:9/530: chown f8 34985 1 2024-04-25T15:49:35.065 INFO:tasks.workunit.client.1.smithi193.stdout:0/320: dread d0/d49/d5/d7/dd/d16/d26/d2e/f4b [0,4194304] 0 2024-04-25T15:49:35.066 INFO:tasks.workunit.client.0.smithi110.stdout:1/685: unlink d0/d14/d5f/d7d/d72/c85 0 2024-04-25T15:49:35.067 INFO:tasks.workunit.client.0.smithi110.stdout:1/686: fsync d0/d51/f52 0 2024-04-25T15:49:35.068 INFO:tasks.workunit.client.1.smithi193.stdout:0/321: write d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f59 [5112334,55025] 0 2024-04-25T15:49:35.069 INFO:tasks.workunit.client.1.smithi193.stdout:5/267: truncate d17/d18/d30/f35 3274799 0 2024-04-25T15:49:35.075 INFO:tasks.workunit.client.0.smithi110.stdout:6/562: chown d1/d12/c52 1667833 1 2024-04-25T15:49:35.077 INFO:tasks.workunit.client.0.smithi110.stdout:4/604: creat d13/d24/d2d/d44/d90/fc5 x:0 0 0 2024-04-25T15:49:35.078 INFO:tasks.workunit.client.1.smithi193.stdout:7/256: truncate d12/f1a 4212959 0 2024-04-25T15:49:35.078 INFO:tasks.workunit.client.1.smithi193.stdout:7/257: fsync d12/d13/f17 0 2024-04-25T15:49:35.081 INFO:tasks.workunit.client.1.smithi193.stdout:4/306: rmdir db/d13/d1e/d1f/d56/d57/d65 0 2024-04-25T15:49:35.083 INFO:tasks.workunit.client.1.smithi193.stdout:7/258: dwrite d12/d13/f14 [0,4194304] 0 2024-04-25T15:49:35.084 INFO:tasks.workunit.client.0.smithi110.stdout:8/741: unlink d14/d21/d26/d6a/d7e/da6/de8/d55/l9e 0 2024-04-25T15:49:35.085 INFO:tasks.workunit.client.1.smithi193.stdout:2/295: creat d0/d9/d15/f78 x:0 0 0 2024-04-25T15:49:35.086 INFO:tasks.workunit.client.0.smithi110.stdout:9/531: mknod d9/db/d58/cae 0 2024-04-25T15:49:35.090 INFO:tasks.workunit.client.1.smithi193.stdout:1/257: mknod dc/d35/d3d/c52 0 2024-04-25T15:49:35.092 INFO:tasks.workunit.client.0.smithi110.stdout:1/687: symlink d0/d14/d5f/d7d/d72/d61/ld1 0 2024-04-25T15:49:35.093 INFO:tasks.workunit.client.1.smithi193.stdout:9/315: truncate df/d2c/d2f/d4e/d1c/f3d 872261 0 2024-04-25T15:49:35.093 INFO:tasks.workunit.client.0.smithi110.stdout:1/688: chown d0/d14/d5f/d6f/c6a 1889466379 1 2024-04-25T15:49:35.093 INFO:tasks.workunit.client.0.smithi110.stdout:1/689: chown d0/d25/c39 543555784 1 2024-04-25T15:49:35.094 INFO:tasks.workunit.client.1.smithi193.stdout:6/301: creat dd/d12/d19/d45/f55 x:0 0 0 2024-04-25T15:49:35.097 INFO:tasks.workunit.client.0.smithi110.stdout:6/563: fsync d1/db/d22/d33/f3b 0 2024-04-25T15:49:35.098 INFO:tasks.workunit.client.0.smithi110.stdout:6/564: dread - d1/db/d22/d90/d76/fca zero size 2024-04-25T15:49:35.100 INFO:tasks.workunit.client.0.smithi110.stdout:4/605: symlink d13/d24/d2d/d55/lc6 0 2024-04-25T15:49:35.100 INFO:tasks.workunit.client.0.smithi110.stdout:6/565: write d1/db/d22/d90/fcb [160514,50692] 0 2024-04-25T15:49:35.102 INFO:tasks.workunit.client.1.smithi193.stdout:5/268: rename d17/d18/l19 to d17/d18/d30/d34/d37/l61 0 2024-04-25T15:49:35.102 INFO:tasks.workunit.client.1.smithi193.stdout:5/269: truncate f16 5008504 0 2024-04-25T15:49:35.103 INFO:tasks.workunit.client.0.smithi110.stdout:2/561: creat de/d40/d6e/d52/d6d/d99/da1/faa x:0 0 0 2024-04-25T15:49:35.103 INFO:tasks.workunit.client.1.smithi193.stdout:8/269: link d2/d3/ff d2/d3/d3a/f54 0 2024-04-25T15:49:35.104 INFO:tasks.workunit.client.0.smithi110.stdout:2/562: chown de/d40/d6e/f8e 0 1 2024-04-25T15:49:35.105 INFO:tasks.workunit.client.0.smithi110.stdout:8/742: mkdir d14/d21/d26/d6a/d7e/da6/de8/d71/df8 0 2024-04-25T15:49:35.114 INFO:tasks.workunit.client.0.smithi110.stdout:1/690: rename d0/d25/d33/f24 to d0/d25/d33/fd2 0 2024-04-25T15:49:35.115 INFO:tasks.workunit.client.1.smithi193.stdout:4/307: mknod db/d4b/d51/d54/c66 0 2024-04-25T15:49:35.116 INFO:tasks.workunit.client.1.smithi193.stdout:4/308: write db/d13/d1e/d1f/d22/f5a [342116,69974] 0 2024-04-25T15:49:35.120 INFO:tasks.workunit.client.0.smithi110.stdout:4/606: mknod d13/d24/d2d/d44/d65/d99/daf/cc7 0 2024-04-25T15:49:35.123 INFO:tasks.workunit.client.0.smithi110.stdout:2/563: unlink de/d40/c78 0 2024-04-25T15:49:35.123 INFO:tasks.workunit.client.0.smithi110.stdout:9/532: dread d9/db/df/d2c/d3c/fa4 [0,4194304] 0 2024-04-25T15:49:35.124 INFO:tasks.workunit.client.0.smithi110.stdout:9/533: dread - d9/f95 zero size 2024-04-25T15:49:35.125 INFO:tasks.workunit.client.0.smithi110.stdout:8/743: mkdir d14/d32/d36/db7/dc9/df9 0 2024-04-25T15:49:35.127 INFO:tasks.workunit.client.1.smithi193.stdout:6/302: unlink dd/f11 0 2024-04-25T15:49:35.128 INFO:tasks.workunit.client.1.smithi193.stdout:3/341: sync 2024-04-25T15:49:35.129 INFO:tasks.workunit.client.1.smithi193.stdout:3/342: write d8/d13/d1f/f16 [2064489,61013] 0 2024-04-25T15:49:35.129 INFO:tasks.workunit.client.0.smithi110.stdout:7/614: getdents d1b/d29/d38/d55 0 2024-04-25T15:49:35.130 INFO:tasks.workunit.client.1.smithi193.stdout:3/343: chown d8/d30/f47 3879 1 2024-04-25T15:49:35.130 INFO:tasks.workunit.client.1.smithi193.stdout:3/344: truncate d8/d13/f41 5097482 0 2024-04-25T15:49:35.131 INFO:tasks.workunit.client.1.smithi193.stdout:3/345: read d8/d13/d4b/f5d [1429644,11238] 0 2024-04-25T15:49:35.133 INFO:tasks.workunit.client.0.smithi110.stdout:1/691: creat d0/d14/d5f/d6f/fd3 x:0 0 0 2024-04-25T15:49:35.134 INFO:tasks.workunit.client.1.smithi193.stdout:1/258: rename dc/d35/d3d/f42 to dc/d4a/d50/f53 0 2024-04-25T15:49:35.135 INFO:tasks.workunit.client.1.smithi193.stdout:2/296: truncate d0/df/f10 5024790 0 2024-04-25T15:49:35.135 INFO:tasks.workunit.client.0.smithi110.stdout:9/534: dread d9/db/df/d25/f51 [0,4194304] 0 2024-04-25T15:49:35.136 INFO:tasks.workunit.client.0.smithi110.stdout:6/566: fdatasync d1/db/d24/d2d/f89 0 2024-04-25T15:49:35.136 INFO:tasks.workunit.client.0.smithi110.stdout:9/535: dread - d9/f8c zero size 2024-04-25T15:49:35.136 INFO:tasks.workunit.client.1.smithi193.stdout:2/297: truncate d0/d42/f5b 997433 0 2024-04-25T15:49:35.137 INFO:tasks.workunit.client.1.smithi193.stdout:3/346: dwrite d8/d13/d4b/f5d [0,4194304] 0 2024-04-25T15:49:35.138 INFO:tasks.workunit.client.1.smithi193.stdout:8/270: fdatasync d2/d3/d20/f22 0 2024-04-25T15:49:35.142 INFO:tasks.workunit.client.1.smithi193.stdout:4/309: creat db/d13/f67 x:0 0 0 2024-04-25T15:49:35.144 INFO:tasks.workunit.client.1.smithi193.stdout:7/259: mknod d12/d31/d39/d60/c61 0 2024-04-25T15:49:35.150 INFO:tasks.workunit.client.0.smithi110.stdout:7/615: creat d1b/d29/d38/d55/d95/fd1 x:0 0 0 2024-04-25T15:49:35.155 INFO:tasks.workunit.client.1.smithi193.stdout:0/322: creat d0/d49/d5/d7/f71 x:0 0 0 2024-04-25T15:49:35.157 INFO:tasks.workunit.client.0.smithi110.stdout:9/536: creat d9/d13/d84/faf x:0 0 0 2024-04-25T15:49:35.161 INFO:tasks.workunit.client.0.smithi110.stdout:6/567: rename d1/db/d22/d33/d63/dbd to d1/dd/d3c/d91/d8d/dce 0 2024-04-25T15:49:35.164 INFO:tasks.workunit.client.1.smithi193.stdout:5/270: rename d17/d18/d30/d38/l50 to d17/d18/d30/d34/d52/l62 0 2024-04-25T15:49:35.165 INFO:tasks.workunit.client.1.smithi193.stdout:5/271: read fe [611535,24910] 0 2024-04-25T15:49:35.166 INFO:tasks.workunit.client.1.smithi193.stdout:3/347: creat d8/d13/d14/d51/f6a x:0 0 0 2024-04-25T15:49:35.167 INFO:tasks.workunit.client.0.smithi110.stdout:8/744: getdents d14/d21/d26/d6a/d7e/da6/de8/d71/df8 0 2024-04-25T15:49:35.169 INFO:tasks.workunit.client.0.smithi110.stdout:7/616: mknod d1b/d29/d38/d55/dab/db2/db8/cd2 0 2024-04-25T15:49:35.170 INFO:tasks.workunit.client.1.smithi193.stdout:8/271: mknod d2/d36/c55 0 2024-04-25T15:49:35.175 INFO:tasks.workunit.client.1.smithi193.stdout:4/310: fdatasync db/d13/d1e/d1f/d22/f5d 0 2024-04-25T15:49:35.181 INFO:tasks.workunit.client.1.smithi193.stdout:4/311: dwrite db/d13/d1e/d1f/d22/f2c [0,4194304] 0 2024-04-25T15:49:35.182 INFO:tasks.workunit.client.1.smithi193.stdout:4/312: stat db/d32/c3e 0 2024-04-25T15:49:35.182 INFO:tasks.workunit.client.1.smithi193.stdout:4/313: stat db/d13/l3f 0 2024-04-25T15:49:35.183 INFO:tasks.workunit.client.1.smithi193.stdout:0/323: unlink d0/d49/d5/d7/dd/d16/f1c 0 2024-04-25T15:49:35.183 INFO:tasks.workunit.client.1.smithi193.stdout:0/324: chown d0/c23 128 1 2024-04-25T15:49:35.186 INFO:tasks.workunit.client.0.smithi110.stdout:8/745: creat d14/d32/d36/d48/d8f/ffa x:0 0 0 2024-04-25T15:49:35.194 INFO:tasks.workunit.client.0.smithi110.stdout:9/537: rename d9/l20 to d9/d13/lb0 0 2024-04-25T15:49:35.195 INFO:tasks.workunit.client.0.smithi110.stdout:9/538: write d9/db/d7d/f8b [1030255,63378] 0 2024-04-25T15:49:35.196 INFO:tasks.workunit.client.1.smithi193.stdout:6/303: rename dd/d12/d2d/f37 to dd/d2f/f56 0 2024-04-25T15:49:35.197 INFO:tasks.workunit.client.0.smithi110.stdout:4/607: getdents d13 0 2024-04-25T15:49:35.197 INFO:tasks.workunit.client.0.smithi110.stdout:6/568: dread d1/f26 [0,4194304] 0 2024-04-25T15:49:35.198 INFO:tasks.workunit.client.1.smithi193.stdout:7/260: dread d12/d1c/d45/f48 [0,4194304] 0 2024-04-25T15:49:35.199 INFO:tasks.workunit.client.1.smithi193.stdout:7/261: write d12/d31/f58 [546803,67110] 0 2024-04-25T15:49:35.199 INFO:tasks.workunit.client.0.smithi110.stdout:2/564: link de/d40/d6e/l71 de/d40/d70/lab 0 2024-04-25T15:49:35.201 INFO:tasks.workunit.client.0.smithi110.stdout:8/746: symlink d14/d21/d26/d6a/d7e/da6/lfb 0 2024-04-25T15:49:35.201 INFO:tasks.workunit.client.0.smithi110.stdout:1/692: write d0/d25/d33/f50 [1654582,1672] 0 2024-04-25T15:49:35.201 INFO:tasks.workunit.client.0.smithi110.stdout:8/747: fsync d14/f81 0 2024-04-25T15:49:35.203 INFO:tasks.workunit.client.0.smithi110.stdout:8/748: fsync d14/d21/d23/f7c 0 2024-04-25T15:49:35.204 INFO:tasks.workunit.client.0.smithi110.stdout:7/617: link d1b/d29/d46/f47 d1b/d29/d38/d8b/d89/dc6/fd3 0 2024-04-25T15:49:35.207 INFO:tasks.workunit.client.0.smithi110.stdout:9/539: mknod d9/d44/d5e/d80/cb1 0 2024-04-25T15:49:35.207 INFO:tasks.workunit.client.0.smithi110.stdout:9/540: stat d9/db/df/d4d/f78 0 2024-04-25T15:49:35.209 INFO:tasks.workunit.client.0.smithi110.stdout:6/569: read d1/dd/d3c/d91/f5a [1233948,113309] 0 2024-04-25T15:49:35.213 INFO:tasks.workunit.client.0.smithi110.stdout:4/608: truncate d13/d24/d2d/d44/d65/d99/daf/d56/d88/f41 1586525 0 2024-04-25T15:49:35.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/259: truncate f6 989905 0 2024-04-25T15:49:35.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/260: stat dc/f1e 0 2024-04-25T15:49:35.217 INFO:tasks.workunit.client.1.smithi193.stdout:9/316: getdents df/d2c/d2f/d4e/d1c/d26/d3f/d41 0 2024-04-25T15:49:35.217 INFO:tasks.workunit.client.0.smithi110.stdout:1/693: write d0/d23/f8a [5673412,95107] 0 2024-04-25T15:49:35.234 INFO:tasks.workunit.client.0.smithi110.stdout:2/565: mkdir de/d40/d5b/d66/da9/dac 0 2024-04-25T15:49:35.236 INFO:tasks.workunit.client.0.smithi110.stdout:7/618: write d1b/d29/d38/d8b/d89/dc6/fd3 [2588108,55989] 0 2024-04-25T15:49:35.239 INFO:tasks.workunit.client.0.smithi110.stdout:7/619: truncate d1b/d29/d38/d55/f8d 279480 0 2024-04-25T15:49:35.239 INFO:tasks.workunit.client.0.smithi110.stdout:6/570: rmdir d1/db/d22/d90 39 2024-04-25T15:49:35.240 INFO:tasks.workunit.client.0.smithi110.stdout:7/620: write d1b/d29/fa4 [827418,110265] 0 2024-04-25T15:49:35.240 INFO:tasks.workunit.client.0.smithi110.stdout:6/571: write d1/db/da0/fb6 [2289917,25049] 0 2024-04-25T15:49:35.241 INFO:tasks.workunit.client.0.smithi110.stdout:7/621: chown d1b/c49 2803661 1 2024-04-25T15:49:35.244 INFO:tasks.workunit.client.1.smithi193.stdout:0/325: dwrite d0/f1e [0,4194304] 0 2024-04-25T15:49:35.244 INFO:tasks.workunit.client.1.smithi193.stdout:2/298: rename d0/d44/f47 to d0/d9/d15/f79 0 2024-04-25T15:49:35.245 INFO:tasks.workunit.client.1.smithi193.stdout:2/299: fsync d0/d37/f67 0 2024-04-25T15:49:35.247 INFO:tasks.workunit.client.0.smithi110.stdout:1/694: unlink d0/d25/f3d 0 2024-04-25T15:49:35.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/300: write d0/d42/f5b [237654,61288] 0 2024-04-25T15:49:35.249 INFO:tasks.workunit.client.0.smithi110.stdout:8/749: mknod d14/d21/d26/d6a/de9/cfc 0 2024-04-25T15:49:35.250 INFO:tasks.workunit.client.0.smithi110.stdout:4/609: write d13/d24/d2d/f4f [4211411,96301] 0 2024-04-25T15:49:35.250 INFO:tasks.workunit.client.0.smithi110.stdout:8/750: chown d14/f5c 1740413 1 2024-04-25T15:49:35.251 INFO:tasks.workunit.client.1.smithi193.stdout:7/262: mkdir d12/d31/d37/d62 0 2024-04-25T15:49:35.251 INFO:tasks.workunit.client.0.smithi110.stdout:6/572: read d1/db/f44 [815550,7983] 0 2024-04-25T15:49:35.252 INFO:tasks.workunit.client.1.smithi193.stdout:7/263: truncate d12/d1e/f3c 178188 0 2024-04-25T15:49:35.255 INFO:tasks.workunit.client.1.smithi193.stdout:8/272: mkdir d2/d3/d3a/d50/d56 0 2024-04-25T15:49:35.257 INFO:tasks.workunit.client.1.smithi193.stdout:4/314: symlink db/l68 0 2024-04-25T15:49:35.259 INFO:tasks.workunit.client.0.smithi110.stdout:9/541: dread d9/db/f1b [4194304,4194304] 0 2024-04-25T15:49:35.263 INFO:tasks.workunit.client.0.smithi110.stdout:8/751: write d14/d21/d26/d6a/d7e/da6/de8/d71/fd5 [1591434,44704] 0 2024-04-25T15:49:35.268 INFO:tasks.workunit.client.0.smithi110.stdout:8/752: write d14/d21/d26/d6a/d7e/da6/de8/d55/f7d [1739488,29189] 0 2024-04-25T15:49:35.268 INFO:tasks.workunit.client.0.smithi110.stdout:4/610: truncate d13/d24/d2d/d44/d65/d99/dbc/fc4 437974 0 2024-04-25T15:49:35.268 INFO:tasks.workunit.client.1.smithi193.stdout:7/264: mkdir d12/d1c/d63 0 2024-04-25T15:49:35.269 INFO:tasks.workunit.client.1.smithi193.stdout:7/265: fdatasync d12/d1c/f3b 0 2024-04-25T15:49:35.269 INFO:tasks.workunit.client.1.smithi193.stdout:7/266: chown d12/d1c/c46 9580 1 2024-04-25T15:49:35.272 INFO:tasks.workunit.client.0.smithi110.stdout:9/542: dwrite d9/d13/f27 [0,4194304] 0 2024-04-25T15:49:35.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/348: creat d8/d30/f6b x:0 0 0 2024-04-25T15:49:35.275 INFO:tasks.workunit.client.1.smithi193.stdout:9/317: dread df/d2d/f39 [0,4194304] 0 2024-04-25T15:49:35.277 INFO:tasks.workunit.client.1.smithi193.stdout:7/267: dwrite d12/d31/d39/f3a [4194304,4194304] 0 2024-04-25T15:49:35.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/273: unlink d2/d3/d3a/l44 0 2024-04-25T15:49:35.281 INFO:tasks.workunit.client.0.smithi110.stdout:1/695: dread d0/d14/d5f/d7d/d9c/fa0 [0,4194304] 0 2024-04-25T15:49:35.282 INFO:tasks.workunit.client.1.smithi193.stdout:8/274: chown d2/d3/d3a/f4c 1634 1 2024-04-25T15:49:35.284 INFO:tasks.workunit.client.1.smithi193.stdout:5/272: dwrite d17/f1f [0,4194304] 0 2024-04-25T15:49:35.284 INFO:tasks.workunit.client.1.smithi193.stdout:4/315: rmdir db/d32 39 2024-04-25T15:49:35.284 INFO:tasks.workunit.client.1.smithi193.stdout:2/301: dread d0/df/f10 [0,4194304] 0 2024-04-25T15:49:35.285 INFO:tasks.workunit.client.1.smithi193.stdout:5/273: read d17/f2a [2260455,11671] 0 2024-04-25T15:49:35.285 INFO:tasks.workunit.client.1.smithi193.stdout:2/302: readlink d0/df/l68 0 2024-04-25T15:49:35.291 INFO:tasks.workunit.client.1.smithi193.stdout:6/304: mkdir dd/d2f/d57 0 2024-04-25T15:49:35.297 INFO:tasks.workunit.client.1.smithi193.stdout:9/318: mknod df/d11/c56 0 2024-04-25T15:49:35.300 INFO:tasks.workunit.client.1.smithi193.stdout:7/268: creat d12/d31/d39/d60/f64 x:0 0 0 2024-04-25T15:49:35.304 INFO:tasks.workunit.client.1.smithi193.stdout:8/275: unlink d2/d3/d27/c2d 0 2024-04-25T15:49:35.308 INFO:tasks.workunit.client.1.smithi193.stdout:4/316: mkdir db/d13/d1e/d1f/d56/d69 0 2024-04-25T15:49:35.315 INFO:tasks.workunit.client.1.smithi193.stdout:8/276: rename d2/fa to d2/d3/d20/d45/d29/f57 0 2024-04-25T15:49:35.321 INFO:tasks.workunit.client.1.smithi193.stdout:1/261: sync 2024-04-25T15:49:35.322 INFO:tasks.workunit.client.1.smithi193.stdout:2/303: mknod d0/c7a 0 2024-04-25T15:49:35.322 INFO:tasks.workunit.client.1.smithi193.stdout:1/262: chown dc/f30 219304 1 2024-04-25T15:49:35.325 INFO:tasks.workunit.client.1.smithi193.stdout:4/317: dwrite db/d32/d45/f4f [0,4194304] 0 2024-04-25T15:49:35.326 INFO:tasks.workunit.client.1.smithi193.stdout:4/318: write db/d13/d1e/d1f/d22/f2c [726928,46425] 0 2024-04-25T15:49:35.326 INFO:tasks.workunit.client.1.smithi193.stdout:7/269: creat d12/d1c/d63/f65 x:0 0 0 2024-04-25T15:49:35.328 INFO:tasks.workunit.client.1.smithi193.stdout:4/319: dread - db/d13/d1e/d1f/f63 zero size 2024-04-25T15:49:35.332 INFO:tasks.workunit.client.1.smithi193.stdout:2/304: mkdir d0/d9/d15/d7b 0 2024-04-25T15:49:35.338 INFO:tasks.workunit.client.1.smithi193.stdout:1/263: dwrite dc/d35/f3a [0,4194304] 0 2024-04-25T15:49:35.338 INFO:tasks.workunit.client.1.smithi193.stdout:9/319: creat df/f57 x:0 0 0 2024-04-25T15:49:35.339 INFO:tasks.workunit.client.0.smithi110.stdout:8/753: symlink d14/d32/dd6/lfd 0 2024-04-25T15:49:35.340 INFO:tasks.workunit.client.1.smithi193.stdout:1/264: chown dc/f12 5529487 1 2024-04-25T15:49:35.340 INFO:tasks.workunit.client.1.smithi193.stdout:9/320: fdatasync df/d2d/f31 0 2024-04-25T15:49:35.340 INFO:tasks.workunit.client.1.smithi193.stdout:1/265: write dc/f36 [3341694,53815] 0 2024-04-25T15:49:35.340 INFO:tasks.workunit.client.1.smithi193.stdout:1/266: readlink l3 0 2024-04-25T15:49:35.340 INFO:tasks.workunit.client.1.smithi193.stdout:9/321: write df/d2c/d2f/f49 [22591,107734] 0 2024-04-25T15:49:35.343 INFO:tasks.workunit.client.0.smithi110.stdout:4/611: mkdir d13/d24/d2d/d44/d65/d99/dbc/dc8 0 2024-04-25T15:49:35.344 INFO:tasks.workunit.client.0.smithi110.stdout:4/612: chown d13/d24/d2d/f62 6 1 2024-04-25T15:49:35.345 INFO:tasks.workunit.client.1.smithi193.stdout:8/277: creat d2/d3/d20/d45/d4a/d53/f58 x:0 0 0 2024-04-25T15:49:35.347 INFO:tasks.workunit.client.1.smithi193.stdout:7/270: symlink d12/d1c/d45/l66 0 2024-04-25T15:49:35.356 INFO:tasks.workunit.client.1.smithi193.stdout:1/267: creat dc/d4a/f54 x:0 0 0 2024-04-25T15:49:35.359 INFO:tasks.workunit.client.0.smithi110.stdout:9/543: symlink d9/lb2 0 2024-04-25T15:49:35.360 INFO:tasks.workunit.client.1.smithi193.stdout:8/278: mknod d2/d3/d20/d45/d29/c59 0 2024-04-25T15:49:35.361 INFO:tasks.workunit.client.0.smithi110.stdout:1/696: creat d0/d14/d5f/d7d/d72/d88/db6/fd4 x:0 0 0 2024-04-25T15:49:35.363 INFO:tasks.workunit.client.1.smithi193.stdout:7/271: symlink d12/d31/d39/l67 0 2024-04-25T15:49:35.363 INFO:tasks.workunit.client.1.smithi193.stdout:6/305: dwrite dd/d12/f4d [4194304,4194304] 0 2024-04-25T15:49:35.364 INFO:tasks.workunit.client.0.smithi110.stdout:2/566: getdents de/d40/d5b 0 2024-04-25T15:49:35.366 INFO:tasks.workunit.client.0.smithi110.stdout:7/622: getdents d1b/d29/d38 0 2024-04-25T15:49:35.367 INFO:tasks.workunit.client.0.smithi110.stdout:9/544: dwrite d9/db/df/d2c/f70 [0,4194304] 0 2024-04-25T15:49:35.368 INFO:tasks.workunit.client.1.smithi193.stdout:2/305: mknod d0/c7c 0 2024-04-25T15:49:35.369 INFO:tasks.workunit.client.1.smithi193.stdout:6/306: dwrite dd/d12/d19/f51 [0,4194304] 0 2024-04-25T15:49:35.370 INFO:tasks.workunit.client.1.smithi193.stdout:0/326: sync 2024-04-25T15:49:35.370 INFO:tasks.workunit.client.1.smithi193.stdout:6/307: chown dd/d12/d19/d1c/f20 2018390 1 2024-04-25T15:49:35.371 INFO:tasks.workunit.client.1.smithi193.stdout:6/308: truncate dd/d2f/f50 608958 0 2024-04-25T15:49:35.373 INFO:tasks.workunit.client.0.smithi110.stdout:6/573: truncate d1/db/da0/fc8 2053208 0 2024-04-25T15:49:35.375 INFO:tasks.workunit.client.1.smithi193.stdout:1/268: write dc/f12 [3690469,59634] 0 2024-04-25T15:49:35.377 INFO:tasks.workunit.client.1.smithi193.stdout:6/309: dwrite dd/d12/d19/d1c/f22 [0,4194304] 0 2024-04-25T15:49:35.379 INFO:tasks.workunit.client.0.smithi110.stdout:1/697: rename d0/d14/d5f/d6f/fd3 to d0/d51/dc8/fd5 0 2024-04-25T15:49:35.383 INFO:tasks.workunit.client.0.smithi110.stdout:2/567: mkdir de/d40/d5b/d66/dad 0 2024-04-25T15:49:35.384 INFO:tasks.workunit.client.1.smithi193.stdout:4/320: creat db/f6a x:0 0 0 2024-04-25T15:49:35.385 INFO:tasks.workunit.client.0.smithi110.stdout:7/623: mknod d1b/d29/d87/d8e/cd4 0 2024-04-25T15:49:35.385 INFO:tasks.workunit.client.0.smithi110.stdout:7/624: readlink d1b/d29/d46/l66 0 2024-04-25T15:49:35.391 INFO:tasks.workunit.client.0.smithi110.stdout:6/574: chown d1/dd/d3c/lb1 368 1 2024-04-25T15:49:35.393 INFO:tasks.workunit.client.0.smithi110.stdout:4/613: dwrite d13/d24/d2d/d5e/f98 [0,4194304] 0 2024-04-25T15:49:35.402 INFO:tasks.workunit.client.0.smithi110.stdout:6/575: dwrite d1/db/d24/d2d/fba [0,4194304] 0 2024-04-25T15:49:35.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/274: sync 2024-04-25T15:49:35.403 INFO:tasks.workunit.client.1.smithi193.stdout:3/349: sync 2024-04-25T15:49:35.404 INFO:tasks.workunit.client.1.smithi193.stdout:9/322: truncate df/d2c/d2f/d4e/d1c/f21 2609127 0 2024-04-25T15:49:35.405 INFO:tasks.workunit.client.0.smithi110.stdout:8/754: dwrite d14/d21/d26/d6a/d7e/da6/de8/d55/fe7 [0,4194304] 0 2024-04-25T15:49:35.405 INFO:tasks.workunit.client.1.smithi193.stdout:6/310: mknod dd/d12/d19/d1c/d40/d3c/c58 0 2024-04-25T15:49:35.406 INFO:tasks.workunit.client.1.smithi193.stdout:6/311: fdatasync dd/d12/d19/d1c/f2c 0 2024-04-25T15:49:35.410 INFO:tasks.workunit.client.1.smithi193.stdout:7/272: write f2 [2996260,62316] 0 2024-04-25T15:49:35.411 INFO:tasks.workunit.client.1.smithi193.stdout:7/273: readlink d12/d1c/l57 0 2024-04-25T15:49:35.411 INFO:tasks.workunit.client.1.smithi193.stdout:3/350: dwrite d8/d13/f2d [0,4194304] 0 2024-04-25T15:49:35.413 INFO:tasks.workunit.client.1.smithi193.stdout:4/321: creat db/d13/d1e/d1f/d22/f6b x:0 0 0 2024-04-25T15:49:35.415 INFO:tasks.workunit.client.1.smithi193.stdout:4/322: dread db/d32/d45/f60 [0,4194304] 0 2024-04-25T15:49:35.418 INFO:tasks.workunit.client.1.smithi193.stdout:2/306: truncate d0/df/f26 765320 0 2024-04-25T15:49:35.426 INFO:tasks.workunit.client.0.smithi110.stdout:2/568: mknod de/d40/d5b/d66/da9/dac/cae 0 2024-04-25T15:49:35.428 INFO:tasks.workunit.client.1.smithi193.stdout:5/275: mknod d17/d18/d3e/c63 0 2024-04-25T15:49:35.431 INFO:tasks.workunit.client.1.smithi193.stdout:8/279: dwrite d2/f48 [0,4194304] 0 2024-04-25T15:49:35.434 INFO:tasks.workunit.client.0.smithi110.stdout:9/545: rename d9/dad to d9/d13/d5b/d61/db3 0 2024-04-25T15:49:35.436 INFO:tasks.workunit.client.0.smithi110.stdout:9/546: write d9/db/df/d2c/d3c/f89 [1566018,39188] 0 2024-04-25T15:49:35.437 INFO:tasks.workunit.client.0.smithi110.stdout:9/547: chown d9/la3 79832 1 2024-04-25T15:49:35.438 INFO:tasks.workunit.client.0.smithi110.stdout:4/614: mkdir d13/d24/d2d/d44/d65/dc9 0 2024-04-25T15:49:35.441 INFO:tasks.workunit.client.1.smithi193.stdout:7/274: mkdir d12/d1e/d68 0 2024-04-25T15:49:35.445 INFO:tasks.workunit.client.1.smithi193.stdout:1/269: truncate dc/d35/f3b 265277 0 2024-04-25T15:49:35.448 INFO:tasks.workunit.client.0.smithi110.stdout:8/755: write d14/d21/f72 [579899,63485] 0 2024-04-25T15:49:35.451 INFO:tasks.workunit.client.0.smithi110.stdout:2/569: mknod de/d40/d5b/d5c/caf 0 2024-04-25T15:49:35.451 INFO:tasks.workunit.client.0.smithi110.stdout:3/663: sync 2024-04-25T15:49:35.451 INFO:tasks.workunit.client.0.smithi110.stdout:2/570: readlink de/d40/d6e/d52/d6d/l7c 0 2024-04-25T15:49:35.452 INFO:tasks.workunit.client.1.smithi193.stdout:4/323: unlink db/d13/d1e/d1f/d22/f27 0 2024-04-25T15:49:35.452 INFO:tasks.workunit.client.0.smithi110.stdout:3/664: chown d0/d2e/d88/dbf/le2 6 1 2024-04-25T15:49:35.453 INFO:tasks.workunit.client.0.smithi110.stdout:7/625: write d1b/d29/d46/f6a [3429926,19603] 0 2024-04-25T15:49:35.453 INFO:tasks.workunit.client.0.smithi110.stdout:3/665: truncate d0/fc9 4753084 0 2024-04-25T15:49:35.454 INFO:tasks.workunit.client.0.smithi110.stdout:7/626: chown f2 0 1 2024-04-25T15:49:35.456 INFO:tasks.workunit.client.1.smithi193.stdout:0/327: link d0/d49/l4a d0/d49/d5/d7/dd/d46/l72 0 2024-04-25T15:49:35.460 INFO:tasks.workunit.client.1.smithi193.stdout:5/276: rename d17/d18/d30/d34/d37/l55 to d17/d18/d30/d3b/l64 0 2024-04-25T15:49:35.461 INFO:tasks.workunit.client.1.smithi193.stdout:5/277: fdatasync d17/d18/f21 0 2024-04-25T15:49:35.461 INFO:tasks.workunit.client.0.smithi110.stdout:4/615: creat d13/d24/d2d/d44/da4/fca x:0 0 0 2024-04-25T15:49:35.462 INFO:tasks.workunit.client.1.smithi193.stdout:3/351: write d8/d30/f36 [603240,6600] 0 2024-04-25T15:49:35.462 INFO:tasks.workunit.client.1.smithi193.stdout:9/323: getdents df/d2c/d2f/d4e/d1c/d26/d3f/d41 0 2024-04-25T15:49:35.463 INFO:tasks.workunit.client.0.smithi110.stdout:4/616: write d13/d24/d2d/d7d/d92/f96 [36686,64725] 0 2024-04-25T15:49:35.465 INFO:tasks.workunit.client.0.smithi110.stdout:1/698: getdents d0/d25 0 2024-04-25T15:49:35.466 INFO:tasks.workunit.client.0.smithi110.stdout:6/576: write d1/db/d22/d90/d76/f81 [1022567,33829] 0 2024-04-25T15:49:35.466 INFO:tasks.workunit.client.1.smithi193.stdout:8/280: creat d2/d3/d20/d45/f5a x:0 0 0 2024-04-25T15:49:35.467 INFO:tasks.workunit.client.1.smithi193.stdout:3/352: dwrite d8/d30/f36 [0,4194304] 0 2024-04-25T15:49:35.468 INFO:tasks.workunit.client.1.smithi193.stdout:8/281: write d2/d1f/f2b [4871708,129461] 0 2024-04-25T15:49:35.469 INFO:tasks.workunit.client.1.smithi193.stdout:8/282: write d2/d1f/f2b [752552,119067] 0 2024-04-25T15:49:35.473 INFO:tasks.workunit.client.1.smithi193.stdout:3/353: dwrite f2 [0,4194304] 0 2024-04-25T15:49:35.475 INFO:tasks.workunit.client.1.smithi193.stdout:1/270: mkdir dc/d35/d3d/d55 0 2024-04-25T15:49:35.476 INFO:tasks.workunit.client.1.smithi193.stdout:1/271: write dc/f17 [672068,110053] 0 2024-04-25T15:49:35.477 INFO:tasks.workunit.client.0.smithi110.stdout:4/617: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcb 0 2024-04-25T15:49:35.478 INFO:tasks.workunit.client.1.smithi193.stdout:1/272: chown dc/f16 12426614 1 2024-04-25T15:49:35.479 INFO:tasks.workunit.client.1.smithi193.stdout:3/354: dread d8/d30/d31/f53 [0,4194304] 0 2024-04-25T15:49:35.480 INFO:tasks.workunit.client.1.smithi193.stdout:4/324: mknod db/d13/d1e/c6c 0 2024-04-25T15:49:35.480 INFO:tasks.workunit.client.1.smithi193.stdout:3/355: write d8/d30/f6b [437236,22116] 0 2024-04-25T15:49:35.486 INFO:tasks.workunit.client.1.smithi193.stdout:5/278: creat d17/d18/d3e/f65 x:0 0 0 2024-04-25T15:49:35.488 INFO:tasks.workunit.client.0.smithi110.stdout:8/756: link d14/d21/d26/d6a/d7e/da6/de8/d55/f88 d14/d32/d36/db7/ffe 0 2024-04-25T15:49:35.490 INFO:tasks.workunit.client.1.smithi193.stdout:6/312: creat dd/f59 x:0 0 0 2024-04-25T15:49:35.491 INFO:tasks.workunit.client.0.smithi110.stdout:2/571: link de/d40/d5b/l98 de/d40/lb0 0 2024-04-25T15:49:35.492 INFO:tasks.workunit.client.0.smithi110.stdout:1/699: getdents d0/d23 0 2024-04-25T15:49:35.494 INFO:tasks.workunit.client.0.smithi110.stdout:8/757: dwrite d14/d21/d26/dab/fb1 [0,4194304] 0 2024-04-25T15:49:35.495 INFO:tasks.workunit.client.0.smithi110.stdout:2/572: symlink de/d40/d5b/d66/lb1 0 2024-04-25T15:49:35.501 INFO:tasks.workunit.client.1.smithi193.stdout:1/273: symlink dc/d4a/d50/l56 0 2024-04-25T15:49:35.502 INFO:tasks.workunit.client.0.smithi110.stdout:8/758: fdatasync d14/d21/d26/d6a/d7e/da6/de8/f57 0 2024-04-25T15:49:35.505 INFO:tasks.workunit.client.1.smithi193.stdout:4/325: unlink l7 0 2024-04-25T15:49:35.505 INFO:tasks.workunit.client.0.smithi110.stdout:6/577: dread d1/d12/f1f [0,4194304] 0 2024-04-25T15:49:35.507 INFO:tasks.workunit.client.1.smithi193.stdout:3/356: creat d8/d13/d14/d51/f6c x:0 0 0 2024-04-25T15:49:35.507 INFO:tasks.workunit.client.0.smithi110.stdout:2/573: dwrite de/f28 [4194304,4194304] 0 2024-04-25T15:49:35.512 INFO:tasks.workunit.client.0.smithi110.stdout:2/574: rmdir de/d40/d6e/d52/d6d/d7d 39 2024-04-25T15:49:35.515 INFO:tasks.workunit.client.0.smithi110.stdout:8/759: rename d14/d21/d26/d6a/d7e/da6/de8/d55/f88 to d14/d32/d36/dde/fff 0 2024-04-25T15:49:35.515 INFO:tasks.workunit.client.1.smithi193.stdout:7/275: link d12/d1e/c47 d12/d1c/d45/c69 0 2024-04-25T15:49:35.517 INFO:tasks.workunit.client.1.smithi193.stdout:1/274: readlink dc/l1d 0 2024-04-25T15:49:35.518 INFO:tasks.workunit.client.0.smithi110.stdout:1/700: dread d0/d51/d6b/f7c [0,4194304] 0 2024-04-25T15:49:35.519 INFO:tasks.workunit.client.1.smithi193.stdout:2/307: getdents d0/d37/d25 0 2024-04-25T15:49:35.520 INFO:tasks.workunit.client.1.smithi193.stdout:4/326: creat db/d13/d1e/f6d x:0 0 0 2024-04-25T15:49:35.522 INFO:tasks.workunit.client.1.smithi193.stdout:0/328: rename d0/d49/d5/d7/dd/d16/d18/f4e to d0/d49/d5/d7/dd/f73 0 2024-04-25T15:49:35.523 INFO:tasks.workunit.client.1.smithi193.stdout:0/329: readlink d0/d49/d5/d7/dd/l25 0 2024-04-25T15:49:35.523 INFO:tasks.workunit.client.0.smithi110.stdout:1/701: creat d0/d25/d33/d8e/fd6 x:0 0 0 2024-04-25T15:49:35.525 INFO:tasks.workunit.client.0.smithi110.stdout:8/760: dread d14/d21/d23/f24 [0,4194304] 0 2024-04-25T15:49:35.526 INFO:tasks.workunit.client.0.smithi110.stdout:1/702: mknod d0/d51/dc8/cd7 0 2024-04-25T15:49:35.527 INFO:tasks.workunit.client.1.smithi193.stdout:4/327: dwrite db/d13/d1e/f6d [0,4194304] 0 2024-04-25T15:49:35.527 INFO:tasks.workunit.client.1.smithi193.stdout:2/308: mkdir d0/d9/d3d/d7d 0 2024-04-25T15:49:35.530 INFO:tasks.workunit.client.0.smithi110.stdout:8/761: dread d14/d21/d26/dab/fb1 [0,4194304] 0 2024-04-25T15:49:35.531 INFO:tasks.workunit.client.1.smithi193.stdout:5/279: creat d17/f66 x:0 0 0 2024-04-25T15:49:35.533 INFO:tasks.workunit.client.0.smithi110.stdout:1/703: dwrite d0/d51/f98 [0,4194304] 0 2024-04-25T15:49:35.537 INFO:tasks.workunit.client.0.smithi110.stdout:1/704: mkdir d0/d51/dd8 0 2024-04-25T15:49:35.538 INFO:tasks.workunit.client.0.smithi110.stdout:1/705: write d0/d25/d33/f50 [2017141,58647] 0 2024-04-25T15:49:35.541 INFO:tasks.workunit.client.0.smithi110.stdout:1/706: creat d0/d23/fd9 x:0 0 0 2024-04-25T15:49:35.543 INFO:tasks.workunit.client.0.smithi110.stdout:9/548: write d9/f39 [347769,59332] 0 2024-04-25T15:49:35.545 INFO:tasks.workunit.client.0.smithi110.stdout:9/549: chown d9/db/df/d2c/d3c/d56/da0 1364481 1 2024-04-25T15:49:35.547 INFO:tasks.workunit.client.0.smithi110.stdout:9/550: readlink d9/db/df/l5c 0 2024-04-25T15:49:35.547 INFO:tasks.workunit.client.1.smithi193.stdout:1/275: unlink dc/f21 0 2024-04-25T15:49:35.548 INFO:tasks.workunit.client.1.smithi193.stdout:2/309: mkdir d0/d42/d51/d4f/d65/d7e 0 2024-04-25T15:49:35.549 INFO:tasks.workunit.client.0.smithi110.stdout:9/551: chown d9/db/df/c14 53517 1 2024-04-25T15:49:35.549 INFO:tasks.workunit.client.0.smithi110.stdout:3/666: dwrite d0/d2e/f71 [0,4194304] 0 2024-04-25T15:49:35.551 INFO:tasks.workunit.client.0.smithi110.stdout:1/707: mkdir d0/d14/d5f/d7d/dda 0 2024-04-25T15:49:35.553 INFO:tasks.workunit.client.1.smithi193.stdout:7/276: dread d12/f2e [0,4194304] 0 2024-04-25T15:49:35.554 INFO:tasks.workunit.client.1.smithi193.stdout:7/277: dread - d12/d31/d39/d60/f64 zero size 2024-04-25T15:49:35.554 INFO:tasks.workunit.client.1.smithi193.stdout:7/278: chown d12/f2c 307705522 1 2024-04-25T15:49:35.557 INFO:tasks.workunit.client.1.smithi193.stdout:6/313: getdents dd/d12/d2d 0 2024-04-25T15:49:35.558 INFO:tasks.workunit.client.1.smithi193.stdout:6/314: write dd/d12/d19/d1c/d40/f23 [3434837,100649] 0 2024-04-25T15:49:35.558 INFO:tasks.workunit.client.0.smithi110.stdout:9/552: dwrite d9/f39 [0,4194304] 0 2024-04-25T15:49:35.561 INFO:tasks.workunit.client.0.smithi110.stdout:2/575: rename de/d40/d5b/d66 to de/d40/d5b/d5c/d5f/db2 0 2024-04-25T15:49:35.564 INFO:tasks.workunit.client.0.smithi110.stdout:3/667: unlink d0/f70 0 2024-04-25T15:49:35.568 INFO:tasks.workunit.client.0.smithi110.stdout:1/708: mkdir d0/d25/d33/d94/ddb 0 2024-04-25T15:49:35.569 INFO:tasks.workunit.client.1.smithi193.stdout:8/283: sync 2024-04-25T15:49:35.569 INFO:tasks.workunit.client.0.smithi110.stdout:1/709: write d0/d25/d33/f49 [2600112,126677] 0 2024-04-25T15:49:35.570 INFO:tasks.workunit.client.1.smithi193.stdout:3/357: getdents d8 0 2024-04-25T15:49:35.571 INFO:tasks.workunit.client.0.smithi110.stdout:9/553: mkdir d9/d13/d5b/d61/db3/db4 0 2024-04-25T15:49:35.571 INFO:tasks.workunit.client.1.smithi193.stdout:4/328: dread f9 [0,4194304] 0 2024-04-25T15:49:35.573 INFO:tasks.workunit.client.1.smithi193.stdout:2/310: mknod d0/d9/d15/c7f 0 2024-04-25T15:49:35.575 INFO:tasks.workunit.client.1.smithi193.stdout:1/276: creat dc/d4a/d4d/f57 x:0 0 0 2024-04-25T15:49:35.578 INFO:tasks.workunit.client.1.smithi193.stdout:7/279: creat d12/d1c/d45/f6a x:0 0 0 2024-04-25T15:49:35.578 INFO:tasks.workunit.client.0.smithi110.stdout:6/578: write d1/dd/d3c/f7c [197593,99883] 0 2024-04-25T15:49:35.580 INFO:tasks.workunit.client.0.smithi110.stdout:3/668: creat d0/d7/dd0/ddf/fe3 x:0 0 0 2024-04-25T15:49:35.580 INFO:tasks.workunit.client.1.smithi193.stdout:9/324: write df/d2c/d2f/d4e/d1c/d26/f35 [133916,35598] 0 2024-04-25T15:49:35.580 INFO:tasks.workunit.client.1.smithi193.stdout:6/315: rmdir dd/d12/d19/d1c/d40/d3c 39 2024-04-25T15:49:35.582 INFO:tasks.workunit.client.1.smithi193.stdout:9/325: fdatasync df/d2c/d2f/d4e/d1c/d26/d3f/f43 0 2024-04-25T15:49:35.582 INFO:tasks.workunit.client.1.smithi193.stdout:6/316: write dd/d12/d19/d1c/f26 [496192,129252] 0 2024-04-25T15:49:35.584 INFO:tasks.workunit.client.0.smithi110.stdout:9/554: dread d9/d44/fab [0,4194304] 0 2024-04-25T15:49:35.588 INFO:tasks.workunit.client.1.smithi193.stdout:3/358: mkdir d8/d13/d14/d27/d6d 0 2024-04-25T15:49:35.588 INFO:tasks.workunit.client.1.smithi193.stdout:3/359: fdatasync d8/d13/f64 0 2024-04-25T15:49:35.589 INFO:tasks.workunit.client.1.smithi193.stdout:3/360: write d8/d13/d14/d51/f6a [180757,93352] 0 2024-04-25T15:49:35.590 INFO:tasks.workunit.client.1.smithi193.stdout:4/329: write db/d32/d45/f60 [165567,121768] 0 2024-04-25T15:49:35.591 INFO:tasks.workunit.client.1.smithi193.stdout:4/330: chown db/d4b/d51/d54/f5c 3487 1 2024-04-25T15:49:35.592 INFO:tasks.workunit.client.0.smithi110.stdout:3/669: creat d0/d16/dcb/d2f/d4c/db2/fe4 x:0 0 0 2024-04-25T15:49:35.593 INFO:tasks.workunit.client.1.smithi193.stdout:2/311: creat d0/d9/d15/d31/d50/f80 x:0 0 0 2024-04-25T15:49:35.595 INFO:tasks.workunit.client.1.smithi193.stdout:0/330: sync 2024-04-25T15:49:35.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/555: mkdir d9/d44/d5e/db5 0 2024-04-25T15:49:35.596 INFO:tasks.workunit.client.1.smithi193.stdout:0/331: chown d0/d49/d5/d7/dd/d16/d18/d28/c4f 13 1 2024-04-25T15:49:35.597 INFO:tasks.workunit.client.0.smithi110.stdout:9/556: write d9/d13/d5b/d61/f63 [789851,12687] 0 2024-04-25T15:49:35.598 INFO:tasks.workunit.client.0.smithi110.stdout:9/557: chown d9/d13/d5b/d61/db3 593695759 1 2024-04-25T15:49:35.603 INFO:tasks.workunit.client.1.smithi193.stdout:5/280: write d17/d18/f24 [4393747,100710] 0 2024-04-25T15:49:35.605 INFO:tasks.workunit.client.0.smithi110.stdout:3/670: fsync d0/d16/dcb/d2f/d33/f9e 0 2024-04-25T15:49:35.606 INFO:tasks.workunit.client.0.smithi110.stdout:3/671: stat d0/d2e/d88 0 2024-04-25T15:49:35.607 INFO:tasks.workunit.client.1.smithi193.stdout:1/277: dread dc/fe [0,4194304] 0 2024-04-25T15:49:35.609 INFO:tasks.workunit.client.1.smithi193.stdout:3/361: creat d8/d13/d4b/f6e x:0 0 0 2024-04-25T15:49:35.610 INFO:tasks.workunit.client.0.smithi110.stdout:1/710: write d0/d14/d5f/faa [459220,103963] 0 2024-04-25T15:49:35.611 INFO:tasks.workunit.client.1.smithi193.stdout:4/331: creat db/d32/f6e x:0 0 0 2024-04-25T15:49:35.612 INFO:tasks.workunit.client.0.smithi110.stdout:2/576: truncate de/f24 5217988 0 2024-04-25T15:49:35.612 INFO:tasks.workunit.client.1.smithi193.stdout:4/332: chown db/d4b/d51/d54/c66 839 1 2024-04-25T15:49:35.613 INFO:tasks.workunit.client.1.smithi193.stdout:4/333: chown db/d32/d40/c4a 10148 1 2024-04-25T15:49:35.613 INFO:tasks.workunit.client.1.smithi193.stdout:4/334: fsync db/d13/f67 0 2024-04-25T15:49:35.616 INFO:tasks.workunit.client.0.smithi110.stdout:1/711: write d0/d51/d6b/f7c [4328158,76410] 0 2024-04-25T15:49:35.617 INFO:tasks.workunit.client.1.smithi193.stdout:4/335: dread db/d13/f1d [0,4194304] 0 2024-04-25T15:49:35.618 INFO:tasks.workunit.client.0.smithi110.stdout:6/579: dwrite d1/db/d22/f38 [0,4194304] 0 2024-04-25T15:49:35.620 INFO:tasks.workunit.client.1.smithi193.stdout:0/332: readlink d0/d49/d5/d7/dd/d16/d18/d28/d56/l5f 0 2024-04-25T15:49:35.621 INFO:tasks.workunit.client.1.smithi193.stdout:0/333: write d0/d49/d5/d7/dd/d16/f33 [5411714,42341] 0 2024-04-25T15:49:35.622 INFO:tasks.workunit.client.0.smithi110.stdout:3/672: creat d0/d7/fe5 x:0 0 0 2024-04-25T15:49:35.625 INFO:tasks.workunit.client.0.smithi110.stdout:1/712: mknod d0/d14/d5f/d7d/d72/d88/cdc 0 2024-04-25T15:49:35.628 INFO:tasks.workunit.client.1.smithi193.stdout:5/281: symlink d17/d18/d30/d34/d37/d5d/l67 0 2024-04-25T15:49:35.628 INFO:tasks.workunit.client.0.smithi110.stdout:9/558: dwrite d9/db/df/f33 [0,4194304] 0 2024-04-25T15:49:35.628 INFO:tasks.workunit.client.1.smithi193.stdout:5/282: chown f11 6 1 2024-04-25T15:49:35.630 INFO:tasks.workunit.client.0.smithi110.stdout:6/580: unlink d1/db/d22/d33/d63/c71 0 2024-04-25T15:49:35.635 INFO:tasks.workunit.client.1.smithi193.stdout:8/284: rmdir d2/d3/d3a/d50/d56 0 2024-04-25T15:49:35.636 INFO:tasks.workunit.client.1.smithi193.stdout:7/280: truncate d12/d13/f14 2374123 0 2024-04-25T15:49:35.636 INFO:tasks.workunit.client.0.smithi110.stdout:9/559: dwrite d9/d44/f53 [0,4194304] 0 2024-04-25T15:49:35.637 INFO:tasks.workunit.client.0.smithi110.stdout:9/560: write d9/d13/f1c [3181051,130036] 0 2024-04-25T15:49:35.639 INFO:tasks.workunit.client.1.smithi193.stdout:1/278: readlink dc/l28 0 2024-04-25T15:49:35.639 INFO:tasks.workunit.client.0.smithi110.stdout:9/561: write d9/f96 [446985,120673] 0 2024-04-25T15:49:35.639 INFO:tasks.workunit.client.0.smithi110.stdout:2/577: write de/d40/d5b/d5c/d5f/f63 [9184445,22822] 0 2024-04-25T15:49:35.643 INFO:tasks.workunit.client.0.smithi110.stdout:1/713: rename d0/f18 to d0/d25/d33/fdd 0 2024-04-25T15:49:35.645 INFO:tasks.workunit.client.1.smithi193.stdout:5/283: dread d17/d18/d45/f47 [0,4194304] 0 2024-04-25T15:49:35.647 INFO:tasks.workunit.client.1.smithi193.stdout:2/312: symlink d0/l81 0 2024-04-25T15:49:35.648 INFO:tasks.workunit.client.1.smithi193.stdout:2/313: dread - d0/d9/d15/d31/d50/f6b zero size 2024-04-25T15:49:35.648 INFO:tasks.workunit.client.0.smithi110.stdout:6/581: creat d1/d48/fcf x:0 0 0 2024-04-25T15:49:35.648 INFO:tasks.workunit.client.0.smithi110.stdout:6/582: chown d1/d12/db8 396 1 2024-04-25T15:49:35.649 INFO:tasks.workunit.client.1.smithi193.stdout:2/314: fsync d0/d37/d25/d5f/d3a/f4a 0 2024-04-25T15:49:35.653 INFO:tasks.workunit.client.1.smithi193.stdout:4/336: mkdir db/d4b/d51/d54/d6f 0 2024-04-25T15:49:35.658 INFO:tasks.workunit.client.0.smithi110.stdout:1/714: mknod d0/d25/d33/d8e/cde 0 2024-04-25T15:49:35.658 INFO:tasks.workunit.client.0.smithi110.stdout:3/673: dwrite d0/d7/f2a [0,4194304] 0 2024-04-25T15:49:35.658 INFO:tasks.workunit.client.1.smithi193.stdout:4/337: dwrite db/d32/f5f [0,4194304] 0 2024-04-25T15:49:35.660 INFO:tasks.workunit.client.0.smithi110.stdout:6/583: truncate d1/d48/f6c 4521554 0 2024-04-25T15:49:35.661 INFO:tasks.workunit.client.0.smithi110.stdout:6/584: chown d1/d12/c52 6329 1 2024-04-25T15:49:35.662 INFO:tasks.workunit.client.1.smithi193.stdout:8/285: mknod d2/d3/d20/d45/d29/c5b 0 2024-04-25T15:49:35.662 INFO:tasks.workunit.client.1.smithi193.stdout:8/286: readlink d2/d3/lc 0 2024-04-25T15:49:35.663 INFO:tasks.workunit.client.1.smithi193.stdout:7/281: mknod d12/d13/c6b 0 2024-04-25T15:49:35.665 INFO:tasks.workunit.client.1.smithi193.stdout:7/282: dread d12/d1e/f3c [0,4194304] 0 2024-04-25T15:49:35.666 INFO:tasks.workunit.client.1.smithi193.stdout:1/279: mknod dc/d3c/c58 0 2024-04-25T15:49:35.667 INFO:tasks.workunit.client.0.smithi110.stdout:1/715: dread d0/d23/f30 [0,4194304] 0 2024-04-25T15:49:35.669 INFO:tasks.workunit.client.1.smithi193.stdout:7/283: dread f8 [0,4194304] 0 2024-04-25T15:49:35.670 INFO:tasks.workunit.client.1.smithi193.stdout:5/284: creat d17/d18/d30/d34/d37/d5d/f68 x:0 0 0 2024-04-25T15:49:35.680 INFO:tasks.workunit.client.0.smithi110.stdout:5/640: sync 2024-04-25T15:49:35.680 INFO:tasks.workunit.client.0.smithi110.stdout:0/576: sync 2024-04-25T15:49:35.682 INFO:tasks.workunit.client.0.smithi110.stdout:0/577: truncate d1/d6c/fd5 392742 0 2024-04-25T15:49:35.683 INFO:tasks.workunit.client.0.smithi110.stdout:2/578: creat de/d40/d6e/d52/d6d/fb3 x:0 0 0 2024-04-25T15:49:35.688 INFO:tasks.workunit.client.1.smithi193.stdout:5/285: dread d17/d18/d30/f32 [0,4194304] 0 2024-04-25T15:49:35.692 INFO:tasks.workunit.client.0.smithi110.stdout:1/716: creat d0/d51/fdf x:0 0 0 2024-04-25T15:49:35.693 INFO:tasks.workunit.client.0.smithi110.stdout:9/562: truncate d9/db/df/f33 500834 0 2024-04-25T15:49:35.695 INFO:tasks.workunit.client.1.smithi193.stdout:0/334: creat d0/d49/d5/d61/f74 x:0 0 0 2024-04-25T15:49:35.696 INFO:tasks.workunit.client.1.smithi193.stdout:0/335: write d0/d49/d5/d7/dd/f6e [110403,108477] 0 2024-04-25T15:49:35.697 INFO:tasks.workunit.client.0.smithi110.stdout:3/674: rename d0/c13 to d0/d16/dcb/d2f/d4c/d63/ce6 0 2024-04-25T15:49:35.697 INFO:tasks.workunit.client.1.smithi193.stdout:0/336: fsync d0/fc 0 2024-04-25T15:49:35.698 INFO:tasks.workunit.client.1.smithi193.stdout:9/326: getdents df/d2c/d2f/d4e 0 2024-04-25T15:49:35.698 INFO:tasks.workunit.client.1.smithi193.stdout:0/337: fsync d0/d49/d5/d7/dd/d16/f33 0 2024-04-25T15:49:35.698 INFO:tasks.workunit.client.1.smithi193.stdout:9/327: readlink l0 0 2024-04-25T15:49:35.700 INFO:tasks.workunit.client.1.smithi193.stdout:6/317: getdents dd/d12/d19/d1c/d40 0 2024-04-25T15:49:35.701 INFO:tasks.workunit.client.1.smithi193.stdout:6/318: dread dd/d2f/f50 [0,4194304] 0 2024-04-25T15:49:35.702 INFO:tasks.workunit.client.1.smithi193.stdout:6/319: write dd/d12/d19/d45/f55 [379944,115320] 0 2024-04-25T15:49:35.705 INFO:tasks.workunit.client.1.smithi193.stdout:8/287: rmdir d2/d3/d20 39 2024-04-25T15:49:35.705 INFO:tasks.workunit.client.1.smithi193.stdout:8/288: chown d2 190164 1 2024-04-25T15:49:35.706 INFO:tasks.workunit.client.1.smithi193.stdout:8/289: stat d2/d18/c39 0 2024-04-25T15:49:35.707 INFO:tasks.workunit.client.0.smithi110.stdout:2/579: creat de/d6f/fb4 x:0 0 0 2024-04-25T15:49:35.709 INFO:tasks.workunit.client.0.smithi110.stdout:9/563: creat d9/db/d7d/fb6 x:0 0 0 2024-04-25T15:49:35.709 INFO:tasks.workunit.client.0.smithi110.stdout:9/564: chown d9/d13 0 1 2024-04-25T15:49:35.711 INFO:tasks.workunit.client.0.smithi110.stdout:1/717: rename d0/d25/c69 to d0/d25/d33/d8e/ce0 0 2024-04-25T15:49:35.711 INFO:tasks.workunit.client.0.smithi110.stdout:5/641: write d4/d1d/d1f/d35/f9a [84588,50074] 0 2024-04-25T15:49:35.713 INFO:tasks.workunit.client.0.smithi110.stdout:6/585: rmdir d1/db/d24/d69/d75/dcc 0 2024-04-25T15:49:35.714 INFO:tasks.workunit.client.1.smithi193.stdout:3/362: link d8/d13/l2a d8/d30/d31/l6f 0 2024-04-25T15:49:35.714 INFO:tasks.workunit.client.1.smithi193.stdout:3/363: fdatasync f2 0 2024-04-25T15:49:35.718 INFO:tasks.workunit.client.0.smithi110.stdout:2/580: chown de/d40/d6e/d52/d6d/d7d/f85 2 1 2024-04-25T15:49:35.719 INFO:tasks.workunit.client.1.smithi193.stdout:2/315: creat d0/d42/d51/d4f/d65/d7e/f82 x:0 0 0 2024-04-25T15:49:35.719 INFO:tasks.workunit.client.1.smithi193.stdout:2/316: readlink d0/d37/d4d/l75 0 2024-04-25T15:49:35.719 INFO:tasks.workunit.client.0.smithi110.stdout:6/586: dwrite d1/db/d24/d2d/f7b [0,4194304] 0 2024-04-25T15:49:35.720 INFO:tasks.workunit.client.0.smithi110.stdout:9/565: creat d9/d13/d7c/fb7 x:0 0 0 2024-04-25T15:49:35.723 INFO:tasks.workunit.client.0.smithi110.stdout:1/718: rename d0/d14/c44 to d0/d14/d5f/d7d/d9c/ce1 0 2024-04-25T15:49:35.725 INFO:tasks.workunit.client.0.smithi110.stdout:5/642: mkdir d4/d1d/d1f/d35/d3d/d52/dd2 0 2024-04-25T15:49:35.730 INFO:tasks.workunit.client.0.smithi110.stdout:0/578: link d1/d47/d54/l5a d1/d23/d30/d33/d35/da1/db4/ld6 0 2024-04-25T15:49:35.736 INFO:tasks.workunit.client.0.smithi110.stdout:2/581: rename de/d40/c41 to de/d40/d5b/d5c/d5f/db2/cb5 0 2024-04-25T15:49:35.737 INFO:tasks.workunit.client.0.smithi110.stdout:2/582: write de/d40/d5b/d5c/d5f/db2/f9c [678954,10337] 0 2024-04-25T15:49:35.738 INFO:tasks.workunit.client.0.smithi110.stdout:1/719: mknod d0/d14/ce2 0 2024-04-25T15:49:35.744 INFO:tasks.workunit.client.0.smithi110.stdout:0/579: unlink d1/d6c/d7f/l6e 0 2024-04-25T15:49:35.747 INFO:tasks.workunit.client.0.smithi110.stdout:9/566: fdatasync d9/d2d/f38 0 2024-04-25T15:49:35.748 INFO:tasks.workunit.client.0.smithi110.stdout:6/587: write d1/f5e [851905,99560] 0 2024-04-25T15:49:35.749 INFO:tasks.workunit.client.1.smithi193.stdout:4/338: mkdir db/d13/d1e/d1f/d56/d69/d70 0 2024-04-25T15:49:35.750 INFO:tasks.workunit.client.0.smithi110.stdout:2/583: creat de/d40/d86/fb6 x:0 0 0 2024-04-25T15:49:35.753 INFO:tasks.workunit.client.1.smithi193.stdout:4/339: dread db/d32/d45/f4f [0,4194304] 0 2024-04-25T15:49:35.753 INFO:tasks.workunit.client.0.smithi110.stdout:2/584: write de/d40/d6e/d50/fa4 [671055,94850] 0 2024-04-25T15:49:35.754 INFO:tasks.workunit.client.0.smithi110.stdout:5/643: write d4/d1d/d1f/d35/d3d/f5d [8023416,73259] 0 2024-04-25T15:49:35.756 INFO:tasks.workunit.client.0.smithi110.stdout:6/588: dwrite d1/db/d24/d2d/f7b [0,4194304] 0 2024-04-25T15:49:35.760 INFO:tasks.workunit.client.0.smithi110.stdout:1/720: dread d0/d14/f62 [0,4194304] 0 2024-04-25T15:49:35.760 INFO:tasks.workunit.client.1.smithi193.stdout:3/364: creat d8/d30/d45/f70 x:0 0 0 2024-04-25T15:49:35.761 INFO:tasks.workunit.client.0.smithi110.stdout:0/580: symlink d1/d23/d30/ld7 0 2024-04-25T15:49:35.761 INFO:tasks.workunit.client.1.smithi193.stdout:0/338: dwrite d0/d49/d5/d7/fb [0,4194304] 0 2024-04-25T15:49:35.761 INFO:tasks.workunit.client.1.smithi193.stdout:3/365: chown d8/d13/d14/c5b 246163 1 2024-04-25T15:49:35.762 INFO:tasks.workunit.client.0.smithi110.stdout:0/581: dread - d1/d23/d30/d33/d5b/d72/d77/fd4 zero size 2024-04-25T15:49:35.763 INFO:tasks.workunit.client.0.smithi110.stdout:6/589: read f0 [2893882,116721] 0 2024-04-25T15:49:35.764 INFO:tasks.workunit.client.0.smithi110.stdout:5/644: mkdir d4/d1d/d1f/d35/d3d/d52/d75/dd3 0 2024-04-25T15:49:35.764 INFO:tasks.workunit.client.0.smithi110.stdout:0/582: truncate d1/d23/d81/fc3 105463 0 2024-04-25T15:49:35.767 INFO:tasks.workunit.client.1.smithi193.stdout:9/328: symlink df/d2c/d2f/l58 0 2024-04-25T15:49:35.768 INFO:tasks.workunit.client.1.smithi193.stdout:9/329: chown df/d2c/d34/f46 373026 1 2024-04-25T15:49:35.772 INFO:tasks.workunit.client.0.smithi110.stdout:5/645: dwrite d4/d1d/d1f/d35/f76 [0,4194304] 0 2024-04-25T15:49:35.775 INFO:tasks.workunit.client.1.smithi193.stdout:4/340: creat db/d4b/f71 x:0 0 0 2024-04-25T15:49:35.778 INFO:tasks.workunit.client.0.smithi110.stdout:2/585: getdents de/d40/d6e/d52 0 2024-04-25T15:49:35.780 INFO:tasks.workunit.client.1.smithi193.stdout:7/284: link d12/d13/c3e d12/c6c 0 2024-04-25T15:49:35.783 INFO:tasks.workunit.client.0.smithi110.stdout:9/567: getdents d9/db/df/d25 0 2024-04-25T15:49:35.785 INFO:tasks.workunit.client.1.smithi193.stdout:0/339: unlink d0/f1e 0 2024-04-25T15:49:35.786 INFO:tasks.workunit.client.0.smithi110.stdout:2/586: rmdir de/d40/d6e 39 2024-04-25T15:49:35.788 INFO:tasks.workunit.client.1.smithi193.stdout:5/286: dwrite d17/d18/d30/f35 [0,4194304] 0 2024-04-25T15:49:35.788 INFO:tasks.workunit.client.1.smithi193.stdout:3/366: mkdir d8/d30/d31/d71 0 2024-04-25T15:49:35.789 INFO:tasks.workunit.client.1.smithi193.stdout:6/320: sync 2024-04-25T15:49:35.790 INFO:tasks.workunit.client.1.smithi193.stdout:5/287: write d17/d18/d45/f56 [97620,77004] 0 2024-04-25T15:49:35.790 INFO:tasks.workunit.client.1.smithi193.stdout:2/317: symlink d0/d9/l83 0 2024-04-25T15:49:35.791 INFO:tasks.workunit.client.0.smithi110.stdout:9/568: mknod d9/d2d/cb8 0 2024-04-25T15:49:35.793 INFO:tasks.workunit.client.0.smithi110.stdout:9/569: write d9/db/d7d/fb6 [666846,107352] 0 2024-04-25T15:49:35.793 INFO:tasks.workunit.client.1.smithi193.stdout:0/340: dread d0/d49/d5/d7/dd/d16/d18/f22 [0,4194304] 0 2024-04-25T15:49:35.794 INFO:tasks.workunit.client.0.smithi110.stdout:1/721: dwrite d0/d14/fc0 [0,4194304] 0 2024-04-25T15:49:35.798 INFO:tasks.workunit.client.1.smithi193.stdout:3/367: dwrite d8/d13/d1f/f1c [0,4194304] 0 2024-04-25T15:49:35.800 INFO:tasks.workunit.client.0.smithi110.stdout:5/646: getdents d4/d1d/d1f/d35/d3d/d52/d97 0 2024-04-25T15:49:35.800 INFO:tasks.workunit.client.1.smithi193.stdout:3/368: chown d8/f39 0 1 2024-04-25T15:49:35.801 INFO:tasks.workunit.client.0.smithi110.stdout:5/647: stat d4/d1d/d1f/d35/fc6 0 2024-04-25T15:49:35.803 INFO:tasks.workunit.client.1.smithi193.stdout:1/280: getdents dc/d35/d3d 0 2024-04-25T15:49:35.804 INFO:tasks.workunit.client.1.smithi193.stdout:1/281: readlink dc/d4a/d50/l56 0 2024-04-25T15:49:35.804 INFO:tasks.workunit.client.1.smithi193.stdout:7/285: creat d12/d1c/f6d x:0 0 0 2024-04-25T15:49:35.805 INFO:tasks.workunit.client.1.smithi193.stdout:7/286: fdatasync d12/d13/f40 0 2024-04-25T15:49:35.806 INFO:tasks.workunit.client.0.smithi110.stdout:0/583: dread d1/d6c/d7f/fb5 [0,4194304] 0 2024-04-25T15:49:35.806 INFO:tasks.workunit.client.1.smithi193.stdout:7/287: readlink d12/d13/l2f 0 2024-04-25T15:49:35.808 INFO:tasks.workunit.client.0.smithi110.stdout:2/587: dwrite de/d40/f33 [0,4194304] 0 2024-04-25T15:49:35.809 INFO:tasks.workunit.client.0.smithi110.stdout:5/648: write d4/d1d/d1f/f64 [2518010,84187] 0 2024-04-25T15:49:35.809 INFO:tasks.workunit.client.0.smithi110.stdout:0/584: fdatasync d1/d23/d30/d33/d5b/d72/d77/fd4 0 2024-04-25T15:49:35.810 INFO:tasks.workunit.client.0.smithi110.stdout:9/570: symlink d9/d13/d5b/d61/lb9 0 2024-04-25T15:49:35.812 INFO:tasks.workunit.client.1.smithi193.stdout:6/321: read dd/f2e [4479997,31662] 0 2024-04-25T15:49:35.813 INFO:tasks.workunit.client.0.smithi110.stdout:5/649: dread - d4/d1d/d1f/d35/d7f/fa7 zero size 2024-04-25T15:49:35.814 INFO:tasks.workunit.client.1.smithi193.stdout:5/288: creat d17/d18/d30/d38/f69 x:0 0 0 2024-04-25T15:49:35.815 INFO:tasks.workunit.client.1.smithi193.stdout:2/318: symlink d0/d37/d25/d5f/l84 0 2024-04-25T15:49:35.818 INFO:tasks.workunit.client.1.smithi193.stdout:8/290: link d2/d3/d20/d45/d29/c5b d2/d3/d20/d45/d4a/d53/c5c 0 2024-04-25T15:49:35.819 INFO:tasks.workunit.client.1.smithi193.stdout:4/341: dread db/d13/f21 [0,4194304] 0 2024-04-25T15:49:35.821 INFO:tasks.workunit.client.0.smithi110.stdout:9/571: dwrite d9/d13/d84/faf [0,4194304] 0 2024-04-25T15:49:35.822 INFO:tasks.workunit.client.1.smithi193.stdout:7/288: unlink d12/d31/d37/f5e 0 2024-04-25T15:49:35.824 INFO:tasks.workunit.client.1.smithi193.stdout:7/289: chown d12/d13 61 1 2024-04-25T15:49:35.825 INFO:tasks.workunit.client.0.smithi110.stdout:5/650: rename d4/d1d/d1f/d56/d6b to d4/d1d/d1f/d35/d3d/d52/dd2/dd4 0 2024-04-25T15:49:35.826 INFO:tasks.workunit.client.1.smithi193.stdout:4/342: write db/f59 [1029784,58598] 0 2024-04-25T15:49:35.827 INFO:tasks.workunit.client.0.smithi110.stdout:0/585: fsync d1/d23/d30/d33/f88 0 2024-04-25T15:49:35.830 INFO:tasks.workunit.client.0.smithi110.stdout:9/572: dwrite d9/d13/f8e [4194304,4194304] 0 2024-04-25T15:49:35.831 INFO:tasks.workunit.client.1.smithi193.stdout:5/289: mknod d17/d18/d3e/c6a 0 2024-04-25T15:49:35.831 INFO:tasks.workunit.client.1.smithi193.stdout:2/319: dwrite d0/d9/d15/d31/d50/f6b [0,4194304] 0 2024-04-25T15:49:35.832 INFO:tasks.workunit.client.1.smithi193.stdout:1/282: mkdir dc/d59 0 2024-04-25T15:49:35.834 INFO:tasks.workunit.client.1.smithi193.stdout:9/330: dread df/d2c/d2f/d4e/f1a [0,4194304] 0 2024-04-25T15:49:35.834 INFO:tasks.workunit.client.1.smithi193.stdout:1/283: chown dc/d4a/d50 58796956 1 2024-04-25T15:49:35.834 INFO:tasks.workunit.client.0.smithi110.stdout:2/588: rename de/d40/d6e/d52/d6d/d99/da1 to de/d40/d6e/d50/db7 0 2024-04-25T15:49:35.835 INFO:tasks.workunit.client.1.smithi193.stdout:8/291: dwrite d2/d1f/f2b [0,4194304] 0 2024-04-25T15:49:35.836 INFO:tasks.workunit.client.1.smithi193.stdout:9/331: fsync df/d2c/d2f/d4e/f3a 0 2024-04-25T15:49:35.840 INFO:tasks.workunit.client.0.smithi110.stdout:2/589: rmdir de/d40/d5b/d5c/d5f/db2 39 2024-04-25T15:49:35.841 INFO:tasks.workunit.client.0.smithi110.stdout:2/590: write de/d6f/fb4 [633683,46048] 0 2024-04-25T15:49:35.842 INFO:tasks.workunit.client.0.smithi110.stdout:2/591: chown de/d40/d4f/f49 1 1 2024-04-25T15:49:35.843 INFO:tasks.workunit.client.1.smithi193.stdout:6/322: mknod dd/d12/d19/c5a 0 2024-04-25T15:49:35.845 INFO:tasks.workunit.client.0.smithi110.stdout:0/586: rename d1/d47/d54/lae to d1/d6c/d7f/ld8 0 2024-04-25T15:49:35.846 INFO:tasks.workunit.client.0.smithi110.stdout:0/587: chown d1/d23/d30/d33/d35/da1/db4/db9 219406 1 2024-04-25T15:49:35.847 INFO:tasks.workunit.client.1.smithi193.stdout:2/320: dwrite d0/d37/d25/d5f/d3a/f4a [0,4194304] 0 2024-04-25T15:49:35.849 INFO:tasks.workunit.client.0.smithi110.stdout:1/722: truncate d0/d14/d5f/d7d/d9c/fa0 2468907 0 2024-04-25T15:49:35.849 INFO:tasks.workunit.client.1.smithi193.stdout:8/292: dwrite d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:35.850 INFO:tasks.workunit.client.1.smithi193.stdout:0/341: link d0/d49/d5/d7/dd/d16/d26/d2e/d3d/l54 d0/d49/d5/d7/dd/d46/d48/l75 0 2024-04-25T15:49:35.850 INFO:tasks.workunit.client.0.smithi110.stdout:2/592: write de/d40/d6e/f21 [370206,1467] 0 2024-04-25T15:49:35.852 INFO:tasks.workunit.client.1.smithi193.stdout:4/343: fsync db/d13/f1d 0 2024-04-25T15:49:35.853 INFO:tasks.workunit.client.0.smithi110.stdout:9/573: creat d9/fba x:0 0 0 2024-04-25T15:49:35.853 INFO:tasks.workunit.client.0.smithi110.stdout:9/574: chown d9/db/df/d2c/f70 15691280 1 2024-04-25T15:49:35.854 INFO:tasks.workunit.client.0.smithi110.stdout:9/575: fdatasync d9/d13/d7c/fb7 0 2024-04-25T15:49:35.855 INFO:tasks.workunit.client.1.smithi193.stdout:5/290: symlink d17/d18/d30/d34/l6b 0 2024-04-25T15:49:35.855 INFO:tasks.workunit.client.0.smithi110.stdout:0/588: rmdir d1/d23/d30/d33/d5b/d78 39 2024-04-25T15:49:35.856 INFO:tasks.workunit.client.1.smithi193.stdout:5/291: chown d17/d18/d45/l51 2743 1 2024-04-25T15:49:35.860 INFO:tasks.workunit.client.1.smithi193.stdout:9/332: fsync df/d2c/d2f/d4e/f19 0 2024-04-25T15:49:35.862 INFO:tasks.workunit.client.0.smithi110.stdout:0/589: dwrite d1/fbc [0,4194304] 0 2024-04-25T15:49:35.879 INFO:tasks.workunit.client.1.smithi193.stdout:3/369: sync 2024-04-25T15:49:35.886 INFO:tasks.workunit.client.1.smithi193.stdout:6/323: creat dd/d2f/f5b x:0 0 0 2024-04-25T15:49:35.888 INFO:tasks.workunit.client.1.smithi193.stdout:7/290: mkdir d12/d31/d37/d62/d6e 0 2024-04-25T15:49:35.889 INFO:tasks.workunit.client.0.smithi110.stdout:9/576: dwrite d9/db/df/d25/f51 [0,4194304] 0 2024-04-25T15:49:35.890 INFO:tasks.workunit.client.1.smithi193.stdout:2/321: chown d0/l12 4964 1 2024-04-25T15:49:35.891 INFO:tasks.workunit.client.1.smithi193.stdout:2/322: write d0/d9/d15/f1b [6051933,25717] 0 2024-04-25T15:49:35.893 INFO:tasks.workunit.client.1.smithi193.stdout:8/293: symlink d2/d3/d27/l5d 0 2024-04-25T15:49:35.897 INFO:tasks.workunit.client.0.smithi110.stdout:1/723: dwrite d0/d25/d33/f48 [4194304,4194304] 0 2024-04-25T15:49:35.899 INFO:tasks.workunit.client.1.smithi193.stdout:0/342: creat d0/d49/d5/d7/dd/d16/d26/d2e/f76 x:0 0 0 2024-04-25T15:49:35.900 INFO:tasks.workunit.client.1.smithi193.stdout:0/343: rename d0/d49/d5/d7/dd/d46 to d0/d49/d5/d7/dd/d46/d77 22 2024-04-25T15:49:35.903 INFO:tasks.workunit.client.1.smithi193.stdout:4/344: unlink db/d13/f67 0 2024-04-25T15:49:35.904 INFO:tasks.workunit.client.0.smithi110.stdout:0/590: fdatasync d1/d23/fc2 0 2024-04-25T15:49:35.904 INFO:tasks.workunit.client.1.smithi193.stdout:4/345: write db/f61 [941602,71033] 0 2024-04-25T15:49:35.906 INFO:tasks.workunit.client.1.smithi193.stdout:5/292: readlink d17/d18/l1b 0 2024-04-25T15:49:35.909 INFO:tasks.workunit.client.0.smithi110.stdout:2/593: truncate de/d40/f3e 2732276 0 2024-04-25T15:49:35.910 INFO:tasks.workunit.client.0.smithi110.stdout:2/594: fsync de/d40/d6e/d50/f7b 0 2024-04-25T15:49:35.911 INFO:tasks.workunit.client.1.smithi193.stdout:4/346: dwrite db/d13/d1e/d1f/d22/f2a [0,4194304] 0 2024-04-25T15:49:35.911 INFO:tasks.workunit.client.1.smithi193.stdout:4/347: chown f2 2129 1 2024-04-25T15:49:35.912 INFO:tasks.workunit.client.1.smithi193.stdout:4/348: write db/f3b [152855,37572] 0 2024-04-25T15:49:35.912 INFO:tasks.workunit.client.0.smithi110.stdout:0/591: symlink d1/d6c/d7f/ld9 0 2024-04-25T15:49:35.914 INFO:tasks.workunit.client.1.smithi193.stdout:4/349: truncate db/f59 1727293 0 2024-04-25T15:49:35.917 INFO:tasks.workunit.client.1.smithi193.stdout:7/291: write d12/d1e/f34 [4861365,70516] 0 2024-04-25T15:49:35.920 INFO:tasks.workunit.client.0.smithi110.stdout:1/724: creat d0/d25/fe3 x:0 0 0 2024-04-25T15:49:35.920 INFO:tasks.workunit.client.0.smithi110.stdout:9/577: dwrite d9/db/df/d2c/d3c/f87 [0,4194304] 0 2024-04-25T15:49:35.921 INFO:tasks.workunit.client.1.smithi193.stdout:6/324: dwrite dd/d12/d19/d1c/f26 [0,4194304] 0 2024-04-25T15:49:35.921 INFO:tasks.workunit.client.1.smithi193.stdout:2/323: mkdir d0/d37/d25/d5f/d3a/d6d/d85 0 2024-04-25T15:49:35.922 INFO:tasks.workunit.client.1.smithi193.stdout:6/325: dread - dd/f59 zero size 2024-04-25T15:49:35.922 INFO:tasks.workunit.client.0.smithi110.stdout:9/578: fsync d9/d13/d5b/d61/f63 0 2024-04-25T15:49:35.926 INFO:tasks.workunit.client.1.smithi193.stdout:8/294: creat d2/d3/d3a/f5e x:0 0 0 2024-04-25T15:49:35.928 INFO:tasks.workunit.client.1.smithi193.stdout:8/295: write d2/d3/d20/d45/d4a/f4e [489885,14191] 0 2024-04-25T15:49:35.930 INFO:tasks.workunit.client.0.smithi110.stdout:1/725: dwrite d0/d14/d5f/d7d/d72/d88/db6/fd4 [0,4194304] 0 2024-04-25T15:49:35.930 INFO:tasks.workunit.client.1.smithi193.stdout:8/296: readlink d2/d1f/l30 0 2024-04-25T15:49:35.930 INFO:tasks.workunit.client.0.smithi110.stdout:9/579: mkdir d9/d44/d5e/d80/dbb 0 2024-04-25T15:49:35.934 INFO:tasks.workunit.client.1.smithi193.stdout:3/370: dwrite d8/d13/d4b/f5d [0,4194304] 0 2024-04-25T15:49:35.934 INFO:tasks.workunit.client.0.smithi110.stdout:1/726: symlink d0/d14/d5f/d7d/d72/d61/le4 0 2024-04-25T15:49:35.935 INFO:tasks.workunit.client.0.smithi110.stdout:9/580: read - d9/f34 zero size 2024-04-25T15:49:35.938 INFO:tasks.workunit.client.1.smithi193.stdout:4/350: rmdir db/d13/d1e 39 2024-04-25T15:49:35.940 INFO:tasks.workunit.client.1.smithi193.stdout:3/371: dread d8/d13/d14/d51/f6a [0,4194304] 0 2024-04-25T15:49:35.940 INFO:tasks.workunit.client.0.smithi110.stdout:1/727: dwrite d0/d23/f5c [0,4194304] 0 2024-04-25T15:49:35.940 INFO:tasks.workunit.client.1.smithi193.stdout:2/324: creat d0/d42/d51/d4f/d65/d7e/f86 x:0 0 0 2024-04-25T15:49:35.942 INFO:tasks.workunit.client.1.smithi193.stdout:0/344: mkdir d0/d49/d5/d7/dd/d78 0 2024-04-25T15:49:35.946 INFO:tasks.workunit.client.0.smithi110.stdout:1/728: creat d0/d14/d5f/d7d/d72/d88/dbf/fe5 x:0 0 0 2024-04-25T15:49:35.946 INFO:tasks.workunit.client.1.smithi193.stdout:1/284: getdents dc/d4a/d50 0 2024-04-25T15:49:35.951 INFO:tasks.workunit.client.0.smithi110.stdout:2/595: dwrite de/d40/d5b/d5c/d5f/f7a [0,4194304] 0 2024-04-25T15:49:35.953 INFO:tasks.workunit.client.1.smithi193.stdout:4/351: dwrite f2 [0,4194304] 0 2024-04-25T15:49:35.955 INFO:tasks.workunit.client.0.smithi110.stdout:1/729: dwrite d0/d14/fc0 [0,4194304] 0 2024-04-25T15:49:35.956 INFO:tasks.workunit.client.1.smithi193.stdout:9/333: getdents df/d2c/d2f/d4e/d1c/d26/d3f 0 2024-04-25T15:49:35.958 INFO:tasks.workunit.client.0.smithi110.stdout:1/730: readlink d0/d14/d5f/d7d/d72/d61/ld1 0 2024-04-25T15:49:35.959 INFO:tasks.workunit.client.1.smithi193.stdout:7/292: link d12/c29 d12/d13/c6f 0 2024-04-25T15:49:35.960 INFO:tasks.workunit.client.1.smithi193.stdout:4/352: dwrite f2 [0,4194304] 0 2024-04-25T15:49:35.961 INFO:tasks.workunit.client.0.smithi110.stdout:1/731: chown d0/d14/d5f/d7d/cd0 124940 1 2024-04-25T15:49:35.961 INFO:tasks.workunit.client.1.smithi193.stdout:3/372: mkdir d8/d30/d72 0 2024-04-25T15:49:35.961 INFO:tasks.workunit.client.1.smithi193.stdout:8/297: creat d2/d3/f5f x:0 0 0 2024-04-25T15:49:35.963 INFO:tasks.workunit.client.1.smithi193.stdout:3/373: chown d8/d13/f32 0 1 2024-04-25T15:49:35.963 INFO:tasks.workunit.client.1.smithi193.stdout:3/374: stat d8/d4d/f67 0 2024-04-25T15:49:35.964 INFO:tasks.workunit.client.0.smithi110.stdout:2/596: dwrite de/d40/d6e/f9b [0,4194304] 0 2024-04-25T15:49:35.967 INFO:tasks.workunit.client.1.smithi193.stdout:9/334: creat df/d2c/d2f/d4e/d1c/d26/d3f/d41/f59 x:0 0 0 2024-04-25T15:49:35.967 INFO:tasks.workunit.client.0.smithi110.stdout:1/732: creat d0/d25/db4/dcd/fe6 x:0 0 0 2024-04-25T15:49:35.969 INFO:tasks.workunit.client.0.smithi110.stdout:2/597: mknod de/d40/d6e/d50/db7/cb8 0 2024-04-25T15:49:35.972 INFO:tasks.workunit.client.1.smithi193.stdout:4/353: write db/d13/d1e/d1f/d22/f2c [5181307,109595] 0 2024-04-25T15:49:35.973 INFO:tasks.workunit.client.1.smithi193.stdout:2/325: link d0/d37/d25/l2a d0/d42/d51/d4f/d65/d6a/l87 0 2024-04-25T15:49:35.973 INFO:tasks.workunit.client.0.smithi110.stdout:1/733: rename d0/d14/d5f/fba to d0/d14/fe7 0 2024-04-25T15:49:35.973 INFO:tasks.workunit.client.1.smithi193.stdout:7/293: dwrite d12/f2c [0,4194304] 0 2024-04-25T15:49:35.975 INFO:tasks.workunit.client.0.smithi110.stdout:2/598: symlink de/d40/d70/lb9 0 2024-04-25T15:49:35.977 INFO:tasks.workunit.client.1.smithi193.stdout:1/285: dread dc/f19 [0,4194304] 0 2024-04-25T15:49:35.977 INFO:tasks.workunit.client.1.smithi193.stdout:3/375: creat d8/d13/d14/d27/f73 x:0 0 0 2024-04-25T15:49:35.981 INFO:tasks.workunit.client.1.smithi193.stdout:9/335: unlink df/d2c/d34/f46 0 2024-04-25T15:49:35.982 INFO:tasks.workunit.client.1.smithi193.stdout:9/336: fdatasync df/d2c/d2f/d4e/d1c/d26/f4f 0 2024-04-25T15:49:35.985 INFO:tasks.workunit.client.0.smithi110.stdout:1/734: symlink d0/d14/d5f/d7d/d72/le8 0 2024-04-25T15:49:35.986 INFO:tasks.workunit.client.1.smithi193.stdout:7/294: rmdir d12/d1e 39 2024-04-25T15:49:35.988 INFO:tasks.workunit.client.0.smithi110.stdout:2/599: creat de/d40/d5b/d5c/fba x:0 0 0 2024-04-25T15:49:35.989 INFO:tasks.workunit.client.1.smithi193.stdout:4/354: mknod db/d4b/d51/d54/c72 0 2024-04-25T15:49:35.989 INFO:tasks.workunit.client.0.smithi110.stdout:2/600: write de/d40/d86/fb6 [447691,46219] 0 2024-04-25T15:49:35.990 INFO:tasks.workunit.client.1.smithi193.stdout:4/355: chown db/f61 250520 1 2024-04-25T15:49:35.990 INFO:tasks.workunit.client.0.smithi110.stdout:1/735: creat d0/d25/d33/d8e/fe9 x:0 0 0 2024-04-25T15:49:35.991 INFO:tasks.workunit.client.1.smithi193.stdout:4/356: write db/d13/d1e/f48 [207187,108099] 0 2024-04-25T15:49:35.991 INFO:tasks.workunit.client.0.smithi110.stdout:1/736: chown d0/d14/d5f/d7d/d72/d61/l9a 402000 1 2024-04-25T15:49:35.992 INFO:tasks.workunit.client.1.smithi193.stdout:8/298: unlink d2/d3/d20/d45/d4a/d53/c5c 0 2024-04-25T15:49:35.992 INFO:tasks.workunit.client.1.smithi193.stdout:8/299: fsync d2/f48 0 2024-04-25T15:49:35.993 INFO:tasks.workunit.client.0.smithi110.stdout:1/737: dread - d0/d25/db4/dcd/fe6 zero size 2024-04-25T15:49:35.993 INFO:tasks.workunit.client.0.smithi110.stdout:2/601: mkdir de/d6f/dbb 0 2024-04-25T15:49:35.994 INFO:tasks.workunit.client.0.smithi110.stdout:2/602: write de/d40/f33 [144000,93731] 0 2024-04-25T15:49:35.995 INFO:tasks.workunit.client.1.smithi193.stdout:1/286: fdatasync dc/f4b 0 2024-04-25T15:49:35.997 INFO:tasks.workunit.client.0.smithi110.stdout:1/738: dread d0/d14/f62 [0,4194304] 0 2024-04-25T15:49:35.999 INFO:tasks.workunit.client.1.smithi193.stdout:3/376: read d8/d13/f56 [763834,29847] 0 2024-04-25T15:49:36.000 INFO:tasks.workunit.client.1.smithi193.stdout:9/337: readlink df/d2c/d2f/d4e/d1c/d26/l48 0 2024-04-25T15:49:36.001 INFO:tasks.workunit.client.1.smithi193.stdout:5/293: sync 2024-04-25T15:49:36.003 INFO:tasks.workunit.client.0.smithi110.stdout:2/603: mknod de/d40/cbc 0 2024-04-25T15:49:36.007 INFO:tasks.workunit.client.0.smithi110.stdout:2/604: rmdir de/d40/d5b/d5c/d5f 39 2024-04-25T15:49:36.007 INFO:tasks.workunit.client.1.smithi193.stdout:0/345: dwrite d0/d49/d5/d7/dd/d16/d18/f2a [0,4194304] 0 2024-04-25T15:49:36.011 INFO:tasks.workunit.client.1.smithi193.stdout:1/287: readlink la 0 2024-04-25T15:49:36.012 INFO:tasks.workunit.client.0.smithi110.stdout:1/739: getdents d0/d25/db4/dcd 0 2024-04-25T15:49:36.015 INFO:tasks.workunit.client.1.smithi193.stdout:9/338: symlink df/d2c/d2f/d33/l5a 0 2024-04-25T15:49:36.017 INFO:tasks.workunit.client.0.smithi110.stdout:1/740: rename d0/d14/c6d to d0/d23/cea 0 2024-04-25T15:49:36.017 INFO:tasks.workunit.client.1.smithi193.stdout:5/294: creat d17/d18/d30/d34/f6c x:0 0 0 2024-04-25T15:49:36.020 INFO:tasks.workunit.client.1.smithi193.stdout:7/295: rename d12/d1e to d12/d1c/d63/d70 0 2024-04-25T15:49:36.020 INFO:tasks.workunit.client.0.smithi110.stdout:1/741: mknod d0/d14/d5f/d7d/d72/d88/db6/ceb 0 2024-04-25T15:49:36.022 INFO:tasks.workunit.client.0.smithi110.stdout:2/605: mkdir de/d40/d5b/d5c/d5f/db2/dad/dbd 0 2024-04-25T15:49:36.025 INFO:tasks.workunit.client.0.smithi110.stdout:1/742: symlink d0/d14/d5f/d7d/dda/lec 0 2024-04-25T15:49:36.026 INFO:tasks.workunit.client.0.smithi110.stdout:2/606: dread de/d40/f36 [0,4194304] 0 2024-04-25T15:49:36.028 INFO:tasks.workunit.client.1.smithi193.stdout:9/339: creat df/d2c/d2f/d4e/d1c/d30/d3b/f5b x:0 0 0 2024-04-25T15:49:36.028 INFO:tasks.workunit.client.0.smithi110.stdout:1/743: readlink d0/d14/d5f/d7d/d9c/lcf 0 2024-04-25T15:49:36.029 INFO:tasks.workunit.client.1.smithi193.stdout:9/340: write df/d2c/d2f/f49 [965113,50444] 0 2024-04-25T15:49:36.031 INFO:tasks.workunit.client.1.smithi193.stdout:4/357: rename db/d32/c47 to db/d13/d1e/d1f/c73 0 2024-04-25T15:49:36.031 INFO:tasks.workunit.client.1.smithi193.stdout:7/296: creat d12/d13/f71 x:0 0 0 2024-04-25T15:49:36.032 INFO:tasks.workunit.client.0.smithi110.stdout:2/607: mkdir de/d40/d5b/d5c/dbe 0 2024-04-25T15:49:36.032 INFO:tasks.workunit.client.1.smithi193.stdout:5/295: dread d17/f2a [0,4194304] 0 2024-04-25T15:49:36.032 INFO:tasks.workunit.client.1.smithi193.stdout:0/346: truncate d0/d49/d5/d7/dd/d16/f41 1074497 0 2024-04-25T15:49:36.033 INFO:tasks.workunit.client.0.smithi110.stdout:1/744: truncate d0/d25/d33/d8e/fa6 880811 0 2024-04-25T15:49:36.033 INFO:tasks.workunit.client.0.smithi110.stdout:1/745: chown d0/d14/f4b 33222645 1 2024-04-25T15:49:36.033 INFO:tasks.workunit.client.1.smithi193.stdout:1/288: symlink dc/d35/l5a 0 2024-04-25T15:49:36.034 INFO:tasks.workunit.client.0.smithi110.stdout:2/608: rmdir de/d6f 39 2024-04-25T15:49:36.034 INFO:tasks.workunit.client.0.smithi110.stdout:1/746: write d0/d14/d5f/faa [1015872,25897] 0 2024-04-25T15:49:36.035 INFO:tasks.workunit.client.1.smithi193.stdout:3/377: link d8/d13/d14/d51/f6a d8/d30/d31/f74 0 2024-04-25T15:49:36.035 INFO:tasks.workunit.client.0.smithi110.stdout:2/609: dread - de/d40/d6e/d52/d6d/fb3 zero size 2024-04-25T15:49:36.037 INFO:tasks.workunit.client.0.smithi110.stdout:1/747: creat d0/d51/dc8/fed x:0 0 0 2024-04-25T15:49:36.039 INFO:tasks.workunit.client.1.smithi193.stdout:7/297: rename d12/d1c/f5a to d12/d31/d37/d62/d6e/f72 0 2024-04-25T15:49:36.039 INFO:tasks.workunit.client.1.smithi193.stdout:5/296: creat d17/d18/d45/f6d x:0 0 0 2024-04-25T15:49:36.041 INFO:tasks.workunit.client.1.smithi193.stdout:0/347: unlink d0/d49/d5/d7/dd/d16/d18/d28/f52 0 2024-04-25T15:49:36.042 INFO:tasks.workunit.client.1.smithi193.stdout:1/289: fsync dc/d35/d3d/f43 0 2024-04-25T15:49:36.044 INFO:tasks.workunit.client.1.smithi193.stdout:4/358: dwrite db/d32/d45/f4f [0,4194304] 0 2024-04-25T15:49:36.047 INFO:tasks.workunit.client.1.smithi193.stdout:5/297: dread d17/f59 [0,4194304] 0 2024-04-25T15:49:36.048 INFO:tasks.workunit.client.1.smithi193.stdout:7/298: dwrite d12/d1c/f28 [4194304,4194304] 0 2024-04-25T15:49:36.048 INFO:tasks.workunit.client.1.smithi193.stdout:4/359: creat db/d13/d1e/d1f/d56/f74 x:0 0 0 2024-04-25T15:49:36.050 INFO:tasks.workunit.client.1.smithi193.stdout:7/299: fdatasync fc 0 2024-04-25T15:49:36.050 INFO:tasks.workunit.client.1.smithi193.stdout:4/360: chown db/d13/d1e/d1f/d22/f6b 54140543 1 2024-04-25T15:49:36.051 INFO:tasks.workunit.client.1.smithi193.stdout:0/348: getdents d0/d49/d5/d7/dd/d16 0 2024-04-25T15:49:36.051 INFO:tasks.workunit.client.1.smithi193.stdout:5/298: dread d17/f59 [0,4194304] 0 2024-04-25T15:49:36.053 INFO:tasks.workunit.client.1.smithi193.stdout:7/300: read - d12/d31/d39/d60/f64 zero size 2024-04-25T15:49:36.055 INFO:tasks.workunit.client.1.smithi193.stdout:5/299: chown d17/l1c 37 1 2024-04-25T15:49:36.057 INFO:tasks.workunit.client.1.smithi193.stdout:7/301: chown d12/d13/c15 4670743 1 2024-04-25T15:49:36.058 INFO:tasks.workunit.client.1.smithi193.stdout:0/349: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/l79 0 2024-04-25T15:49:36.058 INFO:tasks.workunit.client.1.smithi193.stdout:7/302: read d12/d38/f4e [1504565,15177] 0 2024-04-25T15:49:36.059 INFO:tasks.workunit.client.1.smithi193.stdout:0/350: write d0/d49/d5/d7/f71 [81704,31508] 0 2024-04-25T15:49:36.060 INFO:tasks.workunit.client.1.smithi193.stdout:7/303: symlink d12/d1c/d45/l73 0 2024-04-25T15:49:36.061 INFO:tasks.workunit.client.1.smithi193.stdout:0/351: truncate d0/d49/d5/d7/dd/d16/f6f 106127 0 2024-04-25T15:49:36.061 INFO:tasks.workunit.client.1.smithi193.stdout:5/300: rename d17/d18/d30/d3b/l64 to d17/d18/d30/d38/d48/l6e 0 2024-04-25T15:49:36.063 INFO:tasks.workunit.client.1.smithi193.stdout:6/326: sync 2024-04-25T15:49:36.064 INFO:tasks.workunit.client.1.smithi193.stdout:7/304: link d12/d1c/f28 d12/d38/f74 0 2024-04-25T15:49:36.067 INFO:tasks.workunit.client.1.smithi193.stdout:6/327: creat dd/d2f/d57/f5c x:0 0 0 2024-04-25T15:49:36.068 INFO:tasks.workunit.client.1.smithi193.stdout:0/352: getdents d0/d49/d5/d7/dd/d16/d26/d2e/d3d 0 2024-04-25T15:49:36.068 INFO:tasks.workunit.client.1.smithi193.stdout:6/328: creat dd/d12/f5d x:0 0 0 2024-04-25T15:49:36.069 INFO:tasks.workunit.client.0.smithi110.stdout:7/627: sync 2024-04-25T15:49:36.069 INFO:tasks.workunit.client.1.smithi193.stdout:0/353: mknod d0/d49/d5/d7/dd/d16/d26/d5e/c7a 0 2024-04-25T15:49:36.072 INFO:tasks.workunit.client.0.smithi110.stdout:7/628: rename d1b/d29/d38/d55/d95/fd1 to d1b/d29/db5/dca/fd5 0 2024-04-25T15:49:36.074 INFO:tasks.workunit.client.1.smithi193.stdout:6/329: dwrite dd/d12/f4d [0,4194304] 0 2024-04-25T15:49:36.077 INFO:tasks.workunit.client.1.smithi193.stdout:0/354: dwrite d0/d49/d5/d7/dd/d16/d18/d28/f3e [0,4194304] 0 2024-04-25T15:49:36.077 INFO:tasks.workunit.client.0.smithi110.stdout:7/629: dwrite d1b/d29/d38/d55/d7f/fb1 [0,4194304] 0 2024-04-25T15:49:36.083 INFO:tasks.workunit.client.1.smithi193.stdout:0/355: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/f69 [0,4194304] 0 2024-04-25T15:49:36.090 INFO:tasks.workunit.client.1.smithi193.stdout:2/326: sync 2024-04-25T15:49:36.091 INFO:tasks.workunit.client.1.smithi193.stdout:2/327: write d0/d9/d15/d31/d50/f6b [2604433,126172] 0 2024-04-25T15:49:36.101 INFO:tasks.workunit.client.1.smithi193.stdout:6/330: chown dd/d12/d19/l2a 4530 1 2024-04-25T15:49:36.102 INFO:tasks.workunit.client.1.smithi193.stdout:6/331: chown dd/d12/d19/d1c/d40/l33 6982 1 2024-04-25T15:49:36.107 INFO:tasks.workunit.client.0.smithi110.stdout:7/630: truncate d1b/d29/f57 2601593 0 2024-04-25T15:49:36.107 INFO:tasks.workunit.client.1.smithi193.stdout:6/332: mknod dd/d2f/c5e 0 2024-04-25T15:49:36.107 INFO:tasks.workunit.client.1.smithi193.stdout:3/378: dread d8/d13/d14/f59 [0,4194304] 0 2024-04-25T15:49:36.108 INFO:tasks.workunit.client.1.smithi193.stdout:6/333: fsync dd/d2f/f5b 0 2024-04-25T15:49:36.111 INFO:tasks.workunit.client.1.smithi193.stdout:0/356: symlink d0/d49/d5/d7/dd/l7b 0 2024-04-25T15:49:36.116 INFO:tasks.workunit.client.1.smithi193.stdout:3/379: unlink d8/d30/c3b 0 2024-04-25T15:49:36.117 INFO:tasks.workunit.client.1.smithi193.stdout:8/300: write d2/d3/d20/f22 [3549916,6361] 0 2024-04-25T15:49:36.120 INFO:tasks.workunit.client.1.smithi193.stdout:2/328: creat d0/d9/f88 x:0 0 0 2024-04-25T15:49:36.122 INFO:tasks.workunit.client.1.smithi193.stdout:8/301: creat d2/d3/d20/d45/d29/f60 x:0 0 0 2024-04-25T15:49:36.123 INFO:tasks.workunit.client.1.smithi193.stdout:8/302: dread - d2/d3/d20/d45/d29/f60 zero size 2024-04-25T15:49:36.124 INFO:tasks.workunit.client.1.smithi193.stdout:2/329: mknod d0/d42/c89 0 2024-04-25T15:49:36.125 INFO:tasks.workunit.client.1.smithi193.stdout:3/380: mknod d8/d13/d14/c75 0 2024-04-25T15:49:36.126 INFO:tasks.workunit.client.1.smithi193.stdout:8/303: creat d2/d3/d3a/f61 x:0 0 0 2024-04-25T15:49:36.127 INFO:tasks.workunit.client.1.smithi193.stdout:2/330: mknod d0/d42/d51/c8a 0 2024-04-25T15:49:36.127 INFO:tasks.workunit.client.1.smithi193.stdout:0/357: dread d0/d49/d5/f24 [0,4194304] 0 2024-04-25T15:49:36.128 INFO:tasks.workunit.client.1.smithi193.stdout:3/381: truncate d8/f11 4216577 0 2024-04-25T15:49:36.129 INFO:tasks.workunit.client.1.smithi193.stdout:2/331: write d0/d9/d15/d31/d50/f80 [467022,129665] 0 2024-04-25T15:49:36.129 INFO:tasks.workunit.client.1.smithi193.stdout:8/304: symlink d2/d3/d20/d45/d4a/l62 0 2024-04-25T15:49:36.132 INFO:tasks.workunit.client.1.smithi193.stdout:3/382: read d8/d13/d4b/f5d [1464512,28299] 0 2024-04-25T15:49:36.132 INFO:tasks.workunit.client.1.smithi193.stdout:6/334: dread dd/fe [0,4194304] 0 2024-04-25T15:49:36.132 INFO:tasks.workunit.client.1.smithi193.stdout:0/358: write d0/d49/d5/d7/dd/d16/d26/d5e/f69 [3181728,96635] 0 2024-04-25T15:49:36.132 INFO:tasks.workunit.client.0.smithi110.stdout:1/748: read d0/d25/d33/d8e/fa6 [541896,85316] 0 2024-04-25T15:49:36.132 INFO:tasks.workunit.client.1.smithi193.stdout:2/332: write d0/d9/d15/d31/d50/f6b [989058,66599] 0 2024-04-25T15:49:36.134 INFO:tasks.workunit.client.0.smithi110.stdout:1/749: write d0/d51/dc8/fed [976493,126893] 0 2024-04-25T15:49:36.134 INFO:tasks.workunit.client.1.smithi193.stdout:0/359: write d0/d49/d5/d7/dd/d16/d26/d2e/f34 [4804103,52923] 0 2024-04-25T15:49:36.137 INFO:tasks.workunit.client.1.smithi193.stdout:9/341: sync 2024-04-25T15:49:36.141 INFO:tasks.workunit.client.1.smithi193.stdout:2/333: mkdir d0/d44/d8b 0 2024-04-25T15:49:36.141 INFO:tasks.workunit.client.1.smithi193.stdout:1/290: write dc/d35/d3d/f43 [1867537,62260] 0 2024-04-25T15:49:36.141 INFO:tasks.workunit.client.1.smithi193.stdout:2/334: dread - d0/d9/f88 zero size 2024-04-25T15:49:36.141 INFO:tasks.workunit.client.1.smithi193.stdout:1/291: readlink dc/l32 0 2024-04-25T15:49:36.142 INFO:tasks.workunit.client.1.smithi193.stdout:2/335: chown d0/d42/c89 681571867 1 2024-04-25T15:49:36.146 INFO:tasks.workunit.client.0.smithi110.stdout:1/750: link d0/d25/lb8 d0/lee 0 2024-04-25T15:49:36.148 INFO:tasks.workunit.client.1.smithi193.stdout:3/383: creat d8/d30/d72/f76 x:0 0 0 2024-04-25T15:49:36.148 INFO:tasks.workunit.client.1.smithi193.stdout:3/384: fsync d8/d30/f36 0 2024-04-25T15:49:36.149 INFO:tasks.workunit.client.1.smithi193.stdout:9/342: unlink df/d2c/d2f/d33/l5a 0 2024-04-25T15:49:36.150 INFO:tasks.workunit.client.1.smithi193.stdout:6/335: rename dd/l4b to dd/d12/d2d/l5f 0 2024-04-25T15:49:36.152 INFO:tasks.workunit.client.0.smithi110.stdout:1/751: rename d0/d25/d33/f47 to d0/d51/dd8/fef 0 2024-04-25T15:49:36.152 INFO:tasks.workunit.client.1.smithi193.stdout:4/361: dwrite db/d13/f21 [0,4194304] 0 2024-04-25T15:49:36.155 INFO:tasks.workunit.client.1.smithi193.stdout:4/362: read db/d13/d1e/f48 [174753,114220] 0 2024-04-25T15:49:36.155 INFO:tasks.workunit.client.1.smithi193.stdout:4/363: readlink db/d32/d40/l5e 0 2024-04-25T15:49:36.157 INFO:tasks.workunit.client.1.smithi193.stdout:2/336: dread d0/d42/d51/f46 [0,4194304] 0 2024-04-25T15:49:36.158 INFO:tasks.workunit.client.1.smithi193.stdout:2/337: write d0/d9/d15/f1b [698477,53136] 0 2024-04-25T15:49:36.161 INFO:tasks.workunit.client.0.smithi110.stdout:1/752: mkdir d0/d51/dd8/df0 0 2024-04-25T15:49:36.161 INFO:tasks.workunit.client.1.smithi193.stdout:7/305: write f8 [1884974,127236] 0 2024-04-25T15:49:36.162 INFO:tasks.workunit.client.1.smithi193.stdout:5/301: dwrite d17/d18/d3e/f4c [0,4194304] 0 2024-04-25T15:49:36.165 INFO:tasks.workunit.client.0.smithi110.stdout:1/753: read d0/d14/f57 [718892,123271] 0 2024-04-25T15:49:36.168 INFO:tasks.workunit.client.1.smithi193.stdout:3/385: rename d8/d30/d31 to d8/d13/d1f/d69/d77 0 2024-04-25T15:49:36.170 INFO:tasks.workunit.client.1.smithi193.stdout:5/302: dwrite d17/d18/d3e/f4c [0,4194304] 0 2024-04-25T15:49:36.175 INFO:tasks.workunit.client.0.smithi110.stdout:1/754: fsync d0/d14/f4b 0 2024-04-25T15:49:36.177 INFO:tasks.workunit.client.1.smithi193.stdout:4/364: rmdir db/d32/d40 39 2024-04-25T15:49:36.178 INFO:tasks.workunit.client.1.smithi193.stdout:4/365: chown db/d4b/d51/d54/c72 198248 1 2024-04-25T15:49:36.182 INFO:tasks.workunit.client.0.smithi110.stdout:1/755: mkdir d0/d51/d6b/df1 0 2024-04-25T15:49:36.183 INFO:tasks.workunit.client.0.smithi110.stdout:7/631: write f14 [828903,102031] 0 2024-04-25T15:49:36.183 INFO:tasks.workunit.client.0.smithi110.stdout:1/756: readlink d0/d14/d5f/d7d/l8c 0 2024-04-25T15:49:36.185 INFO:tasks.workunit.client.1.smithi193.stdout:4/366: dwrite db/d4b/f71 [0,4194304] 0 2024-04-25T15:49:36.186 INFO:tasks.workunit.client.1.smithi193.stdout:4/367: chown db/d13/l19 249855845 1 2024-04-25T15:49:36.187 INFO:tasks.workunit.client.1.smithi193.stdout:2/338: rmdir d0/d9/d3d 39 2024-04-25T15:49:36.190 INFO:tasks.workunit.client.1.smithi193.stdout:0/360: creat d0/d49/d5/d7/dd/d16/d18/f7c x:0 0 0 2024-04-25T15:49:36.191 INFO:tasks.workunit.client.0.smithi110.stdout:7/632: dwrite d1b/d29/d38/d55/dab/fb0 [0,4194304] 0 2024-04-25T15:49:36.193 INFO:tasks.workunit.client.1.smithi193.stdout:7/306: symlink d12/d31/d39/l75 0 2024-04-25T15:49:36.194 INFO:tasks.workunit.client.0.smithi110.stdout:7/633: write d1b/d29/d38/d8b/f92 [1163159,54888] 0 2024-04-25T15:49:36.199 INFO:tasks.workunit.client.1.smithi193.stdout:9/343: mkdir df/d2c/d2f/d5c 0 2024-04-25T15:49:36.202 INFO:tasks.workunit.client.1.smithi193.stdout:8/305: write d2/d36/f40 [1014806,60848] 0 2024-04-25T15:49:36.202 INFO:tasks.workunit.client.0.smithi110.stdout:7/634: mknod d1b/cd6 0 2024-04-25T15:49:36.206 INFO:tasks.workunit.client.0.smithi110.stdout:7/635: creat d1b/d29/d38/d8b/fd7 x:0 0 0 2024-04-25T15:49:36.207 INFO:tasks.workunit.client.1.smithi193.stdout:1/292: getdents dc/d35/d3d 0 2024-04-25T15:49:36.208 INFO:tasks.workunit.client.1.smithi193.stdout:1/293: dread - dc/d35/d3d/f41 zero size 2024-04-25T15:49:36.209 INFO:tasks.workunit.client.1.smithi193.stdout:8/306: dwrite d2/d3/d20/d45/f5a [0,4194304] 0 2024-04-25T15:49:36.210 INFO:tasks.workunit.client.0.smithi110.stdout:7/636: unlink d1b/cd6 0 2024-04-25T15:49:36.214 INFO:tasks.workunit.client.1.smithi193.stdout:2/339: rmdir d0/d9/d15/d31/d50 39 2024-04-25T15:49:36.217 INFO:tasks.workunit.client.1.smithi193.stdout:0/361: unlink d0/d49/d5/d7/l21 0 2024-04-25T15:49:36.218 INFO:tasks.workunit.client.0.smithi110.stdout:7/637: creat d1b/d29/d38/d8b/d89/fd8 x:0 0 0 2024-04-25T15:49:36.221 INFO:tasks.workunit.client.0.smithi110.stdout:7/638: creat d1b/d29/d38/d55/fd9 x:0 0 0 2024-04-25T15:49:36.222 INFO:tasks.workunit.client.1.smithi193.stdout:9/344: mkdir df/d2c/d2f/d4e/d1c/d26/d3f/d41/d5d 0 2024-04-25T15:49:36.226 INFO:tasks.workunit.client.1.smithi193.stdout:5/303: truncate d17/d18/f21 7476055 0 2024-04-25T15:49:36.227 INFO:tasks.workunit.client.1.smithi193.stdout:6/336: link dd/d12/f1e dd/d12/d19/d1c/d40/d3c/d3f/d4f/f60 0 2024-04-25T15:49:36.227 INFO:tasks.workunit.client.1.smithi193.stdout:9/345: dwrite df/d2c/d2f/d4e/f3a [0,4194304] 0 2024-04-25T15:49:36.228 INFO:tasks.workunit.client.1.smithi193.stdout:5/304: chown d17/d18/d30/d34/d37/d5d/f68 232783 1 2024-04-25T15:49:36.229 INFO:tasks.workunit.client.1.smithi193.stdout:5/305: read d17/f2a [835858,33444] 0 2024-04-25T15:49:36.230 INFO:tasks.workunit.client.1.smithi193.stdout:4/368: symlink db/d13/l75 0 2024-04-25T15:49:36.232 INFO:tasks.workunit.client.1.smithi193.stdout:2/340: stat d0/d9/d15/d31/d50/f80 0 2024-04-25T15:49:36.232 INFO:tasks.workunit.client.1.smithi193.stdout:0/362: fsync d0/f17 0 2024-04-25T15:49:36.234 INFO:tasks.workunit.client.1.smithi193.stdout:6/337: dread dd/fe [4194304,4194304] 0 2024-04-25T15:49:36.235 INFO:tasks.workunit.client.1.smithi193.stdout:6/338: chown dd/d12/d19/c5a 8929866 1 2024-04-25T15:49:36.246 INFO:tasks.workunit.client.1.smithi193.stdout:9/346: fsync df/d2c/d2f/d4e/f3a 0 2024-04-25T15:49:36.274 INFO:tasks.workunit.client.0.smithi110.stdout:4/618: sync 2024-04-25T15:49:36.305 INFO:tasks.workunit.client.1.smithi193.stdout:7/307: mknod d12/c76 0 2024-04-25T15:49:36.408 INFO:tasks.workunit.client.1.smithi193.stdout:5/306: readlink l2 0 2024-04-25T15:49:36.409 INFO:tasks.workunit.client.0.smithi110.stdout:1/757: write d0/d14/f2c [7864098,111233] 0 2024-04-25T15:49:36.410 INFO:tasks.workunit.client.1.smithi193.stdout:5/307: read f16 [2642514,6290] 0 2024-04-25T15:49:36.423 INFO:tasks.workunit.client.1.smithi193.stdout:6/339: creat dd/d12/d19/d1c/d40/d3c/d3f/f61 x:0 0 0 2024-04-25T15:49:36.424 INFO:tasks.workunit.client.0.smithi110.stdout:7/639: rmdir d1b 39 2024-04-25T15:49:36.427 INFO:tasks.workunit.client.1.smithi193.stdout:7/308: stat d12/d1c/d63/d70/c1f 0 2024-04-25T15:49:36.430 INFO:tasks.workunit.client.1.smithi193.stdout:1/294: creat dc/f5b x:0 0 0 2024-04-25T15:49:36.430 INFO:tasks.workunit.client.1.smithi193.stdout:1/295: fdatasync dc/f12 0 2024-04-25T15:49:36.433 INFO:tasks.workunit.client.1.smithi193.stdout:8/307: dwrite d2/d3/d3a/f54 [0,4194304] 0 2024-04-25T15:49:36.435 INFO:tasks.workunit.client.1.smithi193.stdout:8/308: read d2/d3/d20/d45/f5a [1652646,98576] 0 2024-04-25T15:49:36.435 INFO:tasks.workunit.client.0.smithi110.stdout:1/758: creat d0/d51/dd8/ff2 x:0 0 0 2024-04-25T15:49:36.438 INFO:tasks.workunit.client.0.smithi110.stdout:7/640: write d1b/d29/d38/d55/dab/db2/fb7 [185608,9452] 0 2024-04-25T15:49:36.439 INFO:tasks.workunit.client.1.smithi193.stdout:4/369: creat db/d13/d1e/d1f/d56/d69/d70/f76 x:0 0 0 2024-04-25T15:49:36.439 INFO:tasks.workunit.client.0.smithi110.stdout:7/641: chown d1b/d29/f2b 18 1 2024-04-25T15:49:36.440 INFO:tasks.workunit.client.1.smithi193.stdout:4/370: dread - db/d13/d1e/d1f/f63 zero size 2024-04-25T15:49:36.440 INFO:tasks.workunit.client.1.smithi193.stdout:4/371: readlink db/d13/l1b 0 2024-04-25T15:49:36.441 INFO:tasks.workunit.client.0.smithi110.stdout:1/759: creat d0/d25/d33/ff3 x:0 0 0 2024-04-25T15:49:36.442 INFO:tasks.workunit.client.0.smithi110.stdout:7/642: truncate d1b/d29/d38/d55/dab/db2/db8/fbb 54782 0 2024-04-25T15:49:36.444 INFO:tasks.workunit.client.1.smithi193.stdout:6/340: mknod dd/d12/d19/d1c/c62 0 2024-04-25T15:49:36.445 INFO:tasks.workunit.client.1.smithi193.stdout:6/341: readlink dd/d12/d19/d1c/l49 0 2024-04-25T15:49:36.445 INFO:tasks.workunit.client.1.smithi193.stdout:6/342: stat dd/d12/d19/d1c/d40/f25 0 2024-04-25T15:49:36.448 INFO:tasks.workunit.client.0.smithi110.stdout:1/760: dwrite d0/d14/fc0 [0,4194304] 0 2024-04-25T15:49:36.450 INFO:tasks.workunit.client.0.smithi110.stdout:8/762: sync 2024-04-25T15:49:36.451 INFO:tasks.workunit.client.0.smithi110.stdout:8/763: dread - d14/d32/d36/d48/d8f/ffa zero size 2024-04-25T15:49:36.456 INFO:tasks.workunit.client.1.smithi193.stdout:1/296: write dc/f40 [701648,80287] 0 2024-04-25T15:49:36.458 INFO:tasks.workunit.client.0.smithi110.stdout:7/643: dread d1b/d29/d38/d8b/d89/dc6/fd3 [0,4194304] 0 2024-04-25T15:49:36.462 INFO:tasks.workunit.client.0.smithi110.stdout:1/761: creat d0/d25/ff4 x:0 0 0 2024-04-25T15:49:36.463 INFO:tasks.workunit.client.1.smithi193.stdout:8/309: mkdir d2/d3/d20/d45/d4a/d63 0 2024-04-25T15:49:36.464 INFO:tasks.workunit.client.1.smithi193.stdout:8/310: chown d2/d3/d20/d45/d4a/d53 10 1 2024-04-25T15:49:36.465 INFO:tasks.workunit.client.0.smithi110.stdout:8/764: creat d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/ddd/f100 x:0 0 0 2024-04-25T15:49:36.466 INFO:tasks.workunit.client.0.smithi110.stdout:4/619: dwrite d13/d24/d2d/f37 [0,4194304] 0 2024-04-25T15:49:36.467 INFO:tasks.workunit.client.0.smithi110.stdout:7/644: mknod d1b/d29/d38/d55/dab/cda 0 2024-04-25T15:49:36.468 INFO:tasks.workunit.client.1.smithi193.stdout:4/372: chown db/d13/d1e/d1f/c73 70457213 1 2024-04-25T15:49:36.469 INFO:tasks.workunit.client.1.smithi193.stdout:4/373: chown db/d4b/d51/d54/c58 326996 1 2024-04-25T15:49:36.469 INFO:tasks.workunit.client.1.smithi193.stdout:3/386: dwrite d8/f11 [0,4194304] 0 2024-04-25T15:49:36.470 INFO:tasks.workunit.client.0.smithi110.stdout:1/762: creat d0/d14/d5f/d7d/d72/ff5 x:0 0 0 2024-04-25T15:49:36.473 INFO:tasks.workunit.client.1.smithi193.stdout:0/363: rename d0/d49/d5/l68 to d0/d49/l7d 0 2024-04-25T15:49:36.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/341: link d0/d44/c4e d0/d37/d4d/c8c 0 2024-04-25T15:49:36.476 INFO:tasks.workunit.client.1.smithi193.stdout:4/374: dwrite db/d13/d1e/d1f/f63 [0,4194304] 0 2024-04-25T15:49:36.478 INFO:tasks.workunit.client.1.smithi193.stdout:6/343: readlink dd/d12/d19/l28 0 2024-04-25T15:49:36.478 INFO:tasks.workunit.client.1.smithi193.stdout:2/342: write d0/d42/d51/d4f/d65/d6a/f3b [477879,46916] 0 2024-04-25T15:49:36.485 INFO:tasks.workunit.client.1.smithi193.stdout:7/309: creat d12/d1c/d63/d70/d68/f77 x:0 0 0 2024-04-25T15:49:36.485 INFO:tasks.workunit.client.1.smithi193.stdout:5/308: dwrite d17/d18/f20 [0,4194304] 0 2024-04-25T15:49:36.491 INFO:tasks.workunit.client.1.smithi193.stdout:4/375: truncate db/d13/d1e/f41 995471 0 2024-04-25T15:49:36.493 INFO:tasks.workunit.client.1.smithi193.stdout:6/344: read dd/f1b [1279792,41265] 0 2024-04-25T15:49:36.493 INFO:tasks.workunit.client.1.smithi193.stdout:6/345: fdatasync dd/d12/f4d 0 2024-04-25T15:49:36.493 INFO:tasks.workunit.client.0.smithi110.stdout:7/645: dwrite d1b/f45 [0,4194304] 0 2024-04-25T15:49:36.493 INFO:tasks.workunit.client.0.smithi110.stdout:1/763: write d0/d14/d5f/d7d/f67 [971821,115095] 0 2024-04-25T15:49:36.494 INFO:tasks.workunit.client.1.smithi193.stdout:0/364: read d0/f12 [316981,3026] 0 2024-04-25T15:49:36.496 INFO:tasks.workunit.client.1.smithi193.stdout:9/347: getdents df/d2c/d2f 0 2024-04-25T15:49:36.496 INFO:tasks.workunit.client.1.smithi193.stdout:1/297: write dc/d4a/d50/f53 [3230914,49397] 0 2024-04-25T15:49:36.496 INFO:tasks.workunit.client.0.smithi110.stdout:7/646: fsync d1b/d29/d46/fcf 0 2024-04-25T15:49:36.498 INFO:tasks.workunit.client.0.smithi110.stdout:7/647: fdatasync d1b/d29/d38/d8b/f92 0 2024-04-25T15:49:36.498 INFO:tasks.workunit.client.1.smithi193.stdout:2/343: rename d0/df/f19 to d0/d37/d25/d5f/d3a/d6d/d85/f8d 0 2024-04-25T15:49:36.499 INFO:tasks.workunit.client.0.smithi110.stdout:4/620: dwrite d13/d24/d2d/f62 [0,4194304] 0 2024-04-25T15:49:36.502 INFO:tasks.workunit.client.1.smithi193.stdout:6/346: dwrite dd/d12/d19/d1c/f2c [0,4194304] 0 2024-04-25T15:49:36.504 INFO:tasks.workunit.client.1.smithi193.stdout:7/310: mkdir d12/d13/d78 0 2024-04-25T15:49:36.506 INFO:tasks.workunit.client.1.smithi193.stdout:6/347: dread - dd/d12/d19/d1c/f46 zero size 2024-04-25T15:49:36.506 INFO:tasks.workunit.client.1.smithi193.stdout:7/311: readlink d12/d31/d39/l67 0 2024-04-25T15:49:36.506 INFO:tasks.workunit.client.0.smithi110.stdout:1/764: dwrite d0/d14/d5f/d7d/fc1 [0,4194304] 0 2024-04-25T15:49:36.507 INFO:tasks.workunit.client.1.smithi193.stdout:1/298: dwrite dc/d35/d3d/f43 [0,4194304] 0 2024-04-25T15:49:36.508 INFO:tasks.workunit.client.0.smithi110.stdout:1/765: truncate d0/d25/ff4 480137 0 2024-04-25T15:49:36.515 INFO:tasks.workunit.client.1.smithi193.stdout:5/309: write f16 [3957626,32949] 0 2024-04-25T15:49:36.516 INFO:tasks.workunit.client.1.smithi193.stdout:5/310: read - d17/d18/d30/d38/f69 zero size 2024-04-25T15:49:36.519 INFO:tasks.workunit.client.0.smithi110.stdout:4/621: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc 0 2024-04-25T15:49:36.519 INFO:tasks.workunit.client.1.smithi193.stdout:4/376: mkdir db/d4b/d51/d54/d77 0 2024-04-25T15:49:36.522 INFO:tasks.workunit.client.1.smithi193.stdout:0/365: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e 0 2024-04-25T15:49:36.523 INFO:tasks.workunit.client.0.smithi110.stdout:4/622: mkdir d13/d24/d26/dcd 0 2024-04-25T15:49:36.524 INFO:tasks.workunit.client.1.smithi193.stdout:4/377: dread db/d32/f5f [0,4194304] 0 2024-04-25T15:49:36.524 INFO:tasks.workunit.client.0.smithi110.stdout:1/766: creat d0/d25/d33/d94/ddb/ff6 x:0 0 0 2024-04-25T15:49:36.525 INFO:tasks.workunit.client.1.smithi193.stdout:2/344: fsync d0/d42/d51/f40 0 2024-04-25T15:49:36.525 INFO:tasks.workunit.client.1.smithi193.stdout:4/378: write db/d13/d1e/d1f/d56/f74 [243385,98031] 0 2024-04-25T15:49:36.526 INFO:tasks.workunit.client.1.smithi193.stdout:4/379: chown db/f61 837529861 1 2024-04-25T15:49:36.528 INFO:tasks.workunit.client.0.smithi110.stdout:1/767: chown d0/d25/c71 4640 1 2024-04-25T15:49:36.529 INFO:tasks.workunit.client.0.smithi110.stdout:4/623: symlink d13/d24/d2d/d44/d65/lce 0 2024-04-25T15:49:36.532 INFO:tasks.workunit.client.0.smithi110.stdout:7/648: dwrite d1b/d29/d38/f3d [0,4194304] 0 2024-04-25T15:49:36.532 INFO:tasks.workunit.client.1.smithi193.stdout:7/312: rename d12/d38 to d12/d31/d37/d62/d6e/d79 0 2024-04-25T15:49:36.532 INFO:tasks.workunit.client.1.smithi193.stdout:2/345: dwrite d0/d37/d25/d5f/d3a/d6d/d6e/f3e [0,4194304] 0 2024-04-25T15:49:36.534 INFO:tasks.workunit.client.1.smithi193.stdout:1/299: mkdir dc/d4a/d50/d5c 0 2024-04-25T15:49:36.535 INFO:tasks.workunit.client.1.smithi193.stdout:3/387: fdatasync d8/d13/d14/d23/f48 0 2024-04-25T15:49:36.536 INFO:tasks.workunit.client.1.smithi193.stdout:1/300: truncate dc/d4a/f54 318153 0 2024-04-25T15:49:36.540 INFO:tasks.workunit.client.0.smithi110.stdout:4/624: creat d13/d24/d2d/d36/d51/fcf x:0 0 0 2024-04-25T15:49:36.542 INFO:tasks.workunit.client.0.smithi110.stdout:7/649: fdatasync d1b/f79 0 2024-04-25T15:49:36.545 INFO:tasks.workunit.client.1.smithi193.stdout:1/301: dwrite dc/d4a/f54 [0,4194304] 0 2024-04-25T15:49:36.546 INFO:tasks.workunit.client.0.smithi110.stdout:1/768: creat d0/d14/d5f/d7d/d9c/dcc/ff7 x:0 0 0 2024-04-25T15:49:36.548 INFO:tasks.workunit.client.1.smithi193.stdout:9/348: mkdir df/d2c/d34/d3e/d4d/d5e 0 2024-04-25T15:49:36.549 INFO:tasks.workunit.client.0.smithi110.stdout:4/625: dwrite d13/d24/d2d/d44/d65/d99/dbc/fc4 [0,4194304] 0 2024-04-25T15:49:36.554 INFO:tasks.workunit.client.0.smithi110.stdout:7/650: mknod d1b/d29/d87/d8e/cdb 0 2024-04-25T15:49:36.555 INFO:tasks.workunit.client.1.smithi193.stdout:5/311: dread d17/d18/f24 [0,4194304] 0 2024-04-25T15:49:36.558 INFO:tasks.workunit.client.0.smithi110.stdout:7/651: dread d1b/d29/d38/f3d [0,4194304] 0 2024-04-25T15:49:36.559 INFO:tasks.workunit.client.1.smithi193.stdout:2/346: dread d0/d42/d51/f40 [4194304,4194304] 0 2024-04-25T15:49:36.560 INFO:tasks.workunit.client.1.smithi193.stdout:1/302: dwrite dc/d4a/d4d/f57 [0,4194304] 0 2024-04-25T15:49:36.561 INFO:tasks.workunit.client.0.smithi110.stdout:3/675: sync 2024-04-25T15:49:36.562 INFO:tasks.workunit.client.0.smithi110.stdout:4/626: mknod d13/d24/d2d/d7d/d92/cd0 0 2024-04-25T15:49:36.562 INFO:tasks.workunit.client.1.smithi193.stdout:7/313: unlink d12/d1c/d63/d70/d68/f77 0 2024-04-25T15:49:36.563 INFO:tasks.workunit.client.1.smithi193.stdout:7/314: readlink d12/d1c/l51 0 2024-04-25T15:49:36.563 INFO:tasks.workunit.client.1.smithi193.stdout:1/303: truncate dc/f49 1648239 0 2024-04-25T15:49:36.563 INFO:tasks.workunit.client.1.smithi193.stdout:9/349: dwrite df/d2d/f31 [0,4194304] 0 2024-04-25T15:49:36.566 INFO:tasks.workunit.client.0.smithi110.stdout:7/652: mknod d1b/d29/d38/d55/cdc 0 2024-04-25T15:49:36.571 INFO:tasks.workunit.client.1.smithi193.stdout:2/347: dwrite d0/d9/f39 [0,4194304] 0 2024-04-25T15:49:36.572 INFO:tasks.workunit.client.1.smithi193.stdout:9/350: truncate df/d2c/d2f/d4e/d1c/d26/d3f/d41/f4b 438686 0 2024-04-25T15:49:36.573 INFO:tasks.workunit.client.1.smithi193.stdout:6/348: write dd/d12/f1e [1119126,118890] 0 2024-04-25T15:49:36.573 INFO:tasks.workunit.client.0.smithi110.stdout:4/627: truncate d13/d24/d2d/d44/d65/d99/daf/d56/f8e 996284 0 2024-04-25T15:49:36.574 INFO:tasks.workunit.client.0.smithi110.stdout:7/653: dwrite d1b/d29/d38/d55/d7f/f78 [0,4194304] 0 2024-04-25T15:49:36.577 INFO:tasks.workunit.client.1.smithi193.stdout:9/351: write df/d2c/d2f/d4e/d1c/d26/d3f/d41/f59 [686140,97420] 0 2024-04-25T15:49:36.578 INFO:tasks.workunit.client.1.smithi193.stdout:4/380: dwrite db/fc [0,4194304] 0 2024-04-25T15:49:36.583 INFO:tasks.workunit.client.0.smithi110.stdout:1/769: dwrite d0/fa [0,4194304] 0 2024-04-25T15:49:36.585 INFO:tasks.workunit.client.0.smithi110.stdout:7/654: truncate d1b/d29/d46/f5c 596953 0 2024-04-25T15:49:36.586 INFO:tasks.workunit.client.0.smithi110.stdout:6/590: sync 2024-04-25T15:49:36.586 INFO:tasks.workunit.client.0.smithi110.stdout:5/651: sync 2024-04-25T15:49:36.587 INFO:tasks.workunit.client.1.smithi193.stdout:5/312: symlink d17/d18/d30/d34/d52/l6f 0 2024-04-25T15:49:36.587 INFO:tasks.workunit.client.0.smithi110.stdout:0/592: sync 2024-04-25T15:49:36.587 INFO:tasks.workunit.client.0.smithi110.stdout:9/581: sync 2024-04-25T15:49:36.587 INFO:tasks.workunit.client.0.smithi110.stdout:2/610: sync 2024-04-25T15:49:36.592 INFO:tasks.workunit.client.0.smithi110.stdout:1/770: rename d0/d25 to d0/d14/d81/df8 0 2024-04-25T15:49:36.593 INFO:tasks.workunit.client.0.smithi110.stdout:2/611: chown de/d40/d5b/d5c/d5f/fa8 680933337 1 2024-04-25T15:49:36.596 INFO:tasks.workunit.client.1.smithi193.stdout:1/304: rename dc/d35 to dc/d4a/d50/d5c/d5d 0 2024-04-25T15:49:36.601 INFO:tasks.workunit.client.0.smithi110.stdout:7/655: creat d1b/d29/d38/d8b/fdd x:0 0 0 2024-04-25T15:49:36.602 INFO:tasks.workunit.client.1.smithi193.stdout:2/348: mkdir d0/d37/d25/d5f/d3a/d6d/d6e/d8e 0 2024-04-25T15:49:36.603 INFO:tasks.workunit.client.0.smithi110.stdout:9/582: dwrite d9/d13/d7c/fb7 [0,4194304] 0 2024-04-25T15:49:36.604 INFO:tasks.workunit.client.0.smithi110.stdout:9/583: fsync d9/db/df/d2c/f70 0 2024-04-25T15:49:36.606 INFO:tasks.workunit.client.0.smithi110.stdout:9/584: stat d9/db/df/d2c/d3c/l4e 0 2024-04-25T15:49:36.606 INFO:tasks.workunit.client.1.smithi193.stdout:9/352: mknod df/d2c/d2f/c5f 0 2024-04-25T15:49:36.607 INFO:tasks.workunit.client.1.smithi193.stdout:2/349: dwrite d0/fc [0,4194304] 0 2024-04-25T15:49:36.607 INFO:tasks.workunit.client.0.smithi110.stdout:2/612: symlink de/d40/d4f/lbf 0 2024-04-25T15:49:36.608 INFO:tasks.workunit.client.0.smithi110.stdout:2/613: fsync de/d40/d5b/d5c/fba 0 2024-04-25T15:49:36.609 INFO:tasks.workunit.client.0.smithi110.stdout:9/585: truncate d9/db/df/d25/f9c 24555 0 2024-04-25T15:49:36.609 INFO:tasks.workunit.client.0.smithi110.stdout:1/771: mkdir d0/d14/d5f/d7d/d72/d88/dbf/df9 0 2024-04-25T15:49:36.611 INFO:tasks.workunit.client.1.smithi193.stdout:5/313: creat d17/d18/d45/f70 x:0 0 0 2024-04-25T15:49:36.611 INFO:tasks.workunit.client.0.smithi110.stdout:9/586: write d9/db/df/d2c/d3c/fac [681619,18016] 0 2024-04-25T15:49:36.613 INFO:tasks.workunit.client.0.smithi110.stdout:5/652: rename d4/d1d/l8c to d4/d1d/d1f/d35/d7f/d92/d9b/dcb/ld5 0 2024-04-25T15:49:36.615 INFO:tasks.workunit.client.1.smithi193.stdout:1/305: creat dc/d4a/d4d/f5e x:0 0 0 2024-04-25T15:49:36.616 INFO:tasks.workunit.client.1.smithi193.stdout:1/306: write dc/d4a/d4d/f57 [915324,124121] 0 2024-04-25T15:49:36.618 INFO:tasks.workunit.client.0.smithi110.stdout:0/593: creat d1/d23/d9e/fda x:0 0 0 2024-04-25T15:49:36.618 INFO:tasks.workunit.client.0.smithi110.stdout:3/676: write d0/f3 [779199,29481] 0 2024-04-25T15:49:36.619 INFO:tasks.workunit.client.1.smithi193.stdout:6/349: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/c63 0 2024-04-25T15:49:36.619 INFO:tasks.workunit.client.0.smithi110.stdout:4/628: write d13/d24/f48 [638993,23812] 0 2024-04-25T15:49:36.622 INFO:tasks.workunit.client.0.smithi110.stdout:0/594: dread d1/d6c/d7f/fb5 [0,4194304] 0 2024-04-25T15:49:36.624 INFO:tasks.workunit.client.0.smithi110.stdout:0/595: truncate d1/d23/d30/d33/d5b/d72/d77/fd4 649562 0 2024-04-25T15:49:36.625 INFO:tasks.workunit.client.0.smithi110.stdout:0/596: readlink d1/d23/d30/d33/d35/l55 0 2024-04-25T15:49:36.626 INFO:tasks.workunit.client.0.smithi110.stdout:2/614: symlink de/d40/d86/lc0 0 2024-04-25T15:49:36.627 INFO:tasks.workunit.client.1.smithi193.stdout:5/314: mkdir d17/d18/d30/d38/d71 0 2024-04-25T15:49:36.628 INFO:tasks.workunit.client.1.smithi193.stdout:5/315: truncate d17/d18/d30/d38/f69 680854 0 2024-04-25T15:49:36.628 INFO:tasks.workunit.client.0.smithi110.stdout:9/587: mknod d9/d44/d5e/cbc 0 2024-04-25T15:49:36.629 INFO:tasks.workunit.client.1.smithi193.stdout:1/307: unlink dc/f49 0 2024-04-25T15:49:36.633 INFO:tasks.workunit.client.0.smithi110.stdout:7/656: mkdir d1b/dde 0 2024-04-25T15:49:36.634 INFO:tasks.workunit.client.1.smithi193.stdout:2/350: mknod d0/d9/c8f 0 2024-04-25T15:49:36.636 INFO:tasks.workunit.client.0.smithi110.stdout:3/677: mknod d0/d7/dd0/ddf/ce7 0 2024-04-25T15:49:36.643 INFO:tasks.workunit.client.0.smithi110.stdout:2/615: chown de/d40/d5b/d5c/d5f/db2/cb5 8151 1 2024-04-25T15:49:36.643 INFO:tasks.workunit.client.0.smithi110.stdout:3/678: dwrite d0/d16/f3d [4194304,4194304] 0 2024-04-25T15:49:36.645 INFO:tasks.workunit.client.0.smithi110.stdout:0/597: dread d1/d23/d30/d33/f88 [0,4194304] 0 2024-04-25T15:49:36.645 INFO:tasks.workunit.client.0.smithi110.stdout:3/679: write d0/d7/d7d/fd3 [388351,123735] 0 2024-04-25T15:49:36.648 INFO:tasks.workunit.client.0.smithi110.stdout:9/588: creat d9/db/d58/fbd x:0 0 0 2024-04-25T15:49:36.651 INFO:tasks.workunit.client.0.smithi110.stdout:2/616: write de/d40/d5b/d5c/d5f/db2/f9c [164098,79642] 0 2024-04-25T15:49:36.651 INFO:tasks.workunit.client.0.smithi110.stdout:6/591: truncate d1/db/d24/d2d/fba 293455 0 2024-04-25T15:49:36.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/350: dwrite dd/d12/d19/f43 [4194304,4194304] 0 2024-04-25T15:49:36.652 INFO:tasks.workunit.client.0.smithi110.stdout:6/592: chown d1/d12/l86 543 1 2024-04-25T15:49:36.653 INFO:tasks.workunit.client.0.smithi110.stdout:5/653: mknod d4/d1d/d1f/cd6 0 2024-04-25T15:49:36.654 INFO:tasks.workunit.client.1.smithi193.stdout:5/316: symlink d17/d18/d30/d38/d71/l72 0 2024-04-25T15:49:36.655 INFO:tasks.workunit.client.0.smithi110.stdout:5/654: fsync d4/d1d/d1f/d35/d3d/d52/d75/fc0 0 2024-04-25T15:49:36.657 INFO:tasks.workunit.client.1.smithi193.stdout:1/308: mknod dc/c5f 0 2024-04-25T15:49:36.659 INFO:tasks.workunit.client.1.smithi193.stdout:4/381: dwrite db/f15 [0,4194304] 0 2024-04-25T15:49:36.659 INFO:tasks.workunit.client.0.smithi110.stdout:8/765: sync 2024-04-25T15:49:36.659 INFO:tasks.workunit.client.1.smithi193.stdout:8/311: sync 2024-04-25T15:49:36.660 INFO:tasks.workunit.client.0.smithi110.stdout:7/657: symlink d1b/d29/d38/d55/dab/db2/db8/ldf 0 2024-04-25T15:49:36.661 INFO:tasks.workunit.client.0.smithi110.stdout:7/658: write d1b/f88 [282962,14906] 0 2024-04-25T15:49:36.663 INFO:tasks.workunit.client.0.smithi110.stdout:7/659: write d1b/d29/d38/d8b/d89/f8c [147472,101892] 0 2024-04-25T15:49:36.669 INFO:tasks.workunit.client.1.smithi193.stdout:6/351: symlink dd/d12/l64 0 2024-04-25T15:49:36.670 INFO:tasks.workunit.client.0.smithi110.stdout:0/598: chown d1/d47/f8e 2281 1 2024-04-25T15:49:36.671 INFO:tasks.workunit.client.1.smithi193.stdout:5/317: creat d17/d18/d30/d34/d37/f73 x:0 0 0 2024-04-25T15:49:36.672 INFO:tasks.workunit.client.1.smithi193.stdout:8/312: read d2/d3/d20/f22 [192349,89186] 0 2024-04-25T15:49:36.673 INFO:tasks.workunit.client.0.smithi110.stdout:0/599: dread - d1/d47/d54/d99/fc1 zero size 2024-04-25T15:49:36.673 INFO:tasks.workunit.client.1.smithi193.stdout:8/313: chown d2/d3/d3a/c4d 16 1 2024-04-25T15:49:36.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/314: write d2/d3/d20/f22 [2154744,23647] 0 2024-04-25T15:49:36.676 INFO:tasks.workunit.client.0.smithi110.stdout:1/772: dwrite d0/d14/d81/df8/f6 [0,4194304] 0 2024-04-25T15:49:36.680 INFO:tasks.workunit.client.0.smithi110.stdout:3/680: rename d0/d7/dd0/ddf/fe0 to d0/d16/dcb/db0/fe8 0 2024-04-25T15:49:36.681 INFO:tasks.workunit.client.1.smithi193.stdout:8/315: dwrite d2/d3/d20/d45/d4a/f4e [0,4194304] 0 2024-04-25T15:49:36.684 INFO:tasks.workunit.client.0.smithi110.stdout:1/773: dread d0/d14/d5f/f65 [0,4194304] 0 2024-04-25T15:49:36.690 INFO:tasks.workunit.client.0.smithi110.stdout:7/660: stat d1b/c20 0 2024-04-25T15:49:36.691 INFO:tasks.workunit.client.0.smithi110.stdout:3/681: dwrite d0/d7/d40/fd1 [0,4194304] 0 2024-04-25T15:49:36.693 INFO:tasks.workunit.client.0.smithi110.stdout:5/655: dread d4/f10 [0,4194304] 0 2024-04-25T15:49:36.693 INFO:tasks.workunit.client.0.smithi110.stdout:5/656: fsync d4/d2c/d2a/f5c 0 2024-04-25T15:49:36.694 INFO:tasks.workunit.client.0.smithi110.stdout:4/629: link d13/l3f d13/d24/d2d/d44/d65/dc9/ld1 0 2024-04-25T15:49:36.695 INFO:tasks.workunit.client.0.smithi110.stdout:4/630: readlink d13/l28 0 2024-04-25T15:49:36.696 INFO:tasks.workunit.client.0.smithi110.stdout:3/682: truncate d0/d2e/d88/dbf/d92/fd2 1250737 0 2024-04-25T15:49:36.697 INFO:tasks.workunit.client.0.smithi110.stdout:3/683: fsync d0/d7/f2a 0 2024-04-25T15:49:36.698 INFO:tasks.workunit.client.1.smithi193.stdout:5/318: creat d17/d18/d30/d38/d71/f74 x:0 0 0 2024-04-25T15:49:36.699 INFO:tasks.workunit.client.0.smithi110.stdout:3/684: write d0/d16/dcb/d2f/d4c/db2/fe4 [636351,109255] 0 2024-04-25T15:49:36.701 INFO:tasks.workunit.client.0.smithi110.stdout:0/600: unlink d1/d23/d30/d33/d35/c97 0 2024-04-25T15:49:36.702 INFO:tasks.workunit.client.1.smithi193.stdout:1/309: symlink dc/d4a/d50/d5c/d5d/l60 0 2024-04-25T15:49:36.704 INFO:tasks.workunit.client.1.smithi193.stdout:8/316: symlink d2/d3/d27/l64 0 2024-04-25T15:49:36.705 INFO:tasks.workunit.client.1.smithi193.stdout:9/353: write df/d2c/d2f/d4e/d1c/f3d [1659236,35302] 0 2024-04-25T15:49:36.706 INFO:tasks.workunit.client.1.smithi193.stdout:4/382: dwrite db/d13/d1e/f48 [0,4194304] 0 2024-04-25T15:49:36.707 INFO:tasks.workunit.client.0.smithi110.stdout:9/589: rename d9/d13/d84/d98 to d9/d44/d5e/d80/dbb/dbe 0 2024-04-25T15:49:36.708 INFO:tasks.workunit.client.1.smithi193.stdout:4/383: stat db/d13/d1e/d1f/d56/d69/d70/f76 0 2024-04-25T15:49:36.709 INFO:tasks.workunit.client.0.smithi110.stdout:1/774: chown d0/d14/d5f/d7d/cc5 6 1 2024-04-25T15:49:36.714 INFO:tasks.workunit.client.1.smithi193.stdout:7/315: sync 2024-04-25T15:49:36.714 INFO:tasks.workunit.client.1.smithi193.stdout:3/388: sync 2024-04-25T15:49:36.714 INFO:tasks.workunit.client.1.smithi193.stdout:0/366: sync 2024-04-25T15:49:36.715 INFO:tasks.workunit.client.0.smithi110.stdout:3/685: creat d0/d2e/d88/dbf/d99/dad/fe9 x:0 0 0 2024-04-25T15:49:36.717 INFO:tasks.workunit.client.1.smithi193.stdout:1/310: chown dc/c25 183638 1 2024-04-25T15:49:36.718 INFO:tasks.workunit.client.1.smithi193.stdout:1/311: rename dc to dc/d4a/d50/d5c/d5d/d3d/d55/d61 22 2024-04-25T15:49:36.720 INFO:tasks.workunit.client.0.smithi110.stdout:9/590: unlink d9/db/d58/l86 0 2024-04-25T15:49:36.721 INFO:tasks.workunit.client.0.smithi110.stdout:1/775: rmdir d0/d14/d81/df8/d33/da9 39 2024-04-25T15:49:36.722 INFO:tasks.workunit.client.1.smithi193.stdout:9/354: mkdir df/d2c/d2f/d4e/d1c/d30/d60 0 2024-04-25T15:49:36.724 INFO:tasks.workunit.client.0.smithi110.stdout:5/657: creat d4/d1d/d1f/d35/d3d/d93/dc4/fd7 x:0 0 0 2024-04-25T15:49:36.726 INFO:tasks.workunit.client.0.smithi110.stdout:5/658: dread d4/d1d/d1f/d35/d3d/fc5 [0,4194304] 0 2024-04-25T15:49:36.729 INFO:tasks.workunit.client.0.smithi110.stdout:0/601: mkdir d1/d23/d30/d33/d5b/d78/ddb 0 2024-04-25T15:49:36.730 INFO:tasks.workunit.client.0.smithi110.stdout:0/602: read d1/d23/d30/d33/d5b/d72/d77/fd4 [191230,23833] 0 2024-04-25T15:49:36.732 INFO:tasks.workunit.client.1.smithi193.stdout:6/352: dwrite dd/f24 [0,4194304] 0 2024-04-25T15:49:36.733 INFO:tasks.workunit.client.0.smithi110.stdout:1/776: write d0/d51/dd8/fef [1850695,49740] 0 2024-04-25T15:49:36.735 INFO:tasks.workunit.client.0.smithi110.stdout:5/659: symlink d4/d1d/d1f/d35/d7f/d92/d9b/ld8 0 2024-04-25T15:49:36.737 INFO:tasks.workunit.client.0.smithi110.stdout:1/777: dread d0/d23/f5c [0,4194304] 0 2024-04-25T15:49:36.739 INFO:tasks.workunit.client.0.smithi110.stdout:9/591: dread d9/db/df/d25/f37 [0,4194304] 0 2024-04-25T15:49:36.744 INFO:tasks.workunit.client.0.smithi110.stdout:2/617: rename de/d40/l46 to de/d40/d6e/d52/d6d/lc1 0 2024-04-25T15:49:36.747 INFO:tasks.workunit.client.1.smithi193.stdout:7/316: creat d12/d1c/d45/f7a x:0 0 0 2024-04-25T15:49:36.748 INFO:tasks.workunit.client.1.smithi193.stdout:1/312: mknod dc/d4a/d50/d5c/c62 0 2024-04-25T15:49:36.748 INFO:tasks.workunit.client.1.smithi193.stdout:9/355: creat df/d2d/f61 x:0 0 0 2024-04-25T15:49:36.748 INFO:tasks.workunit.client.1.smithi193.stdout:4/384: symlink db/l78 0 2024-04-25T15:49:36.749 INFO:tasks.workunit.client.1.smithi193.stdout:3/389: write d8/d13/d4b/f5d [1390692,36176] 0 2024-04-25T15:49:36.751 INFO:tasks.workunit.client.0.smithi110.stdout:5/660: chown d4/d15/l48 46544608 1 2024-04-25T15:49:36.752 INFO:tasks.workunit.client.0.smithi110.stdout:0/603: dread d1/d23/d30/d33/d5b/d83/f5d [0,4194304] 0 2024-04-25T15:49:36.753 INFO:tasks.workunit.client.1.smithi193.stdout:8/317: rmdir d2/d3/d3a/d50 0 2024-04-25T15:49:36.757 INFO:tasks.workunit.client.0.smithi110.stdout:9/592: fdatasync d9/d13/f29 0 2024-04-25T15:49:36.757 INFO:tasks.workunit.client.0.smithi110.stdout:3/686: dwrite d0/d2e/d88/dbf/d92/fbb [0,4194304] 0 2024-04-25T15:49:36.758 INFO:tasks.workunit.client.0.smithi110.stdout:2/618: dwrite de/d40/d70/f7f [0,4194304] 0 2024-04-25T15:49:36.759 INFO:tasks.workunit.client.0.smithi110.stdout:9/593: readlink d9/d44/d5e/la6 0 2024-04-25T15:49:36.760 INFO:tasks.workunit.client.1.smithi193.stdout:7/317: symlink d12/d31/d37/l7b 0 2024-04-25T15:49:36.764 INFO:tasks.workunit.client.0.smithi110.stdout:6/593: rename d1/db/d22/d90/c74 to d1/dd/d3c/d91/cd0 0 2024-04-25T15:49:36.769 INFO:tasks.workunit.client.0.smithi110.stdout:9/594: dwrite d9/db/df/f21 [0,4194304] 0 2024-04-25T15:49:36.770 INFO:tasks.workunit.client.1.smithi193.stdout:1/313: mkdir dc/d4a/d50/d5c/d63 0 2024-04-25T15:49:36.771 INFO:tasks.workunit.client.1.smithi193.stdout:5/319: getdents d17/d18/d30/d34/d37 0 2024-04-25T15:49:36.773 INFO:tasks.workunit.client.1.smithi193.stdout:5/320: write d17/d18/d3e/f4c [3640088,121427] 0 2024-04-25T15:49:36.780 INFO:tasks.workunit.client.0.smithi110.stdout:0/604: mknod d1/d20/d69/cdc 0 2024-04-25T15:49:36.781 INFO:tasks.workunit.client.1.smithi193.stdout:1/314: symlink dc/d4a/d50/l64 0 2024-04-25T15:49:36.782 INFO:tasks.workunit.client.0.smithi110.stdout:2/619: mknod de/d40/d6e/d52/cc2 0 2024-04-25T15:49:36.786 INFO:tasks.workunit.client.0.smithi110.stdout:4/631: rename d13/d24/d2d/d44/d65/dc9 to d13/d24/d2d/d44/d90/db8/dbb/dd2 0 2024-04-25T15:49:36.787 INFO:tasks.workunit.client.1.smithi193.stdout:5/321: write d17/f1d [2801747,282] 0 2024-04-25T15:49:36.787 INFO:tasks.workunit.client.0.smithi110.stdout:6/594: creat d1/db/d22/d33/d43/fd1 x:0 0 0 2024-04-25T15:49:36.788 INFO:tasks.workunit.client.0.smithi110.stdout:6/595: stat d1/dd/d3c/d91/lab 0 2024-04-25T15:49:36.789 INFO:tasks.workunit.client.1.smithi193.stdout:4/385: link db/d13/d1e/d1f/f63 db/d4b/d51/d54/d77/f79 0 2024-04-25T15:49:36.789 INFO:tasks.workunit.client.1.smithi193.stdout:4/386: chown f9 0 1 2024-04-25T15:49:36.790 INFO:tasks.workunit.client.0.smithi110.stdout:9/595: creat d9/d13/fbf x:0 0 0 2024-04-25T15:49:36.790 INFO:tasks.workunit.client.0.smithi110.stdout:5/661: dread d4/d1d/d1f/f8f [0,4194304] 0 2024-04-25T15:49:36.790 INFO:tasks.workunit.client.1.smithi193.stdout:4/387: chown db/d13/l19 377174055 1 2024-04-25T15:49:36.792 INFO:tasks.workunit.client.0.smithi110.stdout:0/605: unlink d1/f1c 0 2024-04-25T15:49:36.794 INFO:tasks.workunit.client.1.smithi193.stdout:8/318: fsync d2/d3/d20/d45/d29/f57 0 2024-04-25T15:49:36.795 INFO:tasks.workunit.client.1.smithi193.stdout:8/319: chown d2/d3/d20/d45/c3e 109 1 2024-04-25T15:49:36.795 INFO:tasks.workunit.client.1.smithi193.stdout:8/320: write d2/f48 [4860285,40925] 0 2024-04-25T15:49:36.797 INFO:tasks.workunit.client.1.smithi193.stdout:2/351: sync 2024-04-25T15:49:36.799 INFO:tasks.workunit.client.1.smithi193.stdout:2/352: fsync d0/d37/d25/d5f/d3a/f4a 0 2024-04-25T15:49:36.801 INFO:tasks.workunit.client.0.smithi110.stdout:1/778: truncate d0/d23/f3b 2361625 0 2024-04-25T15:49:36.802 INFO:tasks.workunit.client.1.smithi193.stdout:9/356: dwrite f9 [0,4194304] 0 2024-04-25T15:49:36.802 INFO:tasks.workunit.client.0.smithi110.stdout:1/779: read - d0/d14/fe7 zero size 2024-04-25T15:49:36.802 INFO:tasks.workunit.client.0.smithi110.stdout:1/780: chown d0/d14/fad 362903439 1 2024-04-25T15:49:36.803 INFO:tasks.workunit.client.0.smithi110.stdout:6/596: truncate d1/db/d22/d90/d9e/f9f 613454 0 2024-04-25T15:49:36.804 INFO:tasks.workunit.client.0.smithi110.stdout:1/781: write d0/d14/d5f/d7d/d72/ff5 [822179,52349] 0 2024-04-25T15:49:36.805 INFO:tasks.workunit.client.0.smithi110.stdout:9/596: symlink d9/db/df/d2c/d3c/d56/lc0 0 2024-04-25T15:49:36.811 INFO:tasks.workunit.client.0.smithi110.stdout:5/662: mkdir d4/d1d/d1f/d35/d3d/d52/d97/dd9 0 2024-04-25T15:49:36.813 INFO:tasks.workunit.client.0.smithi110.stdout:5/663: chown d4/d1d/d1f/f8a 72 1 2024-04-25T15:49:36.813 INFO:tasks.workunit.client.1.smithi193.stdout:4/388: truncate db/d13/f1d 832970 0 2024-04-25T15:49:36.814 INFO:tasks.workunit.client.0.smithi110.stdout:9/597: dwrite d9/d13/d84/f9b [0,4194304] 0 2024-04-25T15:49:36.818 INFO:tasks.workunit.client.0.smithi110.stdout:2/620: symlink de/d40/d5b/d5c/d5f/lc3 0 2024-04-25T15:49:36.818 INFO:tasks.workunit.client.1.smithi193.stdout:6/353: link dd/d12/d19/l32 dd/d12/d19/d1c/d40/d3c/d3f/d4f/l65 0 2024-04-25T15:49:36.820 INFO:tasks.workunit.client.1.smithi193.stdout:3/390: truncate d8/d13/d1f/f22 2276989 0 2024-04-25T15:49:36.820 INFO:tasks.workunit.client.0.smithi110.stdout:4/632: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/fd3 x:0 0 0 2024-04-25T15:49:36.821 INFO:tasks.workunit.client.1.smithi193.stdout:3/391: dread - d8/d30/d45/f68 zero size 2024-04-25T15:49:36.822 INFO:tasks.workunit.client.1.smithi193.stdout:8/321: mknod d2/d3/d20/d45/d29/c65 0 2024-04-25T15:49:36.822 INFO:tasks.workunit.client.1.smithi193.stdout:3/392: truncate d8/f39 5170925 0 2024-04-25T15:49:36.825 INFO:tasks.workunit.client.1.smithi193.stdout:8/322: read d2/d3/d3a/f4c [50343,62391] 0 2024-04-25T15:49:36.825 INFO:tasks.workunit.client.1.smithi193.stdout:6/354: dwrite dd/d12/d19/f51 [0,4194304] 0 2024-04-25T15:49:36.827 INFO:tasks.workunit.client.0.smithi110.stdout:1/782: creat d0/d14/d81/df8/d33/d8e/ffa x:0 0 0 2024-04-25T15:49:36.828 INFO:tasks.workunit.client.1.smithi193.stdout:7/318: truncate f8 2320702 0 2024-04-25T15:49:36.832 INFO:tasks.workunit.client.1.smithi193.stdout:0/367: sync 2024-04-25T15:49:36.832 INFO:tasks.workunit.client.1.smithi193.stdout:8/323: dwrite d2/d3/d20/d45/d4a/d53/f58 [0,4194304] 0 2024-04-25T15:49:36.833 INFO:tasks.workunit.client.1.smithi193.stdout:1/315: creat dc/d59/f65 x:0 0 0 2024-04-25T15:49:36.834 INFO:tasks.workunit.client.1.smithi193.stdout:1/316: fsync dc/d4a/d50/d5c/d5d/f3a 0 2024-04-25T15:49:36.835 INFO:tasks.workunit.client.0.smithi110.stdout:0/606: symlink d1/d23/d30/ldd 0 2024-04-25T15:49:36.838 INFO:tasks.workunit.client.1.smithi193.stdout:9/357: symlink df/d2c/d2f/d4e/l62 0 2024-04-25T15:49:36.838 INFO:tasks.workunit.client.0.smithi110.stdout:0/607: read d1/f8b [1738522,24369] 0 2024-04-25T15:49:36.838 INFO:tasks.workunit.client.0.smithi110.stdout:5/664: mknod d4/d15/d51/cda 0 2024-04-25T15:49:36.841 INFO:tasks.workunit.client.0.smithi110.stdout:0/608: write d1/d47/d54/d99/fb7 [5154888,62856] 0 2024-04-25T15:49:36.841 INFO:tasks.workunit.client.0.smithi110.stdout:5/665: dread - d4/d1d/d1f/d35/d3d/d93/dc4/fd7 zero size 2024-04-25T15:49:36.841 INFO:tasks.workunit.client.1.smithi193.stdout:8/324: dwrite d2/f48 [0,4194304] 0 2024-04-25T15:49:36.844 INFO:tasks.workunit.client.1.smithi193.stdout:5/322: dwrite d17/d18/f2c [0,4194304] 0 2024-04-25T15:49:36.844 INFO:tasks.workunit.client.0.smithi110.stdout:9/598: dread d9/db/df/f33 [0,4194304] 0 2024-04-25T15:49:36.844 INFO:tasks.workunit.client.0.smithi110.stdout:5/666: chown d4/d1d/d1f/f8f 22604007 1 2024-04-25T15:49:36.844 INFO:tasks.workunit.client.0.smithi110.stdout:0/609: chown d1/d23/d30/d33/d5b/d72/d77/fd4 43450671 1 2024-04-25T15:49:36.845 INFO:tasks.workunit.client.0.smithi110.stdout:2/621: creat de/d40/d5b/d5c/d5f/db2/fc4 x:0 0 0 2024-04-25T15:49:36.853 INFO:tasks.workunit.client.1.smithi193.stdout:3/393: fdatasync d8/d13/d14/f59 0 2024-04-25T15:49:36.855 INFO:tasks.workunit.client.0.smithi110.stdout:1/783: symlink d0/d14/d5f/d7d/d72/d88/db6/lfb 0 2024-04-25T15:49:36.858 INFO:tasks.workunit.client.1.smithi193.stdout:6/355: rmdir dd/d12 39 2024-04-25T15:49:36.861 INFO:tasks.workunit.client.1.smithi193.stdout:2/353: mkdir d0/d9/d15/d31/d90 0 2024-04-25T15:49:36.862 INFO:tasks.workunit.client.1.smithi193.stdout:2/354: stat d0/d42/d51/d4f/d65 0 2024-04-25T15:49:36.863 INFO:tasks.workunit.client.1.smithi193.stdout:7/319: symlink d12/d31/d37/d62/d6e/d79/l7c 0 2024-04-25T15:49:36.863 INFO:tasks.workunit.client.0.smithi110.stdout:4/633: dread d13/d24/d2d/d36/f4d [0,4194304] 0 2024-04-25T15:49:36.865 INFO:tasks.workunit.client.1.smithi193.stdout:2/355: dread d0/fc [0,4194304] 0 2024-04-25T15:49:36.867 INFO:tasks.workunit.client.1.smithi193.stdout:0/368: rmdir d0/d49/d5/d7/dd/d16/d26 39 2024-04-25T15:49:36.868 INFO:tasks.workunit.client.1.smithi193.stdout:0/369: fsync d0/d49/d5/d7/dd/d16/f33 0 2024-04-25T15:49:36.881 INFO:tasks.workunit.client.0.smithi110.stdout:9/599: creat d9/d44/d5e/fc1 x:0 0 0 2024-04-25T15:49:36.882 INFO:tasks.workunit.client.1.smithi193.stdout:1/317: mknod dc/d3c/c66 0 2024-04-25T15:49:36.883 INFO:tasks.workunit.client.1.smithi193.stdout:1/318: fdatasync dc/d4a/d4d/f57 0 2024-04-25T15:49:36.885 INFO:tasks.workunit.client.0.smithi110.stdout:6/597: creat d1/db/d22/fd2 x:0 0 0 2024-04-25T15:49:36.888 INFO:tasks.workunit.client.1.smithi193.stdout:4/389: link db/d32/d45/f60 db/d13/d1e/d1f/d22/f7a 0 2024-04-25T15:49:36.889 INFO:tasks.workunit.client.1.smithi193.stdout:4/390: stat db/d32/c39 0 2024-04-25T15:49:36.890 INFO:tasks.workunit.client.1.smithi193.stdout:8/325: rename d2/d3/lc to d2/d1f/l66 0 2024-04-25T15:49:36.891 INFO:tasks.workunit.client.1.smithi193.stdout:8/326: chown d2/d3/d20/f22 164 1 2024-04-25T15:49:36.892 INFO:tasks.workunit.client.0.smithi110.stdout:4/634: rename d13/d24/d2d/d7d/d92/f96 to d13/d24/d26/dcd/fd4 0 2024-04-25T15:49:36.893 INFO:tasks.workunit.client.0.smithi110.stdout:4/635: readlink d13/d24/d2d/d44/d90/db8/dbb/dd2/ld1 0 2024-04-25T15:49:36.893 INFO:tasks.workunit.client.1.smithi193.stdout:5/323: unlink l2 0 2024-04-25T15:49:36.895 INFO:tasks.workunit.client.0.smithi110.stdout:9/600: dread d9/d2d/f5d [0,4194304] 0 2024-04-25T15:49:36.896 INFO:tasks.workunit.client.0.smithi110.stdout:0/610: symlink d1/d23/d30/lde 0 2024-04-25T15:49:36.898 INFO:tasks.workunit.client.1.smithi193.stdout:5/324: dwrite d17/d18/d3e/f4c [0,4194304] 0 2024-04-25T15:49:36.899 INFO:tasks.workunit.client.1.smithi193.stdout:3/394: creat d8/d13/d1f/f78 x:0 0 0 2024-04-25T15:49:36.904 INFO:tasks.workunit.client.0.smithi110.stdout:6/598: write f0 [6356879,105678] 0 2024-04-25T15:49:36.905 INFO:tasks.workunit.client.1.smithi193.stdout:3/395: dwrite d8/d30/d45/f68 [0,4194304] 0 2024-04-25T15:49:36.906 INFO:tasks.workunit.client.0.smithi110.stdout:1/784: write d0/d14/f53 [1233879,51616] 0 2024-04-25T15:49:36.908 INFO:tasks.workunit.client.1.smithi193.stdout:0/370: fsync d0/d49/d5/d7/dd/d16/d26/d2e/f76 0 2024-04-25T15:49:36.911 INFO:tasks.workunit.client.1.smithi193.stdout:0/371: fdatasync d0/d49/d5/d7/dd/d46/f6b 0 2024-04-25T15:49:36.911 INFO:tasks.workunit.client.0.smithi110.stdout:4/636: fdatasync d13/d24/d2d/d44/f60 0 2024-04-25T15:49:36.913 INFO:tasks.workunit.client.0.smithi110.stdout:4/637: write d13/d24/d2d/d44/d65/d99/daf/d56/f82 [3959067,93071] 0 2024-04-25T15:49:36.914 INFO:tasks.workunit.client.1.smithi193.stdout:9/358: link df/d2c/d2f/d4e/d1c/f3d df/d2c/d34/d3e/f63 0 2024-04-25T15:49:36.914 INFO:tasks.workunit.client.1.smithi193.stdout:7/320: dwrite d12/d31/d37/d62/d6e/f72 [0,4194304] 0 2024-04-25T15:49:36.914 INFO:tasks.workunit.client.0.smithi110.stdout:5/667: creat d4/d2c/fdb x:0 0 0 2024-04-25T15:49:36.916 INFO:tasks.workunit.client.0.smithi110.stdout:5/668: write d4/d1d/d1f/d35/fc6 [518462,70689] 0 2024-04-25T15:49:36.916 INFO:tasks.workunit.client.0.smithi110.stdout:9/601: rmdir d9/d13/d7c 39 2024-04-25T15:49:36.920 INFO:tasks.workunit.client.1.smithi193.stdout:4/391: creat db/d4b/f7b x:0 0 0 2024-04-25T15:49:36.922 INFO:tasks.workunit.client.0.smithi110.stdout:6/599: mknod d1/dd/cd3 0 2024-04-25T15:49:36.924 INFO:tasks.workunit.client.0.smithi110.stdout:0/611: dread d1/f8b [0,4194304] 0 2024-04-25T15:49:36.925 INFO:tasks.workunit.client.1.smithi193.stdout:8/327: unlink d2/d3/d20/f43 0 2024-04-25T15:49:36.925 INFO:tasks.workunit.client.1.smithi193.stdout:9/359: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/f5b [0,4194304] 0 2024-04-25T15:49:36.926 INFO:tasks.workunit.client.0.smithi110.stdout:1/785: creat d0/d14/d5f/d7d/d72/d88/dbf/ffc x:0 0 0 2024-04-25T15:49:36.931 INFO:tasks.workunit.client.1.smithi193.stdout:8/328: dwrite d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:36.933 INFO:tasks.workunit.client.0.smithi110.stdout:4/638: symlink d13/dba/ld5 0 2024-04-25T15:49:36.936 INFO:tasks.workunit.client.1.smithi193.stdout:1/319: write dc/f2b [463008,78917] 0 2024-04-25T15:49:36.937 INFO:tasks.workunit.client.0.smithi110.stdout:5/669: mknod d4/d2c/d83/cdc 0 2024-04-25T15:49:36.941 INFO:tasks.workunit.client.0.smithi110.stdout:6/600: creat d1/db/d22/d33/d63/fd4 x:0 0 0 2024-04-25T15:49:36.942 INFO:tasks.workunit.client.0.smithi110.stdout:6/601: fdatasync d1/db/da0/fb6 0 2024-04-25T15:49:36.943 INFO:tasks.workunit.client.1.smithi193.stdout:7/321: unlink d12/d1c/d63/d70/f3c 0 2024-04-25T15:49:36.943 INFO:tasks.workunit.client.0.smithi110.stdout:6/602: write d1/db/d22/d90/fcb [1207987,111501] 0 2024-04-25T15:49:36.950 INFO:tasks.workunit.client.0.smithi110.stdout:9/602: dread d9/d2d/f30 [0,4194304] 0 2024-04-25T15:49:36.951 INFO:tasks.workunit.client.1.smithi193.stdout:4/392: mknod db/d4b/d51/d54/c7c 0 2024-04-25T15:49:36.951 INFO:tasks.workunit.client.0.smithi110.stdout:9/603: chown d9/db/df/d55/f60 1048 1 2024-04-25T15:49:36.951 INFO:tasks.workunit.client.1.smithi193.stdout:4/393: fdatasync db/d13/d1e/d1f/f64 0 2024-04-25T15:49:36.952 INFO:tasks.workunit.client.0.smithi110.stdout:2/622: getdents de/d6f 0 2024-04-25T15:49:36.952 INFO:tasks.workunit.client.1.smithi193.stdout:4/394: truncate db/d13/d1e/f6d 4538269 0 2024-04-25T15:49:36.958 INFO:tasks.workunit.client.0.smithi110.stdout:6/603: creat d1/db/d24/d69/d75/fd5 x:0 0 0 2024-04-25T15:49:36.959 INFO:tasks.workunit.client.1.smithi193.stdout:9/360: dwrite df/d2c/d2f/d33/f3c [0,4194304] 0 2024-04-25T15:49:36.960 INFO:tasks.workunit.client.0.smithi110.stdout:6/604: write d1/db/da0/fb6 [4606597,86764] 0 2024-04-25T15:49:36.961 INFO:tasks.workunit.client.0.smithi110.stdout:6/605: write d1/db/d24/f29 [527189,60636] 0 2024-04-25T15:49:36.962 INFO:tasks.workunit.client.1.smithi193.stdout:7/322: symlink d12/d1c/l7d 0 2024-04-25T15:49:36.963 INFO:tasks.workunit.client.0.smithi110.stdout:4/639: truncate d13/d24/d2d/d44/d65/d99/daf/d56/f8e 1132320 0 2024-04-25T15:49:36.966 INFO:tasks.workunit.client.1.smithi193.stdout:2/356: rename d0/d37/d25/d5f/d3a/f4a to d0/d9/d15/d31/f91 0 2024-04-25T15:49:36.968 INFO:tasks.workunit.client.0.smithi110.stdout:9/604: creat d9/db/d58/fc2 x:0 0 0 2024-04-25T15:49:36.968 INFO:tasks.workunit.client.1.smithi193.stdout:5/325: truncate d17/d18/d3e/f4c 1714624 0 2024-04-25T15:49:36.969 INFO:tasks.workunit.client.1.smithi193.stdout:4/395: symlink db/d32/d45/l7d 0 2024-04-25T15:49:36.970 INFO:tasks.workunit.client.1.smithi193.stdout:4/396: readlink db/d32/d45/l53 0 2024-04-25T15:49:36.970 INFO:tasks.workunit.client.0.smithi110.stdout:2/623: mknod de/d40/d5b/d5c/d5f/db2/da9/cc5 0 2024-04-25T15:49:36.970 INFO:tasks.workunit.client.1.smithi193.stdout:4/397: readlink db/d13/l75 0 2024-04-25T15:49:36.972 INFO:tasks.workunit.client.1.smithi193.stdout:6/356: creat dd/d12/d19/f66 x:0 0 0 2024-04-25T15:49:36.972 INFO:tasks.workunit.client.0.smithi110.stdout:1/786: link d0/d51/dc8/cd7 d0/d51/d6b/cfd 0 2024-04-25T15:49:36.974 INFO:tasks.workunit.client.0.smithi110.stdout:6/606: mknod d1/db/d22/d90/d76/dad/cd6 0 2024-04-25T15:49:36.974 INFO:tasks.workunit.client.1.smithi193.stdout:7/323: creat d12/d1c/f7e x:0 0 0 2024-04-25T15:49:36.975 INFO:tasks.workunit.client.1.smithi193.stdout:0/372: sync 2024-04-25T15:49:36.975 INFO:tasks.workunit.client.0.smithi110.stdout:2/624: chown de/f24 139941 1 2024-04-25T15:49:36.977 INFO:tasks.workunit.client.1.smithi193.stdout:3/396: rename d8/d13/d14/f20 to d8/d13/d1f/f79 0 2024-04-25T15:49:36.977 INFO:tasks.workunit.client.1.smithi193.stdout:8/329: rename d2 to d2/d36/d67 22 2024-04-25T15:49:36.978 INFO:tasks.workunit.client.0.smithi110.stdout:0/612: rename d1/d47/d54/d99/caf to d1/d23/d30/cdf 0 2024-04-25T15:49:36.978 INFO:tasks.workunit.client.0.smithi110.stdout:4/640: rename d13/d24/d26 to d13/d24/d26/dcd/dd6 22 2024-04-25T15:49:36.979 INFO:tasks.workunit.client.0.smithi110.stdout:6/607: fdatasync d1/db/f44 0 2024-04-25T15:49:36.981 INFO:tasks.workunit.client.1.smithi193.stdout:5/326: fdatasync fe 0 2024-04-25T15:49:36.981 INFO:tasks.workunit.client.1.smithi193.stdout:7/324: dwrite d12/d1c/f3b [0,4194304] 0 2024-04-25T15:49:36.982 INFO:tasks.workunit.client.0.smithi110.stdout:9/605: symlink d9/d13/d5b/d61/db3/db4/lc3 0 2024-04-25T15:49:36.982 INFO:tasks.workunit.client.1.smithi193.stdout:5/327: write d17/d18/d30/f35 [1550303,67073] 0 2024-04-25T15:49:36.983 INFO:tasks.workunit.client.1.smithi193.stdout:4/398: dread - db/d13/d1e/d1f/f28 zero size 2024-04-25T15:49:36.983 INFO:tasks.workunit.client.0.smithi110.stdout:2/625: mkdir de/d40/d70/dc6 0 2024-04-25T15:49:36.985 INFO:tasks.workunit.client.0.smithi110.stdout:9/606: write d9/db/df/d2c/d3c/fa4 [4416108,82158] 0 2024-04-25T15:49:36.985 INFO:tasks.workunit.client.1.smithi193.stdout:4/399: write db/d32/d45/f4f [3845008,125545] 0 2024-04-25T15:49:36.985 INFO:tasks.workunit.client.1.smithi193.stdout:9/361: symlink df/d2c/d2f/d4e/d1c/d26/d3f/d41/d5d/l64 0 2024-04-25T15:49:36.986 INFO:tasks.workunit.client.1.smithi193.stdout:9/362: readlink df/d11/l54 0 2024-04-25T15:49:36.991 INFO:tasks.workunit.client.1.smithi193.stdout:6/357: dread dd/d2f/f56 [0,4194304] 0 2024-04-25T15:49:36.991 INFO:tasks.workunit.client.0.smithi110.stdout:1/787: dread d0/d14/d81/df8/d33/fd2 [0,4194304] 0 2024-04-25T15:49:36.997 INFO:tasks.workunit.client.0.smithi110.stdout:6/608: dread d1/db/d24/f9c [0,4194304] 0 2024-04-25T15:49:36.998 INFO:tasks.workunit.client.1.smithi193.stdout:3/397: symlink d8/d13/d14/d27/l7a 0 2024-04-25T15:49:37.002 INFO:tasks.workunit.client.0.smithi110.stdout:2/626: dread de/d40/d6e/f21 [0,4194304] 0 2024-04-25T15:49:37.004 INFO:tasks.workunit.client.1.smithi193.stdout:7/325: unlink fc 0 2024-04-25T15:49:37.004 INFO:tasks.workunit.client.1.smithi193.stdout:7/326: dread - d12/d1c/f6d zero size 2024-04-25T15:49:37.005 INFO:tasks.workunit.client.0.smithi110.stdout:6/609: dwrite d1/db/d24/f29 [0,4194304] 0 2024-04-25T15:49:37.006 INFO:tasks.workunit.client.1.smithi193.stdout:7/327: write d12/d1c/d63/f65 [608558,7306] 0 2024-04-25T15:49:37.006 INFO:tasks.workunit.client.0.smithi110.stdout:6/610: chown d1 0 1 2024-04-25T15:49:37.006 INFO:tasks.workunit.client.0.smithi110.stdout:6/611: chown d1/dd 407 1 2024-04-25T15:49:37.008 INFO:tasks.workunit.client.0.smithi110.stdout:9/607: dread d9/d13/f27 [0,4194304] 0 2024-04-25T15:49:37.010 INFO:tasks.workunit.client.0.smithi110.stdout:4/641: mkdir d13/d24/d2d/db5/dd7 0 2024-04-25T15:49:37.013 INFO:tasks.workunit.client.0.smithi110.stdout:1/788: creat d0/d51/d6b/ffe x:0 0 0 2024-04-25T15:49:37.016 INFO:tasks.workunit.client.1.smithi193.stdout:9/363: symlink df/d2c/d2f/d4e/d1c/d30/l65 0 2024-04-25T15:49:37.017 INFO:tasks.workunit.client.1.smithi193.stdout:9/364: read df/d2c/d2f/d4e/d1c/d26/d3f/d41/f4b [260885,27952] 0 2024-04-25T15:49:37.018 INFO:tasks.workunit.client.1.smithi193.stdout:9/365: fdatasync df/d2c/d2f/d4e/d1c/f23 0 2024-04-25T15:49:37.019 INFO:tasks.workunit.client.1.smithi193.stdout:9/366: write df/d2c/d2f/d4e/d1c/d26/d3f/d41/f4b [525580,14605] 0 2024-04-25T15:49:37.020 INFO:tasks.workunit.client.1.smithi193.stdout:1/320: rename dc/d4a/d50/d5c/d5d/f3b to dc/d4a/d50/d5c/d63/f67 0 2024-04-25T15:49:37.020 INFO:tasks.workunit.client.0.smithi110.stdout:1/789: dwrite d0/d14/d81/df8/db4/dcd/fe6 [0,4194304] 0 2024-04-25T15:49:37.021 INFO:tasks.workunit.client.1.smithi193.stdout:6/358: mknod dd/d12/d19/d1c/d40/d3c/c67 0 2024-04-25T15:49:37.022 INFO:tasks.workunit.client.1.smithi193.stdout:6/359: write dd/d12/d19/f66 [196104,34451] 0 2024-04-25T15:49:37.027 INFO:tasks.workunit.client.1.smithi193.stdout:2/357: creat d0/d37/f92 x:0 0 0 2024-04-25T15:49:37.027 INFO:tasks.workunit.client.0.smithi110.stdout:2/627: unlink de/c2b 0 2024-04-25T15:49:37.028 INFO:tasks.workunit.client.0.smithi110.stdout:2/628: read de/d40/f33 [2955815,73839] 0 2024-04-25T15:49:37.029 INFO:tasks.workunit.client.1.smithi193.stdout:3/398: unlink d8/d30/d45/f68 0 2024-04-25T15:49:37.030 INFO:tasks.workunit.client.0.smithi110.stdout:0/613: truncate d1/d23/d30/d33/d35/da1/db4/db9/fd3 276917 0 2024-04-25T15:49:37.030 INFO:tasks.workunit.client.0.smithi110.stdout:0/614: chown d1/d23/f44 10617164 1 2024-04-25T15:49:37.032 INFO:tasks.workunit.client.1.smithi193.stdout:5/328: mkdir d17/d18/d75 0 2024-04-25T15:49:37.033 INFO:tasks.workunit.client.1.smithi193.stdout:4/400: mknod db/d13/c7e 0 2024-04-25T15:49:37.036 INFO:tasks.workunit.client.0.smithi110.stdout:1/790: rename d0/l43 to d0/d51/lff 0 2024-04-25T15:49:37.037 INFO:tasks.workunit.client.0.smithi110.stdout:4/642: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/f76 [4194304,4194304] 0 2024-04-25T15:49:37.037 INFO:tasks.workunit.client.1.smithi193.stdout:5/329: dwrite d17/d18/f2c [0,4194304] 0 2024-04-25T15:49:37.038 INFO:tasks.workunit.client.0.smithi110.stdout:1/791: write d0/d14/d5f/d7d/fbd [488245,40086] 0 2024-04-25T15:49:37.038 INFO:tasks.workunit.client.1.smithi193.stdout:5/330: chown d17/d18/f29 18406744 1 2024-04-25T15:49:37.038 INFO:tasks.workunit.client.0.smithi110.stdout:4/643: truncate d13/f22 5008997 0 2024-04-25T15:49:37.040 INFO:tasks.workunit.client.0.smithi110.stdout:2/629: symlink de/d40/d6e/d50/lc7 0 2024-04-25T15:49:37.040 INFO:tasks.workunit.client.0.smithi110.stdout:1/792: stat d0/d14/d5f/d7d/d72/d88/dbf/ffc 0 2024-04-25T15:49:37.042 INFO:tasks.workunit.client.0.smithi110.stdout:1/793: chown d0/d14/d81/df8/f6 7552 1 2024-04-25T15:49:37.049 INFO:tasks.workunit.client.0.smithi110.stdout:1/794: dwrite d0/d23/f92 [4194304,4194304] 0 2024-04-25T15:49:37.050 INFO:tasks.workunit.client.1.smithi193.stdout:6/360: dread dd/f2b [0,4194304] 0 2024-04-25T15:49:37.051 INFO:tasks.workunit.client.1.smithi193.stdout:8/330: creat d2/d3/f68 x:0 0 0 2024-04-25T15:49:37.053 INFO:tasks.workunit.client.0.smithi110.stdout:4/644: mkdir d13/dba/dd8 0 2024-04-25T15:49:37.053 INFO:tasks.workunit.client.1.smithi193.stdout:8/331: truncate d2/d3/f37 796402 0 2024-04-25T15:49:37.053 INFO:tasks.workunit.client.0.smithi110.stdout:4/645: write d13/d24/d2d/d5e/f7b [175017,108122] 0 2024-04-25T15:49:37.053 INFO:tasks.workunit.client.1.smithi193.stdout:8/332: truncate d2/d3/ff 4928961 0 2024-04-25T15:49:37.054 INFO:tasks.workunit.client.1.smithi193.stdout:8/333: chown d2/d3/ff 1 1 2024-04-25T15:49:37.056 INFO:tasks.workunit.client.0.smithi110.stdout:2/630: rename de/d40/d4f/f4e to de/d40/d6e/d52/fc8 0 2024-04-25T15:49:37.058 INFO:tasks.workunit.client.1.smithi193.stdout:2/358: mkdir d0/d42/d51/d4f/d93 0 2024-04-25T15:49:37.058 INFO:tasks.workunit.client.1.smithi193.stdout:0/373: sync 2024-04-25T15:49:37.066 INFO:tasks.workunit.client.0.smithi110.stdout:1/795: write d0/d14/d81/df8/db4/dcd/fce [1253985,4929] 0 2024-04-25T15:49:37.068 INFO:tasks.workunit.client.0.smithi110.stdout:4/646: unlink d13/d24/f58 0 2024-04-25T15:49:37.071 INFO:tasks.workunit.client.0.smithi110.stdout:2/631: fsync de/f24 0 2024-04-25T15:49:37.072 INFO:tasks.workunit.client.1.smithi193.stdout:5/331: chown d17/d18/f21 825267 1 2024-04-25T15:49:37.073 INFO:tasks.workunit.client.0.smithi110.stdout:6/612: getdents d1/db/da0/da5 0 2024-04-25T15:49:37.074 INFO:tasks.workunit.client.1.smithi193.stdout:9/367: symlink df/d2c/d2f/l66 0 2024-04-25T15:49:37.075 INFO:tasks.workunit.client.0.smithi110.stdout:9/608: getdents d9/db/df/d2c/d3c 0 2024-04-25T15:49:37.080 INFO:tasks.workunit.client.1.smithi193.stdout:6/361: creat dd/d12/d19/d1c/d40/d3c/f68 x:0 0 0 2024-04-25T15:49:37.081 INFO:tasks.workunit.client.1.smithi193.stdout:6/362: stat dd/d2f/f56 0 2024-04-25T15:49:37.081 INFO:tasks.workunit.client.1.smithi193.stdout:6/363: write dd/d12/f35 [163273,68441] 0 2024-04-25T15:49:37.082 INFO:tasks.workunit.client.1.smithi193.stdout:8/334: mkdir d2/d3/d27/d69 0 2024-04-25T15:49:37.085 INFO:tasks.workunit.client.1.smithi193.stdout:2/359: creat d0/d42/d51/f94 x:0 0 0 2024-04-25T15:49:37.085 INFO:tasks.workunit.client.0.smithi110.stdout:2/632: rmdir de/d6f 39 2024-04-25T15:49:37.086 INFO:tasks.workunit.client.1.smithi193.stdout:0/374: fsync d0/d49/d5/d7/dd/d16/d26/d2e/f4b 0 2024-04-25T15:49:37.087 INFO:tasks.workunit.client.0.smithi110.stdout:6/613: mknod d1/db/d24/daa/cd7 0 2024-04-25T15:49:37.088 INFO:tasks.workunit.client.0.smithi110.stdout:9/609: creat d9/db/df/d25/fc4 x:0 0 0 2024-04-25T15:49:37.088 INFO:tasks.workunit.client.1.smithi193.stdout:7/328: link d12/d1c/d63/d70/c59 d12/d1c/d63/d70/c7f 0 2024-04-25T15:49:37.091 INFO:tasks.workunit.client.0.smithi110.stdout:1/796: symlink d0/d14/d5f/l100 0 2024-04-25T15:49:37.092 INFO:tasks.workunit.client.1.smithi193.stdout:5/332: write d17/d18/d30/f32 [2976802,48860] 0 2024-04-25T15:49:37.099 INFO:tasks.workunit.client.1.smithi193.stdout:6/364: creat dd/d2f/d57/f69 x:0 0 0 2024-04-25T15:49:37.103 INFO:tasks.workunit.client.0.smithi110.stdout:9/610: dwrite d9/d13/f27 [0,4194304] 0 2024-04-25T15:49:37.103 INFO:tasks.workunit.client.1.smithi193.stdout:8/335: fsync d2/d3/d3a/f4c 0 2024-04-25T15:49:37.105 INFO:tasks.workunit.client.0.smithi110.stdout:4/647: truncate d13/d24/d2d/d44/d65/d99/dbc/fc4 3292551 0 2024-04-25T15:49:37.110 INFO:tasks.workunit.client.0.smithi110.stdout:4/648: dread d13/d24/d2d/d36/f4d [0,4194304] 0 2024-04-25T15:49:37.110 INFO:tasks.workunit.client.0.smithi110.stdout:9/611: mkdir d9/d13/d5b/d61/db3/db4/dc5 0 2024-04-25T15:49:37.112 INFO:tasks.workunit.client.1.smithi193.stdout:0/375: rename d0/d49/d5/d7/dd/d16/d26/d2e/d3d/c43 to d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/c7f 0 2024-04-25T15:49:37.113 INFO:tasks.workunit.client.1.smithi193.stdout:7/329: creat d12/d1c/d63/f80 x:0 0 0 2024-04-25T15:49:37.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/401: creat db/d32/f7f x:0 0 0 2024-04-25T15:49:37.114 INFO:tasks.workunit.client.1.smithi193.stdout:4/402: chown db/f61 1025786738 1 2024-04-25T15:49:37.114 INFO:tasks.workunit.client.1.smithi193.stdout:4/403: chown db/d13/d1e/f48 121 1 2024-04-25T15:49:37.115 INFO:tasks.workunit.client.0.smithi110.stdout:9/612: rename d9/db/d58/f82 to d9/db/df/d2c/d3c/d56/da0/fc6 0 2024-04-25T15:49:37.116 INFO:tasks.workunit.client.1.smithi193.stdout:1/321: getdents dc/d4a/d50/d5c/d5d 0 2024-04-25T15:49:37.117 INFO:tasks.workunit.client.1.smithi193.stdout:8/336: fdatasync d2/d3/d3a/f4c 0 2024-04-25T15:49:37.118 INFO:tasks.workunit.client.1.smithi193.stdout:3/399: getdents d8/d13/d14/d51 0 2024-04-25T15:49:37.119 INFO:tasks.workunit.client.1.smithi193.stdout:0/376: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f59 [0,4194304] 0 2024-04-25T15:49:37.120 INFO:tasks.workunit.client.1.smithi193.stdout:1/322: mknod dc/d4a/d50/d5c/d5d/c68 0 2024-04-25T15:49:37.121 INFO:tasks.workunit.client.1.smithi193.stdout:8/337: rmdir d2/d36 39 2024-04-25T15:49:37.123 INFO:tasks.workunit.client.1.smithi193.stdout:2/360: getdents d0/d44 0 2024-04-25T15:49:37.124 INFO:tasks.workunit.client.0.smithi110.stdout:9/613: chown d9/db/df 16146269 1 2024-04-25T15:49:37.125 INFO:tasks.workunit.client.0.smithi110.stdout:9/614: write d9/d13/f54 [953197,32172] 0 2024-04-25T15:49:37.126 INFO:tasks.workunit.client.1.smithi193.stdout:6/365: rename dd/d12/d19/d1c/f38 to dd/d12/d19/f6a 0 2024-04-25T15:49:37.127 INFO:tasks.workunit.client.0.smithi110.stdout:6/614: getdents d1/dd/d3c/d91/d8d/dce 0 2024-04-25T15:49:37.127 INFO:tasks.workunit.client.1.smithi193.stdout:9/368: getdents df/d2c/d2f/d4e/d1c/d26/d3f/d41 0 2024-04-25T15:49:37.127 INFO:tasks.workunit.client.1.smithi193.stdout:6/366: readlink dd/d2f/l36 0 2024-04-25T15:49:37.128 INFO:tasks.workunit.client.1.smithi193.stdout:1/323: mknod dc/d4a/d50/d5c/c69 0 2024-04-25T15:49:37.132 INFO:tasks.workunit.client.1.smithi193.stdout:8/338: symlink d2/d3/d27/l6a 0 2024-04-25T15:49:37.132 INFO:tasks.workunit.client.1.smithi193.stdout:8/339: truncate d2/d3/f16 5033564 0 2024-04-25T15:49:37.134 INFO:tasks.workunit.client.0.smithi110.stdout:9/615: dwrite d9/d13/d84/f9b [0,4194304] 0 2024-04-25T15:49:37.138 INFO:tasks.workunit.client.1.smithi193.stdout:7/330: rename d12/d13/d78 to d12/d31/d37/d62/d81 0 2024-04-25T15:49:37.140 INFO:tasks.workunit.client.0.smithi110.stdout:1/797: dwrite d0/d23/f3b [0,4194304] 0 2024-04-25T15:49:37.141 INFO:tasks.workunit.client.1.smithi193.stdout:9/369: mknod df/d2d/c67 0 2024-04-25T15:49:37.142 INFO:tasks.workunit.client.1.smithi193.stdout:9/370: fsync df/d2d/f61 0 2024-04-25T15:49:37.142 INFO:tasks.workunit.client.0.smithi110.stdout:9/616: mkdir d9/d2d/dc7 0 2024-04-25T15:49:37.145 INFO:tasks.workunit.client.1.smithi193.stdout:3/400: link d8/d13/d14/f59 d8/d13/f7b 0 2024-04-25T15:49:37.146 INFO:tasks.workunit.client.0.smithi110.stdout:4/649: truncate d13/d24/f48 512552 0 2024-04-25T15:49:37.146 INFO:tasks.workunit.client.1.smithi193.stdout:3/401: write d8/d30/f47 [2119850,124823] 0 2024-04-25T15:49:37.147 INFO:tasks.workunit.client.0.smithi110.stdout:1/798: symlink d0/d51/dc8/l101 0 2024-04-25T15:49:37.148 INFO:tasks.workunit.client.1.smithi193.stdout:3/402: dread - d8/d13/f61 zero size 2024-04-25T15:49:37.149 INFO:tasks.workunit.client.1.smithi193.stdout:1/324: creat dc/d4a/d50/f6a x:0 0 0 2024-04-25T15:49:37.151 INFO:tasks.workunit.client.1.smithi193.stdout:9/371: dwrite df/d2c/d2f/d4e/d1c/d26/f4f [0,4194304] 0 2024-04-25T15:49:37.152 INFO:tasks.workunit.client.1.smithi193.stdout:5/333: sync 2024-04-25T15:49:37.153 INFO:tasks.workunit.client.1.smithi193.stdout:2/361: link d0/d9/f39 d0/d42/d51/d4f/d65/d7e/f95 0 2024-04-25T15:49:37.155 INFO:tasks.workunit.client.0.smithi110.stdout:9/617: dread d9/d13/f8e [0,4194304] 0 2024-04-25T15:49:37.155 INFO:tasks.workunit.client.1.smithi193.stdout:5/334: dread - d17/d18/d30/d38/d71/f74 zero size 2024-04-25T15:49:37.157 INFO:tasks.workunit.client.0.smithi110.stdout:6/615: dread d1/db/d24/d2d/f34 [0,4194304] 0 2024-04-25T15:49:37.157 INFO:tasks.workunit.client.1.smithi193.stdout:4/404: dwrite db/d13/d1e/d1f/f63 [0,4194304] 0 2024-04-25T15:49:37.157 INFO:tasks.workunit.client.1.smithi193.stdout:5/335: dread d17/f59 [0,4194304] 0 2024-04-25T15:49:37.157 INFO:tasks.workunit.client.1.smithi193.stdout:4/405: stat db/cd 0 2024-04-25T15:49:37.160 INFO:tasks.workunit.client.0.smithi110.stdout:1/799: symlink d0/d14/d5f/d7d/l102 0 2024-04-25T15:49:37.164 INFO:tasks.workunit.client.1.smithi193.stdout:0/377: rename d0/d49/d5/d7/dd/d16/d26/f67 to d0/d49/d5/d7/dd/d16/d26/d5e/f80 0 2024-04-25T15:49:37.166 INFO:tasks.workunit.client.1.smithi193.stdout:7/331: unlink d12/d31/d37/c49 0 2024-04-25T15:49:37.167 INFO:tasks.workunit.client.1.smithi193.stdout:7/332: write d12/f52 [649748,18740] 0 2024-04-25T15:49:37.169 INFO:tasks.workunit.client.1.smithi193.stdout:7/333: write d12/d31/d37/d62/d6e/d79/f4a [342230,111505] 0 2024-04-25T15:49:37.172 INFO:tasks.workunit.client.0.smithi110.stdout:6/616: creat d1/db/d22/d33/d63/fd8 x:0 0 0 2024-04-25T15:49:37.172 INFO:tasks.workunit.client.0.smithi110.stdout:6/617: stat d1/d79 0 2024-04-25T15:49:37.180 INFO:tasks.workunit.client.0.smithi110.stdout:4/650: rename d13/d24/d2d/d44/d90/db8 to d13/d24/d2d/dd9 0 2024-04-25T15:49:37.181 INFO:tasks.workunit.client.1.smithi193.stdout:2/362: creat d0/d37/d25/d5f/d3a/d6d/d6e/f96 x:0 0 0 2024-04-25T15:49:37.185 INFO:tasks.workunit.client.1.smithi193.stdout:6/367: rename dd/d2f/c34 to dd/d12/d19/d45/c6b 0 2024-04-25T15:49:37.190 INFO:tasks.workunit.client.1.smithi193.stdout:3/403: mknod d8/d30/c7c 0 2024-04-25T15:49:37.190 INFO:tasks.workunit.client.0.smithi110.stdout:4/651: truncate d13/d24/d2d/d36/d51/f54 2968756 0 2024-04-25T15:49:37.200 INFO:tasks.workunit.client.0.smithi110.stdout:9/618: getdents d9/d13 0 2024-04-25T15:49:37.201 INFO:tasks.workunit.client.1.smithi193.stdout:8/340: rename d2/c1d to d2/d3/d20/d45/d4a/d53/c6b 0 2024-04-25T15:49:37.202 INFO:tasks.workunit.client.0.smithi110.stdout:4/652: creat d13/d24/d2d/d7d/fda x:0 0 0 2024-04-25T15:49:37.203 INFO:tasks.workunit.client.1.smithi193.stdout:8/341: chown d2/d3/d20/d45/d4a/d53/f58 41473 1 2024-04-25T15:49:37.204 INFO:tasks.workunit.client.0.smithi110.stdout:6/618: link d1/dd/d3c/d91/d8d/dce/l46 d1/d12/db8/ld9 0 2024-04-25T15:49:37.206 INFO:tasks.workunit.client.0.smithi110.stdout:1/800: getdents d0/d14/d5f/d7d/d72 0 2024-04-25T15:49:37.207 INFO:tasks.workunit.client.1.smithi193.stdout:3/404: symlink d8/d13/d14/d51/l7d 0 2024-04-25T15:49:37.207 INFO:tasks.workunit.client.0.smithi110.stdout:9/619: rmdir d9/d44/d5e/d80 39 2024-04-25T15:49:37.207 INFO:tasks.workunit.client.1.smithi193.stdout:3/405: dread - d8/d13/f66 zero size 2024-04-25T15:49:37.211 INFO:tasks.workunit.client.1.smithi193.stdout:7/334: rename d12/c76 to d12/d13/c82 0 2024-04-25T15:49:37.214 INFO:tasks.workunit.client.1.smithi193.stdout:3/406: dwrite d8/f11 [4194304,4194304] 0 2024-04-25T15:49:37.214 INFO:tasks.workunit.client.1.smithi193.stdout:9/372: dwrite df/d2c/d2f/d4e/d1c/d26/d3f/d41/f4b [0,4194304] 0 2024-04-25T15:49:37.217 INFO:tasks.workunit.client.1.smithi193.stdout:9/373: read df/d2c/d2f/d4e/d1c/d26/d3f/d41/f59 [726633,4709] 0 2024-04-25T15:49:37.225 INFO:tasks.workunit.client.0.smithi110.stdout:1/801: dread d0/f27 [0,4194304] 0 2024-04-25T15:49:37.226 INFO:tasks.workunit.client.0.smithi110.stdout:1/802: chown d0/d23/f8a 1973243 1 2024-04-25T15:49:37.257 INFO:tasks.workunit.client.1.smithi193.stdout:5/336: sync 2024-04-25T15:49:37.258 INFO:tasks.workunit.client.1.smithi193.stdout:5/337: chown d17/d18/l1a 41294 1 2024-04-25T15:49:37.264 INFO:tasks.workunit.client.1.smithi193.stdout:5/338: dwrite d17/d18/d30/d38/d71/f74 [0,4194304] 0 2024-04-25T15:49:37.265 INFO:tasks.workunit.client.1.smithi193.stdout:5/339: dread - d17/d18/d30/d34/d37/d5d/f68 zero size 2024-04-25T15:49:37.283 INFO:tasks.workunit.client.0.smithi110.stdout:6/619: symlink d1/db/d22/d90/d76/lda 0 2024-04-25T15:49:37.285 INFO:tasks.workunit.client.1.smithi193.stdout:4/406: sync 2024-04-25T15:49:37.285 INFO:tasks.workunit.client.1.smithi193.stdout:2/363: sync 2024-04-25T15:49:37.286 INFO:tasks.workunit.client.0.smithi110.stdout:9/620: symlink d9/db/d58/lc8 0 2024-04-25T15:49:37.286 INFO:tasks.workunit.client.1.smithi193.stdout:4/407: dread - db/f6a zero size 2024-04-25T15:49:37.288 INFO:tasks.workunit.client.1.smithi193.stdout:1/325: link dc/c5f dc/c6b 0 2024-04-25T15:49:37.289 INFO:tasks.workunit.client.0.smithi110.stdout:1/803: mknod d0/d14/d5f/d7d/d9c/c103 0 2024-04-25T15:49:37.290 INFO:tasks.workunit.client.0.smithi110.stdout:1/804: chown d0/d14/d81/df8/d33/d8e/ffa 50 1 2024-04-25T15:49:37.291 INFO:tasks.workunit.client.1.smithi193.stdout:7/335: mknod d12/d31/d37/d62/d6e/d79/c83 0 2024-04-25T15:49:37.292 INFO:tasks.workunit.client.1.smithi193.stdout:7/336: stat d12/d1c/l57 0 2024-04-25T15:49:37.293 INFO:tasks.workunit.client.1.smithi193.stdout:7/337: write d12/d13/f43 [749707,31785] 0 2024-04-25T15:49:37.298 INFO:tasks.workunit.client.1.smithi193.stdout:9/374: mknod df/d2c/d2f/d4e/c68 0 2024-04-25T15:49:37.298 INFO:tasks.workunit.client.1.smithi193.stdout:2/364: dread d0/d9/d15/f79 [0,4194304] 0 2024-04-25T15:49:37.301 INFO:tasks.workunit.client.1.smithi193.stdout:6/368: write dd/f2e [3392851,17056] 0 2024-04-25T15:49:37.303 INFO:tasks.workunit.client.1.smithi193.stdout:5/340: rename d17/d18/f29 to d17/d18/d30/f76 0 2024-04-25T15:49:37.304 INFO:tasks.workunit.client.1.smithi193.stdout:5/341: read d17/d18/f2e [944008,46067] 0 2024-04-25T15:49:37.305 INFO:tasks.workunit.client.1.smithi193.stdout:5/342: dread - d17/d18/d30/d34/f46 zero size 2024-04-25T15:49:37.307 INFO:tasks.workunit.client.1.smithi193.stdout:0/378: truncate d0/f35 2288274 0 2024-04-25T15:49:37.307 INFO:tasks.workunit.client.1.smithi193.stdout:0/379: fsync d0/d49/d5/d7/dd/d46/d48/f5b 0 2024-04-25T15:49:37.318 INFO:tasks.workunit.client.1.smithi193.stdout:9/375: mkdir df/d2c/d2f/d4e/d69 0 2024-04-25T15:49:37.320 INFO:tasks.workunit.client.1.smithi193.stdout:2/365: creat d0/d42/d51/d4f/f97 x:0 0 0 2024-04-25T15:49:37.320 INFO:tasks.workunit.client.1.smithi193.stdout:3/407: write d8/f15 [1087734,123757] 0 2024-04-25T15:49:37.323 INFO:tasks.workunit.client.1.smithi193.stdout:6/369: mknod dd/d12/d19/d1c/d40/d3c/c6c 0 2024-04-25T15:49:37.324 INFO:tasks.workunit.client.1.smithi193.stdout:6/370: write dd/d12/f1e [1014829,93804] 0 2024-04-25T15:49:37.325 INFO:tasks.workunit.client.1.smithi193.stdout:9/376: dwrite df/d2c/d2f/d4e/d1c/d26/d3f/f40 [0,4194304] 0 2024-04-25T15:49:37.326 INFO:tasks.workunit.client.1.smithi193.stdout:8/342: unlink d2/d36/l52 0 2024-04-25T15:49:37.328 INFO:tasks.workunit.client.1.smithi193.stdout:4/408: symlink db/d4b/d51/d54/d77/l80 0 2024-04-25T15:49:37.328 INFO:tasks.workunit.client.1.smithi193.stdout:0/380: dread - d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f3f zero size 2024-04-25T15:49:37.332 INFO:tasks.workunit.client.1.smithi193.stdout:8/343: dread d2/d3/d20/d45/d4a/d53/f58 [0,4194304] 0 2024-04-25T15:49:37.334 INFO:tasks.workunit.client.1.smithi193.stdout:2/366: mkdir d0/d37/d25/d5f/d98 0 2024-04-25T15:49:37.336 INFO:tasks.workunit.client.1.smithi193.stdout:3/408: creat d8/d30/d72/f7e x:0 0 0 2024-04-25T15:49:37.342 INFO:tasks.workunit.client.1.smithi193.stdout:3/409: dwrite d8/d13/f61 [0,4194304] 0 2024-04-25T15:49:37.342 INFO:tasks.workunit.client.1.smithi193.stdout:6/371: chown dd/d12/d19/l32 92 1 2024-04-25T15:49:37.350 INFO:tasks.workunit.client.1.smithi193.stdout:6/372: dwrite dd/d12/d19/d1c/d40/d3c/d3f/f61 [0,4194304] 0 2024-04-25T15:49:37.351 INFO:tasks.workunit.client.1.smithi193.stdout:4/409: rename db/d13/d1e/l2e to db/d4b/d51/d54/d77/l81 0 2024-04-25T15:49:37.353 INFO:tasks.workunit.client.1.smithi193.stdout:0/381: symlink d0/d49/d5/d7/dd/d16/d26/d2e/l81 0 2024-04-25T15:49:37.356 INFO:tasks.workunit.client.1.smithi193.stdout:2/367: fsync d0/df/f10 0 2024-04-25T15:49:37.361 INFO:tasks.workunit.client.1.smithi193.stdout:3/410: symlink d8/d13/d1f/d69/d77/l7f 0 2024-04-25T15:49:37.363 INFO:tasks.workunit.client.0.smithi110.stdout:7/661: sync 2024-04-25T15:49:37.363 INFO:tasks.workunit.client.0.smithi110.stdout:8/766: sync 2024-04-25T15:49:37.363 INFO:tasks.workunit.client.1.smithi193.stdout:7/338: truncate d12/d31/d37/d62/d6e/d79/f4a 30704 0 2024-04-25T15:49:37.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/411: read d8/d30/f36 [4188650,3772] 0 2024-04-25T15:49:37.365 INFO:tasks.workunit.client.1.smithi193.stdout:5/343: link d17/d18/d30/d34/d52/l6f d17/d18/d3e/l77 0 2024-04-25T15:49:37.366 INFO:tasks.workunit.client.1.smithi193.stdout:1/326: dwrite f6 [0,4194304] 0 2024-04-25T15:49:37.375 INFO:tasks.workunit.client.1.smithi193.stdout:8/344: rename d2/d3/c13 to d2/d1f/c6c 0 2024-04-25T15:49:37.377 INFO:tasks.workunit.client.1.smithi193.stdout:9/377: write df/d2c/d2f/d4e/f1a [6132617,91073] 0 2024-04-25T15:49:37.377 INFO:tasks.workunit.client.0.smithi110.stdout:6/620: dread - d1/db/d24/d2d/f96 zero size 2024-04-25T15:49:37.378 INFO:tasks.workunit.client.0.smithi110.stdout:6/621: chown d1/dd/d3c/d91/c65 1 1 2024-04-25T15:49:37.381 INFO:tasks.workunit.client.1.smithi193.stdout:2/368: creat d0/d37/d25/d5f/d3a/d6d/d85/f99 x:0 0 0 2024-04-25T15:49:37.381 INFO:tasks.workunit.client.0.smithi110.stdout:4/653: link d13/d24/d2d/d7d/f81 d13/d24/d2d/d44/d65/d99/dbc/fdb 0 2024-04-25T15:49:37.385 INFO:tasks.workunit.client.1.smithi193.stdout:7/339: mknod d12/d1c/d63/c84 0 2024-04-25T15:49:37.388 INFO:tasks.workunit.client.0.smithi110.stdout:8/767: unlink d14/d21/f67 0 2024-04-25T15:49:37.389 INFO:tasks.workunit.client.0.smithi110.stdout:8/768: write d14/d21/d26/d6a/d7e/da6/de8/d55/fb4 [5171772,68445] 0 2024-04-25T15:49:37.397 INFO:tasks.workunit.client.0.smithi110.stdout:4/654: creat d13/d24/d26/dcd/fdc x:0 0 0 2024-04-25T15:49:37.402 INFO:tasks.workunit.client.0.smithi110.stdout:7/662: rename c5 to d1b/d29/d38/d55/d7f/dc8/ce0 0 2024-04-25T15:49:37.405 INFO:tasks.workunit.client.0.smithi110.stdout:8/769: mkdir d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101 0 2024-04-25T15:49:37.405 INFO:tasks.workunit.client.1.smithi193.stdout:0/382: write d0/f17 [140557,112699] 0 2024-04-25T15:49:37.406 INFO:tasks.workunit.client.1.smithi193.stdout:7/340: rename c11 to d12/d31/d37/d62/d6e/d79/c85 0 2024-04-25T15:49:37.407 INFO:tasks.workunit.client.1.smithi193.stdout:0/383: truncate d0/d49/d5/d7/dd/d46/d48/f5b 461538 0 2024-04-25T15:49:37.408 INFO:tasks.workunit.client.0.smithi110.stdout:3/687: sync 2024-04-25T15:49:37.409 INFO:tasks.workunit.client.0.smithi110.stdout:9/621: creat d9/d44/d5e/fc9 x:0 0 0 2024-04-25T15:49:37.409 INFO:tasks.workunit.client.1.smithi193.stdout:0/384: truncate d0/d49/d5/d7/dd/d16/d26/d2e/f76 534833 0 2024-04-25T15:49:37.410 INFO:tasks.workunit.client.1.smithi193.stdout:0/385: stat d0/d49/d5/d7/dd/f40 0 2024-04-25T15:49:37.410 INFO:tasks.workunit.client.0.smithi110.stdout:4/655: mknod d13/cdd 0 2024-04-25T15:49:37.411 INFO:tasks.workunit.client.0.smithi110.stdout:9/622: truncate d9/db/df/d55/f60 4899168 0 2024-04-25T15:49:37.412 INFO:tasks.workunit.client.1.smithi193.stdout:1/327: mknod dc/c6c 0 2024-04-25T15:49:37.413 INFO:tasks.workunit.client.0.smithi110.stdout:9/623: chown d9/d2d/cb8 551 1 2024-04-25T15:49:37.414 INFO:tasks.workunit.client.0.smithi110.stdout:1/805: dwrite d0/d23/f5c [4194304,4194304] 0 2024-04-25T15:49:37.414 INFO:tasks.workunit.client.0.smithi110.stdout:9/624: dread - d9/d13/d5b/d61/f9d zero size 2024-04-25T15:49:37.416 INFO:tasks.workunit.client.1.smithi193.stdout:4/410: creat db/d13/f82 x:0 0 0 2024-04-25T15:49:37.418 INFO:tasks.workunit.client.1.smithi193.stdout:1/328: dwrite dc/f5b [0,4194304] 0 2024-04-25T15:49:37.418 INFO:tasks.workunit.client.1.smithi193.stdout:9/378: write df/d2c/d2f/d4e/d1c/d26/d3f/d41/f59 [1264713,113761] 0 2024-04-25T15:49:37.418 INFO:tasks.workunit.client.0.smithi110.stdout:6/622: link d1/db/d22/d33/d43/f4e d1/db/d24/daa/fdb 0 2024-04-25T15:49:37.422 INFO:tasks.workunit.client.1.smithi193.stdout:2/369: link d0/d37/d25/d5f/d3a/d6d/d6e/f96 d0/d9/d15/d31/d50/f9a 0 2024-04-25T15:49:37.423 INFO:tasks.workunit.client.0.smithi110.stdout:1/806: dwrite d0/d14/d5f/d7d/d72/d88/dbf/ffc [0,4194304] 0 2024-04-25T15:49:37.423 INFO:tasks.workunit.client.0.smithi110.stdout:4/656: creat d13/d24/d2d/d36/fde x:0 0 0 2024-04-25T15:49:37.427 INFO:tasks.workunit.client.1.smithi193.stdout:7/341: mknod d12/d31/d37/d62/d6e/d79/c86 0 2024-04-25T15:49:37.430 INFO:tasks.workunit.client.0.smithi110.stdout:6/623: dread d1/dd/d3c/d91/d8d/dce/f9d [0,4194304] 0 2024-04-25T15:49:37.430 INFO:tasks.workunit.client.1.smithi193.stdout:2/370: dwrite d0/d9/d15/d31/d50/f6b [0,4194304] 0 2024-04-25T15:49:37.432 INFO:tasks.workunit.client.0.smithi110.stdout:9/625: creat d9/d13/fca x:0 0 0 2024-04-25T15:49:37.432 INFO:tasks.workunit.client.0.smithi110.stdout:6/624: truncate d1/db/d24/d69/fb3 543244 0 2024-04-25T15:49:37.435 INFO:tasks.workunit.client.1.smithi193.stdout:3/412: creat d8/d13/d14/f80 x:0 0 0 2024-04-25T15:49:37.436 INFO:tasks.workunit.client.0.smithi110.stdout:7/663: symlink d1b/d29/le1 0 2024-04-25T15:49:37.437 INFO:tasks.workunit.client.0.smithi110.stdout:3/688: dread d0/fc9 [0,4194304] 0 2024-04-25T15:49:37.438 INFO:tasks.workunit.client.0.smithi110.stdout:7/664: readlink d1b/d29/d87/l9f 0 2024-04-25T15:49:37.440 INFO:tasks.workunit.client.0.smithi110.stdout:8/770: mkdir d14/d102 0 2024-04-25T15:49:37.443 INFO:tasks.workunit.client.1.smithi193.stdout:6/373: link dd/d2f/l36 dd/l6d 0 2024-04-25T15:49:37.446 INFO:tasks.workunit.client.1.smithi193.stdout:9/379: dread df/d2c/d2f/d4e/d1c/f23 [0,4194304] 0 2024-04-25T15:49:37.447 INFO:tasks.workunit.client.0.smithi110.stdout:1/807: symlink d0/d23/l104 0 2024-04-25T15:49:37.448 INFO:tasks.workunit.client.1.smithi193.stdout:4/411: symlink db/d32/d45/l83 0 2024-04-25T15:49:37.449 INFO:tasks.workunit.client.0.smithi110.stdout:4/657: rename d13/d24/d2d/d55/c5d to d13/d24/d2d/dd9/dbb/cdf 0 2024-04-25T15:49:37.451 INFO:tasks.workunit.client.1.smithi193.stdout:4/412: dread db/d13/d1e/f48 [0,4194304] 0 2024-04-25T15:49:37.452 INFO:tasks.workunit.client.0.smithi110.stdout:6/625: unlink d1/dd/d3c/d91/d8d/dce/f4a 0 2024-04-25T15:49:37.453 INFO:tasks.workunit.client.0.smithi110.stdout:0/615: sync 2024-04-25T15:49:37.453 INFO:tasks.workunit.client.0.smithi110.stdout:5/670: sync 2024-04-25T15:49:37.453 INFO:tasks.workunit.client.0.smithi110.stdout:2/633: sync 2024-04-25T15:49:37.454 INFO:tasks.workunit.client.1.smithi193.stdout:4/413: dread db/d13/d1e/d1f/f63 [0,4194304] 0 2024-04-25T15:49:37.454 INFO:tasks.workunit.client.0.smithi110.stdout:2/634: dread - de/d40/d6e/d50/db7/faa zero size 2024-04-25T15:49:37.458 INFO:tasks.workunit.client.1.smithi193.stdout:2/371: symlink d0/df/l9b 0 2024-04-25T15:49:37.458 INFO:tasks.workunit.client.0.smithi110.stdout:8/771: mknod d14/d32/d36/c103 0 2024-04-25T15:49:37.464 INFO:tasks.workunit.client.0.smithi110.stdout:1/808: mknod d0/d14/d5f/d7d/d72/d88/dbf/c105 0 2024-04-25T15:49:37.464 INFO:tasks.workunit.client.1.smithi193.stdout:2/372: dread d0/d9/d15/f1b [4194304,4194304] 0 2024-04-25T15:49:37.465 INFO:tasks.workunit.client.0.smithi110.stdout:1/809: write d0/d23/f3b [2082321,9004] 0 2024-04-25T15:49:37.466 INFO:tasks.workunit.client.1.smithi193.stdout:9/380: creat df/d2c/d34/d3e/f6a x:0 0 0 2024-04-25T15:49:37.471 INFO:tasks.workunit.client.1.smithi193.stdout:8/345: getdents d2 0 2024-04-25T15:49:37.471 INFO:tasks.workunit.client.1.smithi193.stdout:8/346: readlink d2/d1f/l30 0 2024-04-25T15:49:37.472 INFO:tasks.workunit.client.0.smithi110.stdout:6/626: mknod d1/d79/cdc 0 2024-04-25T15:49:37.473 INFO:tasks.workunit.client.1.smithi193.stdout:1/329: rename dc/c20 to dc/d3c/c6d 0 2024-04-25T15:49:37.474 INFO:tasks.workunit.client.0.smithi110.stdout:9/626: dwrite d9/db/f1b [4194304,4194304] 0 2024-04-25T15:49:37.477 INFO:tasks.workunit.client.0.smithi110.stdout:5/671: rmdir d4/d1d/d1f/d35/d7f 39 2024-04-25T15:49:37.479 INFO:tasks.workunit.client.0.smithi110.stdout:7/665: dwrite f2 [0,4194304] 0 2024-04-25T15:49:37.481 INFO:tasks.workunit.client.0.smithi110.stdout:0/616: mknod d1/d23/d30/d33/d35/ce0 0 2024-04-25T15:49:37.481 INFO:tasks.workunit.client.1.smithi193.stdout:0/386: truncate d0/d49/d5/d7/dd/d16/d18/f2a 1464679 0 2024-04-25T15:49:37.483 INFO:tasks.workunit.client.0.smithi110.stdout:2/635: symlink de/d40/d5b/d5c/d5f/db2/dad/lc9 0 2024-04-25T15:49:37.484 INFO:tasks.workunit.client.1.smithi193.stdout:5/344: link d17/c42 d17/d18/c78 0 2024-04-25T15:49:37.489 INFO:tasks.workunit.client.0.smithi110.stdout:8/772: creat d14/d32/d36/d48/f104 x:0 0 0 2024-04-25T15:49:37.491 INFO:tasks.workunit.client.0.smithi110.stdout:0/617: dwrite d1/d23/d30/d33/d35/f3a [4194304,4194304] 0 2024-04-25T15:49:37.493 INFO:tasks.workunit.client.0.smithi110.stdout:4/658: dwrite d13/d24/d2d/d36/d51/fb6 [0,4194304] 0 2024-04-25T15:49:37.493 INFO:tasks.workunit.client.0.smithi110.stdout:6/627: readlink d1/dd/d3c/d91/d8d/dce/l46 0 2024-04-25T15:49:37.498 INFO:tasks.workunit.client.0.smithi110.stdout:1/810: dread d0/d51/d6b/f7c [0,4194304] 0 2024-04-25T15:49:37.500 INFO:tasks.workunit.client.1.smithi193.stdout:0/387: mknod d0/d49/d5/d7/dd/d16/d26/d5e/c82 0 2024-04-25T15:49:37.500 INFO:tasks.workunit.client.0.smithi110.stdout:4/659: dread d13/d24/d26/dcd/fd4 [0,4194304] 0 2024-04-25T15:49:37.502 INFO:tasks.workunit.client.1.smithi193.stdout:5/345: write d17/d18/f24 [796039,1309] 0 2024-04-25T15:49:37.504 INFO:tasks.workunit.client.1.smithi193.stdout:2/373: mkdir d0/d9/d15/d31/d90/d9c 0 2024-04-25T15:49:37.504 INFO:tasks.workunit.client.0.smithi110.stdout:3/689: link d0/d16/dcb/f5f d0/d16/d8c/fea 0 2024-04-25T15:49:37.508 INFO:tasks.workunit.client.1.smithi193.stdout:6/374: rename dd/l4c to dd/d12/d19/d1c/d40/d3c/d3f/d4f/l6e 0 2024-04-25T15:49:37.508 INFO:tasks.workunit.client.0.smithi110.stdout:5/672: dwrite d4/d1d/d1f/d35/d3d/fc5 [0,4194304] 0 2024-04-25T15:49:37.510 INFO:tasks.workunit.client.0.smithi110.stdout:8/773: mkdir d14/d32/d105 0 2024-04-25T15:49:37.511 INFO:tasks.workunit.client.1.smithi193.stdout:7/342: getdents d12/d1c/d45 0 2024-04-25T15:49:37.512 INFO:tasks.workunit.client.1.smithi193.stdout:0/388: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/l83 0 2024-04-25T15:49:37.512 INFO:tasks.workunit.client.0.smithi110.stdout:0/618: mkdir d1/d23/d30/d33/d5b/d83/de1 0 2024-04-25T15:49:37.513 INFO:tasks.workunit.client.0.smithi110.stdout:0/619: readlink d1/d20/d69/l9d 0 2024-04-25T15:49:37.514 INFO:tasks.workunit.client.1.smithi193.stdout:5/346: mknod d17/d18/d30/c79 0 2024-04-25T15:49:37.515 INFO:tasks.workunit.client.1.smithi193.stdout:0/389: write d0/d49/d5/d7/dd/d16/d26/f55 [78123,42055] 0 2024-04-25T15:49:37.516 INFO:tasks.workunit.client.1.smithi193.stdout:0/390: dread - d0/d49/f2c zero size 2024-04-25T15:49:37.517 INFO:tasks.workunit.client.0.smithi110.stdout:1/811: creat d0/d14/d81/df8/db4/f106 x:0 0 0 2024-04-25T15:49:37.517 INFO:tasks.workunit.client.1.smithi193.stdout:7/343: dread d12/d31/d37/d62/d6e/f72 [0,4194304] 0 2024-04-25T15:49:37.518 INFO:tasks.workunit.client.1.smithi193.stdout:0/391: write d0/d49/d5/d7/dd/f6e [39392,57350] 0 2024-04-25T15:49:37.518 INFO:tasks.workunit.client.0.smithi110.stdout:1/812: write d0/d14/d5f/d7d/d72/f5e [3626396,129564] 0 2024-04-25T15:49:37.519 INFO:tasks.workunit.client.1.smithi193.stdout:8/347: rename d2/d1f to d2/d3/d3a/d6d 0 2024-04-25T15:49:37.526 INFO:tasks.workunit.client.1.smithi193.stdout:4/414: sync 2024-04-25T15:49:37.526 INFO:tasks.workunit.client.1.smithi193.stdout:3/413: sync 2024-04-25T15:49:37.530 INFO:tasks.workunit.client.0.smithi110.stdout:3/690: unlink d0/d16/f3d 0 2024-04-25T15:49:37.531 INFO:tasks.workunit.client.1.smithi193.stdout:3/414: dwrite f2 [0,4194304] 0 2024-04-25T15:49:37.532 INFO:tasks.workunit.client.1.smithi193.stdout:3/415: write d8/d30/f36 [4271908,85409] 0 2024-04-25T15:49:37.532 INFO:tasks.workunit.client.1.smithi193.stdout:2/374: mknod d0/d9/d15/d31/c9d 0 2024-04-25T15:49:37.533 INFO:tasks.workunit.client.0.smithi110.stdout:5/673: creat d4/d1d/d1f/d35/d3d/d52/dd2/fdd x:0 0 0 2024-04-25T15:49:37.533 INFO:tasks.workunit.client.0.smithi110.stdout:5/674: chown d4/d1d/d1f/l33 25251001 1 2024-04-25T15:49:37.533 INFO:tasks.workunit.client.1.smithi193.stdout:9/381: getdents df/d2c/d2f/d4e/d1c/d30/d3b 0 2024-04-25T15:49:37.535 INFO:tasks.workunit.client.1.smithi193.stdout:7/344: dread f2 [0,4194304] 0 2024-04-25T15:49:37.536 INFO:tasks.workunit.client.1.smithi193.stdout:5/347: mknod d17/d18/d30/d34/d37/c7a 0 2024-04-25T15:49:37.537 INFO:tasks.workunit.client.1.smithi193.stdout:7/345: write d12/d31/d39/d60/f64 [942685,27460] 0 2024-04-25T15:49:37.538 INFO:tasks.workunit.client.0.smithi110.stdout:9/627: rename d9/d2d/f38 to d9/db/df/fcb 0 2024-04-25T15:49:37.540 INFO:tasks.workunit.client.0.smithi110.stdout:2/636: write de/d40/d5b/d5c/d5f/f81 [472476,39357] 0 2024-04-25T15:49:37.541 INFO:tasks.workunit.client.0.smithi110.stdout:2/637: chown de/d40/d5b/d5c/d5f/db2/da9/cc5 20 1 2024-04-25T15:49:37.542 INFO:tasks.workunit.client.1.smithi193.stdout:5/348: dread d17/d18/d45/f56 [0,4194304] 0 2024-04-25T15:49:37.544 INFO:tasks.workunit.client.1.smithi193.stdout:0/392: chown d0/d49/d5/d7/dd/d46/d48/l5d 1278145974 1 2024-04-25T15:49:37.547 INFO:tasks.workunit.client.1.smithi193.stdout:5/349: dwrite d17/d18/d30/d34/f6c [0,4194304] 0 2024-04-25T15:49:37.552 INFO:tasks.workunit.client.1.smithi193.stdout:1/330: getdents dc 0 2024-04-25T15:49:37.553 INFO:tasks.workunit.client.1.smithi193.stdout:5/350: dread d17/d18/f20 [4194304,4194304] 0 2024-04-25T15:49:37.553 INFO:tasks.workunit.client.0.smithi110.stdout:6/628: dwrite d1/db/d22/d33/d43/f6a [0,4194304] 0 2024-04-25T15:49:37.554 INFO:tasks.workunit.client.1.smithi193.stdout:6/375: write dd/d2f/f50 [354016,75634] 0 2024-04-25T15:49:37.554 INFO:tasks.workunit.client.1.smithi193.stdout:6/376: chown dd/d12/f35 92178 1 2024-04-25T15:49:37.559 INFO:tasks.workunit.client.0.smithi110.stdout:7/666: dwrite d1b/d29/f57 [0,4194304] 0 2024-04-25T15:49:37.559 INFO:tasks.workunit.client.1.smithi193.stdout:3/416: unlink d8/f39 0 2024-04-25T15:49:37.561 INFO:tasks.workunit.client.1.smithi193.stdout:5/351: dwrite d17/d18/d30/d34/d37/d5d/f68 [0,4194304] 0 2024-04-25T15:49:37.562 INFO:tasks.workunit.client.1.smithi193.stdout:5/352: write f16 [5140756,78528] 0 2024-04-25T15:49:37.563 INFO:tasks.workunit.client.0.smithi110.stdout:4/660: rename d13/d24/d2d/d44/d65/d99/dbc/dc8 to d13/d24/d2d/d36/de0 0 2024-04-25T15:49:37.569 INFO:tasks.workunit.client.0.smithi110.stdout:2/638: unlink de/d40/d5b/d5c/d5f/db2/f94 0 2024-04-25T15:49:37.570 INFO:tasks.workunit.client.0.smithi110.stdout:2/639: stat de/d40/d70/lb9 0 2024-04-25T15:49:37.574 INFO:tasks.workunit.client.0.smithi110.stdout:0/620: dwrite d1/f32 [4194304,4194304] 0 2024-04-25T15:49:37.577 INFO:tasks.workunit.client.0.smithi110.stdout:3/691: link d0/d7/f6c d0/d16/dcb/db0/feb 0 2024-04-25T15:49:37.577 INFO:tasks.workunit.client.0.smithi110.stdout:1/813: write d0/d51/f99 [558493,14798] 0 2024-04-25T15:49:37.578 INFO:tasks.workunit.client.0.smithi110.stdout:3/692: chown d0/f15 5001513 1 2024-04-25T15:49:37.580 INFO:tasks.workunit.client.1.smithi193.stdout:0/393: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/c84 0 2024-04-25T15:49:37.581 INFO:tasks.workunit.client.1.smithi193.stdout:0/394: chown d0/d49/d5/d7/dd/d16 14625 1 2024-04-25T15:49:37.581 INFO:tasks.workunit.client.0.smithi110.stdout:6/629: mknod d1/db/d24/cdd 0 2024-04-25T15:49:37.583 INFO:tasks.workunit.client.1.smithi193.stdout:8/348: fsync d2/d36/f40 0 2024-04-25T15:49:37.583 INFO:tasks.workunit.client.1.smithi193.stdout:0/395: chown d0/d49/d5/d7/dd/d16/d18/d28/d56/f70 182 1 2024-04-25T15:49:37.584 INFO:tasks.workunit.client.1.smithi193.stdout:4/415: mkdir db/d84 0 2024-04-25T15:49:37.586 INFO:tasks.workunit.client.1.smithi193.stdout:1/331: symlink dc/d4a/d50/l6e 0 2024-04-25T15:49:37.589 INFO:tasks.workunit.client.1.smithi193.stdout:3/417: dread d8/d13/d4b/f5d [0,4194304] 0 2024-04-25T15:49:37.591 INFO:tasks.workunit.client.0.smithi110.stdout:4/661: mkdir d13/d24/d2d/dd9/de1 0 2024-04-25T15:49:37.592 INFO:tasks.workunit.client.1.smithi193.stdout:5/353: dread d17/d18/f21 [4194304,4194304] 0 2024-04-25T15:49:37.593 INFO:tasks.workunit.client.0.smithi110.stdout:9/628: symlink d9/db/df/d2c/lcc 0 2024-04-25T15:49:37.593 INFO:tasks.workunit.client.1.smithi193.stdout:4/416: dwrite db/d13/d1e/d1f/d22/f6b [0,4194304] 0 2024-04-25T15:49:37.594 INFO:tasks.workunit.client.1.smithi193.stdout:4/417: fdatasync db/d13/d1e/d1f/d22/f5d 0 2024-04-25T15:49:37.595 INFO:tasks.workunit.client.0.smithi110.stdout:0/621: write d1/f16 [2467848,1835] 0 2024-04-25T15:49:37.597 INFO:tasks.workunit.client.0.smithi110.stdout:3/693: fdatasync d0/fc9 0 2024-04-25T15:49:37.598 INFO:tasks.workunit.client.0.smithi110.stdout:0/622: fsync d1/d23/d30/d33/d35/f12 0 2024-04-25T15:49:37.598 INFO:tasks.workunit.client.0.smithi110.stdout:1/814: mknod d0/d14/d81/df8/d33/d8e/c107 0 2024-04-25T15:49:37.599 INFO:tasks.workunit.client.0.smithi110.stdout:4/662: dwrite d13/d24/f2a [0,4194304] 0 2024-04-25T15:49:37.599 INFO:tasks.workunit.client.1.smithi193.stdout:0/396: dread d0/fc [0,4194304] 0 2024-04-25T15:49:37.600 INFO:tasks.workunit.client.0.smithi110.stdout:0/623: chown d1/d23/d30/d33/d5b/d83/f68 271244 1 2024-04-25T15:49:37.600 INFO:tasks.workunit.client.0.smithi110.stdout:1/815: chown d0/d14/f53 30891 1 2024-04-25T15:49:37.607 INFO:tasks.workunit.client.1.smithi193.stdout:0/397: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/f34 [4194304,4194304] 0 2024-04-25T15:49:37.611 INFO:tasks.workunit.client.1.smithi193.stdout:9/382: sync 2024-04-25T15:49:37.653 INFO:tasks.workunit.client.0.smithi110.stdout:0/624: chown d1/d23/d30/d33/d35/da1/fcb 28585838 1 2024-04-25T15:49:37.653 INFO:tasks.workunit.client.0.smithi110.stdout:1/816: write d0/d14/d81/df8/d33/d8e/ffa [792083,86697] 0 2024-04-25T15:49:37.654 INFO:tasks.workunit.client.1.smithi193.stdout:2/375: creat d0/d44/d8b/f9e x:0 0 0 2024-04-25T15:49:37.656 INFO:tasks.workunit.client.0.smithi110.stdout:7/667: mkdir d1b/d29/d38/d55/dab/db2/dcd/de2 0 2024-04-25T15:49:37.657 INFO:tasks.workunit.client.1.smithi193.stdout:5/354: dread f11 [0,4194304] 0 2024-04-25T15:49:37.658 INFO:tasks.workunit.client.1.smithi193.stdout:5/355: truncate d17/d18/d30/f32 4844981 0 2024-04-25T15:49:37.704 INFO:tasks.workunit.client.0.smithi110.stdout:8/774: rename d14/l34 to d14/d21/d26/d6a/d7e/l106 0 2024-04-25T15:49:37.707 INFO:tasks.workunit.client.0.smithi110.stdout:9/629: fsync d9/db/df/f68 0 2024-04-25T15:49:37.708 INFO:tasks.workunit.client.1.smithi193.stdout:6/377: write dd/d12/d19/d1c/d40/d3c/d3f/f54 [2602589,48413] 0 2024-04-25T15:49:37.708 INFO:tasks.workunit.client.1.smithi193.stdout:6/378: chown l9 3 1 2024-04-25T15:49:37.710 INFO:tasks.workunit.client.1.smithi193.stdout:6/379: read dd/d12/d19/d1c/d40/d3c/d3f/f61 [3786352,10105] 0 2024-04-25T15:49:37.710 INFO:tasks.workunit.client.1.smithi193.stdout:8/349: stat d2/d18 0 2024-04-25T15:49:37.716 INFO:tasks.workunit.client.0.smithi110.stdout:5/675: getdents d4/d1d/d1f/d35/d3d/d52/d97 0 2024-04-25T15:49:37.720 INFO:tasks.workunit.client.1.smithi193.stdout:7/346: dwrite d12/d13/f14 [0,4194304] 0 2024-04-25T15:49:37.720 INFO:tasks.workunit.client.1.smithi193.stdout:7/347: stat d12/d1c/d63/d70/f34 0 2024-04-25T15:49:37.721 INFO:tasks.workunit.client.0.smithi110.stdout:7/668: fdatasync f17 0 2024-04-25T15:49:37.722 INFO:tasks.workunit.client.0.smithi110.stdout:5/676: dwrite d4/d15/f1a [0,4194304] 0 2024-04-25T15:49:37.723 INFO:tasks.workunit.client.0.smithi110.stdout:4/663: write d13/fa3 [994646,121718] 0 2024-04-25T15:49:37.725 INFO:tasks.workunit.client.0.smithi110.stdout:2/640: rename de/d40/f3d to de/d40/d5b/fca 0 2024-04-25T15:49:37.728 INFO:tasks.workunit.client.1.smithi193.stdout:0/398: creat d0/d49/d5/d7/dd/d46/d48/f85 x:0 0 0 2024-04-25T15:49:37.731 INFO:tasks.workunit.client.0.smithi110.stdout:3/694: mknod d0/cec 0 2024-04-25T15:49:37.733 INFO:tasks.workunit.client.1.smithi193.stdout:2/376: mkdir d0/d9/d15/d31/d90/d9f 0 2024-04-25T15:49:37.735 INFO:tasks.workunit.client.0.smithi110.stdout:0/625: symlink d1/d20/le2 0 2024-04-25T15:49:37.736 INFO:tasks.workunit.client.0.smithi110.stdout:1/817: symlink d0/d51/d6b/df1/l108 0 2024-04-25T15:49:37.738 INFO:tasks.workunit.client.0.smithi110.stdout:7/669: mkdir d1b/d29/d38/d55/dab/db2/db8/de3 0 2024-04-25T15:49:37.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:37 smithi193 ceph-mon[33747]: pgmap v35: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 46 MiB/s rd, 173 MiB/s wr, 313 op/s 2024-04-25T15:49:37.739 INFO:tasks.workunit.client.1.smithi193.stdout:5/356: dwrite d17/d18/d3e/f4c [0,4194304] 0 2024-04-25T15:49:37.740 INFO:tasks.workunit.client.0.smithi110.stdout:4/664: mknod d13/d24/d2d/dd9/dbb/dd2/ce2 0 2024-04-25T15:49:37.745 INFO:tasks.workunit.client.0.smithi110.stdout:9/630: creat d9/d44/d5e/d80/dbb/fcd x:0 0 0 2024-04-25T15:49:37.746 INFO:tasks.workunit.client.1.smithi193.stdout:8/350: readlink d2/d3/d27/l6a 0 2024-04-25T15:49:37.747 INFO:tasks.workunit.client.1.smithi193.stdout:6/380: fsync dd/d2f/f56 0 2024-04-25T15:49:37.748 INFO:tasks.workunit.client.1.smithi193.stdout:8/351: dread d2/d3/f37 [0,4194304] 0 2024-04-25T15:49:37.749 INFO:tasks.workunit.client.1.smithi193.stdout:8/352: chown d2/d3/d27/l5d 21 1 2024-04-25T15:49:37.749 INFO:tasks.workunit.client.1.smithi193.stdout:1/332: write dc/d4a/d50/d5c/d5d/f3a [2152941,19314] 0 2024-04-25T15:49:37.750 INFO:tasks.workunit.client.0.smithi110.stdout:9/631: dread d9/f96 [0,4194304] 0 2024-04-25T15:49:37.751 INFO:tasks.workunit.client.0.smithi110.stdout:9/632: read - d9/d44/d5e/fc9 zero size 2024-04-25T15:49:37.752 INFO:tasks.workunit.client.1.smithi193.stdout:3/418: symlink d8/d13/d14/d27/d6d/l81 0 2024-04-25T15:49:37.756 INFO:tasks.workunit.client.1.smithi193.stdout:4/418: symlink db/d32/d40/l85 0 2024-04-25T15:49:37.756 INFO:tasks.workunit.client.0.smithi110.stdout:7/670: rmdir d1b/d29/d38/d55/dab 39 2024-04-25T15:49:37.757 INFO:tasks.workunit.client.1.smithi193.stdout:3/419: dwrite d8/d30/d45/f70 [0,4194304] 0 2024-04-25T15:49:37.760 INFO:tasks.workunit.client.1.smithi193.stdout:4/419: dread db/d32/d45/f60 [0,4194304] 0 2024-04-25T15:49:37.760 INFO:tasks.workunit.client.0.smithi110.stdout:4/665: creat d13/d24/d2d/dd9/dbb/dd2/fe3 x:0 0 0 2024-04-25T15:49:37.761 INFO:tasks.workunit.client.1.smithi193.stdout:4/420: stat fa 0 2024-04-25T15:49:37.762 INFO:tasks.workunit.client.1.smithi193.stdout:9/383: mkdir df/d2c/d2f/d4e/d69/d6b 0 2024-04-25T15:49:37.763 INFO:tasks.workunit.client.0.smithi110.stdout:8/775: rmdir d14/d21/d26/d6a/d7e/da6/de8/d71/df8 0 2024-04-25T15:49:37.764 INFO:tasks.workunit.client.1.smithi193.stdout:2/377: rmdir d0/d37/d25/d5f/d3a 39 2024-04-25T15:49:37.767 INFO:tasks.workunit.client.0.smithi110.stdout:7/671: dread d1b/d29/d87/d8e/f94 [0,4194304] 0 2024-04-25T15:49:37.769 INFO:tasks.workunit.client.0.smithi110.stdout:7/672: truncate d1b/d29/d38/d55/fd9 14879 0 2024-04-25T15:49:37.769 INFO:tasks.workunit.client.0.smithi110.stdout:3/695: link d0/d16/dcb/db0/fe8 d0/d16/dcb/d2f/d4c/d63/fed 0 2024-04-25T15:49:37.772 INFO:tasks.workunit.client.0.smithi110.stdout:2/641: truncate de/d40/d5b/fca 8795641 0 2024-04-25T15:49:37.773 INFO:tasks.workunit.client.1.smithi193.stdout:6/381: mknod dd/d12/d19/d1c/c6f 0 2024-04-25T15:49:37.774 INFO:tasks.workunit.client.0.smithi110.stdout:5/677: truncate d4/d1d/f65 1797851 0 2024-04-25T15:49:37.776 INFO:tasks.workunit.client.0.smithi110.stdout:9/633: unlink d9/db/df/d2c/d3c/d56/da0/fc6 0 2024-04-25T15:49:37.778 INFO:tasks.workunit.client.1.smithi193.stdout:8/353: unlink d2/d3/d20/d45/c3e 0 2024-04-25T15:49:37.779 INFO:tasks.workunit.client.1.smithi193.stdout:1/333: mknod dc/d4a/d4d/c6f 0 2024-04-25T15:49:37.779 INFO:tasks.workunit.client.0.smithi110.stdout:2/642: dwrite de/d40/d4f/d9f/fa3 [0,4194304] 0 2024-04-25T15:49:37.780 INFO:tasks.workunit.client.0.smithi110.stdout:2/643: chown de/d40/d6e/f93 461717 1 2024-04-25T15:49:37.783 INFO:tasks.workunit.client.0.smithi110.stdout:7/673: dread d1b/d29/d38/d55/f6b [4194304,4194304] 0 2024-04-25T15:49:37.784 INFO:tasks.workunit.client.0.smithi110.stdout:8/776: rename d14/d32/d36/d48/d8f/fd1 to d14/d21/d26/d6a/de9/f107 0 2024-04-25T15:49:37.785 INFO:tasks.workunit.client.1.smithi193.stdout:7/348: unlink d12/f1a 0 2024-04-25T15:49:37.787 INFO:tasks.workunit.client.1.smithi193.stdout:3/420: mknod d8/d13/d1f/d69/d77/c82 0 2024-04-25T15:49:37.788 INFO:tasks.workunit.client.1.smithi193.stdout:3/421: write d8/d13/d4b/f57 [1047857,88869] 0 2024-04-25T15:49:37.789 INFO:tasks.workunit.client.0.smithi110.stdout:7/674: dwrite d1b/f26 [0,4194304] 0 2024-04-25T15:49:37.795 INFO:tasks.workunit.client.1.smithi193.stdout:4/421: symlink db/d13/d1e/l86 0 2024-04-25T15:49:37.796 INFO:tasks.workunit.client.0.smithi110.stdout:5/678: mknod d4/d1d/d85/cde 0 2024-04-25T15:49:37.800 INFO:tasks.workunit.client.0.smithi110.stdout:9/634: readlink d9/l6d 0 2024-04-25T15:49:37.803 INFO:tasks.workunit.client.0.smithi110.stdout:1/818: rmdir d0/d14/d5f/d7d/d72/d61/dc4 0 2024-04-25T15:49:37.806 INFO:tasks.workunit.client.0.smithi110.stdout:2/644: creat de/d40/d5b/d5c/d5f/db2/fcb x:0 0 0 2024-04-25T15:49:37.807 INFO:tasks.workunit.client.1.smithi193.stdout:6/382: chown dd/d12/d19/d1c/d40/d3c/d3f/d4f/l6e 0 1 2024-04-25T15:49:37.808 INFO:tasks.workunit.client.0.smithi110.stdout:3/696: mknod d0/d16/dcb/d2f/cee 0 2024-04-25T15:49:37.808 INFO:tasks.workunit.client.1.smithi193.stdout:6/383: readlink dd/d12/l18 0 2024-04-25T15:49:37.809 INFO:tasks.workunit.client.0.smithi110.stdout:3/697: truncate d0/d2e/d88/dbf/d92/fd2 1275985 0 2024-04-25T15:49:37.810 INFO:tasks.workunit.client.1.smithi193.stdout:1/334: mkdir dc/d4a/d50/d5c/d63/d70 0 2024-04-25T15:49:37.811 INFO:tasks.workunit.client.0.smithi110.stdout:0/626: link d1/d47/l59 d1/d20/d69/le3 0 2024-04-25T15:49:37.813 INFO:tasks.workunit.client.1.smithi193.stdout:3/422: mkdir d8/d13/d4b/d83 0 2024-04-25T15:49:37.815 INFO:tasks.workunit.client.0.smithi110.stdout:5/679: dread - d4/d1d/d1f/d35/d7f/fa7 zero size 2024-04-25T15:49:37.815 INFO:tasks.workunit.client.0.smithi110.stdout:0/627: dread d1/d47/d54/f6d [0,4194304] 0 2024-04-25T15:49:37.816 INFO:tasks.workunit.client.1.smithi193.stdout:0/399: dwrite d0/d49/d5/d7/dd/d16/d18/f2a [0,4194304] 0 2024-04-25T15:49:37.818 INFO:tasks.workunit.client.0.smithi110.stdout:1/819: mkdir d0/d14/d81/df8/d33/d8e/d109 0 2024-04-25T15:49:37.819 INFO:tasks.workunit.client.0.smithi110.stdout:1/820: readlink d0/d51/dc8/l101 0 2024-04-25T15:49:37.819 INFO:tasks.workunit.client.0.smithi110.stdout:1/821: write d0/d14/d5f/faa [1911667,119547] 0 2024-04-25T15:49:37.821 INFO:tasks.workunit.client.0.smithi110.stdout:1/822: write d0/d51/dd8/ff2 [48614,66918] 0 2024-04-25T15:49:37.822 INFO:tasks.workunit.client.1.smithi193.stdout:9/384: getdents df/d2c/d2f/d4e/d69/d6b 0 2024-04-25T15:49:37.823 INFO:tasks.workunit.client.1.smithi193.stdout:5/357: truncate d17/d18/d30/d38/d71/f74 3873499 0 2024-04-25T15:49:37.826 INFO:tasks.workunit.client.0.smithi110.stdout:3/698: creat d0/d16/dcb/db0/fef x:0 0 0 2024-04-25T15:49:37.827 INFO:tasks.workunit.client.0.smithi110.stdout:4/666: dread d13/d24/d2d/d44/d65/d99/daf/d56/d88/f41 [0,4194304] 0 2024-04-25T15:49:37.828 INFO:tasks.workunit.client.1.smithi193.stdout:6/384: unlink dd/d12/d19/d1c/d40/d3c/c6c 0 2024-04-25T15:49:37.829 INFO:tasks.workunit.client.0.smithi110.stdout:4/667: write d13/d24/d2d/d36/fde [1047369,85836] 0 2024-04-25T15:49:37.829 INFO:tasks.workunit.client.1.smithi193.stdout:1/335: creat dc/d4a/f71 x:0 0 0 2024-04-25T15:49:37.830 INFO:tasks.workunit.client.0.smithi110.stdout:4/668: chown d13/d24/d2d/dd9/dbb/dd2/ce2 226 1 2024-04-25T15:49:37.831 INFO:tasks.workunit.client.1.smithi193.stdout:3/423: mknod d8/c84 0 2024-04-25T15:49:37.832 INFO:tasks.workunit.client.1.smithi193.stdout:3/424: readlink d8/d13/d1f/d69/d77/l4a 0 2024-04-25T15:49:37.833 INFO:tasks.workunit.client.1.smithi193.stdout:3/425: read d8/d13/f7b [62479,26608] 0 2024-04-25T15:49:37.834 INFO:tasks.workunit.client.0.smithi110.stdout:0/628: symlink d1/d23/d30/d33/d5b/d72/d77/le4 0 2024-04-25T15:49:37.834 INFO:tasks.workunit.client.1.smithi193.stdout:0/400: mkdir d0/d49/d5/d7/dd/d16/d1a/d86 0 2024-04-25T15:49:37.836 INFO:tasks.workunit.client.0.smithi110.stdout:5/680: dread d4/d1d/d1f/f49 [0,4194304] 0 2024-04-25T15:49:37.837 INFO:tasks.workunit.client.1.smithi193.stdout:5/358: creat d17/d18/d30/d38/f7b x:0 0 0 2024-04-25T15:49:37.839 INFO:tasks.workunit.client.1.smithi193.stdout:2/378: symlink d0/d37/d25/d5f/d3a/d6d/d6e/d8e/la0 0 2024-04-25T15:49:37.840 INFO:tasks.workunit.client.1.smithi193.stdout:2/379: write d0/d42/d51/d4f/d65/d6a/f3b [838484,96448] 0 2024-04-25T15:49:37.841 INFO:tasks.workunit.client.1.smithi193.stdout:8/354: creat d2/f6e x:0 0 0 2024-04-25T15:49:37.842 INFO:tasks.workunit.client.0.smithi110.stdout:7/675: link d1b/d29/d38/d55/fbd d1b/d29/d38/d55/d95/dcb/fe4 0 2024-04-25T15:49:37.848 INFO:tasks.workunit.client.0.smithi110.stdout:7/676: dwrite d1b/d29/d38/d55/d7f/fb1 [0,4194304] 0 2024-04-25T15:49:37.850 INFO:tasks.workunit.client.1.smithi193.stdout:1/336: symlink dc/d59/l72 0 2024-04-25T15:49:37.853 INFO:tasks.workunit.client.0.smithi110.stdout:4/669: unlink d13/d24/d2d/d36/f95 0 2024-04-25T15:49:37.854 INFO:tasks.workunit.client.0.smithi110.stdout:4/670: write d13/d24/d2d/d7d/fda [464718,99738] 0 2024-04-25T15:49:37.855 INFO:tasks.workunit.client.0.smithi110.stdout:9/635: link d9/db/df/d2c/d3c/d56/c59 d9/d44/d5e/d80/cce 0 2024-04-25T15:49:37.856 INFO:tasks.workunit.client.1.smithi193.stdout:1/337: dwrite dc/f5b [0,4194304] 0 2024-04-25T15:49:37.856 INFO:tasks.workunit.client.0.smithi110.stdout:9/636: read - d9/faa zero size 2024-04-25T15:49:37.857 INFO:tasks.workunit.client.0.smithi110.stdout:9/637: chown d9/db/df/f6c 1 1 2024-04-25T15:49:37.862 INFO:tasks.workunit.client.1.smithi193.stdout:4/422: creat db/f87 x:0 0 0 2024-04-25T15:49:37.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:37 smithi110 ceph-mon[26344]: pgmap v35: 65 pgs: 65 active+clean; 1.9 GiB data, 7.0 GiB used, 529 GiB / 536 GiB avail; 46 MiB/s rd, 173 MiB/s wr, 313 op/s 2024-04-25T15:49:37.865 INFO:tasks.workunit.client.1.smithi193.stdout:0/401: symlink d0/d49/d5/d7/dd/d16/d1a/l87 0 2024-04-25T15:49:37.868 INFO:tasks.workunit.client.1.smithi193.stdout:5/359: write d17/d18/d30/f76 [487459,98360] 0 2024-04-25T15:49:37.869 INFO:tasks.workunit.client.1.smithi193.stdout:5/360: fdatasync d17/d18/d30/d38/f7b 0 2024-04-25T15:49:37.870 INFO:tasks.workunit.client.1.smithi193.stdout:5/361: readlink d17/l1c 0 2024-04-25T15:49:37.870 INFO:tasks.workunit.client.1.smithi193.stdout:5/362: chown d17/d18/d30/d3b/f60 103 1 2024-04-25T15:49:37.871 INFO:tasks.workunit.client.0.smithi110.stdout:1/823: write d0/d14/d5f/d7d/d72/f7e [1162211,91471] 0 2024-04-25T15:49:37.872 INFO:tasks.workunit.client.0.smithi110.stdout:3/699: mknod d0/d2e/d88/dbf/cf0 0 2024-04-25T15:49:37.873 INFO:tasks.workunit.client.1.smithi193.stdout:2/380: chown d0/d9/d3d 88 1 2024-04-25T15:49:37.875 INFO:tasks.workunit.client.0.smithi110.stdout:7/677: creat d1b/d29/d46/fe5 x:0 0 0 2024-04-25T15:49:37.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/355: rmdir d2 39 2024-04-25T15:49:37.880 INFO:tasks.workunit.client.1.smithi193.stdout:6/385: mknod dd/d12/d19/c70 0 2024-04-25T15:49:37.881 INFO:tasks.workunit.client.1.smithi193.stdout:7/349: getdents d12/d13 0 2024-04-25T15:49:37.881 INFO:tasks.workunit.client.1.smithi193.stdout:6/386: truncate dd/d12/f5d 720140 0 2024-04-25T15:49:37.886 INFO:tasks.workunit.client.0.smithi110.stdout:2/645: link de/d40/d6e/d50/l74 de/d6f/dbb/lcc 0 2024-04-25T15:49:37.889 INFO:tasks.workunit.client.0.smithi110.stdout:5/681: dwrite d4/fd [0,4194304] 0 2024-04-25T15:49:37.889 INFO:tasks.workunit.client.0.smithi110.stdout:0/629: dwrite d1/d20/f96 [0,4194304] 0 2024-04-25T15:49:37.891 INFO:tasks.workunit.client.0.smithi110.stdout:0/630: chown d1/d23/d30/d33/d5b/d72/d77 61 1 2024-04-25T15:49:37.892 INFO:tasks.workunit.client.1.smithi193.stdout:4/423: unlink db/d4b/d51/d54/f5c 0 2024-04-25T15:49:37.895 INFO:tasks.workunit.client.0.smithi110.stdout:3/700: dread - d0/d2e/d88/dbf/d92/f89 zero size 2024-04-25T15:49:37.896 INFO:tasks.workunit.client.1.smithi193.stdout:0/402: mknod d0/d49/d5/d7/dd/d16/d1a/c88 0 2024-04-25T15:49:37.897 INFO:tasks.workunit.client.0.smithi110.stdout:2/646: dwrite de/d40/d6e/d50/db7/faa [0,4194304] 0 2024-04-25T15:49:37.898 INFO:tasks.workunit.client.1.smithi193.stdout:0/403: chown d0/d49/d5/d7/dd/d16/d18/d28/l32 241876 1 2024-04-25T15:49:37.900 INFO:tasks.workunit.client.1.smithi193.stdout:5/363: mknod d17/d18/d30/d34/c7c 0 2024-04-25T15:49:37.901 INFO:tasks.workunit.client.0.smithi110.stdout:4/671: link d13/d24/d2d/dd9/dbb/dd2/fe3 d13/d24/d2d/d36/d51/fe4 0 2024-04-25T15:49:37.903 INFO:tasks.workunit.client.1.smithi193.stdout:9/385: sync 2024-04-25T15:49:37.903 INFO:tasks.workunit.client.1.smithi193.stdout:9/386: fdatasync df/f57 0 2024-04-25T15:49:37.904 INFO:tasks.workunit.client.1.smithi193.stdout:2/381: rename d0/d9/d15/l6f to d0/d42/d51/d4f/d65/la1 0 2024-04-25T15:49:37.905 INFO:tasks.workunit.client.1.smithi193.stdout:2/382: fdatasync d0/d42/d51/d4f/d65/d7e/f82 0 2024-04-25T15:49:37.906 INFO:tasks.workunit.client.1.smithi193.stdout:9/387: write df/d2c/d2f/d4e/d1c/d26/d3f/d41/f59 [1214866,20223] 0 2024-04-25T15:49:37.908 INFO:tasks.workunit.client.0.smithi110.stdout:1/824: creat d0/d23/db1/f10a x:0 0 0 2024-04-25T15:49:37.910 INFO:tasks.workunit.client.1.smithi193.stdout:6/387: rename dd/d12/d19/d1c/f2c to dd/d12/d19/d1c/f71 0 2024-04-25T15:49:37.911 INFO:tasks.workunit.client.0.smithi110.stdout:5/682: mkdir d4/d1d/d1f/d35/d3d/d52/ddf 0 2024-04-25T15:49:37.917 INFO:tasks.workunit.client.0.smithi110.stdout:3/701: rmdir d0/d16 39 2024-04-25T15:49:37.922 INFO:tasks.workunit.client.0.smithi110.stdout:9/638: truncate d9/db/d7d/fb6 546482 0 2024-04-25T15:49:37.923 INFO:tasks.workunit.client.1.smithi193.stdout:5/364: mknod d17/d18/d30/d3b/c7d 0 2024-04-25T15:49:37.924 INFO:tasks.workunit.client.1.smithi193.stdout:5/365: chown d17/d18/d30/d3b/f60 75 1 2024-04-25T15:49:37.924 INFO:tasks.workunit.client.0.smithi110.stdout:4/672: creat d13/d24/d2d/db5/fe5 x:0 0 0 2024-04-25T15:49:37.924 INFO:tasks.workunit.client.1.smithi193.stdout:5/366: chown d17/d18/f2c 7 1 2024-04-25T15:49:37.926 INFO:tasks.workunit.client.0.smithi110.stdout:1/825: mknod d0/d14/d5f/d6f/c10b 0 2024-04-25T15:49:37.927 INFO:tasks.workunit.client.0.smithi110.stdout:0/631: write d1/d47/d54/f6d [908671,105305] 0 2024-04-25T15:49:37.928 INFO:tasks.workunit.client.1.smithi193.stdout:7/350: dwrite d12/d31/d37/d62/d6e/d79/f4e [0,4194304] 0 2024-04-25T15:49:37.929 INFO:tasks.workunit.client.1.smithi193.stdout:7/351: fdatasync d12/d1c/d63/f65 0 2024-04-25T15:49:37.930 INFO:tasks.workunit.client.1.smithi193.stdout:4/424: write db/d13/d1e/d1f/d22/f7a [232973,84877] 0 2024-04-25T15:49:37.931 INFO:tasks.workunit.client.1.smithi193.stdout:4/425: read - db/d4b/f7b zero size 2024-04-25T15:49:37.932 INFO:tasks.workunit.client.0.smithi110.stdout:7/678: creat d1b/d29/d87/fe6 x:0 0 0 2024-04-25T15:49:37.932 INFO:tasks.workunit.client.1.smithi193.stdout:4/426: write db/d13/d1e/f41 [4838564,112226] 0 2024-04-25T15:49:37.933 INFO:tasks.workunit.client.0.smithi110.stdout:7/679: write d1b/d29/d87/fe6 [993977,111548] 0 2024-04-25T15:49:37.936 INFO:tasks.workunit.client.0.smithi110.stdout:2/647: mkdir de/d40/dcd 0 2024-04-25T15:49:37.938 INFO:tasks.workunit.client.1.smithi193.stdout:5/367: dread d17/d18/d30/f35 [0,4194304] 0 2024-04-25T15:49:37.938 INFO:tasks.workunit.client.0.smithi110.stdout:9/639: unlink d9/db/d7d/l8f 0 2024-04-25T15:49:37.940 INFO:tasks.workunit.client.1.smithi193.stdout:1/338: creat dc/d4a/d50/d5c/d5d/d3d/d55/f73 x:0 0 0 2024-04-25T15:49:37.940 INFO:tasks.workunit.client.0.smithi110.stdout:4/673: mknod d13/ce6 0 2024-04-25T15:49:37.941 INFO:tasks.workunit.client.1.smithi193.stdout:3/426: link d8/d13/d14/f17 d8/d30/f85 0 2024-04-25T15:49:37.941 INFO:tasks.workunit.client.1.smithi193.stdout:3/427: chown d8/d13/f2d 28434844 1 2024-04-25T15:49:37.943 INFO:tasks.workunit.client.0.smithi110.stdout:5/683: mknod d4/d1d/d1f/d35/d3d/dc8/ce0 0 2024-04-25T15:49:37.946 INFO:tasks.workunit.client.1.smithi193.stdout:8/356: read d2/d3/f37 [138690,69165] 0 2024-04-25T15:49:37.947 INFO:tasks.workunit.client.1.smithi193.stdout:8/357: write d2/d3/d3a/f4c [1033469,107217] 0 2024-04-25T15:49:37.949 INFO:tasks.workunit.client.0.smithi110.stdout:7/680: dread d1b/f88 [0,4194304] 0 2024-04-25T15:49:37.954 INFO:tasks.workunit.client.1.smithi193.stdout:9/388: creat df/d2c/d34/d3e/d4d/d5e/f6c x:0 0 0 2024-04-25T15:49:37.954 INFO:tasks.workunit.client.1.smithi193.stdout:9/389: write f9 [2467080,11180] 0 2024-04-25T15:49:37.955 INFO:tasks.workunit.client.0.smithi110.stdout:4/674: mknod d13/d24/d2d/d44/d65/d99/ce7 0 2024-04-25T15:49:37.956 INFO:tasks.workunit.client.1.smithi193.stdout:7/352: symlink d12/d13/l87 0 2024-04-25T15:49:37.956 INFO:tasks.workunit.client.1.smithi193.stdout:7/353: chown d12/d31/l54 548845 1 2024-04-25T15:49:37.959 INFO:tasks.workunit.client.0.smithi110.stdout:5/684: fdatasync d4/d1d/f70 0 2024-04-25T15:49:37.962 INFO:tasks.workunit.client.0.smithi110.stdout:0/632: rename d1/d23/d30/d33/d35/da1/ca7 to d1/d23/d30/d33/d5b/dbb/ce5 0 2024-04-25T15:49:37.965 INFO:tasks.workunit.client.0.smithi110.stdout:3/702: creat d0/d7/d40/ff1 x:0 0 0 2024-04-25T15:49:37.966 INFO:tasks.workunit.client.0.smithi110.stdout:1/826: dwrite d0/d14/d5f/f65 [0,4194304] 0 2024-04-25T15:49:37.966 INFO:tasks.workunit.client.1.smithi193.stdout:6/388: fsync dd/d12/d19/f1d 0 2024-04-25T15:49:37.971 INFO:tasks.workunit.client.1.smithi193.stdout:3/428: truncate d8/d13/d1f/d69/d77/f53 5071919 0 2024-04-25T15:49:37.972 INFO:tasks.workunit.client.0.smithi110.stdout:2/648: mkdir de/d40/d5b/d5c/d5f/db2/dad/dbd/dce 0 2024-04-25T15:49:37.973 INFO:tasks.workunit.client.0.smithi110.stdout:2/649: write de/d40/d5b/d5c/d5f/f7a [2414294,85961] 0 2024-04-25T15:49:37.975 INFO:tasks.workunit.client.1.smithi193.stdout:2/383: dwrite d0/df/f26 [0,4194304] 0 2024-04-25T15:49:37.976 INFO:tasks.workunit.client.1.smithi193.stdout:6/389: dwrite dd/d12/d19/d1c/f29 [0,4194304] 0 2024-04-25T15:49:37.977 INFO:tasks.workunit.client.0.smithi110.stdout:5/685: symlink d4/d1d/d1f/d35/d7f/le1 0 2024-04-25T15:49:37.978 INFO:tasks.workunit.client.1.smithi193.stdout:8/358: mkdir d2/d3/d27/d6f 0 2024-04-25T15:49:37.978 INFO:tasks.workunit.client.1.smithi193.stdout:5/368: write d17/d18/f20 [4269821,51084] 0 2024-04-25T15:49:37.979 INFO:tasks.workunit.client.0.smithi110.stdout:4/675: rename d13/d24/d26/c2f to d13/ce8 0 2024-04-25T15:49:37.980 INFO:tasks.workunit.client.0.smithi110.stdout:7/681: write d1b/f4b [239606,26506] 0 2024-04-25T15:49:37.984 INFO:tasks.workunit.client.0.smithi110.stdout:0/633: creat d1/d23/d30/d33/d5b/dbb/da8/fe6 x:0 0 0 2024-04-25T15:49:37.986 INFO:tasks.workunit.client.1.smithi193.stdout:5/369: truncate d17/d18/d45/f6d 919506 0 2024-04-25T15:49:37.988 INFO:tasks.workunit.client.1.smithi193.stdout:4/427: mkdir db/d32/d88 0 2024-04-25T15:49:37.988 INFO:tasks.workunit.client.1.smithi193.stdout:6/390: dread dd/d12/f1e [0,4194304] 0 2024-04-25T15:49:37.989 INFO:tasks.workunit.client.1.smithi193.stdout:4/428: chown db/fc 90844979 1 2024-04-25T15:49:37.990 INFO:tasks.workunit.client.1.smithi193.stdout:2/384: dwrite d0/d42/d51/d4f/d65/d6a/f3b [0,4194304] 0 2024-04-25T15:49:37.990 INFO:tasks.workunit.client.1.smithi193.stdout:1/339: mknod dc/d4a/d50/d5c/d63/d70/c74 0 2024-04-25T15:49:37.992 INFO:tasks.workunit.client.1.smithi193.stdout:1/340: chown dc/d4a/d50/l6e 11086 1 2024-04-25T15:49:37.993 INFO:tasks.workunit.client.0.smithi110.stdout:0/634: dwrite d1/f16 [4194304,4194304] 0 2024-04-25T15:49:37.993 INFO:tasks.workunit.client.0.smithi110.stdout:3/703: dwrite d0/d7/f68 [4194304,4194304] 0 2024-04-25T15:49:37.996 INFO:tasks.workunit.client.1.smithi193.stdout:0/404: getdents d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53 0 2024-04-25T15:49:37.996 INFO:tasks.workunit.client.1.smithi193.stdout:0/405: fdatasync d0/d49/d5/d7/dd/d46/f6b 0 2024-04-25T15:49:38.002 INFO:tasks.workunit.client.0.smithi110.stdout:7/682: truncate d1b/d29/d38/d55/d7f/f7a 912778 0 2024-04-25T15:49:38.003 INFO:tasks.workunit.client.0.smithi110.stdout:7/683: chown d1b/d29/d38/d55/d7f/l85 1950 1 2024-04-25T15:49:38.005 INFO:tasks.workunit.client.1.smithi193.stdout:8/359: symlink d2/d3/d20/d45/l70 0 2024-04-25T15:49:38.006 INFO:tasks.workunit.client.0.smithi110.stdout:9/640: getdents d9/db/df/d2c/d3c/d56 0 2024-04-25T15:49:38.013 INFO:tasks.workunit.client.0.smithi110.stdout:0/635: stat d1/d23/d30/d33/l67 0 2024-04-25T15:49:38.014 INFO:tasks.workunit.client.0.smithi110.stdout:0/636: chown d1/d23/d30/d33/d35/l55 113960284 1 2024-04-25T15:49:38.015 INFO:tasks.workunit.client.0.smithi110.stdout:0/637: write d1/d23/d81/fc3 [262923,121751] 0 2024-04-25T15:49:38.016 INFO:tasks.workunit.client.1.smithi193.stdout:5/370: rename d17/d18/l1b to d17/d18/d30/d3b/l7e 0 2024-04-25T15:49:38.018 INFO:tasks.workunit.client.0.smithi110.stdout:4/676: unlink d13/d24/c61 0 2024-04-25T15:49:38.018 INFO:tasks.workunit.client.1.smithi193.stdout:7/354: write d12/d31/d37/d62/d6e/d79/f3f [975517,118950] 0 2024-04-25T15:49:38.021 INFO:tasks.workunit.client.1.smithi193.stdout:4/429: rmdir db/d32 39 2024-04-25T15:49:38.021 INFO:tasks.workunit.client.0.smithi110.stdout:0/638: dwrite d1/f8d [4194304,4194304] 0 2024-04-25T15:49:38.023 INFO:tasks.workunit.client.1.smithi193.stdout:1/341: creat dc/d4a/f75 x:0 0 0 2024-04-25T15:49:38.023 INFO:tasks.workunit.client.1.smithi193.stdout:6/391: dwrite dd/d12/d19/d1c/f71 [0,4194304] 0 2024-04-25T15:49:38.027 INFO:tasks.workunit.client.0.smithi110.stdout:5/686: write d4/d15/d51/d90/fa3 [168178,115499] 0 2024-04-25T15:49:38.028 INFO:tasks.workunit.client.1.smithi193.stdout:3/429: creat d8/d13/d1f/d69/d77/d71/f86 x:0 0 0 2024-04-25T15:49:38.030 INFO:tasks.workunit.client.0.smithi110.stdout:3/704: mkdir d0/d16/dcb/d2f/d4c/db2/df2 0 2024-04-25T15:49:38.035 INFO:tasks.workunit.client.1.smithi193.stdout:3/430: dwrite d8/d13/f60 [0,4194304] 0 2024-04-25T15:49:38.036 INFO:tasks.workunit.client.1.smithi193.stdout:3/431: chown d8/d13/d4b/f6e 1631409 1 2024-04-25T15:49:38.037 INFO:tasks.workunit.client.1.smithi193.stdout:9/390: creat df/d2c/d2f/d5c/f6d x:0 0 0 2024-04-25T15:49:38.038 INFO:tasks.workunit.client.0.smithi110.stdout:4/677: creat d13/d24/d26/fe9 x:0 0 0 2024-04-25T15:49:38.040 INFO:tasks.workunit.client.0.smithi110.stdout:4/678: chown d13/d24/d2d/d44/da4/fb3 0 1 2024-04-25T15:49:38.041 INFO:tasks.workunit.client.0.smithi110.stdout:2/650: dwrite de/d40/d6e/f39 [0,4194304] 0 2024-04-25T15:49:38.043 INFO:tasks.workunit.client.0.smithi110.stdout:2/651: write de/d40/d5b/d5c/d5f/db2/f95 [1178507,129872] 0 2024-04-25T15:49:38.044 INFO:tasks.workunit.client.1.smithi193.stdout:2/385: rename d0/d37/d25/d5f/l84 to d0/d37/d25/d5f/d3a/d6d/la2 0 2024-04-25T15:49:38.045 INFO:tasks.workunit.client.1.smithi193.stdout:7/355: creat d12/d31/d37/d62/d81/f88 x:0 0 0 2024-04-25T15:49:38.046 INFO:tasks.workunit.client.1.smithi193.stdout:7/356: dread - d12/d31/d39/f53 zero size 2024-04-25T15:49:38.048 INFO:tasks.workunit.client.1.smithi193.stdout:4/430: write db/d32/d45/f60 [1333485,35820] 0 2024-04-25T15:49:38.049 INFO:tasks.workunit.client.1.smithi193.stdout:1/342: unlink dc/c1c 0 2024-04-25T15:49:38.049 INFO:tasks.workunit.client.0.smithi110.stdout:5/687: truncate d4/f44 3846838 0 2024-04-25T15:49:38.050 INFO:tasks.workunit.client.1.smithi193.stdout:6/392: creat dd/d2f/f72 x:0 0 0 2024-04-25T15:49:38.051 INFO:tasks.workunit.client.1.smithi193.stdout:0/406: symlink d0/d49/d5/d7/dd/d46/d48/d62/l89 0 2024-04-25T15:49:38.051 INFO:tasks.workunit.client.0.smithi110.stdout:2/652: dwrite de/d40/d5b/d5c/d5f/db2/fa0 [0,4194304] 0 2024-04-25T15:49:38.052 INFO:tasks.workunit.client.1.smithi193.stdout:4/431: truncate db/d13/d1e/d1f/d56/d69/d70/f76 903833 0 2024-04-25T15:49:38.053 INFO:tasks.workunit.client.1.smithi193.stdout:6/393: truncate dd/d12/d19/d1c/f46 9986 0 2024-04-25T15:49:38.054 INFO:tasks.workunit.client.1.smithi193.stdout:6/394: readlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/l65 0 2024-04-25T15:49:38.054 INFO:tasks.workunit.client.1.smithi193.stdout:6/395: readlink l9 0 2024-04-25T15:49:38.055 INFO:tasks.workunit.client.0.smithi110.stdout:0/639: mkdir d1/d23/d30/d33/d5b/d78/ddb/de7 0 2024-04-25T15:49:38.055 INFO:tasks.workunit.client.1.smithi193.stdout:6/396: fdatasync dd/d12/d19/d1c/f22 0 2024-04-25T15:49:38.055 INFO:tasks.workunit.client.0.smithi110.stdout:0/640: chown d1/d20/l70 1023 1 2024-04-25T15:49:38.059 INFO:tasks.workunit.client.0.smithi110.stdout:0/641: write d1/d6c/fd5 [933336,71911] 0 2024-04-25T15:49:38.061 INFO:tasks.workunit.client.1.smithi193.stdout:7/357: dread d12/f30 [0,4194304] 0 2024-04-25T15:49:38.061 INFO:tasks.workunit.client.1.smithi193.stdout:5/371: symlink d17/l7f 0 2024-04-25T15:49:38.061 INFO:tasks.workunit.client.0.smithi110.stdout:2/653: dwrite de/d40/d4f/d9f/fa3 [0,4194304] 0 2024-04-25T15:49:38.063 INFO:tasks.workunit.client.0.smithi110.stdout:7/684: creat d1b/d29/d87/fe7 x:0 0 0 2024-04-25T15:49:38.063 INFO:tasks.workunit.client.0.smithi110.stdout:2/654: chown de/d40/d5b/d5c/d5f/db2/cb5 43 1 2024-04-25T15:49:38.065 INFO:tasks.workunit.client.1.smithi193.stdout:8/360: rename d2/d3/d20/d45/d29/c65 to d2/d3/d20/c71 0 2024-04-25T15:49:38.065 INFO:tasks.workunit.client.0.smithi110.stdout:7/685: read - d1b/d29/d38/d8b/fc7 zero size 2024-04-25T15:49:38.065 INFO:tasks.workunit.client.0.smithi110.stdout:2/655: fdatasync de/d40/d6e/d52/d6d/fb3 0 2024-04-25T15:49:38.066 INFO:tasks.workunit.client.1.smithi193.stdout:4/432: dwrite db/d13/d1e/d1f/f64 [0,4194304] 0 2024-04-25T15:49:38.067 INFO:tasks.workunit.client.0.smithi110.stdout:9/641: truncate d9/db/df/d2c/d3c/f87 2068370 0 2024-04-25T15:49:38.069 INFO:tasks.workunit.client.0.smithi110.stdout:7/686: fsync d1b/d29/f2b 0 2024-04-25T15:49:38.069 INFO:tasks.workunit.client.1.smithi193.stdout:9/391: dwrite df/d2c/d2f/d4e/f25 [4194304,4194304] 0 2024-04-25T15:49:38.069 INFO:tasks.workunit.client.1.smithi193.stdout:4/433: write db/f6a [287091,58641] 0 2024-04-25T15:49:38.071 INFO:tasks.workunit.client.1.smithi193.stdout:2/386: creat d0/d9/d3d/fa3 x:0 0 0 2024-04-25T15:49:38.078 INFO:tasks.workunit.client.1.smithi193.stdout:7/358: dread d12/d31/f58 [0,4194304] 0 2024-04-25T15:49:38.080 INFO:tasks.workunit.client.1.smithi193.stdout:0/407: unlink d0/d49/d5/d7/dd/d16/c50 0 2024-04-25T15:49:38.081 INFO:tasks.workunit.client.0.smithi110.stdout:4/679: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/d88/f41 [0,4194304] 0 2024-04-25T15:49:38.083 INFO:tasks.workunit.client.1.smithi193.stdout:4/434: dwrite db/d13/d1e/d1f/d22/f6b [0,4194304] 0 2024-04-25T15:49:38.084 INFO:tasks.workunit.client.0.smithi110.stdout:0/642: mknod d1/d23/d30/d33/d5b/d78/ddb/ce8 0 2024-04-25T15:49:38.086 INFO:tasks.workunit.client.1.smithi193.stdout:6/397: dread dd/f2e [0,4194304] 0 2024-04-25T15:49:38.088 INFO:tasks.workunit.client.1.smithi193.stdout:4/435: write db/d13/d1e/d1f/d22/f2c [3613161,63093] 0 2024-04-25T15:49:38.088 INFO:tasks.workunit.client.1.smithi193.stdout:4/436: write db/fc [4807330,54216] 0 2024-04-25T15:49:38.093 INFO:tasks.workunit.client.0.smithi110.stdout:2/656: rename c9 to de/d40/d70/dc6/ccf 0 2024-04-25T15:49:38.094 INFO:tasks.workunit.client.0.smithi110.stdout:2/657: dread - de/d40/d5b/d5c/fa5 zero size 2024-04-25T15:49:38.095 INFO:tasks.workunit.client.0.smithi110.stdout:2/658: chown de/d40/f96 26985 1 2024-04-25T15:49:38.095 INFO:tasks.workunit.client.0.smithi110.stdout:7/687: fdatasync d1b/f79 0 2024-04-25T15:49:38.096 INFO:tasks.workunit.client.1.smithi193.stdout:6/398: dwrite dd/d12/d19/d1c/d40/d3c/f68 [0,4194304] 0 2024-04-25T15:49:38.096 INFO:tasks.workunit.client.0.smithi110.stdout:3/705: link d0/d16/ldc d0/d7/dd0/lf3 0 2024-04-25T15:49:38.100 INFO:tasks.workunit.client.0.smithi110.stdout:4/680: creat d13/d24/d26/fea x:0 0 0 2024-04-25T15:49:38.102 INFO:tasks.workunit.client.1.smithi193.stdout:8/361: dwrite d2/f48 [0,4194304] 0 2024-04-25T15:49:38.105 INFO:tasks.workunit.client.1.smithi193.stdout:8/362: read d2/d3/d3a/f54 [2070976,106364] 0 2024-04-25T15:49:38.109 INFO:tasks.workunit.client.0.smithi110.stdout:3/706: symlink d0/d2e/d88/dbf/d99/dad/daf/lf4 0 2024-04-25T15:49:38.110 INFO:tasks.workunit.client.0.smithi110.stdout:3/707: truncate d0/d7/f6c 2032817 0 2024-04-25T15:49:38.110 INFO:tasks.workunit.client.1.smithi193.stdout:5/372: mknod d17/d18/c80 0 2024-04-25T15:49:38.112 INFO:tasks.workunit.client.0.smithi110.stdout:4/681: creat d13/d24/d2d/d36/feb x:0 0 0 2024-04-25T15:49:38.116 INFO:tasks.workunit.client.1.smithi193.stdout:5/373: dwrite d17/d18/f20 [4194304,4194304] 0 2024-04-25T15:49:38.117 INFO:tasks.workunit.client.1.smithi193.stdout:6/399: creat dd/d12/d19/d1c/d40/d3c/d3f/f73 x:0 0 0 2024-04-25T15:49:38.117 INFO:tasks.workunit.client.1.smithi193.stdout:5/374: truncate d17/d18/d30/d34/f6c 4761927 0 2024-04-25T15:49:38.118 INFO:tasks.workunit.client.0.smithi110.stdout:7/688: unlink d1b/d29/l2c 0 2024-04-25T15:49:38.119 INFO:tasks.workunit.client.1.smithi193.stdout:2/387: mknod d0/d9/d3d/d7d/ca4 0 2024-04-25T15:49:38.119 INFO:tasks.workunit.client.1.smithi193.stdout:5/375: truncate d17/f59 163317 0 2024-04-25T15:49:38.120 INFO:tasks.workunit.client.0.smithi110.stdout:3/708: creat d0/d16/dcb/d2f/d33/ff5 x:0 0 0 2024-04-25T15:49:38.122 INFO:tasks.workunit.client.1.smithi193.stdout:1/343: creat dc/d4a/d50/d5c/d63/f76 x:0 0 0 2024-04-25T15:49:38.125 INFO:tasks.workunit.client.1.smithi193.stdout:6/400: dwrite dd/d12/d19/d1c/f71 [0,4194304] 0 2024-04-25T15:49:38.125 INFO:tasks.workunit.client.0.smithi110.stdout:7/689: dwrite d1b/d29/d38/d55/d7f/fb1 [0,4194304] 0 2024-04-25T15:49:38.126 INFO:tasks.workunit.client.0.smithi110.stdout:0/643: rmdir d1/d23/d30/d33/d80 0 2024-04-25T15:49:38.127 INFO:tasks.workunit.client.1.smithi193.stdout:1/344: read dc/d4a/d4d/f57 [2996290,44578] 0 2024-04-25T15:49:38.130 INFO:tasks.workunit.client.1.smithi193.stdout:3/432: getdents d8/d13/d14/d27/d54 0 2024-04-25T15:49:38.133 INFO:tasks.workunit.client.1.smithi193.stdout:6/401: dwrite dd/d12/f35 [0,4194304] 0 2024-04-25T15:49:38.137 INFO:tasks.workunit.client.0.smithi110.stdout:7/690: dwrite d1b/d29/d38/d55/fd9 [0,4194304] 0 2024-04-25T15:49:38.139 INFO:tasks.workunit.client.0.smithi110.stdout:7/691: write d1b/d29/d38/d8b/d89/fd8 [1010508,107083] 0 2024-04-25T15:49:38.142 INFO:tasks.workunit.client.1.smithi193.stdout:3/433: dwrite d8/f15 [0,4194304] 0 2024-04-25T15:49:38.144 INFO:tasks.workunit.client.1.smithi193.stdout:3/434: chown d8/d13/d14/f80 70670042 1 2024-04-25T15:49:38.145 INFO:tasks.workunit.client.1.smithi193.stdout:9/392: creat df/d2c/d2f/f6e x:0 0 0 2024-04-25T15:49:38.146 INFO:tasks.workunit.client.1.smithi193.stdout:9/393: readlink l0 0 2024-04-25T15:49:38.151 INFO:tasks.workunit.client.1.smithi193.stdout:2/388: readlink d0/d42/d51/d4f/d65/la1 0 2024-04-25T15:49:38.153 INFO:tasks.workunit.client.1.smithi193.stdout:5/376: mknod d17/d18/d30/d38/d48/c81 0 2024-04-25T15:49:38.153 INFO:tasks.workunit.client.0.smithi110.stdout:0/644: dread d1/d23/d30/d33/d35/da1/db4/db9/fd3 [0,4194304] 0 2024-04-25T15:49:38.154 INFO:tasks.workunit.client.0.smithi110.stdout:0/645: stat d1/d47/d54/f9c 0 2024-04-25T15:49:38.154 INFO:tasks.workunit.client.1.smithi193.stdout:3/435: dwrite d8/d13/d14/d27/f5a [0,4194304] 0 2024-04-25T15:49:38.154 INFO:tasks.workunit.client.0.smithi110.stdout:0/646: truncate d1/d23/d30/fca 69896 0 2024-04-25T15:49:38.156 INFO:tasks.workunit.client.1.smithi193.stdout:5/377: write d17/d18/d30/f32 [3732068,41078] 0 2024-04-25T15:49:38.156 INFO:tasks.workunit.client.1.smithi193.stdout:3/436: write d8/d30/d72/f76 [982011,115695] 0 2024-04-25T15:49:38.158 INFO:tasks.workunit.client.0.smithi110.stdout:0/647: dread d1/d6c/fd5 [0,4194304] 0 2024-04-25T15:49:38.160 INFO:tasks.workunit.client.1.smithi193.stdout:8/363: dread d2/d3/d20/f22 [0,4194304] 0 2024-04-25T15:49:38.162 INFO:tasks.workunit.client.0.smithi110.stdout:3/709: truncate d0/d7/fa6 479829 0 2024-04-25T15:49:38.164 INFO:tasks.workunit.client.1.smithi193.stdout:1/345: mkdir dc/d4a/d77 0 2024-04-25T15:49:38.165 INFO:tasks.workunit.client.1.smithi193.stdout:8/364: dread d2/d3/d20/d45/d29/f57 [0,4194304] 0 2024-04-25T15:49:38.168 INFO:tasks.workunit.client.0.smithi110.stdout:9/642: getdents d9/d13/d5b/d61 0 2024-04-25T15:49:38.169 INFO:tasks.workunit.client.0.smithi110.stdout:9/643: write d9/db/df/f21 [627451,1496] 0 2024-04-25T15:49:38.170 INFO:tasks.workunit.client.0.smithi110.stdout:2/659: write de/d40/d4f/f47 [1354357,73472] 0 2024-04-25T15:49:38.177 INFO:tasks.workunit.client.0.smithi110.stdout:3/710: rmdir d0/d16/dcb/d50 39 2024-04-25T15:49:38.177 INFO:tasks.workunit.client.0.smithi110.stdout:0/648: dread d1/d23/f89 [0,4194304] 0 2024-04-25T15:49:38.178 INFO:tasks.workunit.client.1.smithi193.stdout:5/378: mknod d17/d18/d30/d38/d48/d4f/c82 0 2024-04-25T15:49:38.183 INFO:tasks.workunit.client.1.smithi193.stdout:7/359: sync 2024-04-25T15:49:38.183 INFO:tasks.workunit.client.1.smithi193.stdout:3/437: creat d8/d4d/f87 x:0 0 0 2024-04-25T15:49:38.183 INFO:tasks.workunit.client.1.smithi193.stdout:0/408: link d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/c7f d0/d49/d5/d7/dd/d16/d18/d28/d56/c8a 0 2024-04-25T15:49:38.184 INFO:tasks.workunit.client.1.smithi193.stdout:3/438: fdatasync d8/d13/d1f/d69/d77/d71/f86 0 2024-04-25T15:49:38.186 INFO:tasks.workunit.client.0.smithi110.stdout:4/682: dwrite d13/d24/d2d/d44/d65/f72 [0,4194304] 0 2024-04-25T15:49:38.186 INFO:tasks.workunit.client.1.smithi193.stdout:1/346: mknod dc/d4a/c78 0 2024-04-25T15:49:38.187 INFO:tasks.workunit.client.1.smithi193.stdout:1/347: fsync dc/f16 0 2024-04-25T15:49:38.188 INFO:tasks.workunit.client.0.smithi110.stdout:7/692: creat d1b/d29/d38/d55/dab/db2/dcd/de2/fe8 x:0 0 0 2024-04-25T15:49:38.189 INFO:tasks.workunit.client.1.smithi193.stdout:4/437: write db/d13/f1d [674951,36620] 0 2024-04-25T15:49:38.195 INFO:tasks.workunit.client.0.smithi110.stdout:0/649: truncate d1/f38 41371 0 2024-04-25T15:49:38.196 INFO:tasks.workunit.client.1.smithi193.stdout:4/438: dwrite db/f38 [0,4194304] 0 2024-04-25T15:49:38.196 INFO:tasks.workunit.client.1.smithi193.stdout:2/389: fdatasync d0/d37/d25/d5f/d3a/d6d/d85/f8d 0 2024-04-25T15:49:38.201 INFO:tasks.workunit.client.1.smithi193.stdout:7/360: creat d12/d31/d37/d62/d81/f89 x:0 0 0 2024-04-25T15:49:38.204 INFO:tasks.workunit.client.1.smithi193.stdout:3/439: creat d8/d13/d4b/f88 x:0 0 0 2024-04-25T15:49:38.205 INFO:tasks.workunit.client.0.smithi110.stdout:5/688: write d4/f44 [878715,107207] 0 2024-04-25T15:49:38.205 INFO:tasks.workunit.client.1.smithi193.stdout:2/390: dwrite d0/d44/d8b/f9e [0,4194304] 0 2024-04-25T15:49:38.206 INFO:tasks.workunit.client.0.smithi110.stdout:5/689: dread - d4/d1d/d1f/d35/d3d/d52/fc9 zero size 2024-04-25T15:49:38.206 INFO:tasks.workunit.client.0.smithi110.stdout:4/683: chown d13/d24/d2d/d44/d65/d99/daf/d56/f76 509725 1 2024-04-25T15:49:38.208 INFO:tasks.workunit.client.0.smithi110.stdout:5/690: chown d4/d1d/d1f/d35/d3d/dc8/ce0 479869 1 2024-04-25T15:49:38.212 INFO:tasks.workunit.client.1.smithi193.stdout:6/402: rename dd/d12/d19/d1c/d40/f25 to dd/f74 0 2024-04-25T15:49:38.214 INFO:tasks.workunit.client.0.smithi110.stdout:2/660: creat de/d40/d6e/d52/d6d/fd0 x:0 0 0 2024-04-25T15:49:38.216 INFO:tasks.workunit.client.1.smithi193.stdout:9/394: dwrite df/d2c/d2f/d4e/f19 [0,4194304] 0 2024-04-25T15:49:38.217 INFO:tasks.workunit.client.1.smithi193.stdout:4/439: symlink db/d13/d1e/d1f/d56/d69/l89 0 2024-04-25T15:49:38.218 INFO:tasks.workunit.client.1.smithi193.stdout:5/379: mkdir d17/d83 0 2024-04-25T15:49:38.219 INFO:tasks.workunit.client.1.smithi193.stdout:5/380: chown d17/d83 505595 1 2024-04-25T15:49:38.220 INFO:tasks.workunit.client.0.smithi110.stdout:4/684: symlink d13/d24/d2d/d44/d90/lec 0 2024-04-25T15:49:38.222 INFO:tasks.workunit.client.1.smithi193.stdout:7/361: chown d12/d1c/d45/f48 260218752 1 2024-04-25T15:49:38.225 INFO:tasks.workunit.client.1.smithi193.stdout:9/395: dwrite df/d2c/d2f/d4e/d1c/d26/d3f/f40 [0,4194304] 0 2024-04-25T15:49:38.226 INFO:tasks.workunit.client.0.smithi110.stdout:6/630: sync 2024-04-25T15:49:38.226 INFO:tasks.workunit.client.1.smithi193.stdout:4/440: dwrite db/d13/d1e/d1f/d22/f2a [4194304,4194304] 0 2024-04-25T15:49:38.228 INFO:tasks.workunit.client.1.smithi193.stdout:4/441: truncate db/d32/f7f 474071 0 2024-04-25T15:49:38.228 INFO:tasks.workunit.client.0.smithi110.stdout:3/711: getdents d0/d2e/d88/dbf/d92 0 2024-04-25T15:49:38.231 INFO:tasks.workunit.client.1.smithi193.stdout:2/391: mkdir d0/df/da5 0 2024-04-25T15:49:38.232 INFO:tasks.workunit.client.1.smithi193.stdout:8/365: write d2/d3/d20/f33 [4391809,40568] 0 2024-04-25T15:49:38.233 INFO:tasks.workunit.client.1.smithi193.stdout:2/392: write d0/d9/d15/d31/d50/f6b [2315989,2903] 0 2024-04-25T15:49:38.234 INFO:tasks.workunit.client.1.smithi193.stdout:8/366: truncate d2/f48 5231181 0 2024-04-25T15:49:38.236 INFO:tasks.workunit.client.0.smithi110.stdout:5/691: symlink d4/le2 0 2024-04-25T15:49:38.237 INFO:tasks.workunit.client.0.smithi110.stdout:5/692: chown d4/d1d/d1f/f42 961621150 1 2024-04-25T15:49:38.237 INFO:tasks.workunit.client.0.smithi110.stdout:5/693: readlink d4/d1d/d1f/l33 0 2024-04-25T15:49:38.238 INFO:tasks.workunit.client.0.smithi110.stdout:9/644: truncate d9/db/df/d25/f51 3096496 0 2024-04-25T15:49:38.238 INFO:tasks.workunit.client.0.smithi110.stdout:5/694: write d4/f22 [3907354,51070] 0 2024-04-25T15:49:38.240 INFO:tasks.workunit.client.0.smithi110.stdout:7/693: getdents d1b/d29/d46 0 2024-04-25T15:49:38.242 INFO:tasks.workunit.client.0.smithi110.stdout:2/661: symlink de/d40/d5b/d5c/d5f/db2/dad/dbd/dce/ld1 0 2024-04-25T15:49:38.244 INFO:tasks.workunit.client.0.smithi110.stdout:6/631: chown d1/dd/d3c/l80 13 1 2024-04-25T15:49:38.250 INFO:tasks.workunit.client.1.smithi193.stdout:9/396: read df/d2c/d2f/d33/f3c [3734665,4105] 0 2024-04-25T15:49:38.250 INFO:tasks.workunit.client.0.smithi110.stdout:0/650: dwrite d1/f19 [0,4194304] 0 2024-04-25T15:49:38.252 INFO:tasks.workunit.client.1.smithi193.stdout:4/442: unlink db/l68 0 2024-04-25T15:49:38.256 INFO:tasks.workunit.client.1.smithi193.stdout:9/397: dwrite df/d2c/d2f/d4e/d1c/d26/f52 [0,4194304] 0 2024-04-25T15:49:38.257 INFO:tasks.workunit.client.0.smithi110.stdout:9/645: fsync d9/f5f 0 2024-04-25T15:49:38.260 INFO:tasks.workunit.client.1.smithi193.stdout:0/409: truncate d0/d49/d5/d7/dd/d16/d18/f2a 1173824 0 2024-04-25T15:49:38.270 INFO:tasks.workunit.client.1.smithi193.stdout:4/443: dread fa [0,4194304] 0 2024-04-25T15:49:38.271 INFO:tasks.workunit.client.1.smithi193.stdout:6/403: chown dd/d12/d19/d1c/d40/d3c/f68 8610 1 2024-04-25T15:49:38.272 INFO:tasks.workunit.client.0.smithi110.stdout:6/632: rename d1/d12/f5b to d1/dd/d3c/d91/d8d/dce/fde 0 2024-04-25T15:49:38.272 INFO:tasks.workunit.client.1.smithi193.stdout:6/404: write dd/d12/f4d [2250801,79694] 0 2024-04-25T15:49:38.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/440: write d8/d13/d1f/f22 [705029,84890] 0 2024-04-25T15:49:38.274 INFO:tasks.workunit.client.0.smithi110.stdout:5/695: dread d4/d2c/f27 [0,4194304] 0 2024-04-25T15:49:38.276 INFO:tasks.workunit.client.0.smithi110.stdout:0/651: creat d1/d23/d30/d33/d5b/d83/fe9 x:0 0 0 2024-04-25T15:49:38.277 INFO:tasks.workunit.client.0.smithi110.stdout:4/685: dwrite d13/d24/d2d/d55/faa [0,4194304] 0 2024-04-25T15:49:38.277 INFO:tasks.workunit.client.0.smithi110.stdout:6/633: write d1/db/d22/d33/d43/f60 [3961057,65848] 0 2024-04-25T15:49:38.277 INFO:tasks.workunit.client.1.smithi193.stdout:2/393: unlink d0/d42/l62 0 2024-04-25T15:49:38.278 INFO:tasks.workunit.client.1.smithi193.stdout:4/444: dwrite db/d13/d1e/d1f/d22/f2c [4194304,4194304] 0 2024-04-25T15:49:38.282 INFO:tasks.workunit.client.0.smithi110.stdout:3/712: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:38.283 INFO:tasks.workunit.client.1.smithi193.stdout:9/398: creat df/d2c/d2f/d33/f6f x:0 0 0 2024-04-25T15:49:38.285 INFO:tasks.workunit.client.1.smithi193.stdout:9/399: dread - df/d2c/d2f/d4e/d1c/d26/d3f/f43 zero size 2024-04-25T15:49:38.285 INFO:tasks.workunit.client.1.smithi193.stdout:8/367: mkdir d2/d36/d72 0 2024-04-25T15:49:38.286 INFO:tasks.workunit.client.0.smithi110.stdout:2/662: unlink de/d40/d5b/fca 0 2024-04-25T15:49:38.288 INFO:tasks.workunit.client.1.smithi193.stdout:5/381: rename d17/d18/d30/d38/f44 to d17/d18/f84 0 2024-04-25T15:49:38.293 INFO:tasks.workunit.client.1.smithi193.stdout:1/348: sync 2024-04-25T15:49:38.294 INFO:tasks.workunit.client.1.smithi193.stdout:0/410: symlink d0/d49/d5/d7/dd/d46/l8b 0 2024-04-25T15:49:38.298 INFO:tasks.workunit.client.1.smithi193.stdout:4/445: mkdir db/d13/d1e/d1f/d56/d69/d70/d8a 0 2024-04-25T15:49:38.300 INFO:tasks.workunit.client.0.smithi110.stdout:7/694: write d1b/d29/d38/d55/d95/dcb/fe4 [348756,55163] 0 2024-04-25T15:49:38.304 INFO:tasks.workunit.client.1.smithi193.stdout:6/405: dread dd/fe [4194304,4194304] 0 2024-04-25T15:49:38.304 INFO:tasks.workunit.client.1.smithi193.stdout:7/362: rename d12/d31/d37/d62/d6e/f72 to d12/d1c/f8a 0 2024-04-25T15:49:38.306 INFO:tasks.workunit.client.1.smithi193.stdout:7/363: dread - d12/d1c/d45/f5d zero size 2024-04-25T15:49:38.306 INFO:tasks.workunit.client.1.smithi193.stdout:2/394: dread d0/d9/d15/d31/f91 [0,4194304] 0 2024-04-25T15:49:38.307 INFO:tasks.workunit.client.1.smithi193.stdout:8/368: mknod d2/d3/c73 0 2024-04-25T15:49:38.309 INFO:tasks.workunit.client.1.smithi193.stdout:9/400: link df/f57 df/d2c/d2f/d4e/d1c/d26/d3f/d41/f70 0 2024-04-25T15:49:38.310 INFO:tasks.workunit.client.1.smithi193.stdout:6/406: readlink dd/d12/d2d/l5f 0 2024-04-25T15:49:38.311 INFO:tasks.workunit.client.1.smithi193.stdout:3/441: rename d8/d30/d45/c4c to d8/d13/d4b/c89 0 2024-04-25T15:49:38.312 INFO:tasks.workunit.client.1.smithi193.stdout:3/442: dread - d8/d13/d14/d51/f5f zero size 2024-04-25T15:49:38.313 INFO:tasks.workunit.client.1.smithi193.stdout:4/446: mknod db/d4b/d51/d54/d6f/c8b 0 2024-04-25T15:49:38.315 INFO:tasks.workunit.client.1.smithi193.stdout:2/395: fsync d0/df/f10 0 2024-04-25T15:49:38.317 INFO:tasks.workunit.client.1.smithi193.stdout:8/369: mknod d2/d3/d20/d45/d4a/d53/c74 0 2024-04-25T15:49:38.318 INFO:tasks.workunit.client.1.smithi193.stdout:9/401: mknod df/d2c/d2f/d4e/d1c/d30/c71 0 2024-04-25T15:49:38.319 INFO:tasks.workunit.client.1.smithi193.stdout:4/447: dwrite db/d13/d1e/d1f/d22/f7a [0,4194304] 0 2024-04-25T15:49:38.322 INFO:tasks.workunit.client.1.smithi193.stdout:5/382: rename d17/d18/d30/d34/d37 to d17/d18/d75/d85 0 2024-04-25T15:49:38.323 INFO:tasks.workunit.client.0.smithi110.stdout:1/827: sync 2024-04-25T15:49:38.323 INFO:tasks.workunit.client.0.smithi110.stdout:8/777: sync 2024-04-25T15:49:38.324 INFO:tasks.workunit.client.1.smithi193.stdout:5/383: write d17/d18/d30/d3b/f60 [428447,116314] 0 2024-04-25T15:49:38.331 INFO:tasks.workunit.client.1.smithi193.stdout:2/396: fsync d0/d37/d25/d5f/d3a/f58 0 2024-04-25T15:49:38.332 INFO:tasks.workunit.client.1.smithi193.stdout:2/397: write d0/d42/d51/d4f/f69 [1694750,17564] 0 2024-04-25T15:49:38.333 INFO:tasks.workunit.client.1.smithi193.stdout:8/370: stat d2/d3/d3a/d6d/l66 0 2024-04-25T15:49:38.335 INFO:tasks.workunit.client.1.smithi193.stdout:6/407: symlink dd/l75 0 2024-04-25T15:49:38.338 INFO:tasks.workunit.client.1.smithi193.stdout:9/402: symlink df/d2c/d2f/d4e/d1c/d26/d3f/l72 0 2024-04-25T15:49:38.342 INFO:tasks.workunit.client.1.smithi193.stdout:7/364: write d12/d1c/f1d [94134,55830] 0 2024-04-25T15:49:38.345 INFO:tasks.workunit.client.1.smithi193.stdout:2/398: dread d0/d37/d25/d5f/d3a/f58 [0,4194304] 0 2024-04-25T15:49:38.346 INFO:tasks.workunit.client.1.smithi193.stdout:2/399: chown d0/d37/d25/d5f/d3a/d6d/d6e 140365 1 2024-04-25T15:49:38.350 INFO:tasks.workunit.client.1.smithi193.stdout:3/443: write d8/d13/d4b/f5d [1300566,41647] 0 2024-04-25T15:49:38.351 INFO:tasks.workunit.client.1.smithi193.stdout:6/408: truncate dd/f24 4426095 0 2024-04-25T15:49:38.351 INFO:tasks.workunit.client.1.smithi193.stdout:2/400: dwrite d0/df/f26 [0,4194304] 0 2024-04-25T15:49:38.354 INFO:tasks.workunit.client.1.smithi193.stdout:4/448: creat db/d13/d1e/d1f/d56/d69/d70/d8a/f8c x:0 0 0 2024-04-25T15:49:38.356 INFO:tasks.workunit.client.1.smithi193.stdout:4/449: dread fa [0,4194304] 0 2024-04-25T15:49:38.356 INFO:tasks.workunit.client.1.smithi193.stdout:7/365: mknod d12/d13/c8b 0 2024-04-25T15:49:38.358 INFO:tasks.workunit.client.1.smithi193.stdout:1/349: rename dc/f2c to dc/d3c/f79 0 2024-04-25T15:49:38.363 INFO:tasks.workunit.client.1.smithi193.stdout:3/444: mknod d8/d13/d14/d27/d54/c8a 0 2024-04-25T15:49:38.363 INFO:tasks.workunit.client.1.smithi193.stdout:3/445: chown d8/d13/d14/d27/d54 381 1 2024-04-25T15:49:38.364 INFO:tasks.workunit.client.1.smithi193.stdout:3/446: chown d8/d13/d1f/d69/d77/f3a 4 1 2024-04-25T15:49:38.364 INFO:tasks.workunit.client.1.smithi193.stdout:9/403: creat df/d2c/d2f/d4e/d1c/d30/d60/f73 x:0 0 0 2024-04-25T15:49:38.366 INFO:tasks.workunit.client.1.smithi193.stdout:6/409: mknod dd/d2f/d57/c76 0 2024-04-25T15:49:38.369 INFO:tasks.workunit.client.1.smithi193.stdout:2/401: write d0/d42/d51/f46 [2628629,67967] 0 2024-04-25T15:49:38.371 INFO:tasks.workunit.client.1.smithi193.stdout:4/450: rmdir db/d4b/d51/d54/d6f 39 2024-04-25T15:49:38.373 INFO:tasks.workunit.client.1.smithi193.stdout:6/410: dwrite dd/d2f/f72 [0,4194304] 0 2024-04-25T15:49:38.377 INFO:tasks.workunit.client.1.smithi193.stdout:5/384: creat d17/f86 x:0 0 0 2024-04-25T15:49:38.379 INFO:tasks.workunit.client.1.smithi193.stdout:8/371: creat d2/f75 x:0 0 0 2024-04-25T15:49:38.382 INFO:tasks.workunit.client.0.smithi110.stdout:9/646: rename d9/db/df/l3b to d9/d2d/lcf 0 2024-04-25T15:49:38.384 INFO:tasks.workunit.client.1.smithi193.stdout:9/404: rename l2 to df/d2c/d2f/d4e/d1c/d26/d3f/d41/l74 0 2024-04-25T15:49:38.393 INFO:tasks.workunit.client.1.smithi193.stdout:8/372: dread d2/d3/d20/d45/f5a [0,4194304] 0 2024-04-25T15:49:38.394 INFO:tasks.workunit.client.1.smithi193.stdout:8/373: readlink d2/d3/d27/l6a 0 2024-04-25T15:49:38.394 INFO:tasks.workunit.client.1.smithi193.stdout:8/374: readlink d2/d3/d3a/d6d/l30 0 2024-04-25T15:49:38.395 INFO:tasks.workunit.client.0.smithi110.stdout:4/686: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded 0 2024-04-25T15:49:38.396 INFO:tasks.workunit.client.1.smithi193.stdout:8/375: write d2/d3/d3a/d6d/f2b [4249971,29098] 0 2024-04-25T15:49:38.396 INFO:tasks.workunit.client.1.smithi193.stdout:8/376: read - d2/f6e zero size 2024-04-25T15:49:38.397 INFO:tasks.workunit.client.1.smithi193.stdout:6/411: dread dd/d12/f1e [0,4194304] 0 2024-04-25T15:49:38.397 INFO:tasks.workunit.client.1.smithi193.stdout:5/385: mknod d17/d18/d30/d38/d71/c87 0 2024-04-25T15:49:38.398 INFO:tasks.workunit.client.0.smithi110.stdout:2/663: mknod de/d40/d6e/d50/db7/cd2 0 2024-04-25T15:49:38.401 INFO:tasks.workunit.client.1.smithi193.stdout:9/405: creat df/d2c/d34/f75 x:0 0 0 2024-04-25T15:49:38.401 INFO:tasks.workunit.client.0.smithi110.stdout:7/695: fdatasync d1b/d29/d38/d55/dab/db2/fc1 0 2024-04-25T15:49:38.402 INFO:tasks.workunit.client.1.smithi193.stdout:9/406: chown df/d2c/d2f/d4e/d1c/d26/d3f/d41/f59 6675 1 2024-04-25T15:49:38.403 INFO:tasks.workunit.client.0.smithi110.stdout:5/696: dread d4/d1d/f34 [0,4194304] 0 2024-04-25T15:49:38.403 INFO:tasks.workunit.client.1.smithi193.stdout:1/350: stat dc/d4a/d50/l56 0 2024-04-25T15:49:38.405 INFO:tasks.workunit.client.1.smithi193.stdout:0/411: sync 2024-04-25T15:49:38.406 INFO:tasks.workunit.client.1.smithi193.stdout:0/412: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53 778 1 2024-04-25T15:49:38.406 INFO:tasks.workunit.client.0.smithi110.stdout:1/828: mknod d0/c10c 0 2024-04-25T15:49:38.408 INFO:tasks.workunit.client.0.smithi110.stdout:1/829: chown d0/d14/d81/df8/d33/f48 29701 1 2024-04-25T15:49:38.408 INFO:tasks.workunit.client.1.smithi193.stdout:7/366: link d12/f30 d12/d31/d37/d62/d6e/f8c 0 2024-04-25T15:49:38.409 INFO:tasks.workunit.client.1.smithi193.stdout:7/367: write d12/d13/f14 [3575140,93769] 0 2024-04-25T15:49:38.411 INFO:tasks.workunit.client.0.smithi110.stdout:5/697: dwrite d4/fd [4194304,4194304] 0 2024-04-25T15:49:38.411 INFO:tasks.workunit.client.0.smithi110.stdout:5/698: stat d4/f22 0 2024-04-25T15:49:38.414 INFO:tasks.workunit.client.1.smithi193.stdout:8/377: creat d2/d3/d3a/f76 x:0 0 0 2024-04-25T15:49:38.415 INFO:tasks.workunit.client.1.smithi193.stdout:8/378: dread - d2/f6e zero size 2024-04-25T15:49:38.415 INFO:tasks.workunit.client.0.smithi110.stdout:0/652: symlink d1/d23/d30/d33/d5b/d83/de1/lea 0 2024-04-25T15:49:38.416 INFO:tasks.workunit.client.1.smithi193.stdout:2/402: write d0/d42/d51/d4f/d65/d7e/f95 [3424353,99629] 0 2024-04-25T15:49:38.418 INFO:tasks.workunit.client.0.smithi110.stdout:4/687: creat d13/d24/d2d/d44/d65/d99/fee x:0 0 0 2024-04-25T15:49:38.421 INFO:tasks.workunit.client.0.smithi110.stdout:2/664: rmdir de/d40/d5b/d5c/d5f/db2/da9 39 2024-04-25T15:49:38.421 INFO:tasks.workunit.client.0.smithi110.stdout:2/665: chown de/d40/d5b 1 1 2024-04-25T15:49:38.426 INFO:tasks.workunit.client.0.smithi110.stdout:7/696: creat d1b/d29/d38/d8b/d89/fe9 x:0 0 0 2024-04-25T15:49:38.427 INFO:tasks.workunit.client.1.smithi193.stdout:1/351: rmdir dc/d4a/d50/d5c/d63 39 2024-04-25T15:49:38.428 INFO:tasks.workunit.client.1.smithi193.stdout:1/352: write dc/d4a/d50/d5c/d5d/f3a [1996682,66051] 0 2024-04-25T15:49:38.431 INFO:tasks.workunit.client.1.smithi193.stdout:1/353: write dc/d4a/d50/d5c/d5d/d3d/f43 [1554200,106435] 0 2024-04-25T15:49:38.432 INFO:tasks.workunit.client.0.smithi110.stdout:3/713: rename d0/d2e/lb6 to d0/d16/dcb/d2f/d4c/d63/lf6 0 2024-04-25T15:49:38.433 INFO:tasks.workunit.client.0.smithi110.stdout:8/778: dwrite f5 [0,4194304] 0 2024-04-25T15:49:38.433 INFO:tasks.workunit.client.0.smithi110.stdout:3/714: chown d0/d7/f2a 225176207 1 2024-04-25T15:49:38.434 INFO:tasks.workunit.client.1.smithi193.stdout:4/451: creat db/f8d x:0 0 0 2024-04-25T15:49:38.435 INFO:tasks.workunit.client.0.smithi110.stdout:3/715: stat d0/d7/d40/l42 0 2024-04-25T15:49:38.438 INFO:tasks.workunit.client.0.smithi110.stdout:5/699: dread d4/d1d/d1f/d35/d3d/d52/f55 [0,4194304] 0 2024-04-25T15:49:38.440 INFO:tasks.workunit.client.0.smithi110.stdout:6/634: creat d1/fdf x:0 0 0 2024-04-25T15:49:38.442 INFO:tasks.workunit.client.1.smithi193.stdout:2/403: fsync d0/df/f10 0 2024-04-25T15:49:38.443 INFO:tasks.workunit.client.1.smithi193.stdout:2/404: chown d0/d42/d51 851294275 1 2024-04-25T15:49:38.443 INFO:tasks.workunit.client.0.smithi110.stdout:0/653: dread d1/d23/d30/d33/d5b/f9b [0,4194304] 0 2024-04-25T15:49:38.443 INFO:tasks.workunit.client.1.smithi193.stdout:9/407: dread df/d11/f2a [0,4194304] 0 2024-04-25T15:49:38.444 INFO:tasks.workunit.client.0.smithi110.stdout:1/830: dread d0/d23/f55 [0,4194304] 0 2024-04-25T15:49:38.445 INFO:tasks.workunit.client.1.smithi193.stdout:6/412: rename dd/d12/d19/d1c/d40/l3b to dd/d12/d19/d1c/d40/d3c/d3f/l77 0 2024-04-25T15:49:38.446 INFO:tasks.workunit.client.1.smithi193.stdout:6/413: stat dd/d12/d19/f66 0 2024-04-25T15:49:38.447 INFO:tasks.workunit.client.0.smithi110.stdout:4/688: stat d13/d24/d26/c6e 0 2024-04-25T15:49:38.447 INFO:tasks.workunit.client.0.smithi110.stdout:4/689: readlink d13/la1 0 2024-04-25T15:49:38.448 INFO:tasks.workunit.client.1.smithi193.stdout:3/447: getdents d8/d13/d14/d27 0 2024-04-25T15:49:38.451 INFO:tasks.workunit.client.0.smithi110.stdout:7/697: dread - d1b/d29/d38/d55/d95/f9e zero size 2024-04-25T15:49:38.452 INFO:tasks.workunit.client.1.smithi193.stdout:3/448: dread d8/d13/d4b/f57 [0,4194304] 0 2024-04-25T15:49:38.452 INFO:tasks.workunit.client.0.smithi110.stdout:1/831: dwrite d0/d23/db1/f10a [0,4194304] 0 2024-04-25T15:49:38.453 INFO:tasks.workunit.client.1.smithi193.stdout:0/413: mknod d0/d49/c8c 0 2024-04-25T15:49:38.454 INFO:tasks.workunit.client.0.smithi110.stdout:3/716: rmdir d0/d2e/d88/dbf/d99 39 2024-04-25T15:49:38.455 INFO:tasks.workunit.client.0.smithi110.stdout:9/647: dwrite d9/db/d7d/fb6 [0,4194304] 0 2024-04-25T15:49:38.458 INFO:tasks.workunit.client.1.smithi193.stdout:5/386: link fe d17/d18/d75/f88 0 2024-04-25T15:49:38.458 INFO:tasks.workunit.client.1.smithi193.stdout:4/452: chown db/d4b/d51/d54/d77/l81 1 1 2024-04-25T15:49:38.459 INFO:tasks.workunit.client.1.smithi193.stdout:0/414: dwrite d0/d49/d5/d7/dd/f5c [0,4194304] 0 2024-04-25T15:49:38.460 INFO:tasks.workunit.client.0.smithi110.stdout:8/779: unlink d14/d21/d26/d6a/d7e/da6/de8/c4e 0 2024-04-25T15:49:38.461 INFO:tasks.workunit.client.1.smithi193.stdout:0/415: read - d0/d49/d5/d61/f74 zero size 2024-04-25T15:49:38.461 INFO:tasks.workunit.client.1.smithi193.stdout:4/453: write db/d13/d1e/d1f/d22/f2c [1968170,12438] 0 2024-04-25T15:49:38.463 INFO:tasks.workunit.client.0.smithi110.stdout:6/635: symlink d1/dd/d3c/d91/le0 0 2024-04-25T15:49:38.465 INFO:tasks.workunit.client.0.smithi110.stdout:0/654: creat d1/d23/d30/d33/d5b/d78/da4/feb x:0 0 0 2024-04-25T15:49:38.465 INFO:tasks.workunit.client.1.smithi193.stdout:2/405: mknod d0/d42/d51/d4f/d65/ca6 0 2024-04-25T15:49:38.468 INFO:tasks.workunit.client.1.smithi193.stdout:9/408: fdatasync df/d2c/d2f/d4e/d1c/f3d 0 2024-04-25T15:49:38.468 INFO:tasks.workunit.client.0.smithi110.stdout:5/700: dread d4/d1d/d1f/d35/d3d/d52/f7a [0,4194304] 0 2024-04-25T15:49:38.470 INFO:tasks.workunit.client.1.smithi193.stdout:5/387: dwrite d17/d18/f20 [4194304,4194304] 0 2024-04-25T15:49:38.471 INFO:tasks.workunit.client.1.smithi193.stdout:7/368: rename d12/d31/d37/d62/d81 to d12/d1c/d63/d70/d68/d8d 0 2024-04-25T15:49:38.472 INFO:tasks.workunit.client.1.smithi193.stdout:6/414: symlink dd/d12/d19/d1c/l78 0 2024-04-25T15:49:38.477 INFO:tasks.workunit.client.0.smithi110.stdout:5/701: dwrite d4/d2c/d2a/f5c [0,4194304] 0 2024-04-25T15:49:38.479 INFO:tasks.workunit.client.0.smithi110.stdout:5/702: fsync d4/f25 0 2024-04-25T15:49:38.480 INFO:tasks.workunit.client.1.smithi193.stdout:3/449: dread d8/d13/d14/f3e [4194304,4194304] 0 2024-04-25T15:49:38.483 INFO:tasks.workunit.client.1.smithi193.stdout:3/450: dread d8/d13/f60 [0,4194304] 0 2024-04-25T15:49:38.483 INFO:tasks.workunit.client.0.smithi110.stdout:5/703: read d4/d1d/d1f/f6e [3866234,105173] 0 2024-04-25T15:49:38.485 INFO:tasks.workunit.client.0.smithi110.stdout:0/655: dread d1/d23/d30/d33/d5b/d78/f7d [4194304,4194304] 0 2024-04-25T15:49:38.491 INFO:tasks.workunit.client.1.smithi193.stdout:4/454: rmdir db/d4b/d51/d54/d77 39 2024-04-25T15:49:38.494 INFO:tasks.workunit.client.1.smithi193.stdout:0/416: creat d0/d49/d5/d7/dd/d16/d26/f8d x:0 0 0 2024-04-25T15:49:38.498 INFO:tasks.workunit.client.0.smithi110.stdout:4/690: write d13/d24/d26/f49 [3492231,55691] 0 2024-04-25T15:49:38.498 INFO:tasks.workunit.client.1.smithi193.stdout:4/455: dwrite db/d13/d1e/d1f/d22/f2c [0,4194304] 0 2024-04-25T15:49:38.498 INFO:tasks.workunit.client.1.smithi193.stdout:4/456: chown db/d13/d1e/c4e 1343 1 2024-04-25T15:49:38.499 INFO:tasks.workunit.client.1.smithi193.stdout:8/379: rename d2/d3/c73 to d2/d3/d3a/d6d/c77 0 2024-04-25T15:49:38.502 INFO:tasks.workunit.client.1.smithi193.stdout:7/369: mkdir d12/d1c/d63/d70/d68/d8e 0 2024-04-25T15:49:38.504 INFO:tasks.workunit.client.1.smithi193.stdout:6/415: rmdir dd/d12/d2d 39 2024-04-25T15:49:38.505 INFO:tasks.workunit.client.0.smithi110.stdout:9/648: write d9/f96 [882647,18776] 0 2024-04-25T15:49:38.506 INFO:tasks.workunit.client.0.smithi110.stdout:9/649: stat d9/db/d58 0 2024-04-25T15:49:38.507 INFO:tasks.workunit.client.0.smithi110.stdout:3/717: fdatasync d0/d16/d8c/fea 0 2024-04-25T15:49:38.508 INFO:tasks.workunit.client.0.smithi110.stdout:3/718: chown d0/d16/dcb/db0 647703 1 2024-04-25T15:49:38.510 INFO:tasks.workunit.client.1.smithi193.stdout:2/406: write d0/fc [1570097,58827] 0 2024-04-25T15:49:38.513 INFO:tasks.workunit.client.0.smithi110.stdout:5/704: creat d4/d1d/d1f/d35/d7f/d92/d9b/dcb/fe3 x:0 0 0 2024-04-25T15:49:38.514 INFO:tasks.workunit.client.1.smithi193.stdout:3/451: unlink d8/f10 0 2024-04-25T15:49:38.520 INFO:tasks.workunit.client.0.smithi110.stdout:7/698: truncate d1b/d29/d38/d8b/d89/fcc 3952566 0 2024-04-25T15:49:38.522 INFO:tasks.workunit.client.0.smithi110.stdout:8/780: mkdir d14/d32/d105/d108 0 2024-04-25T15:49:38.524 INFO:tasks.workunit.client.0.smithi110.stdout:1/832: truncate d0/d14/d5f/d7d/f67 2899978 0 2024-04-25T15:49:38.525 INFO:tasks.workunit.client.1.smithi193.stdout:4/457: mknod db/d13/d1e/d1f/c8e 0 2024-04-25T15:49:38.525 INFO:tasks.workunit.client.0.smithi110.stdout:2/666: getdents de/d40/d6e/d50/db7 0 2024-04-25T15:49:38.527 INFO:tasks.workunit.client.1.smithi193.stdout:0/417: rename d0/d49/d5/d7/dd/d16/f33 to d0/d49/d5/d7/dd/d16/d1a/f8e 0 2024-04-25T15:49:38.530 INFO:tasks.workunit.client.0.smithi110.stdout:4/691: unlink d13/d24/d2d/d36/f5a 0 2024-04-25T15:49:38.530 INFO:tasks.workunit.client.1.smithi193.stdout:7/370: mkdir d12/d1c/d63/d70/d68/d8f 0 2024-04-25T15:49:38.532 INFO:tasks.workunit.client.1.smithi193.stdout:6/416: stat dd/d12/d2d 0 2024-04-25T15:49:38.533 INFO:tasks.workunit.client.1.smithi193.stdout:6/417: chown dd/d2f/f56 56859551 1 2024-04-25T15:49:38.536 INFO:tasks.workunit.client.0.smithi110.stdout:3/719: fsync d0/d2e/f76 0 2024-04-25T15:49:38.541 INFO:tasks.workunit.client.1.smithi193.stdout:8/380: write d2/d3/d20/f22 [260733,37181] 0 2024-04-25T15:49:38.544 INFO:tasks.workunit.client.1.smithi193.stdout:3/452: mkdir d8/d13/d4b/d8b 0 2024-04-25T15:49:38.545 INFO:tasks.workunit.client.0.smithi110.stdout:7/699: truncate f19 2156838 0 2024-04-25T15:49:38.546 INFO:tasks.workunit.client.0.smithi110.stdout:5/705: dread d4/d1d/d1f/d35/d3d/d52/f6f [0,4194304] 0 2024-04-25T15:49:38.546 INFO:tasks.workunit.client.1.smithi193.stdout:8/381: dwrite d2/d36/f40 [4194304,4194304] 0 2024-04-25T15:49:38.546 INFO:tasks.workunit.client.0.smithi110.stdout:7/700: dread - d1b/d29/d38/d55/d95/f9e zero size 2024-04-25T15:49:38.548 INFO:tasks.workunit.client.0.smithi110.stdout:8/781: truncate d14/fb2 773330 0 2024-04-25T15:49:38.548 INFO:tasks.workunit.client.1.smithi193.stdout:1/354: getdents dc/d4a/d50/d5c/d5d/d3d/d55 0 2024-04-25T15:49:38.550 INFO:tasks.workunit.client.1.smithi193.stdout:5/388: creat d17/f89 x:0 0 0 2024-04-25T15:49:38.550 INFO:tasks.workunit.client.1.smithi193.stdout:2/407: write d0/f29 [841574,71823] 0 2024-04-25T15:49:38.551 INFO:tasks.workunit.client.1.smithi193.stdout:5/389: chown d17/l1c 74410250 1 2024-04-25T15:49:38.553 INFO:tasks.workunit.client.1.smithi193.stdout:0/418: creat d0/d49/d5/d7/dd/d46/d48/d62/f8f x:0 0 0 2024-04-25T15:49:38.555 INFO:tasks.workunit.client.1.smithi193.stdout:7/371: mknod d12/d13/c90 0 2024-04-25T15:49:38.556 INFO:tasks.workunit.client.1.smithi193.stdout:7/372: chown d12/d1c/l7d 71 1 2024-04-25T15:49:38.556 INFO:tasks.workunit.client.0.smithi110.stdout:4/692: mkdir d13/d24/d2d/d5e/def 0 2024-04-25T15:49:38.557 INFO:tasks.workunit.client.1.smithi193.stdout:7/373: write d12/d31/d37/d62/d6e/d79/f3f [310069,49496] 0 2024-04-25T15:49:38.561 INFO:tasks.workunit.client.1.smithi193.stdout:1/355: mkdir dc/d4a/d4d/d7a 0 2024-04-25T15:49:38.562 INFO:tasks.workunit.client.1.smithi193.stdout:2/408: rmdir d0/d9/d15 39 2024-04-25T15:49:38.563 INFO:tasks.workunit.client.1.smithi193.stdout:5/390: creat d17/d18/d30/f8a x:0 0 0 2024-04-25T15:49:38.564 INFO:tasks.workunit.client.1.smithi193.stdout:5/391: dread - d17/d18/d75/d85/f73 zero size 2024-04-25T15:49:38.564 INFO:tasks.workunit.client.0.smithi110.stdout:5/706: symlink d4/d15/d51/le4 0 2024-04-25T15:49:38.565 INFO:tasks.workunit.client.1.smithi193.stdout:0/419: rmdir d0/d49/d5/d7/dd/d16/d26/d2e 39 2024-04-25T15:49:38.566 INFO:tasks.workunit.client.1.smithi193.stdout:0/420: write d0/d49/d5/d7/dd/f6e [11325,40572] 0 2024-04-25T15:49:38.568 INFO:tasks.workunit.client.1.smithi193.stdout:5/392: dread d17/d18/d30/d3b/f60 [0,4194304] 0 2024-04-25T15:49:38.600 INFO:tasks.workunit.client.0.smithi110.stdout:7/701: symlink d1b/d29/d38/d55/d7f/lea 0 2024-04-25T15:49:38.601 INFO:tasks.workunit.client.0.smithi110.stdout:7/702: write d1b/d29/d38/d55/d95/dcb/fe4 [507692,63329] 0 2024-04-25T15:49:38.603 INFO:tasks.workunit.client.1.smithi193.stdout:9/409: sync 2024-04-25T15:49:38.610 INFO:tasks.workunit.client.1.smithi193.stdout:9/410: dwrite df/d2c/d34/f75 [0,4194304] 0 2024-04-25T15:49:38.637 INFO:tasks.workunit.client.1.smithi193.stdout:4/458: sync 2024-04-25T15:49:38.638 INFO:tasks.workunit.client.1.smithi193.stdout:4/459: write db/d13/d1e/d1f/d22/f2a [4662681,11211] 0 2024-04-25T15:49:38.640 INFO:tasks.workunit.client.1.smithi193.stdout:4/460: write db/d13/d1e/d1f/d22/f5a [1173326,26488] 0 2024-04-25T15:49:38.641 INFO:tasks.workunit.client.1.smithi193.stdout:4/461: write db/d13/d1e/d1f/d22/f5a [31403,37682] 0 2024-04-25T15:49:38.648 INFO:tasks.workunit.client.1.smithi193.stdout:6/418: fsync dd/f13 0 2024-04-25T15:49:38.652 INFO:tasks.workunit.client.1.smithi193.stdout:7/374: mknod d12/d31/d37/c91 0 2024-04-25T15:49:38.653 INFO:tasks.workunit.client.0.smithi110.stdout:8/782: dread d14/f94 [0,4194304] 0 2024-04-25T15:49:38.654 INFO:tasks.workunit.client.1.smithi193.stdout:3/453: creat d8/d13/d4b/d83/f8c x:0 0 0 2024-04-25T15:49:38.656 INFO:tasks.workunit.client.0.smithi110.stdout:6/636: rename d1/db/d24/d2d/fb0 to d1/d12/db8/fe1 0 2024-04-25T15:49:38.657 INFO:tasks.workunit.client.0.smithi110.stdout:6/637: chown d1/f5e 3768519 1 2024-04-25T15:49:38.658 INFO:tasks.workunit.client.1.smithi193.stdout:8/382: rename d2/d3/d3a/d6d/c77 to d2/d3/d27/d6f/c78 0 2024-04-25T15:49:38.663 INFO:tasks.workunit.client.1.smithi193.stdout:1/356: fdatasync dc/f36 0 2024-04-25T15:49:38.664 INFO:tasks.workunit.client.1.smithi193.stdout:8/383: dwrite d2/d3/d20/d45/d4a/f4e [0,4194304] 0 2024-04-25T15:49:38.665 INFO:tasks.workunit.client.0.smithi110.stdout:9/650: link d9/db/df/c14 d9/d13/d5b/d61/d67/cd0 0 2024-04-25T15:49:38.665 INFO:tasks.workunit.client.1.smithi193.stdout:8/384: stat d2/d3/d27/d6f 0 2024-04-25T15:49:38.666 INFO:tasks.workunit.client.0.smithi110.stdout:9/651: write d9/d44/d5e/fc1 [931093,69678] 0 2024-04-25T15:49:38.669 INFO:tasks.workunit.client.0.smithi110.stdout:9/652: dread d9/f96 [0,4194304] 0 2024-04-25T15:49:38.669 INFO:tasks.workunit.client.0.smithi110.stdout:3/720: mknod d0/d2e/d88/dbf/d99/cf7 0 2024-04-25T15:49:38.672 INFO:tasks.workunit.client.1.smithi193.stdout:8/385: dread d2/d3/d3a/f4c [0,4194304] 0 2024-04-25T15:49:38.672 INFO:tasks.workunit.client.0.smithi110.stdout:0/656: getdents d1/d23/d81 0 2024-04-25T15:49:38.674 INFO:tasks.workunit.client.0.smithi110.stdout:5/707: symlink d4/d1d/d85/le5 0 2024-04-25T15:49:38.677 INFO:tasks.workunit.client.0.smithi110.stdout:5/708: dread d4/d1d/f34 [0,4194304] 0 2024-04-25T15:49:38.697 INFO:tasks.workunit.client.0.smithi110.stdout:4/693: rmdir d13/d24/d2d/d44/d65/d99/daf/d56 39 2024-04-25T15:49:38.699 INFO:tasks.workunit.client.1.smithi193.stdout:4/462: unlink db/d32/d45/f4f 0 2024-04-25T15:49:38.703 INFO:tasks.workunit.client.0.smithi110.stdout:6/638: creat d1/db/d22/d33/d43/fe2 x:0 0 0 2024-04-25T15:49:38.705 INFO:tasks.workunit.client.0.smithi110.stdout:1/833: link d0/c16 d0/d14/d5f/d7d/d72/d61/c10d 0 2024-04-25T15:49:38.709 INFO:tasks.workunit.client.1.smithi193.stdout:4/463: dwrite db/d32/f7f [0,4194304] 0 2024-04-25T15:49:38.710 INFO:tasks.workunit.client.1.smithi193.stdout:4/464: chown db/d32/d88 9 1 2024-04-25T15:49:38.711 INFO:tasks.workunit.client.0.smithi110.stdout:0/657: unlink d1/d20/d69/f91 0 2024-04-25T15:49:38.711 INFO:tasks.workunit.client.1.smithi193.stdout:4/465: write db/f87 [331698,95920] 0 2024-04-25T15:49:38.714 INFO:tasks.workunit.client.1.smithi193.stdout:4/466: chown db/d13/d1e/d1f/c3c 244 1 2024-04-25T15:49:38.722 INFO:tasks.workunit.client.1.smithi193.stdout:4/467: dwrite db/d13/d1e/d1f/d22/f5d [8388608,4194304] 0 2024-04-25T15:49:38.722 INFO:tasks.workunit.client.0.smithi110.stdout:8/783: symlink d14/d21/d26/d6a/d7e/da6/de8/d71/dae/l109 0 2024-04-25T15:49:38.725 INFO:tasks.workunit.client.1.smithi193.stdout:3/454: dread d8/d13/d1f/d69/d77/f53 [0,4194304] 0 2024-04-25T15:49:38.726 INFO:tasks.workunit.client.1.smithi193.stdout:3/455: stat d8/d13/d1f/f78 0 2024-04-25T15:49:38.727 INFO:tasks.workunit.client.1.smithi193.stdout:3/456: truncate d8/d30/d72/f7e 804400 0 2024-04-25T15:49:38.728 INFO:tasks.workunit.client.1.smithi193.stdout:3/457: write d8/d30/f47 [3648214,89689] 0 2024-04-25T15:49:38.730 INFO:tasks.workunit.client.0.smithi110.stdout:2/667: rename de/d40/lb0 to de/d40/d5b/d5c/d5f/db2/da9/ld3 0 2024-04-25T15:49:38.730 INFO:tasks.workunit.client.0.smithi110.stdout:2/668: readlink de/d40/d6e/d50/l9e 0 2024-04-25T15:49:38.733 INFO:tasks.workunit.client.0.smithi110.stdout:6/639: fsync d1/db/d22/d33/d63/f97 0 2024-04-25T15:49:38.737 INFO:tasks.workunit.client.1.smithi193.stdout:0/421: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/c90 0 2024-04-25T15:49:38.742 INFO:tasks.workunit.client.0.smithi110.stdout:1/834: dread d0/d14/d5f/d6f/fa3 [0,4194304] 0 2024-04-25T15:49:38.748 INFO:tasks.workunit.client.0.smithi110.stdout:2/669: mkdir de/d40/d6e/d52/d6d/d7d/dd4 0 2024-04-25T15:49:38.764 INFO:tasks.workunit.client.0.smithi110.stdout:7/703: getdents d1b/d29/d38/d8b/d89 0 2024-04-25T15:49:38.765 INFO:tasks.workunit.client.0.smithi110.stdout:8/784: getdents d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101 0 2024-04-25T15:49:38.774 INFO:tasks.workunit.client.0.smithi110.stdout:1/835: mknod d0/d14/d81/df8/d33/da9/c10e 0 2024-04-25T15:49:38.775 INFO:tasks.workunit.client.0.smithi110.stdout:7/704: fdatasync d1b/d29/d38/d55/d95/f9c 0 2024-04-25T15:49:38.776 INFO:tasks.workunit.client.0.smithi110.stdout:7/705: write d1b/d29/d38/d8b/f92 [1891523,51786] 0 2024-04-25T15:49:38.777 INFO:tasks.workunit.client.0.smithi110.stdout:7/706: write fb [330390,36599] 0 2024-04-25T15:49:38.779 INFO:tasks.workunit.client.1.smithi193.stdout:4/468: chown db/l36 134461713 1 2024-04-25T15:49:38.780 INFO:tasks.workunit.client.1.smithi193.stdout:4/469: chown db/f61 4051803 1 2024-04-25T15:49:38.781 INFO:tasks.workunit.client.0.smithi110.stdout:2/670: rename de/d40/d6e/d52/l7e to de/d40/d4f/ld5 0 2024-04-25T15:49:38.781 INFO:tasks.workunit.client.0.smithi110.stdout:2/671: readlink de/d40/d6e/d52/d6d/l7c 0 2024-04-25T15:49:38.786 INFO:tasks.workunit.client.1.smithi193.stdout:3/458: symlink d8/d13/d1f/d69/d77/l8d 0 2024-04-25T15:49:38.787 INFO:tasks.workunit.client.1.smithi193.stdout:3/459: read d8/d13/d14/f59 [10634,60557] 0 2024-04-25T15:49:38.789 INFO:tasks.workunit.client.0.smithi110.stdout:8/785: mknod d14/c10a 0 2024-04-25T15:49:38.790 INFO:tasks.workunit.client.1.smithi193.stdout:0/422: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/c91 0 2024-04-25T15:49:38.790 INFO:tasks.workunit.client.0.smithi110.stdout:8/786: fdatasync d14/d21/d26/d6a/d7e/da6/de8/d55/fb4 0 2024-04-25T15:49:38.795 INFO:tasks.workunit.client.1.smithi193.stdout:2/409: creat d0/d42/d51/d4f/fa7 x:0 0 0 2024-04-25T15:49:38.797 INFO:tasks.workunit.client.0.smithi110.stdout:8/787: mkdir d14/d21/d26/d6a/d7e/da6/de8/d10b 0 2024-04-25T15:49:38.799 INFO:tasks.workunit.client.1.smithi193.stdout:8/386: rename d2/d36 to d2/d79 0 2024-04-25T15:49:38.807 INFO:tasks.workunit.client.1.smithi193.stdout:0/423: dwrite d0/d49/d5/d7/dd/f73 [0,4194304] 0 2024-04-25T15:49:38.807 INFO:tasks.workunit.client.1.smithi193.stdout:2/410: creat d0/d44/fa8 x:0 0 0 2024-04-25T15:49:38.808 INFO:tasks.workunit.client.1.smithi193.stdout:2/411: dread - d0/d42/d51/d4f/d65/d7e/f82 zero size 2024-04-25T15:49:38.813 INFO:tasks.workunit.client.1.smithi193.stdout:3/460: symlink d8/d13/l8e 0 2024-04-25T15:49:38.815 INFO:tasks.workunit.client.0.smithi110.stdout:7/707: getdents d1b/d29/d38/d8b 0 2024-04-25T15:49:38.816 INFO:tasks.workunit.client.1.smithi193.stdout:4/470: link db/lf db/d4b/d51/l8f 0 2024-04-25T15:49:38.820 INFO:tasks.workunit.client.0.smithi110.stdout:8/788: link d14/d21/d26/d6a/db9/lcc d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/l10c 0 2024-04-25T15:49:38.821 INFO:tasks.workunit.client.0.smithi110.stdout:7/708: dwrite d1b/d29/d46/fb3 [0,4194304] 0 2024-04-25T15:49:38.826 INFO:tasks.workunit.client.1.smithi193.stdout:5/393: write f11 [3020165,13098] 0 2024-04-25T15:49:38.829 INFO:tasks.workunit.client.0.smithi110.stdout:8/789: creat d14/d21/d26/dab/dac/dbb/f10d x:0 0 0 2024-04-25T15:49:38.830 INFO:tasks.workunit.client.0.smithi110.stdout:7/709: dwrite d1b/d29/d46/f6a [0,4194304] 0 2024-04-25T15:49:38.832 INFO:tasks.workunit.client.1.smithi193.stdout:5/394: creat d17/d83/f8b x:0 0 0 2024-04-25T15:49:38.837 INFO:tasks.workunit.client.0.smithi110.stdout:7/710: creat d1b/d29/d38/d55/d95/feb x:0 0 0 2024-04-25T15:49:38.839 INFO:tasks.workunit.client.1.smithi193.stdout:3/461: dread d8/d30/f42 [0,4194304] 0 2024-04-25T15:49:38.840 INFO:tasks.workunit.client.0.smithi110.stdout:8/790: mkdir d14/d32/d36/db7/dc9/df9/d10e 0 2024-04-25T15:49:38.841 INFO:tasks.workunit.client.0.smithi110.stdout:7/711: truncate d1b/d29/f98 252950 0 2024-04-25T15:49:38.845 INFO:tasks.workunit.client.1.smithi193.stdout:5/395: dwrite d17/f26 [0,4194304] 0 2024-04-25T15:49:38.846 INFO:tasks.workunit.client.1.smithi193.stdout:3/462: dwrite d8/d30/d45/f70 [4194304,4194304] 0 2024-04-25T15:49:38.846 INFO:tasks.workunit.client.0.smithi110.stdout:7/712: mkdir d1b/d29/d38/d8b/d89/dc6/dec 0 2024-04-25T15:49:38.847 INFO:tasks.workunit.client.0.smithi110.stdout:7/713: chown d1b/d29/d38/d8b/d89/dc6/dec 1081 1 2024-04-25T15:49:38.858 INFO:tasks.workunit.client.0.smithi110.stdout:7/714: rmdir d1b/d29/d38/d8b/d89/dc6/dec 0 2024-04-25T15:49:38.858 INFO:tasks.workunit.client.1.smithi193.stdout:6/419: write dd/f1b [4711411,35382] 0 2024-04-25T15:49:38.859 INFO:tasks.workunit.client.1.smithi193.stdout:7/375: write d12/f2e [2110615,51156] 0 2024-04-25T15:49:38.860 INFO:tasks.workunit.client.1.smithi193.stdout:7/376: chown d12/d31/l54 7877 1 2024-04-25T15:49:38.861 INFO:tasks.workunit.client.0.smithi110.stdout:7/715: mknod d1b/d29/d38/d55/dab/db2/dcd/ced 0 2024-04-25T15:49:38.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:38 smithi110 ceph-mon[26344]: pgmap v36: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 75 MiB/s rd, 257 MiB/s wr, 482 op/s 2024-04-25T15:49:38.864 INFO:tasks.workunit.client.1.smithi193.stdout:6/420: truncate dd/f24 2652819 0 2024-04-25T15:49:38.867 INFO:tasks.workunit.client.0.smithi110.stdout:7/716: dwrite d1b/d29/fa4 [0,4194304] 0 2024-04-25T15:49:38.868 INFO:tasks.workunit.client.1.smithi193.stdout:4/471: symlink db/d13/d1e/d1f/l90 0 2024-04-25T15:49:38.869 INFO:tasks.workunit.client.1.smithi193.stdout:4/472: fdatasync db/d13/d1e/f48 0 2024-04-25T15:49:38.870 INFO:tasks.workunit.client.1.smithi193.stdout:4/473: chown db/d13/d1e/d1f/f63 20455 1 2024-04-25T15:49:38.872 INFO:tasks.workunit.client.0.smithi110.stdout:5/709: dwrite d4/f8 [0,4194304] 0 2024-04-25T15:49:38.873 INFO:tasks.workunit.client.1.smithi193.stdout:4/474: symlink db/d32/d40/l91 0 2024-04-25T15:49:38.873 INFO:tasks.workunit.client.1.smithi193.stdout:4/475: stat db/d13/l3f 0 2024-04-25T15:49:38.879 INFO:tasks.workunit.client.0.smithi110.stdout:7/717: link d1b/d29/d38/c91 d1b/d29/d38/cee 0 2024-04-25T15:49:38.880 INFO:tasks.workunit.client.0.smithi110.stdout:7/718: readlink d1b/d29/d38/d55/d7f/l9a 0 2024-04-25T15:49:38.883 INFO:tasks.workunit.client.1.smithi193.stdout:8/387: creat d2/d3/d27/f7a x:0 0 0 2024-04-25T15:49:38.883 INFO:tasks.workunit.client.0.smithi110.stdout:5/710: getdents d4/d2c 0 2024-04-25T15:49:38.885 INFO:tasks.workunit.client.1.smithi193.stdout:4/476: link db/d4b/d51/l8f db/d13/d1e/l92 0 2024-04-25T15:49:38.885 INFO:tasks.workunit.client.1.smithi193.stdout:4/477: fsync db/f61 0 2024-04-25T15:49:38.887 INFO:tasks.workunit.client.0.smithi110.stdout:5/711: rename d4/d15/d51/cda to d4/d1d/d1f/d35/d3d/d52/dd2/ce6 0 2024-04-25T15:49:38.887 INFO:tasks.workunit.client.0.smithi110.stdout:5/712: readlink d4/d1d/d1f/l7b 0 2024-04-25T15:49:38.888 INFO:tasks.workunit.client.0.smithi110.stdout:0/658: write d1/d20/d69/fb3 [1047043,8283] 0 2024-04-25T15:49:38.891 INFO:tasks.workunit.client.0.smithi110.stdout:9/653: mknod d9/cd1 0 2024-04-25T15:49:38.891 INFO:tasks.workunit.client.1.smithi193.stdout:4/478: dwrite db/d13/d1e/f48 [0,4194304] 0 2024-04-25T15:49:38.892 INFO:tasks.workunit.client.0.smithi110.stdout:0/659: truncate d1/d23/d30/d33/d5b/d83/f5d 634319 0 2024-04-25T15:49:38.892 INFO:tasks.workunit.client.0.smithi110.stdout:5/713: truncate d4/d1d/d1f/d35/d3d/d52/dd2/dd4/fb7 556335 0 2024-04-25T15:49:38.893 INFO:tasks.workunit.client.0.smithi110.stdout:0/660: write d1/d23/d81/fc3 [1001709,21683] 0 2024-04-25T15:49:38.899 INFO:tasks.workunit.client.0.smithi110.stdout:9/654: dwrite d9/d44/f53 [4194304,4194304] 0 2024-04-25T15:49:38.900 INFO:tasks.workunit.client.0.smithi110.stdout:9/655: stat d9/d44/d5e/l7b 0 2024-04-25T15:49:38.901 INFO:tasks.workunit.client.0.smithi110.stdout:0/661: mknod d1/d23/d30/d33/d35/cec 0 2024-04-25T15:49:38.903 INFO:tasks.workunit.client.0.smithi110.stdout:5/714: creat d4/d15/d51/d99/fe7 x:0 0 0 2024-04-25T15:49:38.908 INFO:tasks.workunit.client.0.smithi110.stdout:0/662: mknod d1/d23/d30/d33/d5b/d72/d77/ced 0 2024-04-25T15:49:38.909 INFO:tasks.workunit.client.1.smithi193.stdout:4/479: dread f4 [0,4194304] 0 2024-04-25T15:49:38.911 INFO:tasks.workunit.client.0.smithi110.stdout:5/715: dwrite d4/d1d/d1f/d35/d7f/d92/d9b/dcb/fce [0,4194304] 0 2024-04-25T15:49:38.911 INFO:tasks.workunit.client.1.smithi193.stdout:4/480: dread db/d13/d1e/d1f/d56/d69/d70/f76 [0,4194304] 0 2024-04-25T15:49:38.913 INFO:tasks.workunit.client.1.smithi193.stdout:9/411: rename f4 to df/d2c/d2f/d4e/d1c/f76 0 2024-04-25T15:49:38.914 INFO:tasks.workunit.client.0.smithi110.stdout:0/663: mkdir d1/d47/d54/d99/dee 0 2024-04-25T15:49:38.917 INFO:tasks.workunit.client.1.smithi193.stdout:4/481: creat db/d32/d40/f93 x:0 0 0 2024-04-25T15:49:38.917 INFO:tasks.workunit.client.0.smithi110.stdout:5/716: mkdir d4/d15/d51/d90/de8 0 2024-04-25T15:49:38.919 INFO:tasks.workunit.client.0.smithi110.stdout:5/717: truncate d4/d1d/d1f/d35/d7f/fa7 913033 0 2024-04-25T15:49:38.921 INFO:tasks.workunit.client.0.smithi110.stdout:0/664: rename d1/d23/d30/d33/d5b/d83/f8a to d1/d47/d54/d99/fef 0 2024-04-25T15:49:38.923 INFO:tasks.workunit.client.1.smithi193.stdout:4/482: dwrite db/d4b/f7b [0,4194304] 0 2024-04-25T15:49:38.923 INFO:tasks.workunit.client.0.smithi110.stdout:5/718: unlink d4/d1d/d1f/d35/f7d 0 2024-04-25T15:49:38.924 INFO:tasks.workunit.client.1.smithi193.stdout:4/483: fdatasync db/d13/d1e/d1f/d56/f74 0 2024-04-25T15:49:38.924 INFO:tasks.workunit.client.1.smithi193.stdout:4/484: stat db/d4b/f71 0 2024-04-25T15:49:38.927 INFO:tasks.workunit.client.0.smithi110.stdout:0/665: rename d1/f4d to d1/d23/d30/d33/d5b/dbb/da8/ff0 0 2024-04-25T15:49:38.929 INFO:tasks.workunit.client.0.smithi110.stdout:5/719: dread d4/d1d/d1f/f64 [4194304,4194304] 0 2024-04-25T15:49:38.934 INFO:tasks.workunit.client.0.smithi110.stdout:0/666: dwrite d1/d23/d30/d33/d5b/d83/fe9 [0,4194304] 0 2024-04-25T15:49:38.935 INFO:tasks.workunit.client.1.smithi193.stdout:4/485: fsync f4 0 2024-04-25T15:49:38.940 INFO:tasks.workunit.client.0.smithi110.stdout:0/667: dread - d1/d23/d30/d33/d35/da1/fcb zero size 2024-04-25T15:49:38.940 INFO:tasks.workunit.client.1.smithi193.stdout:4/486: dread db/d13/d1e/d1f/d22/f2c [4194304,4194304] 0 2024-04-25T15:49:38.942 INFO:tasks.workunit.client.1.smithi193.stdout:1/357: write dc/f4b [1681638,27158] 0 2024-04-25T15:49:38.943 INFO:tasks.workunit.client.0.smithi110.stdout:6/640: truncate d1/db/d24/f29 2851284 0 2024-04-25T15:49:38.944 INFO:tasks.workunit.client.0.smithi110.stdout:0/668: fdatasync d1/d23/d30/d33/d5b/d72/d77/fcc 0 2024-04-25T15:49:38.946 INFO:tasks.workunit.client.0.smithi110.stdout:6/641: creat d1/dd/db5/fe3 x:0 0 0 2024-04-25T15:49:38.949 INFO:tasks.workunit.client.1.smithi193.stdout:1/358: unlink dc/f16 0 2024-04-25T15:49:38.950 INFO:tasks.workunit.client.1.smithi193.stdout:2/412: rename d0/d42/f5b to d0/d37/d25/fa9 0 2024-04-25T15:49:38.952 INFO:tasks.workunit.client.1.smithi193.stdout:4/487: creat db/d32/f94 x:0 0 0 2024-04-25T15:49:38.955 INFO:tasks.workunit.client.0.smithi110.stdout:6/642: rmdir d1/db/d24/d69/d75 39 2024-04-25T15:49:38.956 INFO:tasks.workunit.client.0.smithi110.stdout:2/672: write de/d40/d6e/f9d [1047841,14573] 0 2024-04-25T15:49:38.958 INFO:tasks.workunit.client.0.smithi110.stdout:6/643: readlink d1/db/d24/d69/lbf 0 2024-04-25T15:49:38.960 INFO:tasks.workunit.client.1.smithi193.stdout:4/488: mkdir db/d4b/d51/d95 0 2024-04-25T15:49:38.961 INFO:tasks.workunit.client.1.smithi193.stdout:1/359: dwrite dc/f17 [0,4194304] 0 2024-04-25T15:49:38.962 INFO:tasks.workunit.client.0.smithi110.stdout:6/644: rmdir d1/d12/daf 39 2024-04-25T15:49:38.967 INFO:tasks.workunit.client.0.smithi110.stdout:6/645: unlink d1/db/d22/d33/d43/f53 0 2024-04-25T15:49:38.971 INFO:tasks.workunit.client.0.smithi110.stdout:4/694: symlink d13/d24/d2d/lf0 0 2024-04-25T15:49:38.973 INFO:tasks.workunit.client.0.smithi110.stdout:1/836: write d0/d14/d5f/d7d/f67 [448978,81627] 0 2024-04-25T15:49:38.973 INFO:tasks.workunit.client.1.smithi193.stdout:1/360: chown dc/d4a/d50/d5c/c69 216643765 1 2024-04-25T15:49:38.975 INFO:tasks.workunit.client.1.smithi193.stdout:6/421: rename dd/d12/f4d to dd/d12/d19/f79 0 2024-04-25T15:49:38.975 INFO:tasks.workunit.client.1.smithi193.stdout:8/388: rename d2/d3 to d2/d3/d3a/d7b 22 2024-04-25T15:49:38.977 INFO:tasks.workunit.client.1.smithi193.stdout:2/413: symlink d0/d9/d15/d7b/laa 0 2024-04-25T15:49:38.978 INFO:tasks.workunit.client.0.smithi110.stdout:4/695: dread - d13/d24/d2d/d44/fb4 zero size 2024-04-25T15:49:38.978 INFO:tasks.workunit.client.1.smithi193.stdout:1/361: dread dc/f17 [0,4194304] 0 2024-04-25T15:49:38.980 INFO:tasks.workunit.client.0.smithi110.stdout:1/837: creat d0/d14/d5f/d7d/d72/d88/db6/f10f x:0 0 0 2024-04-25T15:49:38.982 INFO:tasks.workunit.client.1.smithi193.stdout:4/489: dread f1 [0,4194304] 0 2024-04-25T15:49:38.983 INFO:tasks.workunit.client.1.smithi193.stdout:4/490: write db/d32/f94 [400303,57753] 0 2024-04-25T15:49:38.985 INFO:tasks.workunit.client.0.smithi110.stdout:1/838: rmdir d0/d14/d81 39 2024-04-25T15:49:38.986 INFO:tasks.workunit.client.1.smithi193.stdout:8/389: creat d2/d3/d3a/f7c x:0 0 0 2024-04-25T15:49:38.987 INFO:tasks.workunit.client.1.smithi193.stdout:0/424: dwrite d0/d49/d5/d7/dd/f3c [0,4194304] 0 2024-04-25T15:49:38.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:38 smithi193 ceph-mon[33747]: pgmap v36: 65 pgs: 65 active+clean; 2.4 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 75 MiB/s rd, 257 MiB/s wr, 482 op/s 2024-04-25T15:49:38.990 INFO:tasks.workunit.client.1.smithi193.stdout:0/425: truncate d0/d49/d5/d7/dd/d16/d26/f55 706352 0 2024-04-25T15:49:38.993 INFO:tasks.workunit.client.0.smithi110.stdout:1/839: rename d0/d51/d6b/ca2 to d0/d14/d81/c110 0 2024-04-25T15:49:38.994 INFO:tasks.workunit.client.0.smithi110.stdout:1/840: write d0/d14/f53 [3873263,98682] 0 2024-04-25T15:49:38.994 INFO:tasks.workunit.client.0.smithi110.stdout:8/791: write d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/fb0 [4367251,65822] 0 2024-04-25T15:49:38.995 INFO:tasks.workunit.client.0.smithi110.stdout:1/841: chown d0/d14/d81/f8f 115469796 1 2024-04-25T15:49:38.996 INFO:tasks.workunit.client.1.smithi193.stdout:6/422: fsync dd/f74 0 2024-04-25T15:49:38.996 INFO:tasks.workunit.client.0.smithi110.stdout:1/842: chown d0/d51/dd8/fef 5 1 2024-04-25T15:49:38.999 INFO:tasks.workunit.client.1.smithi193.stdout:4/491: rename db/d13/d1e/d1f/d22/c62 to db/d13/d1e/d1f/d22/c96 0 2024-04-25T15:49:38.999 INFO:tasks.workunit.client.1.smithi193.stdout:3/463: write d8/d13/d4b/f57 [179745,116355] 0 2024-04-25T15:49:39.002 INFO:tasks.workunit.client.1.smithi193.stdout:5/396: dwrite d17/d18/d45/f47 [0,4194304] 0 2024-04-25T15:49:39.003 INFO:tasks.workunit.client.1.smithi193.stdout:5/397: dread - d17/f66 zero size 2024-04-25T15:49:39.006 INFO:tasks.workunit.client.0.smithi110.stdout:8/792: symlink d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/l10f 0 2024-04-25T15:49:39.010 INFO:tasks.workunit.client.1.smithi193.stdout:9/412: creat df/d2c/d2f/f77 x:0 0 0 2024-04-25T15:49:39.012 INFO:tasks.workunit.client.1.smithi193.stdout:5/398: dwrite d17/d18/f2c [0,4194304] 0 2024-04-25T15:49:39.012 INFO:tasks.workunit.client.0.smithi110.stdout:8/793: rmdir d14/d21/d26/d6a/d7e/da6/de8/d55 39 2024-04-25T15:49:39.014 INFO:tasks.workunit.client.1.smithi193.stdout:7/377: dwrite d12/f30 [0,4194304] 0 2024-04-25T15:49:39.015 INFO:tasks.workunit.client.1.smithi193.stdout:8/390: dread d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:39.016 INFO:tasks.workunit.client.1.smithi193.stdout:7/378: write d12/d31/d37/d62/d6e/f8c [1586921,19552] 0 2024-04-25T15:49:39.019 INFO:tasks.workunit.client.1.smithi193.stdout:8/391: chown d2/d3/f16 513 1 2024-04-25T15:49:39.021 INFO:tasks.workunit.client.1.smithi193.stdout:4/492: symlink db/d13/d1e/d1f/d56/d69/d70/l97 0 2024-04-25T15:49:39.022 INFO:tasks.workunit.client.0.smithi110.stdout:7/719: dwrite d1b/d29/f5d [0,4194304] 0 2024-04-25T15:49:39.025 INFO:tasks.workunit.client.0.smithi110.stdout:8/794: rmdir d14/d21/d26/d6a/d7e/da6/de8/d10b 0 2024-04-25T15:49:39.027 INFO:tasks.workunit.client.1.smithi193.stdout:5/399: dwrite d17/f86 [0,4194304] 0 2024-04-25T15:49:39.029 INFO:tasks.workunit.client.0.smithi110.stdout:7/720: chown d1b/d29/f98 7 1 2024-04-25T15:49:39.030 INFO:tasks.workunit.client.1.smithi193.stdout:5/400: fsync d17/d18/d75/d85/f73 0 2024-04-25T15:49:39.030 INFO:tasks.workunit.client.1.smithi193.stdout:7/379: dwrite d12/d1c/d45/f6a [0,4194304] 0 2024-04-25T15:49:39.031 INFO:tasks.workunit.client.0.smithi110.stdout:8/795: mkdir d14/d32/d105/d110 0 2024-04-25T15:49:39.033 INFO:tasks.workunit.client.1.smithi193.stdout:2/414: link d0/d37/f48 d0/d9/d3d/d7d/fab 0 2024-04-25T15:49:39.033 INFO:tasks.workunit.client.1.smithi193.stdout:7/380: write d12/d13/f24 [1834170,69875] 0 2024-04-25T15:49:39.036 INFO:tasks.workunit.client.0.smithi110.stdout:9/656: write d9/db/df/d2c/d3c/f87 [2965528,29486] 0 2024-04-25T15:49:39.040 INFO:tasks.workunit.client.0.smithi110.stdout:8/796: mkdir d14/d32/d36/dde/d111 0 2024-04-25T15:49:39.040 INFO:tasks.workunit.client.0.smithi110.stdout:8/797: readlink d14/d32/d36/l52 0 2024-04-25T15:49:39.042 INFO:tasks.workunit.client.0.smithi110.stdout:7/721: rename d1b/lb6 to d1b/d29/d38/d55/dab/db2/dcd/de2/lef 0 2024-04-25T15:49:39.044 INFO:tasks.workunit.client.0.smithi110.stdout:7/722: read d1b/d29/d38/d55/f59 [3745395,92720] 0 2024-04-25T15:49:39.045 INFO:tasks.workunit.client.0.smithi110.stdout:7/723: write d1b/d29/d38/d55/fbd [364869,74005] 0 2024-04-25T15:49:39.046 INFO:tasks.workunit.client.1.smithi193.stdout:9/413: dread df/d2c/d2f/d4e/f37 [0,4194304] 0 2024-04-25T15:49:39.050 INFO:tasks.workunit.client.1.smithi193.stdout:4/493: unlink db/f59 0 2024-04-25T15:49:39.054 INFO:tasks.workunit.client.0.smithi110.stdout:5/720: write d4/d1d/d1f/d35/d3d/d52/f7a [5858,37455] 0 2024-04-25T15:49:39.055 INFO:tasks.workunit.client.0.smithi110.stdout:7/724: symlink d1b/d29/d38/d55/dab/db2/db8/lf0 0 2024-04-25T15:49:39.057 INFO:tasks.workunit.client.1.smithi193.stdout:1/362: link dc/fe dc/d59/f7b 0 2024-04-25T15:49:39.058 INFO:tasks.workunit.client.1.smithi193.stdout:1/363: truncate dc/d4a/f71 159035 0 2024-04-25T15:49:39.059 INFO:tasks.workunit.client.1.smithi193.stdout:5/401: symlink d17/d18/d75/d85/l8c 0 2024-04-25T15:49:39.059 INFO:tasks.workunit.client.1.smithi193.stdout:1/364: write dc/d4a/d50/f6a [111931,112282] 0 2024-04-25T15:49:39.059 INFO:tasks.workunit.client.0.smithi110.stdout:7/725: fdatasync d1b/d29/d38/f61 0 2024-04-25T15:49:39.060 INFO:tasks.workunit.client.0.smithi110.stdout:7/726: fsync d1b/f31 0 2024-04-25T15:49:39.063 INFO:tasks.workunit.client.0.smithi110.stdout:5/721: mknod d4/d1d/d1f/d35/d7f/d92/d9b/dcb/ce9 0 2024-04-25T15:49:39.069 INFO:tasks.workunit.client.1.smithi193.stdout:6/423: rename dd/d12/d19/d1c/d40/l44 to dd/d2f/d57/l7a 0 2024-04-25T15:49:39.073 INFO:tasks.workunit.client.1.smithi193.stdout:1/365: dread dc/d59/f7b [0,4194304] 0 2024-04-25T15:49:39.074 INFO:tasks.workunit.client.1.smithi193.stdout:4/494: creat db/d13/d1e/d1f/f98 x:0 0 0 2024-04-25T15:49:39.075 INFO:tasks.workunit.client.1.smithi193.stdout:4/495: fsync db/fc 0 2024-04-25T15:49:39.075 INFO:tasks.workunit.client.0.smithi110.stdout:8/798: link c7 d14/d32/d105/d110/c112 0 2024-04-25T15:49:39.076 INFO:tasks.workunit.client.1.smithi193.stdout:8/392: mknod d2/d3/d20/d45/d4a/d63/c7d 0 2024-04-25T15:49:39.078 INFO:tasks.workunit.client.0.smithi110.stdout:7/727: rename d1b/c41 to d1b/dde/cf1 0 2024-04-25T15:49:39.080 INFO:tasks.workunit.client.0.smithi110.stdout:5/722: symlink d4/d15/d51/d90/de8/lea 0 2024-04-25T15:49:39.084 INFO:tasks.workunit.client.0.smithi110.stdout:8/799: creat d14/d21/d26/d6a/db9/f113 x:0 0 0 2024-04-25T15:49:39.084 INFO:tasks.workunit.client.1.smithi193.stdout:7/381: getdents d12/d1c/d63/d70/d68/d8e 0 2024-04-25T15:49:39.087 INFO:tasks.workunit.client.0.smithi110.stdout:6/646: dwrite d1/dd/d3c/d91/d8d/dce/fde [0,4194304] 0 2024-04-25T15:49:39.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/414: rename df/d2c/d2f/d4e/l62 to df/d2c/d2f/d4e/d1c/d26/d3f/d41/l78 0 2024-04-25T15:49:39.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/415: stat df 0 2024-04-25T15:49:39.089 INFO:tasks.workunit.client.0.smithi110.stdout:7/728: dread - d1b/fad zero size 2024-04-25T15:49:39.089 INFO:tasks.workunit.client.0.smithi110.stdout:4/696: write d13/d24/d2d/f37 [1685556,54470] 0 2024-04-25T15:49:39.095 INFO:tasks.workunit.client.0.smithi110.stdout:5/723: mknod d4/d1d/d1f/d35/d7f/d92/d9b/dcb/ceb 0 2024-04-25T15:49:39.095 INFO:tasks.workunit.client.0.smithi110.stdout:8/800: dwrite d14/d21/f62 [0,4194304] 0 2024-04-25T15:49:39.096 INFO:tasks.workunit.client.1.smithi193.stdout:1/366: chown dc/l31 0 1 2024-04-25T15:49:39.096 INFO:tasks.workunit.client.0.smithi110.stdout:6/647: dread d1/db/d22/d33/f3b [0,4194304] 0 2024-04-25T15:49:39.096 INFO:tasks.workunit.client.1.smithi193.stdout:1/367: chown dc/d4a/d50/d5c/d5d/d3d/f41 63120 1 2024-04-25T15:49:39.099 INFO:tasks.workunit.client.0.smithi110.stdout:6/648: chown d1/d12/l8a 27 1 2024-04-25T15:49:39.100 INFO:tasks.workunit.client.0.smithi110.stdout:1/843: write d0/d14/d81/df8/d33/d8e/fa6 [1771488,77719] 0 2024-04-25T15:49:39.102 INFO:tasks.workunit.client.0.smithi110.stdout:5/724: write d4/d1d/d1f/d35/fc6 [1038304,130350] 0 2024-04-25T15:49:39.103 INFO:tasks.workunit.client.1.smithi193.stdout:4/496: creat db/d32/d40/f99 x:0 0 0 2024-04-25T15:49:39.104 INFO:tasks.workunit.client.0.smithi110.stdout:6/649: chown d1/db/d24/d2d/f3a 1 1 2024-04-25T15:49:39.104 INFO:tasks.workunit.client.0.smithi110.stdout:7/729: dwrite d1b/d29/d46/fcf [0,4194304] 0 2024-04-25T15:49:39.105 INFO:tasks.workunit.client.1.smithi193.stdout:4/497: chown db/d13/d1e/d1f/d22/c2f 142165 1 2024-04-25T15:49:39.105 INFO:tasks.workunit.client.1.smithi193.stdout:8/393: symlink d2/d3/d27/d6f/l7e 0 2024-04-25T15:49:39.106 INFO:tasks.workunit.client.0.smithi110.stdout:1/844: dread - d0/d14/d81/df8/d33/d94/ddb/ff6 zero size 2024-04-25T15:49:39.106 INFO:tasks.workunit.client.0.smithi110.stdout:7/730: readlink d1b/l1d 0 2024-04-25T15:49:39.107 INFO:tasks.workunit.client.0.smithi110.stdout:1/845: chown d0/d14/d5f/d7d/d72/ff5 1017966 1 2024-04-25T15:49:39.109 INFO:tasks.workunit.client.1.smithi193.stdout:5/402: symlink d17/l8d 0 2024-04-25T15:49:39.110 INFO:tasks.workunit.client.0.smithi110.stdout:7/731: truncate d1b/d29/d38/d55/d95/dcb/fe4 1591987 0 2024-04-25T15:49:39.111 INFO:tasks.workunit.client.0.smithi110.stdout:7/732: truncate d1b/d29/fa4 4510562 0 2024-04-25T15:49:39.112 INFO:tasks.workunit.client.0.smithi110.stdout:8/801: mknod d14/d21/d26/d6a/de9/c114 0 2024-04-25T15:49:39.113 INFO:tasks.workunit.client.1.smithi193.stdout:7/382: mkdir d12/d31/d37/d92 0 2024-04-25T15:49:39.114 INFO:tasks.workunit.client.0.smithi110.stdout:8/802: write d14/d32/d36/d48/fb3 [1343875,77208] 0 2024-04-25T15:49:39.115 INFO:tasks.workunit.client.1.smithi193.stdout:9/416: mknod df/d2c/d2f/d4e/d1c/d30/d3b/c79 0 2024-04-25T15:49:39.115 INFO:tasks.workunit.client.1.smithi193.stdout:3/464: dwrite d8/d13/f56 [0,4194304] 0 2024-04-25T15:49:39.116 INFO:tasks.workunit.client.1.smithi193.stdout:3/465: dread - d8/d13/f66 zero size 2024-04-25T15:49:39.117 INFO:tasks.workunit.client.0.smithi110.stdout:8/803: read d14/d21/d26/d6a/d7e/da6/de8/f50 [8051448,109942] 0 2024-04-25T15:49:39.117 INFO:tasks.workunit.client.1.smithi193.stdout:3/466: dread - d8/d13/d14/d27/d54/f65 zero size 2024-04-25T15:49:39.119 INFO:tasks.workunit.client.1.smithi193.stdout:6/424: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/f7b x:0 0 0 2024-04-25T15:49:39.119 INFO:tasks.workunit.client.0.smithi110.stdout:5/725: mknod d4/d1d/d1f/d9e/cec 0 2024-04-25T15:49:39.121 INFO:tasks.workunit.client.1.smithi193.stdout:0/426: write d0/d49/d5/d7/dd/d46/f4d [1489595,77694] 0 2024-04-25T15:49:39.122 INFO:tasks.workunit.client.0.smithi110.stdout:6/650: fdatasync d1/f26 0 2024-04-25T15:49:39.125 INFO:tasks.workunit.client.0.smithi110.stdout:4/697: symlink d13/d24/d2d/d44/d65/d99/daf/lf1 0 2024-04-25T15:49:39.128 INFO:tasks.workunit.client.0.smithi110.stdout:9/657: truncate d9/db/df/d2c/d3c/fa4 1779675 0 2024-04-25T15:49:39.130 INFO:tasks.workunit.client.0.smithi110.stdout:9/658: write d9/db/d58/fc2 [493117,91384] 0 2024-04-25T15:49:39.132 INFO:tasks.workunit.client.1.smithi193.stdout:4/498: mkdir db/d13/d1e/d1f/d56/d69/d70/d9a 0 2024-04-25T15:49:39.132 INFO:tasks.workunit.client.1.smithi193.stdout:9/417: dread df/d2c/d2f/d33/f3c [0,4194304] 0 2024-04-25T15:49:39.134 INFO:tasks.workunit.client.1.smithi193.stdout:5/403: chown d17/d18/d30/d34/d52/l62 1927638 1 2024-04-25T15:49:39.134 INFO:tasks.workunit.client.1.smithi193.stdout:5/404: stat d17/d18/d45/c49 0 2024-04-25T15:49:39.136 INFO:tasks.workunit.client.1.smithi193.stdout:7/383: creat d12/d31/d39/f93 x:0 0 0 2024-04-25T15:49:39.137 INFO:tasks.workunit.client.1.smithi193.stdout:7/384: readlink d12/d1c/d45/l73 0 2024-04-25T15:49:39.137 INFO:tasks.workunit.client.0.smithi110.stdout:5/726: mkdir d4/d15/d51/d90/ded 0 2024-04-25T15:49:39.141 INFO:tasks.workunit.client.1.smithi193.stdout:6/425: dread dd/d12/d19/d1c/d40/d3c/d3f/f54 [0,4194304] 0 2024-04-25T15:49:39.141 INFO:tasks.workunit.client.0.smithi110.stdout:6/651: mkdir d1/db/d22/d33/de4 0 2024-04-25T15:49:39.142 INFO:tasks.workunit.client.0.smithi110.stdout:8/804: dread f3 [4194304,4194304] 0 2024-04-25T15:49:39.145 INFO:tasks.workunit.client.1.smithi193.stdout:0/427: truncate d0/d49/d5/f2d 698841 0 2024-04-25T15:49:39.149 INFO:tasks.workunit.client.0.smithi110.stdout:9/659: read d9/db/df/d2c/f36 [1267909,128576] 0 2024-04-25T15:49:39.150 INFO:tasks.workunit.client.1.smithi193.stdout:8/394: mkdir d2/d3/d7f 0 2024-04-25T15:49:39.152 INFO:tasks.workunit.client.1.smithi193.stdout:4/499: creat db/d13/d1e/d1f/d56/d57/f9b x:0 0 0 2024-04-25T15:49:39.156 INFO:tasks.workunit.client.1.smithi193.stdout:9/418: creat df/d2c/d2f/d4e/d1c/d30/f7a x:0 0 0 2024-04-25T15:49:39.158 INFO:tasks.workunit.client.1.smithi193.stdout:5/405: mknod d17/d83/c8e 0 2024-04-25T15:49:39.158 INFO:tasks.workunit.client.1.smithi193.stdout:4/500: dwrite db/d13/d1e/f6d [4194304,4194304] 0 2024-04-25T15:49:39.158 INFO:tasks.workunit.client.1.smithi193.stdout:5/406: stat d17 0 2024-04-25T15:49:39.159 INFO:tasks.workunit.client.0.smithi110.stdout:5/727: rmdir d4/d1d/d1f/d35 39 2024-04-25T15:49:39.160 INFO:tasks.workunit.client.0.smithi110.stdout:5/728: write d4/f8 [2843484,13843] 0 2024-04-25T15:49:39.161 INFO:tasks.workunit.client.0.smithi110.stdout:5/729: truncate d4/d15/d51/d99/fe7 598105 0 2024-04-25T15:49:39.162 INFO:tasks.workunit.client.1.smithi193.stdout:7/385: creat d12/d31/d37/d62/d6e/d79/f94 x:0 0 0 2024-04-25T15:49:39.168 INFO:tasks.workunit.client.0.smithi110.stdout:1/846: link d0/d51/dd8/fef d0/d14/d81/df8/d33/f111 0 2024-04-25T15:49:39.169 INFO:tasks.workunit.client.1.smithi193.stdout:6/426: fdatasync dd/d12/d19/f79 0 2024-04-25T15:49:39.172 INFO:tasks.workunit.client.1.smithi193.stdout:6/427: dread dd/d2f/f72 [0,4194304] 0 2024-04-25T15:49:39.173 INFO:tasks.workunit.client.1.smithi193.stdout:6/428: dread - dd/d2f/f5b zero size 2024-04-25T15:49:39.175 INFO:tasks.workunit.client.1.smithi193.stdout:0/428: rename d0/f12 to d0/d49/d5/d7/dd/d46/d48/d62/f92 0 2024-04-25T15:49:39.175 INFO:tasks.workunit.client.0.smithi110.stdout:4/698: dread d13/d24/d2d/d44/d65/d99/daf/d56/d88/f2c [0,4194304] 0 2024-04-25T15:49:39.182 INFO:tasks.workunit.client.1.smithi193.stdout:9/419: mknod df/d2c/d34/d3e/d4d/c7b 0 2024-04-25T15:49:39.182 INFO:tasks.workunit.client.1.smithi193.stdout:2/415: sync 2024-04-25T15:49:39.182 INFO:tasks.workunit.client.0.smithi110.stdout:5/730: rmdir d4/d15/d51 39 2024-04-25T15:49:39.183 INFO:tasks.workunit.client.1.smithi193.stdout:9/420: readlink df/d2c/d2f/d4e/d1c/l2b 0 2024-04-25T15:49:39.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/847: mknod d0/d14/d81/df8/d33/d8e/c112 0 2024-04-25T15:49:39.186 INFO:tasks.workunit.client.1.smithi193.stdout:5/407: creat d17/d18/d30/d38/d48/f8f x:0 0 0 2024-04-25T15:49:39.190 INFO:tasks.workunit.client.0.smithi110.stdout:4/699: creat d13/d24/d2d/d36/ff2 x:0 0 0 2024-04-25T15:49:39.193 INFO:tasks.workunit.client.1.smithi193.stdout:6/429: symlink dd/d2f/d57/l7c 0 2024-04-25T15:49:39.195 INFO:tasks.workunit.client.1.smithi193.stdout:7/386: rename d12/d1c/l57 to d12/d31/d37/d62/l95 0 2024-04-25T15:49:39.196 INFO:tasks.workunit.client.1.smithi193.stdout:1/368: truncate dc/d4a/f54 3313795 0 2024-04-25T15:49:39.197 INFO:tasks.workunit.client.0.smithi110.stdout:9/660: creat d9/d13/fd2 x:0 0 0 2024-04-25T15:49:39.198 INFO:tasks.workunit.client.0.smithi110.stdout:9/661: read f8 [2081115,36688] 0 2024-04-25T15:49:39.199 INFO:tasks.workunit.client.0.smithi110.stdout:4/700: rename d13/d24/d26/dcd/fd4 to d13/d24/d2d/dd9/dbb/dd2/ff3 0 2024-04-25T15:49:39.200 INFO:tasks.workunit.client.1.smithi193.stdout:2/416: rmdir d0/d9/d15/d31/d90 39 2024-04-25T15:49:39.202 INFO:tasks.workunit.client.0.smithi110.stdout:5/731: mknod d4/d1d/d1f/d35/d7f/cee 0 2024-04-25T15:49:39.202 INFO:tasks.workunit.client.0.smithi110.stdout:7/733: dwrite d1b/d29/d38/d55/d7f/f7a [0,4194304] 0 2024-04-25T15:49:39.203 INFO:tasks.workunit.client.1.smithi193.stdout:9/421: unlink df/d2c/d2f/d33/f6f 0 2024-04-25T15:49:39.204 INFO:tasks.workunit.client.1.smithi193.stdout:9/422: chown df/d2c/d2f/d4e/d1c/d30/l65 1 1 2024-04-25T15:49:39.207 INFO:tasks.workunit.client.1.smithi193.stdout:5/408: symlink d17/d18/d30/d38/d71/l90 0 2024-04-25T15:49:39.208 INFO:tasks.workunit.client.1.smithi193.stdout:5/409: write d17/d18/d75/d85/d5d/f68 [4490288,103381] 0 2024-04-25T15:49:39.208 INFO:tasks.workunit.client.1.smithi193.stdout:3/467: write d8/d13/d1f/d69/d77/f74 [79602,61601] 0 2024-04-25T15:49:39.209 INFO:tasks.workunit.client.1.smithi193.stdout:5/410: readlink d17/d18/d75/d85/l54 0 2024-04-25T15:49:39.213 INFO:tasks.workunit.client.0.smithi110.stdout:6/652: dwrite d1/db/d22/f2c [4194304,4194304] 0 2024-04-25T15:49:39.215 INFO:tasks.workunit.client.0.smithi110.stdout:9/662: dread d9/f5f [0,4194304] 0 2024-04-25T15:49:39.215 INFO:tasks.workunit.client.1.smithi193.stdout:3/468: dwrite d8/d30/d72/f7e [0,4194304] 0 2024-04-25T15:49:39.216 INFO:tasks.workunit.client.0.smithi110.stdout:9/663: dread - d9/d44/d5e/fa7 zero size 2024-04-25T15:49:39.216 INFO:tasks.workunit.client.0.smithi110.stdout:7/734: truncate d1b/d29/d38/f53 293140 0 2024-04-25T15:49:39.219 INFO:tasks.workunit.client.1.smithi193.stdout:3/469: dread d8/d13/d14/d51/f6a [0,4194304] 0 2024-04-25T15:49:39.220 INFO:tasks.workunit.client.1.smithi193.stdout:0/429: rename d0/d49/d5/d7/dd/d16/d1a/d86 to d0/d49/d5/d7/dd/d16/d26/d2e/d93 0 2024-04-25T15:49:39.221 INFO:tasks.workunit.client.1.smithi193.stdout:0/430: write d0/d49/d5/d7/dd/d16/d26/d2e/f4b [4085469,89454] 0 2024-04-25T15:49:39.222 INFO:tasks.workunit.client.0.smithi110.stdout:1/848: creat d0/d14/d5f/f113 x:0 0 0 2024-04-25T15:49:39.224 INFO:tasks.workunit.client.1.smithi193.stdout:7/387: unlink d12/d31/d39/c5b 0 2024-04-25T15:49:39.225 INFO:tasks.workunit.client.1.smithi193.stdout:7/388: truncate d12/f2e 2930688 0 2024-04-25T15:49:39.225 INFO:tasks.workunit.client.0.smithi110.stdout:9/664: dwrite d9/db/df/d25/f9c [0,4194304] 0 2024-04-25T15:49:39.227 INFO:tasks.workunit.client.0.smithi110.stdout:9/665: fdatasync d9/d44/d5e/fa7 0 2024-04-25T15:49:39.231 INFO:tasks.workunit.client.1.smithi193.stdout:8/395: creat d2/d3/f80 x:0 0 0 2024-04-25T15:49:39.232 INFO:tasks.workunit.client.1.smithi193.stdout:2/417: chown d0/d9/d15/d31/d90/d9f 1 1 2024-04-25T15:49:39.234 INFO:tasks.workunit.client.1.smithi193.stdout:4/501: write db/d13/d1e/d1f/f63 [690082,96775] 0 2024-04-25T15:49:39.239 INFO:tasks.workunit.client.1.smithi193.stdout:4/502: dread f1 [0,4194304] 0 2024-04-25T15:49:39.244 INFO:tasks.workunit.client.1.smithi193.stdout:5/411: creat d17/d18/d30/f91 x:0 0 0 2024-04-25T15:49:39.245 INFO:tasks.workunit.client.0.smithi110.stdout:1/849: dread d0/d14/f1f [0,4194304] 0 2024-04-25T15:49:39.245 INFO:tasks.workunit.client.1.smithi193.stdout:5/412: stat d17/d18/d75/d85/l4b 0 2024-04-25T15:49:39.245 INFO:tasks.workunit.client.1.smithi193.stdout:0/431: dread d0/d49/d5/f24 [0,4194304] 0 2024-04-25T15:49:39.248 INFO:tasks.workunit.client.0.smithi110.stdout:6/653: symlink d1/db/d24/le5 0 2024-04-25T15:49:39.248 INFO:tasks.workunit.client.1.smithi193.stdout:1/369: dread dc/f12 [0,4194304] 0 2024-04-25T15:49:39.249 INFO:tasks.workunit.client.1.smithi193.stdout:5/413: dread d17/f86 [0,4194304] 0 2024-04-25T15:49:39.250 INFO:tasks.workunit.client.1.smithi193.stdout:5/414: fdatasync d17/d18/d30/f8a 0 2024-04-25T15:49:39.250 INFO:tasks.workunit.client.0.smithi110.stdout:5/732: mkdir d4/d15/d51/d90/ded/def 0 2024-04-25T15:49:39.251 INFO:tasks.workunit.client.0.smithi110.stdout:5/733: chown d4/d1d/d1f/d35/lae 108 1 2024-04-25T15:49:39.254 INFO:tasks.workunit.client.1.smithi193.stdout:3/470: symlink d8/d13/d14/d23/l8f 0 2024-04-25T15:49:39.255 INFO:tasks.workunit.client.1.smithi193.stdout:3/471: fsync d8/d13/d1f/f16 0 2024-04-25T15:49:39.255 INFO:tasks.workunit.client.1.smithi193.stdout:7/389: rmdir d12/d13 39 2024-04-25T15:49:39.256 INFO:tasks.workunit.client.0.smithi110.stdout:9/666: mkdir d9/d13/d5b/d61/db3/dd3 0 2024-04-25T15:49:39.259 INFO:tasks.workunit.client.1.smithi193.stdout:6/430: dwrite dd/f74 [0,4194304] 0 2024-04-25T15:49:39.260 INFO:tasks.workunit.client.0.smithi110.stdout:4/701: dwrite d13/d24/d2d/d44/fb4 [0,4194304] 0 2024-04-25T15:49:39.261 INFO:tasks.workunit.client.0.smithi110.stdout:6/654: readlink d1/l20 0 2024-04-25T15:49:39.264 INFO:tasks.workunit.client.1.smithi193.stdout:2/418: write d0/d37/d25/d5f/d3a/f58 [5090978,90382] 0 2024-04-25T15:49:39.264 INFO:tasks.workunit.client.0.smithi110.stdout:5/734: fsync d4/f7c 0 2024-04-25T15:49:39.265 INFO:tasks.workunit.client.0.smithi110.stdout:7/735: link d1b/d29/d46/c7b d1b/d29/d38/d55/d95/cf2 0 2024-04-25T15:49:39.267 INFO:tasks.workunit.client.1.smithi193.stdout:8/396: dwrite d2/f11 [0,4194304] 0 2024-04-25T15:49:39.267 INFO:tasks.workunit.client.0.smithi110.stdout:7/736: truncate d1b/f4b 1351063 0 2024-04-25T15:49:39.270 INFO:tasks.workunit.client.1.smithi193.stdout:8/397: stat d2/d3/d20/d45/d29/c2c 0 2024-04-25T15:49:39.272 INFO:tasks.workunit.client.1.smithi193.stdout:0/432: creat d0/d49/d5/d7/dd/d16/d18/d28/f94 x:0 0 0 2024-04-25T15:49:39.277 INFO:tasks.workunit.client.0.smithi110.stdout:9/667: fdatasync d9/f5f 0 2024-04-25T15:49:39.277 INFO:tasks.workunit.client.1.smithi193.stdout:9/423: dwrite df/d2c/d2f/d4e/d1c/d26/d3f/d41/f70 [0,4194304] 0 2024-04-25T15:49:39.277 INFO:tasks.workunit.client.0.smithi110.stdout:9/668: fdatasync d9/f8c 0 2024-04-25T15:49:39.278 INFO:tasks.workunit.client.0.smithi110.stdout:9/669: readlink d9/d13/d5b/d61/lb9 0 2024-04-25T15:49:39.280 INFO:tasks.workunit.client.0.smithi110.stdout:9/670: chown d9/db/df/d2c/d3c/l4e 109262821 1 2024-04-25T15:49:39.281 INFO:tasks.workunit.client.0.smithi110.stdout:5/735: read d4/d1d/f70 [175877,100108] 0 2024-04-25T15:49:39.283 INFO:tasks.workunit.client.0.smithi110.stdout:7/737: fsync d1b/f79 0 2024-04-25T15:49:39.286 INFO:tasks.workunit.client.1.smithi193.stdout:7/390: fsync d12/d13/f17 0 2024-04-25T15:49:39.288 INFO:tasks.workunit.client.1.smithi193.stdout:7/391: read d12/d13/f14 [2600820,64912] 0 2024-04-25T15:49:39.288 INFO:tasks.workunit.client.1.smithi193.stdout:7/392: fdatasync d12/f2e 0 2024-04-25T15:49:39.289 INFO:tasks.workunit.client.1.smithi193.stdout:7/393: dread - d12/d31/d39/f93 zero size 2024-04-25T15:49:39.292 INFO:tasks.workunit.client.1.smithi193.stdout:6/431: dread dd/d2f/f72 [0,4194304] 0 2024-04-25T15:49:39.293 INFO:tasks.workunit.client.1.smithi193.stdout:6/432: write dd/f1b [1233791,103366] 0 2024-04-25T15:49:39.295 INFO:tasks.workunit.client.1.smithi193.stdout:1/370: dread dc/f36 [0,4194304] 0 2024-04-25T15:49:39.301 INFO:tasks.workunit.client.1.smithi193.stdout:9/424: mknod df/d2c/d2f/d4e/d69/c7c 0 2024-04-25T15:49:39.301 INFO:tasks.workunit.client.1.smithi193.stdout:9/425: fsync df/d2c/d2f/f6e 0 2024-04-25T15:49:39.302 INFO:tasks.workunit.client.1.smithi193.stdout:9/426: readlink df/d2c/d2f/d4e/d1c/d26/l48 0 2024-04-25T15:49:39.306 INFO:tasks.workunit.client.1.smithi193.stdout:4/503: truncate db/d13/d1e/d1f/d22/f5d 10109835 0 2024-04-25T15:49:39.307 INFO:tasks.workunit.client.1.smithi193.stdout:7/394: mkdir d12/d1c/d96 0 2024-04-25T15:49:39.309 INFO:tasks.workunit.client.1.smithi193.stdout:7/395: truncate d12/d1c/d63/d70/d68/d8d/f89 475026 0 2024-04-25T15:49:39.309 INFO:tasks.workunit.client.1.smithi193.stdout:2/419: truncate d0/d37/f48 3471010 0 2024-04-25T15:49:39.312 INFO:tasks.workunit.client.1.smithi193.stdout:6/433: creat dd/d2f/d57/f7d x:0 0 0 2024-04-25T15:49:39.314 INFO:tasks.workunit.client.1.smithi193.stdout:5/415: truncate d17/f86 629269 0 2024-04-25T15:49:39.317 INFO:tasks.workunit.client.1.smithi193.stdout:9/427: creat df/d2c/d34/d3e/f7d x:0 0 0 2024-04-25T15:49:39.318 INFO:tasks.workunit.client.1.smithi193.stdout:0/433: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f51 [0,4194304] 0 2024-04-25T15:49:39.318 INFO:tasks.workunit.client.1.smithi193.stdout:4/504: fsync db/f15 0 2024-04-25T15:49:39.319 INFO:tasks.workunit.client.1.smithi193.stdout:0/434: stat d0/d49/d5/d7/dd/d16/d26/d2e/f76 0 2024-04-25T15:49:39.319 INFO:tasks.workunit.client.1.smithi193.stdout:7/396: unlink d12/d13/c56 0 2024-04-25T15:49:39.321 INFO:tasks.workunit.client.1.smithi193.stdout:1/371: truncate dc/f19 623707 0 2024-04-25T15:49:39.321 INFO:tasks.workunit.client.0.smithi110.stdout:4/702: truncate d13/d24/f45 4927737 0 2024-04-25T15:49:39.322 INFO:tasks.workunit.client.1.smithi193.stdout:6/434: truncate dd/d12/d19/d1c/d40/d3c/d3f/f41 551704 0 2024-04-25T15:49:39.323 INFO:tasks.workunit.client.1.smithi193.stdout:5/416: mknod d17/d18/d30/d38/c92 0 2024-04-25T15:49:39.326 INFO:tasks.workunit.client.0.smithi110.stdout:1/850: getdents d0/d23/db1 0 2024-04-25T15:49:39.327 INFO:tasks.workunit.client.0.smithi110.stdout:1/851: fdatasync d0/d51/f98 0 2024-04-25T15:49:39.330 INFO:tasks.workunit.client.0.smithi110.stdout:4/703: rmdir d13/d24 39 2024-04-25T15:49:39.339 INFO:tasks.workunit.client.0.smithi110.stdout:1/852: symlink d0/d14/d81/df8/d33/d8e/l114 0 2024-04-25T15:49:39.343 INFO:tasks.workunit.client.0.smithi110.stdout:9/671: rename d9/db/df/ca2 to d9/d13/cd4 0 2024-04-25T15:49:39.344 INFO:tasks.workunit.client.0.smithi110.stdout:4/704: mkdir d13/d24/d2d/d7d/da2/df4 0 2024-04-25T15:49:39.347 INFO:tasks.workunit.client.0.smithi110.stdout:1/853: rename d0/d14/d81/df8/db4/dcd/fe6 to d0/d14/d81/df8/d33/d94/f115 0 2024-04-25T15:49:39.351 INFO:tasks.workunit.client.0.smithi110.stdout:4/705: mknod d13/d24/d2d/d5e/def/cf5 0 2024-04-25T15:49:39.353 INFO:tasks.workunit.client.0.smithi110.stdout:1/854: creat d0/d51/dd8/df0/f116 x:0 0 0 2024-04-25T15:49:39.355 INFO:tasks.workunit.client.0.smithi110.stdout:4/706: creat d13/d24/d2d/d36/ff6 x:0 0 0 2024-04-25T15:49:39.356 INFO:tasks.workunit.client.0.smithi110.stdout:9/672: dread d9/d13/f8e [0,4194304] 0 2024-04-25T15:49:39.357 INFO:tasks.workunit.client.0.smithi110.stdout:9/673: chown d9/db/df/d25 1416249 1 2024-04-25T15:49:39.358 INFO:tasks.workunit.client.0.smithi110.stdout:4/707: symlink d13/d24/d2d/d36/d51/lf7 0 2024-04-25T15:49:39.359 INFO:tasks.workunit.client.0.smithi110.stdout:1/855: truncate d0/d14/d81/df8/d33/fdd 1350584 0 2024-04-25T15:49:39.359 INFO:tasks.workunit.client.0.smithi110.stdout:4/708: write d13/d24/d26/dcd/fdc [425788,81139] 0 2024-04-25T15:49:39.361 INFO:tasks.workunit.client.0.smithi110.stdout:9/674: symlink d9/d13/d5b/ld5 0 2024-04-25T15:49:39.362 INFO:tasks.workunit.client.0.smithi110.stdout:9/675: readlink d9/db/d58/l72 0 2024-04-25T15:49:39.366 INFO:tasks.workunit.client.0.smithi110.stdout:9/676: creat d9/db/df/d2c/d3c/fd6 x:0 0 0 2024-04-25T15:49:39.367 INFO:tasks.workunit.client.0.smithi110.stdout:4/709: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/ff8 x:0 0 0 2024-04-25T15:49:39.368 INFO:tasks.workunit.client.0.smithi110.stdout:1/856: dwrite d0/d14/f1f [0,4194304] 0 2024-04-25T15:49:39.369 INFO:tasks.workunit.client.0.smithi110.stdout:7/738: write d1b/d29/d46/f5c [847879,111396] 0 2024-04-25T15:49:39.370 INFO:tasks.workunit.client.0.smithi110.stdout:4/710: creat d13/d24/d2d/d44/da4/ff9 x:0 0 0 2024-04-25T15:49:39.373 INFO:tasks.workunit.client.1.smithi193.stdout:3/472: link d8/d13/d14/d27/l7a d8/d13/l90 0 2024-04-25T15:49:39.374 INFO:tasks.workunit.client.1.smithi193.stdout:9/428: mkdir df/d2c/d2f/d5c/d7e 0 2024-04-25T15:49:39.374 INFO:tasks.workunit.client.1.smithi193.stdout:7/397: dwrite d12/d1c/d63/d70/d68/d8d/f89 [0,4194304] 0 2024-04-25T15:49:39.375 INFO:tasks.workunit.client.1.smithi193.stdout:2/420: getdents d0/df/da5 0 2024-04-25T15:49:39.378 INFO:tasks.workunit.client.1.smithi193.stdout:6/435: write dd/fe [2102638,60189] 0 2024-04-25T15:49:39.379 INFO:tasks.workunit.client.1.smithi193.stdout:2/421: write d0/d37/d25/d5f/d3a/f58 [317787,116076] 0 2024-04-25T15:49:39.379 INFO:tasks.workunit.client.1.smithi193.stdout:1/372: unlink dc/l1d 0 2024-04-25T15:49:39.380 INFO:tasks.workunit.client.1.smithi193.stdout:3/473: creat d8/d13/d14/d27/d6d/f91 x:0 0 0 2024-04-25T15:49:39.381 INFO:tasks.workunit.client.1.smithi193.stdout:7/398: symlink d12/d13/l97 0 2024-04-25T15:49:39.382 INFO:tasks.workunit.client.1.smithi193.stdout:3/474: write d8/d13/d4b/f5d [2982773,109524] 0 2024-04-25T15:49:39.382 INFO:tasks.workunit.client.1.smithi193.stdout:2/422: truncate d0/d9/d15/d31/d50/f9a 65608 0 2024-04-25T15:49:39.383 INFO:tasks.workunit.client.1.smithi193.stdout:3/475: chown d8/d13/d1f/l34 490095 1 2024-04-25T15:49:39.383 INFO:tasks.workunit.client.1.smithi193.stdout:2/423: dread - d0/d42/d51/f94 zero size 2024-04-25T15:49:39.384 INFO:tasks.workunit.client.1.smithi193.stdout:9/429: symlink df/d2c/d34/d3e/d4d/l7f 0 2024-04-25T15:49:39.389 INFO:tasks.workunit.client.1.smithi193.stdout:1/373: mkdir dc/d4a/d50/d7c 0 2024-04-25T15:49:39.390 INFO:tasks.workunit.client.1.smithi193.stdout:9/430: dread df/d2c/d2f/d4e/d1c/d26/d3f/f4c [0,4194304] 0 2024-04-25T15:49:39.391 INFO:tasks.workunit.client.1.smithi193.stdout:6/436: creat dd/d12/d19/d45/f7e x:0 0 0 2024-04-25T15:49:39.393 INFO:tasks.workunit.client.1.smithi193.stdout:7/399: symlink d12/d1c/d63/d70/d68/d8d/l98 0 2024-04-25T15:49:39.395 INFO:tasks.workunit.client.1.smithi193.stdout:3/476: mknod d8/d30/d72/c92 0 2024-04-25T15:49:39.396 INFO:tasks.workunit.client.1.smithi193.stdout:2/424: creat d0/d42/d51/d4f/fac x:0 0 0 2024-04-25T15:49:39.398 INFO:tasks.workunit.client.1.smithi193.stdout:0/435: getdents d0/d49/d5 0 2024-04-25T15:49:39.399 INFO:tasks.workunit.client.1.smithi193.stdout:5/417: dwrite d17/f86 [0,4194304] 0 2024-04-25T15:49:39.400 INFO:tasks.workunit.client.1.smithi193.stdout:0/436: dread - d0/d49/d5/d61/f74 zero size 2024-04-25T15:49:39.400 INFO:tasks.workunit.client.1.smithi193.stdout:5/418: stat d17/d18/d30/d3b/l43 0 2024-04-25T15:49:39.401 INFO:tasks.workunit.client.1.smithi193.stdout:9/431: symlink df/d2c/d2f/d4e/d1c/d26/l80 0 2024-04-25T15:49:39.402 INFO:tasks.workunit.client.1.smithi193.stdout:5/419: chown d17/d18/d30/d38/d71/l90 2330606 1 2024-04-25T15:49:39.402 INFO:tasks.workunit.client.1.smithi193.stdout:7/400: creat d12/d1c/f99 x:0 0 0 2024-04-25T15:49:39.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/420: readlink d17/d18/d45/l4d 0 2024-04-25T15:49:39.404 INFO:tasks.workunit.client.1.smithi193.stdout:7/401: stat d12/d1c/d63/f80 0 2024-04-25T15:49:39.404 INFO:tasks.workunit.client.1.smithi193.stdout:7/402: fdatasync d12/d13/f24 0 2024-04-25T15:49:39.405 INFO:tasks.workunit.client.1.smithi193.stdout:8/398: sync 2024-04-25T15:49:39.407 INFO:tasks.workunit.client.1.smithi193.stdout:2/425: truncate d0/d9/d15/f1b 1166798 0 2024-04-25T15:49:39.409 INFO:tasks.workunit.client.1.smithi193.stdout:0/437: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/f95 x:0 0 0 2024-04-25T15:49:39.409 INFO:tasks.workunit.client.1.smithi193.stdout:2/426: write d0/d42/d51/d4f/f69 [891916,116403] 0 2024-04-25T15:49:39.411 INFO:tasks.workunit.client.1.smithi193.stdout:5/421: symlink d17/d18/d30/d38/l93 0 2024-04-25T15:49:39.411 INFO:tasks.workunit.client.1.smithi193.stdout:7/403: dread d12/d31/d37/d62/d6e/f8c [0,4194304] 0 2024-04-25T15:49:39.414 INFO:tasks.workunit.client.1.smithi193.stdout:8/399: truncate d2/d3/d3a/f4c 1558717 0 2024-04-25T15:49:39.414 INFO:tasks.workunit.client.1.smithi193.stdout:9/432: truncate df/d11/f2e 7762423 0 2024-04-25T15:49:39.416 INFO:tasks.workunit.client.1.smithi193.stdout:2/427: creat d0/d37/d4d/fad x:0 0 0 2024-04-25T15:49:39.418 INFO:tasks.workunit.client.1.smithi193.stdout:8/400: dread d2/f11 [0,4194304] 0 2024-04-25T15:49:39.418 INFO:tasks.workunit.client.1.smithi193.stdout:2/428: write d0/d37/d25/d5f/d3a/d6d/d6e/f3e [1671376,4084] 0 2024-04-25T15:49:39.420 INFO:tasks.workunit.client.0.smithi110.stdout:7/739: write d1b/d29/db5/dca/fd5 [468702,122312] 0 2024-04-25T15:49:39.420 INFO:tasks.workunit.client.1.smithi193.stdout:2/429: write d0/d42/d51/d4f/d65/d7e/f82 [766973,121532] 0 2024-04-25T15:49:39.421 INFO:tasks.workunit.client.0.smithi110.stdout:7/740: fdatasync f2 0 2024-04-25T15:49:39.423 INFO:tasks.workunit.client.0.smithi110.stdout:1/857: write d0/d14/f29 [4107284,15772] 0 2024-04-25T15:49:39.426 INFO:tasks.workunit.client.0.smithi110.stdout:7/741: dread - d1b/fb4 zero size 2024-04-25T15:49:39.427 INFO:tasks.workunit.client.1.smithi193.stdout:9/433: dwrite df/d2c/d2f/d33/f3c [0,4194304] 0 2024-04-25T15:49:39.432 INFO:tasks.workunit.client.1.smithi193.stdout:0/438: getdents d0/d49/d5/d7/dd/d16/d26 0 2024-04-25T15:49:39.433 INFO:tasks.workunit.client.1.smithi193.stdout:9/434: rename df/d2c/d2f/d4e/d1c/d26/d3f to df/d2c/d34/d3e/d4d/d5e/d81 0 2024-04-25T15:49:39.435 INFO:tasks.workunit.client.1.smithi193.stdout:0/439: truncate d0/d49/d5/d7/dd/d16/f6f 431983 0 2024-04-25T15:49:39.435 INFO:tasks.workunit.client.1.smithi193.stdout:2/430: mknod d0/d37/cae 0 2024-04-25T15:49:39.436 INFO:tasks.workunit.client.1.smithi193.stdout:9/435: symlink df/d2c/d34/d3e/d4d/l82 0 2024-04-25T15:49:39.436 INFO:tasks.workunit.client.1.smithi193.stdout:2/431: truncate d0/d42/d51/d4f/fac 845461 0 2024-04-25T15:49:39.437 INFO:tasks.workunit.client.1.smithi193.stdout:9/436: rename df to df/d2c/d2f/d4e/d83 22 2024-04-25T15:49:39.439 INFO:tasks.workunit.client.1.smithi193.stdout:0/440: symlink d0/d49/d5/d7/l96 0 2024-04-25T15:49:39.441 INFO:tasks.workunit.client.1.smithi193.stdout:9/437: creat df/d11/f84 x:0 0 0 2024-04-25T15:49:39.441 INFO:tasks.workunit.client.1.smithi193.stdout:0/441: dread d0/d49/d5/d7/dd/d16/d26/d2e/f76 [0,4194304] 0 2024-04-25T15:49:39.444 INFO:tasks.workunit.client.1.smithi193.stdout:0/442: truncate d0/d49/d5/d7/dd/d46/d48/f57 988623 0 2024-04-25T15:49:39.444 INFO:tasks.workunit.client.1.smithi193.stdout:2/432: mknod d0/d9/d15/d31/d90/d9c/caf 0 2024-04-25T15:49:39.449 INFO:tasks.workunit.client.1.smithi193.stdout:7/404: truncate d12/d13/f24 1229634 0 2024-04-25T15:49:39.451 INFO:tasks.workunit.client.1.smithi193.stdout:9/438: link df/d2c/d2f/d4e/d1c/d30/d3b/f5b df/d2c/d2f/d4e/d1c/d30/d60/f85 0 2024-04-25T15:49:39.452 INFO:tasks.workunit.client.1.smithi193.stdout:5/422: dread d17/d18/d30/d34/f6c [0,4194304] 0 2024-04-25T15:49:39.454 INFO:tasks.workunit.client.1.smithi193.stdout:3/477: dwrite d8/d13/f32 [0,4194304] 0 2024-04-25T15:49:39.457 INFO:tasks.workunit.client.1.smithi193.stdout:9/439: rmdir df/d2c/d2f/d5c 39 2024-04-25T15:49:39.458 INFO:tasks.workunit.client.1.smithi193.stdout:8/401: write d2/d3/f16 [1825130,103175] 0 2024-04-25T15:49:39.459 INFO:tasks.workunit.client.1.smithi193.stdout:5/423: mknod d17/d18/d75/d85/c94 0 2024-04-25T15:49:39.464 INFO:tasks.workunit.client.1.smithi193.stdout:0/443: dwrite d0/d49/d5/d7/dd/d46/d48/d62/f92 [4194304,4194304] 0 2024-04-25T15:49:39.465 INFO:tasks.workunit.client.1.smithi193.stdout:2/433: dwrite d0/d44/d8b/f9e [0,4194304] 0 2024-04-25T15:49:39.467 INFO:tasks.workunit.client.1.smithi193.stdout:2/434: readlink d0/l2 0 2024-04-25T15:49:39.469 INFO:tasks.workunit.client.1.smithi193.stdout:7/405: dwrite d12/f52 [0,4194304] 0 2024-04-25T15:49:39.470 INFO:tasks.workunit.client.1.smithi193.stdout:5/424: creat d17/d83/f95 x:0 0 0 2024-04-25T15:49:39.472 INFO:tasks.workunit.client.1.smithi193.stdout:7/406: write d12/d31/d39/f93 [95225,105362] 0 2024-04-25T15:49:39.474 INFO:tasks.workunit.client.1.smithi193.stdout:5/425: write d17/d83/f95 [265911,75972] 0 2024-04-25T15:49:39.475 INFO:tasks.workunit.client.1.smithi193.stdout:9/440: rmdir df/d2c/d2f 39 2024-04-25T15:49:39.478 INFO:tasks.workunit.client.1.smithi193.stdout:4/505: sync 2024-04-25T15:49:39.480 INFO:tasks.workunit.client.1.smithi193.stdout:8/402: dwrite d2/d3/f68 [0,4194304] 0 2024-04-25T15:49:39.482 INFO:tasks.workunit.client.1.smithi193.stdout:4/506: read - db/f8d zero size 2024-04-25T15:49:39.483 INFO:tasks.workunit.client.1.smithi193.stdout:7/407: symlink d12/d1c/d45/l9a 0 2024-04-25T15:49:39.487 INFO:tasks.workunit.client.1.smithi193.stdout:8/403: dread - d2/d3/f5f zero size 2024-04-25T15:49:39.488 INFO:tasks.workunit.client.1.smithi193.stdout:2/435: mknod d0/d9/d15/d31/d90/d9f/cb0 0 2024-04-25T15:49:39.488 INFO:tasks.workunit.client.1.smithi193.stdout:3/478: write d8/fc [4662472,2531] 0 2024-04-25T15:49:39.491 INFO:tasks.workunit.client.1.smithi193.stdout:0/444: dwrite d0/d49/d5/d7/dd/d16/d18/d28/d56/f70 [0,4194304] 0 2024-04-25T15:49:39.495 INFO:tasks.workunit.client.1.smithi193.stdout:5/426: dwrite d17/d18/d75/d85/f73 [0,4194304] 0 2024-04-25T15:49:39.497 INFO:tasks.workunit.client.1.smithi193.stdout:6/437: sync 2024-04-25T15:49:39.497 INFO:tasks.workunit.client.1.smithi193.stdout:1/374: sync 2024-04-25T15:49:39.497 INFO:tasks.workunit.client.1.smithi193.stdout:8/404: unlink d2/d3/d27/d34/l35 0 2024-04-25T15:49:39.499 INFO:tasks.workunit.client.1.smithi193.stdout:2/436: mknod d0/d42/d51/d4f/d65/d7e/cb1 0 2024-04-25T15:49:39.503 INFO:tasks.workunit.client.1.smithi193.stdout:9/441: dwrite df/d2c/d34/d3e/d4d/d5e/d81/f43 [0,4194304] 0 2024-04-25T15:49:39.503 INFO:tasks.workunit.client.1.smithi193.stdout:8/405: dread - d2/f75 zero size 2024-04-25T15:49:39.504 INFO:tasks.workunit.client.1.smithi193.stdout:0/445: creat d0/d49/d6a/f97 x:0 0 0 2024-04-25T15:49:39.506 INFO:tasks.workunit.client.1.smithi193.stdout:7/408: dread d12/d31/d39/f3a [0,4194304] 0 2024-04-25T15:49:39.507 INFO:tasks.workunit.client.1.smithi193.stdout:8/406: dread - d2/d3/d27/f7a zero size 2024-04-25T15:49:39.508 INFO:tasks.workunit.client.1.smithi193.stdout:5/427: mkdir d17/d18/d30/d38/d71/d96 0 2024-04-25T15:49:39.508 INFO:tasks.workunit.client.1.smithi193.stdout:0/446: truncate d0/d49/d6a/f97 649693 0 2024-04-25T15:49:39.509 INFO:tasks.workunit.client.1.smithi193.stdout:7/409: fdatasync d12/d1c/d63/d70/f34 0 2024-04-25T15:49:39.510 INFO:tasks.workunit.client.1.smithi193.stdout:5/428: dread - d17/f89 zero size 2024-04-25T15:49:39.512 INFO:tasks.workunit.client.1.smithi193.stdout:3/479: rename d8/f15 to d8/d13/d14/f93 0 2024-04-25T15:49:39.512 INFO:tasks.workunit.client.1.smithi193.stdout:0/447: fsync d0/d49/d5/d7/dd/f5c 0 2024-04-25T15:49:39.514 INFO:tasks.workunit.client.1.smithi193.stdout:7/410: creat d12/d31/d37/d62/f9b x:0 0 0 2024-04-25T15:49:39.516 INFO:tasks.workunit.client.1.smithi193.stdout:6/438: creat dd/d12/d19/f7f x:0 0 0 2024-04-25T15:49:39.517 INFO:tasks.workunit.client.1.smithi193.stdout:8/407: dread d2/d3/f68 [0,4194304] 0 2024-04-25T15:49:39.518 INFO:tasks.workunit.client.1.smithi193.stdout:1/375: rename dc/d4a/d50/f6a to dc/d4a/d77/f7d 0 2024-04-25T15:49:39.518 INFO:tasks.workunit.client.1.smithi193.stdout:3/480: symlink d8/d13/d1f/l94 0 2024-04-25T15:49:39.520 INFO:tasks.workunit.client.1.smithi193.stdout:7/411: truncate d12/d31/d39/f3a 159022 0 2024-04-25T15:49:39.521 INFO:tasks.workunit.client.1.smithi193.stdout:6/439: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80 0 2024-04-25T15:49:39.521 INFO:tasks.workunit.client.1.smithi193.stdout:8/408: symlink d2/d3/d27/d6f/l81 0 2024-04-25T15:49:39.522 INFO:tasks.workunit.client.1.smithi193.stdout:6/440: read - dd/d12/d19/d45/f7e zero size 2024-04-25T15:49:39.523 INFO:tasks.workunit.client.1.smithi193.stdout:8/409: write d2/d3/d20/f33 [3870130,106018] 0 2024-04-25T15:49:39.523 INFO:tasks.workunit.client.1.smithi193.stdout:3/481: creat d8/d13/d14/d27/f95 x:0 0 0 2024-04-25T15:49:39.526 INFO:tasks.workunit.client.1.smithi193.stdout:1/376: rmdir dc/d4a/d50/d5c/d63/d70 39 2024-04-25T15:49:39.527 INFO:tasks.workunit.client.1.smithi193.stdout:6/441: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/l81 0 2024-04-25T15:49:39.527 INFO:tasks.workunit.client.1.smithi193.stdout:1/377: readlink la 0 2024-04-25T15:49:39.528 INFO:tasks.workunit.client.1.smithi193.stdout:5/429: rename d17/d18/d30/f32 to d17/f97 0 2024-04-25T15:49:39.531 INFO:tasks.workunit.client.1.smithi193.stdout:5/430: write d17/d18/d30/d38/d48/f8f [1033245,128344] 0 2024-04-25T15:49:39.531 INFO:tasks.workunit.client.1.smithi193.stdout:4/507: dread db/d32/d45/f60 [0,4194304] 0 2024-04-25T15:49:39.532 INFO:tasks.workunit.client.0.smithi110.stdout:4/711: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/ff8 [0,4194304] 0 2024-04-25T15:49:39.533 INFO:tasks.workunit.client.0.smithi110.stdout:4/712: stat d13/cdd 0 2024-04-25T15:49:39.533 INFO:tasks.workunit.client.1.smithi193.stdout:4/508: truncate db/d13/d1e/d1f/d56/d69/d70/d8a/f8c 585170 0 2024-04-25T15:49:39.534 INFO:tasks.workunit.client.1.smithi193.stdout:4/509: chown db/d13/l19 8688604 1 2024-04-25T15:49:39.534 INFO:tasks.workunit.client.0.smithi110.stdout:9/677: dread d9/db/df/d2c/f36 [0,4194304] 0 2024-04-25T15:49:39.535 INFO:tasks.workunit.client.0.smithi110.stdout:4/713: read - d13/d24/d2d/d36/d51/fe4 zero size 2024-04-25T15:49:39.536 INFO:tasks.workunit.client.1.smithi193.stdout:6/442: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/c82 0 2024-04-25T15:49:39.537 INFO:tasks.workunit.client.0.smithi110.stdout:4/714: dread - d13/d24/d2d/d44/d90/fc5 zero size 2024-04-25T15:49:39.538 INFO:tasks.workunit.client.0.smithi110.stdout:7/742: mkdir d1b/d29/d38/d8b/d89/dc6/df3 0 2024-04-25T15:49:39.539 INFO:tasks.workunit.client.0.smithi110.stdout:4/715: write d13/d24/d26/f49 [3707521,34620] 0 2024-04-25T15:49:39.540 INFO:tasks.workunit.client.0.smithi110.stdout:7/743: truncate d1b/d29/d46/fe5 255962 0 2024-04-25T15:49:39.540 INFO:tasks.workunit.client.1.smithi193.stdout:4/510: dread db/d32/f7f [0,4194304] 0 2024-04-25T15:49:39.541 INFO:tasks.workunit.client.0.smithi110.stdout:1/858: symlink d0/d14/l117 0 2024-04-25T15:49:39.545 INFO:tasks.workunit.client.1.smithi193.stdout:2/437: dwrite d0/d9/d15/d31/d50/f5c [0,4194304] 0 2024-04-25T15:49:39.545 INFO:tasks.workunit.client.0.smithi110.stdout:9/678: rmdir d9/d44/d5e 39 2024-04-25T15:49:39.546 INFO:tasks.workunit.client.1.smithi193.stdout:1/378: mkdir dc/d4a/d50/d7c/d7e 0 2024-04-25T15:49:39.550 INFO:tasks.workunit.client.1.smithi193.stdout:8/410: dread d2/d3/ff [0,4194304] 0 2024-04-25T15:49:39.551 INFO:tasks.workunit.client.1.smithi193.stdout:6/443: rename dd/f2e to dd/d12/d19/d1c/d40/d3c/d3f/f83 0 2024-04-25T15:49:39.553 INFO:tasks.workunit.client.1.smithi193.stdout:6/444: chown dd/d12/d19/d1c 6498 1 2024-04-25T15:49:39.553 INFO:tasks.workunit.client.1.smithi193.stdout:8/411: fsync d2/d3/d3a/d6d/f2b 0 2024-04-25T15:49:39.554 INFO:tasks.workunit.client.1.smithi193.stdout:9/442: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/f5b [0,4194304] 0 2024-04-25T15:49:39.557 INFO:tasks.workunit.client.1.smithi193.stdout:8/412: truncate d2/d3/d3a/f61 421865 0 2024-04-25T15:49:39.557 INFO:tasks.workunit.client.1.smithi193.stdout:0/448: truncate d0/d49/d5/d7/dd/d16/f6f 174243 0 2024-04-25T15:49:39.557 INFO:tasks.workunit.client.1.smithi193.stdout:9/443: write df/d2c/d2f/d4e/d1c/d30/d60/f85 [1855153,78151] 0 2024-04-25T15:49:39.557 INFO:tasks.workunit.client.1.smithi193.stdout:7/412: link d12/d31/d37/d62/d6e/d79/f4a d12/d1c/d63/d70/d68/d8d/f9c 0 2024-04-25T15:49:39.562 INFO:tasks.workunit.client.0.smithi110.stdout:4/716: rename d13/d24/d2d/d7d to d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa 0 2024-04-25T15:49:39.563 INFO:tasks.workunit.client.1.smithi193.stdout:9/444: read df/d2c/d2f/d4e/f3a [531941,122228] 0 2024-04-25T15:49:39.566 INFO:tasks.workunit.client.1.smithi193.stdout:3/482: write d8/d13/d1f/d69/d77/f53 [1548428,60909] 0 2024-04-25T15:49:39.568 INFO:tasks.workunit.client.1.smithi193.stdout:3/483: write d8/d13/d4b/f88 [736327,60154] 0 2024-04-25T15:49:39.570 INFO:tasks.workunit.client.0.smithi110.stdout:9/679: rename d9/db/df/d55 to d9/d13/d5b/d61/d67/dd7 0 2024-04-25T15:49:39.571 INFO:tasks.workunit.client.0.smithi110.stdout:9/680: read - d9/d13/d5b/d61/f9d zero size 2024-04-25T15:49:39.572 INFO:tasks.workunit.client.0.smithi110.stdout:4/717: creat d13/dba/ffb x:0 0 0 2024-04-25T15:49:39.573 INFO:tasks.workunit.client.1.smithi193.stdout:1/379: dwrite dc/f30 [4194304,4194304] 0 2024-04-25T15:49:39.575 INFO:tasks.workunit.client.1.smithi193.stdout:4/511: rename db/d13/f82 to db/d13/d1e/d1f/d56/d69/d70/f9c 0 2024-04-25T15:49:39.578 INFO:tasks.workunit.client.1.smithi193.stdout:6/445: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/l84 0 2024-04-25T15:49:39.578 INFO:tasks.workunit.client.0.smithi110.stdout:4/718: creat d13/d24/d2d/d55/ffc x:0 0 0 2024-04-25T15:49:39.581 INFO:tasks.workunit.client.1.smithi193.stdout:4/512: dread db/f6a [0,4194304] 0 2024-04-25T15:49:39.582 INFO:tasks.workunit.client.1.smithi193.stdout:8/413: symlink d2/d3/d20/d45/d4a/l82 0 2024-04-25T15:49:39.583 INFO:tasks.workunit.client.1.smithi193.stdout:4/513: write db/d13/d1e/f41 [1673097,9855] 0 2024-04-25T15:49:39.583 INFO:tasks.workunit.client.1.smithi193.stdout:8/414: write d2/d3/d20/f22 [1265583,43900] 0 2024-04-25T15:49:39.584 INFO:tasks.workunit.client.1.smithi193.stdout:4/514: write db/d4b/f7b [3989220,126652] 0 2024-04-25T15:49:39.585 INFO:tasks.workunit.client.1.smithi193.stdout:9/445: dread df/d2c/d2f/d4e/f1a [0,4194304] 0 2024-04-25T15:49:39.586 INFO:tasks.workunit.client.1.smithi193.stdout:0/449: creat d0/d49/d5/d7/dd/d46/d48/f98 x:0 0 0 2024-04-25T15:49:39.586 INFO:tasks.workunit.client.0.smithi110.stdout:4/719: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/d88/f20 [0,4194304] 0 2024-04-25T15:49:39.587 INFO:tasks.workunit.client.0.smithi110.stdout:9/681: link d9/d13/d5b/d61/d67/dd7/l7e d9/db/df/d2c/d3c/ld8 0 2024-04-25T15:49:39.587 INFO:tasks.workunit.client.0.smithi110.stdout:7/744: dwrite d1b/d29/d46/f51 [0,4194304] 0 2024-04-25T15:49:39.588 INFO:tasks.workunit.client.1.smithi193.stdout:9/446: write df/d2c/d2f/d4e/d1c/d30/d3b/f5b [4073855,10388] 0 2024-04-25T15:49:39.588 INFO:tasks.workunit.client.1.smithi193.stdout:0/450: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60 143432 1 2024-04-25T15:49:39.590 INFO:tasks.workunit.client.0.smithi110.stdout:7/745: write d1b/d29/d38/d55/fbd [719939,56423] 0 2024-04-25T15:49:39.591 INFO:tasks.workunit.client.0.smithi110.stdout:7/746: chown d1b/d29/db5 765 1 2024-04-25T15:49:39.591 INFO:tasks.workunit.client.1.smithi193.stdout:7/413: chown d12/d1c/d63/d70/d68/d8d/f9c 234481703 1 2024-04-25T15:49:39.594 INFO:tasks.workunit.client.0.smithi110.stdout:1/859: write d0/d23/f13 [3812382,91639] 0 2024-04-25T15:49:39.595 INFO:tasks.workunit.client.0.smithi110.stdout:9/682: stat d9/db/c12 0 2024-04-25T15:49:39.602 INFO:tasks.workunit.client.1.smithi193.stdout:5/431: dwrite d17/d18/d30/d38/d71/f74 [4194304,4194304] 0 2024-04-25T15:49:39.606 INFO:tasks.workunit.client.1.smithi193.stdout:3/484: creat d8/d13/d14/d27/f96 x:0 0 0 2024-04-25T15:49:39.609 INFO:tasks.workunit.client.1.smithi193.stdout:5/432: dread d17/d83/f95 [0,4194304] 0 2024-04-25T15:49:39.612 INFO:tasks.workunit.client.1.smithi193.stdout:5/433: chown d17/d18/d30/d38/d48/d4f/f5f 0 1 2024-04-25T15:49:39.612 INFO:tasks.workunit.client.1.smithi193.stdout:0/451: dread d0/d49/d5/d7/dd/d16/d26/d5e/f69 [0,4194304] 0 2024-04-25T15:49:39.613 INFO:tasks.workunit.client.0.smithi110.stdout:9/683: dwrite d9/d13/fbf [0,4194304] 0 2024-04-25T15:49:39.615 INFO:tasks.workunit.client.1.smithi193.stdout:3/485: dwrite d8/d13/f32 [0,4194304] 0 2024-04-25T15:49:39.617 INFO:tasks.workunit.client.1.smithi193.stdout:8/415: symlink d2/d3/d3a/d6d/l83 0 2024-04-25T15:49:39.618 INFO:tasks.workunit.client.0.smithi110.stdout:9/684: symlink d9/db/d58/ld9 0 2024-04-25T15:49:39.620 INFO:tasks.workunit.client.1.smithi193.stdout:7/414: mkdir d12/d31/d37/d62/d6e/d79/d9d 0 2024-04-25T15:49:39.621 INFO:tasks.workunit.client.0.smithi110.stdout:7/747: dwrite d1b/d29/d38/f54 [0,4194304] 0 2024-04-25T15:49:39.621 INFO:tasks.workunit.client.0.smithi110.stdout:1/860: write d0/f27 [3862142,68398] 0 2024-04-25T15:49:39.623 INFO:tasks.workunit.client.1.smithi193.stdout:8/416: fsync d2/d3/d3a/f76 0 2024-04-25T15:49:39.628 INFO:tasks.workunit.client.0.smithi110.stdout:1/861: symlink d0/d14/d5f/d7d/d72/d88/l118 0 2024-04-25T15:49:39.629 INFO:tasks.workunit.client.1.smithi193.stdout:0/452: dwrite d0/d49/d6a/f97 [0,4194304] 0 2024-04-25T15:49:39.630 INFO:tasks.workunit.client.0.smithi110.stdout:1/862: fsync d0/d14/d5f/d7d/fbd 0 2024-04-25T15:49:39.632 INFO:tasks.workunit.client.1.smithi193.stdout:2/438: dwrite d0/d9/d3d/d7d/fab [0,4194304] 0 2024-04-25T15:49:39.635 INFO:tasks.workunit.client.1.smithi193.stdout:4/515: mkdir db/d13/d9d 0 2024-04-25T15:49:39.636 INFO:tasks.workunit.client.1.smithi193.stdout:9/447: mknod df/d2c/d2f/c86 0 2024-04-25T15:49:39.637 INFO:tasks.workunit.client.1.smithi193.stdout:2/439: write d0/d9/d3d/fa3 [894491,79966] 0 2024-04-25T15:49:39.638 INFO:tasks.workunit.client.1.smithi193.stdout:3/486: symlink d8/d30/d72/l97 0 2024-04-25T15:49:39.638 INFO:tasks.workunit.client.1.smithi193.stdout:8/417: fsync d2/f48 0 2024-04-25T15:49:39.639 INFO:tasks.workunit.client.1.smithi193.stdout:9/448: chown df/f57 0 1 2024-04-25T15:49:39.640 INFO:tasks.workunit.client.1.smithi193.stdout:6/446: dwrite dd/d12/f1e [0,4194304] 0 2024-04-25T15:49:39.642 INFO:tasks.workunit.client.1.smithi193.stdout:0/453: mkdir d0/d49/d5/d7/dd/d16/d26/d5e/d99 0 2024-04-25T15:49:39.642 INFO:tasks.workunit.client.1.smithi193.stdout:5/434: unlink d17/d18/d30/d34/d52/l6f 0 2024-04-25T15:49:39.643 INFO:tasks.workunit.client.1.smithi193.stdout:3/487: readlink d8/d13/d14/d27/d6d/l81 0 2024-04-25T15:49:39.643 INFO:tasks.workunit.client.1.smithi193.stdout:4/516: mknod db/d13/d1e/d1f/d56/c9e 0 2024-04-25T15:49:39.644 INFO:tasks.workunit.client.1.smithi193.stdout:3/488: readlink d8/d30/d72/l97 0 2024-04-25T15:49:39.646 INFO:tasks.workunit.client.1.smithi193.stdout:2/440: dread d0/d37/d25/f77 [0,4194304] 0 2024-04-25T15:49:39.659 INFO:tasks.workunit.client.0.smithi110.stdout:7/748: getdents d1b/d29/d38/d8b/d89/dc6/df3 0 2024-04-25T15:49:39.660 INFO:tasks.workunit.client.0.smithi110.stdout:7/749: write d1b/d29/d38/d55/fbd [656919,92101] 0 2024-04-25T15:49:39.669 INFO:tasks.workunit.client.0.smithi110.stdout:7/750: rename d1b/d29/f57 to d1b/d29/d38/d55/d95/ff4 0 2024-04-25T15:49:39.670 INFO:tasks.workunit.client.0.smithi110.stdout:9/685: write d9/d13/f29 [1552683,54249] 0 2024-04-25T15:49:39.676 INFO:tasks.workunit.client.0.smithi110.stdout:7/751: creat d1b/d29/d38/d55/dab/db2/db8/ff5 x:0 0 0 2024-04-25T15:49:39.678 INFO:tasks.workunit.client.0.smithi110.stdout:7/752: symlink d1b/d29/d38/d55/dab/db2/db8/lf6 0 2024-04-25T15:49:39.681 INFO:tasks.workunit.client.0.smithi110.stdout:7/753: symlink d1b/d29/d38/d55/d7f/lf7 0 2024-04-25T15:49:39.683 INFO:tasks.workunit.client.0.smithi110.stdout:1/863: write d0/d14/d5f/d6f/f82 [488275,47978] 0 2024-04-25T15:49:39.689 INFO:tasks.workunit.client.0.smithi110.stdout:1/864: write d0/d14/d5f/d7d/d9c/fa0 [2346481,21086] 0 2024-04-25T15:49:39.692 INFO:tasks.workunit.client.1.smithi193.stdout:0/454: fdatasync d0/d49/d5/d7/dd/f58 0 2024-04-25T15:49:39.692 INFO:tasks.workunit.client.1.smithi193.stdout:9/449: rename df/d2d/c45 to df/d2c/d34/d3e/d4d/d5e/d81/d41/c87 0 2024-04-25T15:49:39.693 INFO:tasks.workunit.client.1.smithi193.stdout:5/435: chown d17/c2b 139098 1 2024-04-25T15:49:39.693 INFO:tasks.workunit.client.1.smithi193.stdout:4/517: mknod db/d13/d1e/d1f/d22/c9f 0 2024-04-25T15:49:39.693 INFO:tasks.workunit.client.1.smithi193.stdout:1/380: write dc/f19 [1066246,29538] 0 2024-04-25T15:49:39.694 INFO:tasks.workunit.client.1.smithi193.stdout:5/436: fdatasync d17/f89 0 2024-04-25T15:49:39.694 INFO:tasks.workunit.client.0.smithi110.stdout:1/865: dwrite d0/d23/f92 [4194304,4194304] 0 2024-04-25T15:49:39.696 INFO:tasks.workunit.client.1.smithi193.stdout:9/450: chown df/d2c/d2f/d4e/d1c/d26/l36 373314 1 2024-04-25T15:49:39.697 INFO:tasks.workunit.client.0.smithi110.stdout:1/866: creat d0/d14/d81/df8/d33/da9/f119 x:0 0 0 2024-04-25T15:49:39.697 INFO:tasks.workunit.client.1.smithi193.stdout:4/518: truncate db/d32/d40/f93 601521 0 2024-04-25T15:49:39.700 INFO:tasks.workunit.client.0.smithi110.stdout:1/867: symlink d0/d14/d5f/d7d/d9c/dcc/l11a 0 2024-04-25T15:49:39.702 INFO:tasks.workunit.client.0.smithi110.stdout:1/868: write d0/d14/d81/df8/d33/ff3 [191518,59208] 0 2024-04-25T15:49:39.706 INFO:tasks.workunit.client.0.smithi110.stdout:1/869: symlink d0/d14/d5f/d7d/d9c/l11b 0 2024-04-25T15:49:39.712 INFO:tasks.workunit.client.0.smithi110.stdout:1/870: rename d0/d14/d81/f9d to d0/d14/d5f/f11c 0 2024-04-25T15:49:39.716 INFO:tasks.workunit.client.0.smithi110.stdout:1/871: rename d0/d14/d5f/d7d/l102 to d0/d14/d81/df8/l11d 0 2024-04-25T15:49:39.717 INFO:tasks.workunit.client.0.smithi110.stdout:1/872: chown d0/d14/d5f/d7d/d72/d88/db6/fd4 27112 1 2024-04-25T15:49:39.717 INFO:tasks.workunit.client.0.smithi110.stdout:1/873: chown d0/d14/d5f/d7d/d72/d88/db6/ceb 52 1 2024-04-25T15:49:39.722 INFO:tasks.workunit.client.0.smithi110.stdout:1/874: write d0/d23/f83 [660308,20384] 0 2024-04-25T15:49:39.725 INFO:tasks.workunit.client.0.smithi110.stdout:1/875: getdents d0/d14/d5f/d7d/d72/d88/dbf 0 2024-04-25T15:49:39.728 INFO:tasks.workunit.client.0.smithi110.stdout:1/876: getdents d0/d14/d81/df8/d33/d94 0 2024-04-25T15:49:39.729 INFO:tasks.workunit.client.0.smithi110.stdout:1/877: mkdir d0/d23/d11e 0 2024-04-25T15:49:39.730 INFO:tasks.workunit.client.0.smithi110.stdout:1/878: truncate d0/d14/f57 1531662 0 2024-04-25T15:49:39.731 INFO:tasks.workunit.client.0.smithi110.stdout:1/879: fdatasync d0/d14/d5f/d7d/d72/f7e 0 2024-04-25T15:49:39.747 INFO:tasks.workunit.client.0.smithi110.stdout:1/880: dread d0/d14/d5f/f65 [0,4194304] 0 2024-04-25T15:49:39.760 INFO:tasks.workunit.client.1.smithi193.stdout:3/489: creat d8/d30/f98 x:0 0 0 2024-04-25T15:49:39.761 INFO:tasks.workunit.client.1.smithi193.stdout:0/455: creat d0/d49/d5/d7/dd/d16/d26/d5e/d99/f9a x:0 0 0 2024-04-25T15:49:39.761 INFO:tasks.workunit.client.1.smithi193.stdout:9/451: fsync df/d2c/d2f/f49 0 2024-04-25T15:49:39.763 INFO:tasks.workunit.client.1.smithi193.stdout:7/415: dwrite d12/d31/d37/d62/d6e/f8c [0,4194304] 0 2024-04-25T15:49:39.766 INFO:tasks.workunit.client.1.smithi193.stdout:8/418: dwrite d2/d3/f37 [0,4194304] 0 2024-04-25T15:49:39.768 INFO:tasks.workunit.client.1.smithi193.stdout:0/456: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/l83 2 1 2024-04-25T15:49:39.769 INFO:tasks.workunit.client.1.smithi193.stdout:1/381: rename dc/d4a/d50/d5c/d63/d70/c74 to dc/d4a/c7f 0 2024-04-25T15:49:39.771 INFO:tasks.workunit.client.1.smithi193.stdout:9/452: mknod df/d2c/d2f/d4e/c88 0 2024-04-25T15:49:39.771 INFO:tasks.workunit.client.1.smithi193.stdout:2/441: dwrite d0/d9/d15/f1b [0,4194304] 0 2024-04-25T15:49:39.778 INFO:tasks.workunit.client.1.smithi193.stdout:9/453: write df/d2c/d2f/d4e/d1c/d30/d60/f85 [4394609,99867] 0 2024-04-25T15:49:39.780 INFO:tasks.workunit.client.1.smithi193.stdout:8/419: read d2/d3/d20/d45/d4a/d53/f58 [1955427,38689] 0 2024-04-25T15:49:39.782 INFO:tasks.workunit.client.1.smithi193.stdout:9/454: chown df/l10 17 1 2024-04-25T15:49:39.782 INFO:tasks.workunit.client.1.smithi193.stdout:7/416: symlink d12/d1c/d96/l9e 0 2024-04-25T15:49:39.783 INFO:tasks.workunit.client.1.smithi193.stdout:2/442: symlink d0/d9/d15/d31/d50/lb2 0 2024-04-25T15:49:39.785 INFO:tasks.workunit.client.1.smithi193.stdout:3/490: link d8/d13/d14/l3d d8/d30/d45/l99 0 2024-04-25T15:49:39.786 INFO:tasks.workunit.client.1.smithi193.stdout:0/457: dwrite d0/d49/d5/d7/dd/d16/d18/d28/f94 [0,4194304] 0 2024-04-25T15:49:39.786 INFO:tasks.workunit.client.1.smithi193.stdout:2/443: dread - d0/d37/f92 zero size 2024-04-25T15:49:39.789 INFO:tasks.workunit.client.1.smithi193.stdout:7/417: mknod d12/d31/d37/d62/c9f 0 2024-04-25T15:49:39.789 INFO:tasks.workunit.client.1.smithi193.stdout:1/382: dwrite dc/d4a/d50/d5c/d5d/d3d/f43 [4194304,4194304] 0 2024-04-25T15:49:39.792 INFO:tasks.workunit.client.1.smithi193.stdout:9/455: mknod df/d2c/d2f/c89 0 2024-04-25T15:49:39.795 INFO:tasks.workunit.client.1.smithi193.stdout:6/447: sync 2024-04-25T15:49:39.796 INFO:tasks.workunit.client.1.smithi193.stdout:7/418: chown d12/d31/d37/d62/d6e/d79/c5c 797 1 2024-04-25T15:49:39.796 INFO:tasks.workunit.client.1.smithi193.stdout:3/491: read d8/d13/d14/d27/f5a [1801302,61360] 0 2024-04-25T15:49:39.797 INFO:tasks.workunit.client.1.smithi193.stdout:7/419: readlink la 0 2024-04-25T15:49:39.797 INFO:tasks.workunit.client.1.smithi193.stdout:2/444: fsync d0/d9/d15/d31/d50/f5c 0 2024-04-25T15:49:39.797 INFO:tasks.workunit.client.1.smithi193.stdout:1/383: mkdir dc/d4a/d4d/d80 0 2024-04-25T15:49:39.799 INFO:tasks.workunit.client.1.smithi193.stdout:8/420: dwrite d2/d3/f5f [0,4194304] 0 2024-04-25T15:49:39.799 INFO:tasks.workunit.client.1.smithi193.stdout:4/519: truncate db/d13/d1e/d1f/d22/f2a 4762845 0 2024-04-25T15:49:39.800 INFO:tasks.workunit.client.1.smithi193.stdout:6/448: chown dd/d12/d19/f43 23366046 1 2024-04-25T15:49:39.800 INFO:tasks.workunit.client.1.smithi193.stdout:9/456: write df/d2c/d34/d3e/f4a [1935738,109560] 0 2024-04-25T15:49:39.804 INFO:tasks.workunit.client.1.smithi193.stdout:2/445: chown d0/d9/d15/f1b 199093 1 2024-04-25T15:49:39.805 INFO:tasks.workunit.client.1.smithi193.stdout:1/384: symlink dc/d59/l81 0 2024-04-25T15:49:39.806 INFO:tasks.workunit.client.1.smithi193.stdout:8/421: chown d2/d3/ff 1269560 1 2024-04-25T15:49:39.808 INFO:tasks.workunit.client.1.smithi193.stdout:8/422: fsync d2/f6e 0 2024-04-25T15:49:39.808 INFO:tasks.workunit.client.1.smithi193.stdout:4/520: creat db/d4b/fa0 x:0 0 0 2024-04-25T15:49:39.809 INFO:tasks.workunit.client.1.smithi193.stdout:3/492: truncate d8/d30/f40 2859052 0 2024-04-25T15:49:39.812 INFO:tasks.workunit.client.1.smithi193.stdout:6/449: creat dd/d12/d19/d1c/d40/d3c/d3f/f85 x:0 0 0 2024-04-25T15:49:39.812 INFO:tasks.workunit.client.1.smithi193.stdout:1/385: mkdir dc/d4a/d50/d7c/d82 0 2024-04-25T15:49:39.814 INFO:tasks.workunit.client.1.smithi193.stdout:8/423: write d2/d3/d20/f22 [1411298,78649] 0 2024-04-25T15:49:39.815 INFO:tasks.workunit.client.1.smithi193.stdout:0/458: dread d0/d49/d5/d7/dd/d16/f30 [0,4194304] 0 2024-04-25T15:49:39.816 INFO:tasks.workunit.client.1.smithi193.stdout:3/493: truncate d8/d30/f85 1500686 0 2024-04-25T15:49:39.817 INFO:tasks.workunit.client.1.smithi193.stdout:8/424: chown d2/f6e 321608 1 2024-04-25T15:49:39.817 INFO:tasks.workunit.client.1.smithi193.stdout:7/420: dwrite d12/d1c/f99 [0,4194304] 0 2024-04-25T15:49:39.817 INFO:tasks.workunit.client.1.smithi193.stdout:1/386: stat dc/d4a/d4d/f57 0 2024-04-25T15:49:39.819 INFO:tasks.workunit.client.1.smithi193.stdout:4/521: getdents db/d4b/d51 0 2024-04-25T15:49:39.822 INFO:tasks.workunit.client.1.smithi193.stdout:0/459: mknod d0/d49/d6a/c9b 0 2024-04-25T15:49:39.822 INFO:tasks.workunit.client.1.smithi193.stdout:7/421: chown d12/d1c/c21 125 1 2024-04-25T15:49:39.823 INFO:tasks.workunit.client.1.smithi193.stdout:4/522: symlink db/d4b/la1 0 2024-04-25T15:49:39.824 INFO:tasks.workunit.client.1.smithi193.stdout:1/387: truncate dc/d59/f65 438684 0 2024-04-25T15:49:39.824 INFO:tasks.workunit.client.1.smithi193.stdout:8/425: chown d2/d3/d3a/f4c 15356 1 2024-04-25T15:49:39.824 INFO:tasks.workunit.client.1.smithi193.stdout:2/446: dwrite d0/d37/f67 [0,4194304] 0 2024-04-25T15:49:39.826 INFO:tasks.workunit.client.1.smithi193.stdout:7/422: dread - d12/d31/d37/d62/f9b zero size 2024-04-25T15:49:39.827 INFO:tasks.workunit.client.1.smithi193.stdout:9/457: dwrite df/d2c/d2f/d4e/d1c/d30/d60/f85 [0,4194304] 0 2024-04-25T15:49:39.828 INFO:tasks.workunit.client.1.smithi193.stdout:5/437: sync 2024-04-25T15:49:39.830 INFO:tasks.workunit.client.1.smithi193.stdout:3/494: mkdir d8/d13/d4b/d8b/d9a 0 2024-04-25T15:49:39.831 INFO:tasks.workunit.client.1.smithi193.stdout:0/460: readlink d0/d49/d5/d7/dd/d16/d18/d28/d56/l5f 0 2024-04-25T15:49:39.833 INFO:tasks.workunit.client.1.smithi193.stdout:3/495: readlink d8/d13/d1f/d69/d77/l6f 0 2024-04-25T15:49:39.834 INFO:tasks.workunit.client.1.smithi193.stdout:9/458: chown df/d2c/d2f/d4e/d69 133 1 2024-04-25T15:49:39.834 INFO:tasks.workunit.client.1.smithi193.stdout:1/388: creat dc/d4a/d50/d5c/d5d/d3d/f83 x:0 0 0 2024-04-25T15:49:39.836 INFO:tasks.workunit.client.1.smithi193.stdout:6/450: link dd/d2f/d57/l7a dd/d12/d19/d1c/d40/d3c/l86 0 2024-04-25T15:49:39.836 INFO:tasks.workunit.client.1.smithi193.stdout:0/461: dread - d0/d49/d5/d7/dd/d16/d26/d5e/d99/f9a zero size 2024-04-25T15:49:39.837 INFO:tasks.workunit.client.1.smithi193.stdout:6/451: write dd/d2f/d57/f5c [824646,56386] 0 2024-04-25T15:49:39.841 INFO:tasks.workunit.client.1.smithi193.stdout:1/389: chown dc/f2d 21 1 2024-04-25T15:49:39.842 INFO:tasks.workunit.client.1.smithi193.stdout:3/496: dwrite d8/d13/d1f/d69/d77/f53 [4194304,4194304] 0 2024-04-25T15:49:39.843 INFO:tasks.workunit.client.1.smithi193.stdout:4/523: rename db/d13/d1e/c6c to db/d32/ca2 0 2024-04-25T15:49:39.844 INFO:tasks.workunit.client.1.smithi193.stdout:5/438: creat d17/d18/d30/d38/d48/d4f/f98 x:0 0 0 2024-04-25T15:49:39.845 INFO:tasks.workunit.client.1.smithi193.stdout:6/452: readlink dd/d12/d19/d1c/d40/l33 0 2024-04-25T15:49:39.846 INFO:tasks.workunit.client.1.smithi193.stdout:0/462: mknod d0/d49/d5/d7/dd/d16/d26/d5e/d99/c9c 0 2024-04-25T15:49:39.847 INFO:tasks.workunit.client.1.smithi193.stdout:0/463: write d0/fc [3536488,61660] 0 2024-04-25T15:49:39.847 INFO:tasks.workunit.client.1.smithi193.stdout:8/426: rmdir d2/d3/d7f 0 2024-04-25T15:49:39.848 INFO:tasks.workunit.client.1.smithi193.stdout:8/427: chown d2/f75 41521478 1 2024-04-25T15:49:39.850 INFO:tasks.workunit.client.1.smithi193.stdout:2/447: link d0/d42/d51/d4f/fa7 d0/d37/fb3 0 2024-04-25T15:49:39.850 INFO:tasks.workunit.client.1.smithi193.stdout:1/390: mknod dc/d4a/d50/d7c/c84 0 2024-04-25T15:49:39.852 INFO:tasks.workunit.client.1.smithi193.stdout:7/423: link d12/d1c/d45/l66 d12/d31/la0 0 2024-04-25T15:49:39.854 INFO:tasks.workunit.client.1.smithi193.stdout:7/424: stat d12/d1c/d63/f80 0 2024-04-25T15:49:39.854 INFO:tasks.workunit.client.1.smithi193.stdout:8/428: creat d2/d79/f84 x:0 0 0 2024-04-25T15:49:39.856 INFO:tasks.workunit.client.1.smithi193.stdout:5/439: chown d17/d18/d3e/c63 24375 1 2024-04-25T15:49:39.856 INFO:tasks.workunit.client.1.smithi193.stdout:2/448: mknod d0/d44/cb4 0 2024-04-25T15:49:39.858 INFO:tasks.workunit.client.1.smithi193.stdout:0/464: dwrite d0/d49/d5/d7/dd/d16/d26/f55 [0,4194304] 0 2024-04-25T15:49:39.860 INFO:tasks.workunit.client.1.smithi193.stdout:1/391: stat dc/d3c/c6d 0 2024-04-25T15:49:39.862 INFO:tasks.workunit.client.1.smithi193.stdout:5/440: creat d17/d18/d75/f99 x:0 0 0 2024-04-25T15:49:39.864 INFO:tasks.workunit.client.1.smithi193.stdout:7/425: mkdir d12/da1 0 2024-04-25T15:49:39.865 INFO:tasks.workunit.client.1.smithi193.stdout:5/441: write d17/d18/d30/f8a [112892,13597] 0 2024-04-25T15:49:39.868 INFO:tasks.workunit.client.1.smithi193.stdout:2/449: rename d0/c17 to d0/cb5 0 2024-04-25T15:49:39.869 INFO:tasks.workunit.client.1.smithi193.stdout:1/392: dwrite dc/d4a/d50/d5c/d5d/d3d/f43 [0,4194304] 0 2024-04-25T15:49:39.869 INFO:tasks.workunit.client.1.smithi193.stdout:2/450: dread - d0/d37/d4d/fad zero size 2024-04-25T15:49:39.871 INFO:tasks.workunit.client.1.smithi193.stdout:1/393: truncate dc/d4a/f71 941451 0 2024-04-25T15:49:39.871 INFO:tasks.workunit.client.1.smithi193.stdout:0/465: rename d0/d49/d5/d7/l96 to d0/d49/d5/d7/l9d 0 2024-04-25T15:49:39.872 INFO:tasks.workunit.client.1.smithi193.stdout:8/429: getdents d2/d3/d27/d34 0 2024-04-25T15:49:39.873 INFO:tasks.workunit.client.1.smithi193.stdout:2/451: write d0/d42/d51/d4f/d65/d7e/f82 [240301,123606] 0 2024-04-25T15:49:39.874 INFO:tasks.workunit.client.1.smithi193.stdout:5/442: creat d17/d18/d45/f9a x:0 0 0 2024-04-25T15:49:39.876 INFO:tasks.workunit.client.1.smithi193.stdout:2/452: truncate d0/d42/d51/d4f/f97 873906 0 2024-04-25T15:49:39.878 INFO:tasks.workunit.client.1.smithi193.stdout:1/394: chown dc/l37 1240 1 2024-04-25T15:49:39.878 INFO:tasks.workunit.client.1.smithi193.stdout:0/466: chown d0/d49/d5/d7/dd/d16/d18/d28/d56/f70 1875963 1 2024-04-25T15:49:39.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/430: creat d2/d3/d20/d45/d29/f85 x:0 0 0 2024-04-25T15:49:39.882 INFO:tasks.workunit.client.1.smithi193.stdout:4/524: truncate db/d13/d1e/d1f/d22/f6b 3121541 0 2024-04-25T15:49:39.883 INFO:tasks.workunit.client.1.smithi193.stdout:4/525: write db/d32/f94 [802228,14961] 0 2024-04-25T15:49:39.883 INFO:tasks.workunit.client.1.smithi193.stdout:6/453: truncate dd/fe 2602646 0 2024-04-25T15:49:39.885 INFO:tasks.workunit.client.1.smithi193.stdout:8/431: rename d2/d79 to d2/d3/d27/d69/d86 0 2024-04-25T15:49:39.886 INFO:tasks.workunit.client.1.smithi193.stdout:3/497: truncate d8/d13/d1f/f22 104709 0 2024-04-25T15:49:39.886 INFO:tasks.workunit.client.1.smithi193.stdout:1/395: mkdir dc/d4a/d4d/d7a/d85 0 2024-04-25T15:49:39.890 INFO:tasks.workunit.client.1.smithi193.stdout:5/443: getdents d17/d18/d75/d85/d5d 0 2024-04-25T15:49:39.890 INFO:tasks.workunit.client.1.smithi193.stdout:4/526: dwrite db/d13/d1e/d1f/f63 [0,4194304] 0 2024-04-25T15:49:39.891 INFO:tasks.workunit.client.1.smithi193.stdout:6/454: rename dd/d12/d19/f1d to dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f87 0 2024-04-25T15:49:39.891 INFO:tasks.workunit.client.1.smithi193.stdout:4/527: chown db/d13/f1d 174 1 2024-04-25T15:49:39.893 INFO:tasks.workunit.client.1.smithi193.stdout:4/528: readlink db/d13/d1e/d1f/l43 0 2024-04-25T15:49:39.893 INFO:tasks.workunit.client.1.smithi193.stdout:3/498: mknod d8/d13/d14/d51/c9b 0 2024-04-25T15:49:39.894 INFO:tasks.workunit.client.1.smithi193.stdout:1/396: symlink dc/d3c/l86 0 2024-04-25T15:49:39.896 INFO:tasks.workunit.client.1.smithi193.stdout:2/453: getdents d0/d9 0 2024-04-25T15:49:39.898 INFO:tasks.workunit.client.1.smithi193.stdout:6/455: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/c88 0 2024-04-25T15:49:39.900 INFO:tasks.workunit.client.1.smithi193.stdout:4/529: truncate db/d32/f7f 331072 0 2024-04-25T15:49:39.902 INFO:tasks.workunit.client.1.smithi193.stdout:5/444: mkdir d17/d18/d9b 0 2024-04-25T15:49:39.908 INFO:tasks.workunit.client.1.smithi193.stdout:4/530: creat db/d4b/fa3 x:0 0 0 2024-04-25T15:49:39.908 INFO:tasks.workunit.client.1.smithi193.stdout:3/499: dwrite d8/d13/d14/d27/d6d/f91 [0,4194304] 0 2024-04-25T15:49:39.910 INFO:tasks.workunit.client.1.smithi193.stdout:5/445: chown d17/d18/d30/d38/d48/d4f 3548 1 2024-04-25T15:49:39.911 INFO:tasks.workunit.client.1.smithi193.stdout:0/467: truncate d0/fc 3875903 0 2024-04-25T15:49:39.912 INFO:tasks.workunit.client.1.smithi193.stdout:7/426: dwrite d12/d31/d39/f3a [0,4194304] 0 2024-04-25T15:49:39.915 INFO:tasks.workunit.client.1.smithi193.stdout:4/531: chown db/d32/d40/l91 52757396 1 2024-04-25T15:49:39.917 INFO:tasks.workunit.client.1.smithi193.stdout:1/397: dwrite dc/d4a/d50/d5c/d5d/d3d/d55/f73 [0,4194304] 0 2024-04-25T15:49:39.925 INFO:tasks.workunit.client.1.smithi193.stdout:3/500: truncate d8/d13/d1f/f16 6140771 0 2024-04-25T15:49:39.925 INFO:tasks.workunit.client.1.smithi193.stdout:7/427: mknod d12/d31/ca2 0 2024-04-25T15:49:39.926 INFO:tasks.workunit.client.1.smithi193.stdout:1/398: write dc/d4a/f75 [553818,79339] 0 2024-04-25T15:49:39.927 INFO:tasks.workunit.client.1.smithi193.stdout:6/456: dread dd/d12/d19/d1c/d40/d3c/d3f/d4f/f60 [0,4194304] 0 2024-04-25T15:49:39.927 INFO:tasks.workunit.client.1.smithi193.stdout:7/428: stat f7 0 2024-04-25T15:49:39.932 INFO:tasks.workunit.client.1.smithi193.stdout:8/432: dwrite d2/d3/d3a/f54 [0,4194304] 0 2024-04-25T15:49:39.933 INFO:tasks.workunit.client.1.smithi193.stdout:1/399: write dc/f19 [1989280,82576] 0 2024-04-25T15:49:39.934 INFO:tasks.workunit.client.1.smithi193.stdout:6/457: rmdir dd/d12/d19 39 2024-04-25T15:49:39.940 INFO:tasks.workunit.client.1.smithi193.stdout:5/446: dwrite d17/d18/d75/d85/d5d/f68 [4194304,4194304] 0 2024-04-25T15:49:39.940 INFO:tasks.workunit.client.1.smithi193.stdout:8/433: read d2/d3/d20/f22 [2803269,88985] 0 2024-04-25T15:49:39.942 INFO:tasks.workunit.client.1.smithi193.stdout:7/429: rmdir d12/d1c/d63/d70/d68/d8e 0 2024-04-25T15:49:39.943 INFO:tasks.workunit.client.1.smithi193.stdout:0/468: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/f80 [0,4194304] 0 2024-04-25T15:49:39.945 INFO:tasks.workunit.client.1.smithi193.stdout:6/458: unlink dd/d12/d19/l2a 0 2024-04-25T15:49:39.950 INFO:tasks.workunit.client.1.smithi193.stdout:0/469: chown d0/d49/f2c 267018813 1 2024-04-25T15:49:39.950 INFO:tasks.workunit.client.1.smithi193.stdout:7/430: fsync d12/d1c/d63/d70/d68/d8d/f89 0 2024-04-25T15:49:39.950 INFO:tasks.workunit.client.1.smithi193.stdout:1/400: creat dc/f87 x:0 0 0 2024-04-25T15:49:39.950 INFO:tasks.workunit.client.1.smithi193.stdout:5/447: unlink d17/d18/d30/d38/c5c 0 2024-04-25T15:49:39.951 INFO:tasks.workunit.client.1.smithi193.stdout:2/454: dread d0/f11 [0,4194304] 0 2024-04-25T15:49:39.953 INFO:tasks.workunit.client.1.smithi193.stdout:8/434: rename d2/d3/d20/d45/d29/f57 to d2/d3/d27/d69/d86/d72/f87 0 2024-04-25T15:49:39.955 INFO:tasks.workunit.client.1.smithi193.stdout:4/532: dread db/d13/f21 [0,4194304] 0 2024-04-25T15:49:39.955 INFO:tasks.workunit.client.1.smithi193.stdout:0/470: mkdir d0/d49/d5/d7/dd/d16/d26/d9e 0 2024-04-25T15:49:39.955 INFO:tasks.workunit.client.1.smithi193.stdout:2/455: chown d0/d9/d15/d31/d90 10899 1 2024-04-25T15:49:39.956 INFO:tasks.workunit.client.1.smithi193.stdout:7/431: dread - d12/d31/d37/d62/d6e/d79/f94 zero size 2024-04-25T15:49:39.957 INFO:tasks.workunit.client.1.smithi193.stdout:6/459: rename dd/d12/d19/d1c/d40/d3c/d3f/d4f/l84 to dd/d12/d2d/l89 0 2024-04-25T15:49:39.958 INFO:tasks.workunit.client.1.smithi193.stdout:8/435: creat d2/d3/d20/d45/f88 x:0 0 0 2024-04-25T15:49:39.961 INFO:tasks.workunit.client.1.smithi193.stdout:5/448: truncate d17/d18/d30/d34/f46 491283 0 2024-04-25T15:49:39.963 INFO:tasks.workunit.client.1.smithi193.stdout:9/459: sync 2024-04-25T15:49:39.963 INFO:tasks.workunit.client.1.smithi193.stdout:2/456: readlink d0/d37/d25/d5f/d3a/d6d/la2 0 2024-04-25T15:49:39.964 INFO:tasks.workunit.client.1.smithi193.stdout:3/501: dwrite d8/d13/d1f/d69/d77/f50 [0,4194304] 0 2024-04-25T15:49:39.965 INFO:tasks.workunit.client.1.smithi193.stdout:6/460: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/f8a x:0 0 0 2024-04-25T15:49:39.966 INFO:tasks.workunit.client.1.smithi193.stdout:8/436: write d2/d3/d20/d45/f88 [658912,6548] 0 2024-04-25T15:49:39.971 INFO:tasks.workunit.client.1.smithi193.stdout:3/502: creat d8/d30/d72/f9c x:0 0 0 2024-04-25T15:49:39.971 INFO:tasks.workunit.client.1.smithi193.stdout:4/533: truncate db/d13/d1e/d1f/d56/d69/d70/d8a/f8c 1630635 0 2024-04-25T15:49:39.972 INFO:tasks.workunit.client.1.smithi193.stdout:2/457: write d0/d9/d3d/d7d/fab [1504692,47686] 0 2024-04-25T15:49:39.972 INFO:tasks.workunit.client.1.smithi193.stdout:1/401: dwrite dc/d4a/f75 [0,4194304] 0 2024-04-25T15:49:39.973 INFO:tasks.workunit.client.1.smithi193.stdout:8/437: write d2/d3/ff [3916737,75456] 0 2024-04-25T15:49:39.979 INFO:tasks.workunit.client.1.smithi193.stdout:8/438: read d2/d3/d3a/f61 [88329,117625] 0 2024-04-25T15:49:39.979 INFO:tasks.workunit.client.1.smithi193.stdout:7/432: write f7 [4286558,39] 0 2024-04-25T15:49:39.980 INFO:tasks.workunit.client.1.smithi193.stdout:1/402: unlink dc/l31 0 2024-04-25T15:49:39.982 INFO:tasks.workunit.client.1.smithi193.stdout:0/471: write d0/d49/d5/d7/dd/f58 [878469,40466] 0 2024-04-25T15:49:39.986 INFO:tasks.workunit.client.1.smithi193.stdout:8/439: write d2/d3/f16 [5633303,19349] 0 2024-04-25T15:49:39.986 INFO:tasks.workunit.client.1.smithi193.stdout:4/534: dread db/f61 [0,4194304] 0 2024-04-25T15:49:39.990 INFO:tasks.workunit.client.1.smithi193.stdout:7/433: mkdir d12/d1c/da3 0 2024-04-25T15:49:39.991 INFO:tasks.workunit.client.1.smithi193.stdout:6/461: dwrite dd/d12/d19/f43 [4194304,4194304] 0 2024-04-25T15:49:39.993 INFO:tasks.workunit.client.1.smithi193.stdout:0/472: stat d0/d49/d5/d7/dd/d46/d48/d62/f92 0 2024-04-25T15:49:39.993 INFO:tasks.workunit.client.0.smithi110.stdout:3/721: sync 2024-04-25T15:49:39.994 INFO:tasks.workunit.client.1.smithi193.stdout:3/503: dwrite d8/d13/d14/d27/f73 [0,4194304] 0 2024-04-25T15:49:39.995 INFO:tasks.workunit.client.0.smithi110.stdout:3/722: creat d0/d2e/d88/dbf/da5/ff8 x:0 0 0 2024-04-25T15:49:39.996 INFO:tasks.workunit.client.0.smithi110.stdout:3/723: fdatasync d0/d2e/d88/dbf/d99/dad/fe9 0 2024-04-25T15:49:39.996 INFO:tasks.workunit.client.1.smithi193.stdout:9/460: dread df/d2c/d2f/d4e/d1c/d26/f35 [0,4194304] 0 2024-04-25T15:49:39.998 INFO:tasks.workunit.client.1.smithi193.stdout:2/458: rename d0/d42/d51/d4f/d65/d7e/f82 to d0/d37/fb6 0 2024-04-25T15:49:39.999 INFO:tasks.workunit.client.0.smithi110.stdout:3/724: creat d0/d2e/d88/dbf/d92/ff9 x:0 0 0 2024-04-25T15:49:40.000 INFO:tasks.workunit.client.1.smithi193.stdout:1/403: getdents dc/d4a/d50/d7c/d82 0 2024-04-25T15:49:40.000 INFO:tasks.workunit.client.1.smithi193.stdout:9/461: read - df/d2c/d2f/f6e zero size 2024-04-25T15:49:40.001 INFO:tasks.workunit.client.1.smithi193.stdout:2/459: fdatasync d0/d9/f88 0 2024-04-25T15:49:40.002 INFO:tasks.workunit.client.1.smithi193.stdout:5/449: dwrite d17/d18/f21 [4194304,4194304] 0 2024-04-25T15:49:40.004 INFO:tasks.workunit.client.1.smithi193.stdout:1/404: fdatasync dc/d4a/d50/d5c/d5d/d3d/f41 0 2024-04-25T15:49:40.004 INFO:tasks.workunit.client.1.smithi193.stdout:9/462: stat df/d2c/d34/d3e/d4d/d5e/f6c 0 2024-04-25T15:49:40.005 INFO:tasks.workunit.client.1.smithi193.stdout:4/535: link db/d4b/f71 db/d13/d1e/d1f/d56/d57/fa4 0 2024-04-25T15:49:40.006 INFO:tasks.workunit.client.1.smithi193.stdout:7/434: rename d12/d1c/l7d to d12/da1/la4 0 2024-04-25T15:49:40.007 INFO:tasks.workunit.client.1.smithi193.stdout:0/473: mknod d0/d49/d5/d7/dd/d16/d18/d28/d56/c9f 0 2024-04-25T15:49:40.008 INFO:tasks.workunit.client.1.smithi193.stdout:7/435: dread - d12/d31/d37/d62/f9b zero size 2024-04-25T15:49:40.009 INFO:tasks.workunit.client.1.smithi193.stdout:8/440: truncate d2/d3/d20/d45/d4a/d53/f58 2706524 0 2024-04-25T15:49:40.009 INFO:tasks.workunit.client.1.smithi193.stdout:3/504: fsync d8/d13/d1f/f22 0 2024-04-25T15:49:40.010 INFO:tasks.workunit.client.1.smithi193.stdout:5/450: rmdir d17/d18/d45 39 2024-04-25T15:49:40.011 INFO:tasks.workunit.client.1.smithi193.stdout:1/405: rmdir dc/d4a/d4d/d7a 39 2024-04-25T15:49:40.012 INFO:tasks.workunit.client.1.smithi193.stdout:7/436: chown d12/d1c/d45/l50 56696090 1 2024-04-25T15:49:40.014 INFO:tasks.workunit.client.1.smithi193.stdout:2/460: symlink d0/df/da5/lb7 0 2024-04-25T15:49:40.016 INFO:tasks.workunit.client.1.smithi193.stdout:3/505: rmdir d8/d30/d72 39 2024-04-25T15:49:40.019 INFO:tasks.workunit.client.1.smithi193.stdout:6/462: getdents dd/d12/d19/d1c/d40/d3c 0 2024-04-25T15:49:40.022 INFO:tasks.workunit.client.1.smithi193.stdout:4/536: dwrite db/d32/d40/f93 [0,4194304] 0 2024-04-25T15:49:40.022 INFO:tasks.workunit.client.1.smithi193.stdout:7/437: mkdir d12/d31/d37/d62/d6e/da5 0 2024-04-25T15:49:40.023 INFO:tasks.workunit.client.1.smithi193.stdout:5/451: write d17/d18/d45/f47 [5020472,33467] 0 2024-04-25T15:49:40.023 INFO:tasks.workunit.client.0.smithi110.stdout:3/725: dwrite d0/fc9 [0,4194304] 0 2024-04-25T15:49:40.024 INFO:tasks.workunit.client.1.smithi193.stdout:6/463: chown dd/d12/d19/d45/f7e 3546395 1 2024-04-25T15:49:40.029 INFO:tasks.workunit.client.1.smithi193.stdout:3/506: unlink d8/f11 0 2024-04-25T15:49:40.030 INFO:tasks.workunit.client.1.smithi193.stdout:8/441: dwrite d2/d3/d3a/f7c [0,4194304] 0 2024-04-25T15:49:40.030 INFO:tasks.workunit.client.0.smithi110.stdout:3/726: creat d0/d2e/d88/ffa x:0 0 0 2024-04-25T15:49:40.031 INFO:tasks.workunit.client.1.smithi193.stdout:2/461: mkdir d0/d37/d25/d5f/d3a/db8 0 2024-04-25T15:49:40.036 INFO:tasks.workunit.client.0.smithi110.stdout:3/727: symlink d0/d16/dcb/d2f/d4c/db2/df2/lfb 0 2024-04-25T15:49:40.036 INFO:tasks.workunit.client.1.smithi193.stdout:1/406: dread dc/d4a/d50/d5c/d5d/d3d/f43 [0,4194304] 0 2024-04-25T15:49:40.036 INFO:tasks.workunit.client.1.smithi193.stdout:0/474: dread d0/d49/d5/d7/dd/d16/d18/d28/f3e [0,4194304] 0 2024-04-25T15:49:40.041 INFO:tasks.workunit.client.1.smithi193.stdout:5/452: fdatasync d17/f97 0 2024-04-25T15:49:40.042 INFO:tasks.workunit.client.1.smithi193.stdout:3/507: write d8/d13/d1f/d69/d77/d71/f86 [810236,95530] 0 2024-04-25T15:49:40.043 INFO:tasks.workunit.client.1.smithi193.stdout:7/438: dread d12/d1c/f99 [0,4194304] 0 2024-04-25T15:49:40.044 INFO:tasks.workunit.client.1.smithi193.stdout:4/537: truncate db/d13/d1e/d1f/d56/d57/f9b 132407 0 2024-04-25T15:49:40.047 INFO:tasks.workunit.client.1.smithi193.stdout:9/463: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/f47 [0,4194304] 0 2024-04-25T15:49:40.050 INFO:tasks.workunit.client.1.smithi193.stdout:1/407: dwrite dc/d4a/d50/d5c/d63/f76 [0,4194304] 0 2024-04-25T15:49:40.071 INFO:tasks.workunit.client.1.smithi193.stdout:8/442: dread d2/d3/d3a/d6d/f2b [0,4194304] 0 2024-04-25T15:49:40.076 INFO:tasks.workunit.client.1.smithi193.stdout:8/443: dwrite d2/d3/d3a/f76 [0,4194304] 0 2024-04-25T15:49:40.113 INFO:tasks.workunit.client.1.smithi193.stdout:6/464: creat dd/d2f/d57/f8b x:0 0 0 2024-04-25T15:49:40.115 INFO:tasks.workunit.client.1.smithi193.stdout:5/453: dread f16 [0,4194304] 0 2024-04-25T15:49:40.116 INFO:tasks.workunit.client.1.smithi193.stdout:2/462: mkdir d0/d37/d25/d5f/d3a/d6d/d6e/db9 0 2024-04-25T15:49:40.118 INFO:tasks.workunit.client.1.smithi193.stdout:5/454: dread f16 [0,4194304] 0 2024-04-25T15:49:40.119 INFO:tasks.workunit.client.1.smithi193.stdout:5/455: stat d17/d18/d45/c4a 0 2024-04-25T15:49:40.176 INFO:tasks.workunit.client.1.smithi193.stdout:7/439: creat d12/d1c/d96/fa6 x:0 0 0 2024-04-25T15:49:40.178 INFO:tasks.workunit.client.1.smithi193.stdout:9/464: creat df/d2c/d2f/d33/f8a x:0 0 0 2024-04-25T15:49:40.179 INFO:tasks.workunit.client.0.smithi110.stdout:3/728: write d0/d2e/d88/dbf/d99/dad/daf/fd9 [2113907,43298] 0 2024-04-25T15:49:40.181 INFO:tasks.workunit.client.1.smithi193.stdout:8/444: chown d2/d18 994711657 1 2024-04-25T15:49:40.184 INFO:tasks.workunit.client.1.smithi193.stdout:0/475: mknod d0/d49/d5/d7/dd/ca0 0 2024-04-25T15:49:40.185 INFO:tasks.workunit.client.1.smithi193.stdout:2/463: creat d0/d9/d15/d31/d90/d9c/fba x:0 0 0 2024-04-25T15:49:40.186 INFO:tasks.workunit.client.1.smithi193.stdout:2/464: truncate d0/d44/fa8 562934 0 2024-04-25T15:49:40.186 INFO:tasks.workunit.client.1.smithi193.stdout:5/456: creat d17/d18/d75/f9c x:0 0 0 2024-04-25T15:49:40.187 INFO:tasks.workunit.client.1.smithi193.stdout:5/457: write d17/d18/d45/f6d [1941283,57321] 0 2024-04-25T15:49:40.189 INFO:tasks.workunit.client.1.smithi193.stdout:5/458: write d17/d18/d30/d38/d48/d4f/f98 [870431,102569] 0 2024-04-25T15:49:40.190 INFO:tasks.workunit.client.1.smithi193.stdout:7/440: mknod d12/d31/d39/d60/ca7 0 2024-04-25T15:49:40.190 INFO:tasks.workunit.client.1.smithi193.stdout:1/408: creat dc/d4a/d50/d7c/d7e/f88 x:0 0 0 2024-04-25T15:49:40.191 INFO:tasks.workunit.client.1.smithi193.stdout:1/409: fsync dc/f19 0 2024-04-25T15:49:40.192 INFO:tasks.workunit.client.1.smithi193.stdout:9/465: rmdir df/d2c/d2f/d4e/d1c 39 2024-04-25T15:49:40.196 INFO:tasks.workunit.client.1.smithi193.stdout:8/445: symlink d2/d3/d27/d69/l89 0 2024-04-25T15:49:40.202 INFO:tasks.workunit.client.1.smithi193.stdout:4/538: link db/d4b/d51/d54/d77/f79 db/d13/d1e/d1f/fa5 0 2024-04-25T15:49:40.202 INFO:tasks.workunit.client.0.smithi110.stdout:3/729: dwrite d0/d16/dcb/db0/fc2 [0,4194304] 0 2024-04-25T15:49:40.202 INFO:tasks.workunit.client.1.smithi193.stdout:4/539: write db/d13/f1d [791050,103076] 0 2024-04-25T15:49:40.203 INFO:tasks.workunit.client.1.smithi193.stdout:5/459: fdatasync d17/d18/d3e/f3f 0 2024-04-25T15:49:40.204 INFO:tasks.workunit.client.1.smithi193.stdout:5/460: stat d17/d18/d75/d85/l54 0 2024-04-25T15:49:40.205 INFO:tasks.workunit.client.0.smithi110.stdout:3/730: read d0/d16/f1f [3643500,100552] 0 2024-04-25T15:49:40.206 INFO:tasks.workunit.client.0.smithi110.stdout:3/731: stat d0/d2e/fc6 0 2024-04-25T15:49:40.208 INFO:tasks.workunit.client.1.smithi193.stdout:1/410: chown dc/f1b 310417 1 2024-04-25T15:49:40.210 INFO:tasks.workunit.client.0.smithi110.stdout:3/732: chown d0/d16/dcb/d2f/d33/l84 81 1 2024-04-25T15:49:40.211 INFO:tasks.workunit.client.1.smithi193.stdout:6/465: creat dd/d12/d19/f8c x:0 0 0 2024-04-25T15:49:40.211 INFO:tasks.workunit.client.1.smithi193.stdout:5/461: dread d17/d18/d45/f47 [4194304,4194304] 0 2024-04-25T15:49:40.212 INFO:tasks.workunit.client.1.smithi193.stdout:8/446: rmdir d2/d18 39 2024-04-25T15:49:40.213 INFO:tasks.workunit.client.1.smithi193.stdout:6/466: dread - dd/d12/d19/d1c/d40/d3c/d3f/d4f/f7b zero size 2024-04-25T15:49:40.215 INFO:tasks.workunit.client.1.smithi193.stdout:8/447: write d2/d3/d20/d45/d4a/f4e [2245140,85325] 0 2024-04-25T15:49:40.217 INFO:tasks.workunit.client.1.smithi193.stdout:3/508: dwrite d8/d30/f40 [0,4194304] 0 2024-04-25T15:49:40.218 INFO:tasks.workunit.client.1.smithi193.stdout:1/411: dread dc/f15 [0,4194304] 0 2024-04-25T15:49:40.219 INFO:tasks.workunit.client.1.smithi193.stdout:9/466: rename df/d2c/d2f/d4e/d1c/d26 to df/d11/d8b 0 2024-04-25T15:49:40.221 INFO:tasks.workunit.client.1.smithi193.stdout:9/467: chown df/d11 117 1 2024-04-25T15:49:40.223 INFO:tasks.workunit.client.1.smithi193.stdout:7/441: write d12/d13/f41 [4582032,67551] 0 2024-04-25T15:49:40.225 INFO:tasks.workunit.client.1.smithi193.stdout:2/465: dwrite d0/df/f10 [4194304,4194304] 0 2024-04-25T15:49:40.226 INFO:tasks.workunit.client.1.smithi193.stdout:3/509: symlink d8/d13/d4b/d83/l9d 0 2024-04-25T15:49:40.227 INFO:tasks.workunit.client.1.smithi193.stdout:5/462: mknod d17/d53/c9d 0 2024-04-25T15:49:40.229 INFO:tasks.workunit.client.1.smithi193.stdout:4/540: link db/d13/d1e/d1f/d22/c9f db/d13/d9d/ca6 0 2024-04-25T15:49:40.230 INFO:tasks.workunit.client.1.smithi193.stdout:8/448: mknod d2/d18/c8a 0 2024-04-25T15:49:40.231 INFO:tasks.workunit.client.1.smithi193.stdout:8/449: chown d2/d3/d20/d45/f49 2 1 2024-04-25T15:49:40.232 INFO:tasks.workunit.client.1.smithi193.stdout:9/468: rename df/d2c/d2f/c5f to df/d2c/d2f/d4e/d1c/d30/d3b/c8c 0 2024-04-25T15:49:40.233 INFO:tasks.workunit.client.1.smithi193.stdout:3/510: creat d8/d13/d1f/f9e x:0 0 0 2024-04-25T15:49:40.234 INFO:tasks.workunit.client.1.smithi193.stdout:5/463: fsync d17/d18/d45/f47 0 2024-04-25T15:49:40.235 INFO:tasks.workunit.client.1.smithi193.stdout:0/476: link d0/d49/d5/d7/dd/d16/d26/d5e/c82 d0/d49/d5/d7/dd/d16/d26/d5e/d99/ca1 0 2024-04-25T15:49:40.236 INFO:tasks.workunit.client.1.smithi193.stdout:4/541: symlink db/d32/d40/la7 0 2024-04-25T15:49:40.237 INFO:tasks.workunit.client.1.smithi193.stdout:5/464: write d17/d18/d3e/f3f [3294599,113480] 0 2024-04-25T15:49:40.238 INFO:tasks.workunit.client.1.smithi193.stdout:2/466: mkdir d0/d37/dbb 0 2024-04-25T15:49:40.239 INFO:tasks.workunit.client.1.smithi193.stdout:4/542: stat db/d32/d88 0 2024-04-25T15:49:40.240 INFO:tasks.workunit.client.0.smithi110.stdout:3/733: dread d0/d7/d7d/fda [0,4194304] 0 2024-04-25T15:49:40.242 INFO:tasks.workunit.client.1.smithi193.stdout:7/442: dread d12/d1c/d45/f48 [0,4194304] 0 2024-04-25T15:49:40.243 INFO:tasks.workunit.client.1.smithi193.stdout:5/465: rename d17/f66 to d17/d18/d75/f9e 0 2024-04-25T15:49:40.244 INFO:tasks.workunit.client.0.smithi110.stdout:0/669: sync 2024-04-25T15:49:40.244 INFO:tasks.workunit.client.0.smithi110.stdout:2/673: sync 2024-04-25T15:49:40.248 INFO:tasks.workunit.client.1.smithi193.stdout:8/450: link d2/d3/d20/d45/d4a/f4e d2/d3/d20/f8b 0 2024-04-25T15:49:40.251 INFO:tasks.workunit.client.1.smithi193.stdout:3/511: dwrite d8/d30/d72/f76 [0,4194304] 0 2024-04-25T15:49:40.254 INFO:tasks.workunit.client.1.smithi193.stdout:2/467: rename d0/f29 to d0/d44/d8b/fbc 0 2024-04-25T15:49:40.256 INFO:tasks.workunit.client.1.smithi193.stdout:6/467: dwrite dd/f42 [0,4194304] 0 2024-04-25T15:49:40.257 INFO:tasks.workunit.client.1.smithi193.stdout:1/412: dread dc/d4a/f54 [0,4194304] 0 2024-04-25T15:49:40.259 INFO:tasks.workunit.client.1.smithi193.stdout:2/468: write d0/d9/d15/f1b [680605,67214] 0 2024-04-25T15:49:40.262 INFO:tasks.workunit.client.1.smithi193.stdout:3/512: mkdir d8/d13/d1f/d9f 0 2024-04-25T15:49:40.263 INFO:tasks.workunit.client.1.smithi193.stdout:4/543: write db/f6a [96134,99090] 0 2024-04-25T15:49:40.264 INFO:tasks.workunit.client.1.smithi193.stdout:7/443: rename d12/d1c/d45/f48 to d12/d31/d39/d60/fa8 0 2024-04-25T15:49:40.264 INFO:tasks.workunit.client.0.smithi110.stdout:3/734: dwrite d0/d16/dcb/d2f/d4c/fac [0,4194304] 0 2024-04-25T15:49:40.264 INFO:tasks.workunit.client.1.smithi193.stdout:8/451: dread d2/d3/d3a/f54 [0,4194304] 0 2024-04-25T15:49:40.265 INFO:tasks.workunit.client.1.smithi193.stdout:4/544: fdatasync db/d13/d1e/d1f/f64 0 2024-04-25T15:49:40.265 INFO:tasks.workunit.client.1.smithi193.stdout:7/444: write d12/d1c/f3b [2161846,47221] 0 2024-04-25T15:49:40.266 INFO:tasks.workunit.client.0.smithi110.stdout:0/670: write d1/d47/fa9 [430878,92310] 0 2024-04-25T15:49:40.267 INFO:tasks.workunit.client.0.smithi110.stdout:0/671: chown d1/d23/fc2 327 1 2024-04-25T15:49:40.267 INFO:tasks.workunit.client.1.smithi193.stdout:4/545: write db/d13/d1e/d1f/d56/d57/f9b [187258,86239] 0 2024-04-25T15:49:40.269 INFO:tasks.workunit.client.1.smithi193.stdout:4/546: chown db/d13/c23 253974038 1 2024-04-25T15:49:40.271 INFO:tasks.workunit.client.0.smithi110.stdout:3/735: mkdir d0/d2e/dfc 0 2024-04-25T15:49:40.271 INFO:tasks.workunit.client.1.smithi193.stdout:0/477: write d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f3f [36962,111178] 0 2024-04-25T15:49:40.274 INFO:tasks.workunit.client.0.smithi110.stdout:0/672: creat d1/d6c/ff1 x:0 0 0 2024-04-25T15:49:40.276 INFO:tasks.workunit.client.0.smithi110.stdout:2/674: dwrite de/d40/d6e/f8e [0,4194304] 0 2024-04-25T15:49:40.278 INFO:tasks.workunit.client.1.smithi193.stdout:7/445: dwrite d12/d31/d39/d60/f64 [0,4194304] 0 2024-04-25T15:49:40.278 INFO:tasks.workunit.client.1.smithi193.stdout:9/469: getdents df/d11/d8b 0 2024-04-25T15:49:40.280 INFO:tasks.workunit.client.0.smithi110.stdout:6/655: sync 2024-04-25T15:49:40.280 INFO:tasks.workunit.client.0.smithi110.stdout:1/881: sync 2024-04-25T15:49:40.280 INFO:tasks.workunit.client.0.smithi110.stdout:5/736: sync 2024-04-25T15:49:40.280 INFO:tasks.workunit.client.0.smithi110.stdout:9/686: sync 2024-04-25T15:49:40.280 INFO:tasks.workunit.client.0.smithi110.stdout:8/805: sync 2024-04-25T15:49:40.281 INFO:tasks.workunit.client.0.smithi110.stdout:4/720: sync 2024-04-25T15:49:40.281 INFO:tasks.workunit.client.0.smithi110.stdout:7/754: sync 2024-04-25T15:49:40.281 INFO:tasks.workunit.client.1.smithi193.stdout:9/470: chown df/d11/f84 86774111 1 2024-04-25T15:49:40.282 INFO:tasks.workunit.client.0.smithi110.stdout:5/737: readlink d4/d1d/l50 0 2024-04-25T15:49:40.284 INFO:tasks.workunit.client.0.smithi110.stdout:8/806: fdatasync f5 0 2024-04-25T15:49:40.284 INFO:tasks.workunit.client.1.smithi193.stdout:5/466: dwrite d17/d18/d30/d3b/f60 [0,4194304] 0 2024-04-25T15:49:40.286 INFO:tasks.workunit.client.0.smithi110.stdout:4/721: chown d13/cdd 183649172 1 2024-04-25T15:49:40.286 INFO:tasks.workunit.client.1.smithi193.stdout:8/452: fsync d2/d3/d20/f8b 0 2024-04-25T15:49:40.289 INFO:tasks.workunit.client.1.smithi193.stdout:8/453: write d2/f48 [2845864,53419] 0 2024-04-25T15:49:40.293 INFO:tasks.workunit.client.1.smithi193.stdout:9/471: dwrite df/d2c/d34/d3e/d4d/d5e/d81/f43 [0,4194304] 0 2024-04-25T15:49:40.296 INFO:tasks.workunit.client.0.smithi110.stdout:3/736: dread d0/d2e/d88/dbf/d92/f93 [0,4194304] 0 2024-04-25T15:49:40.299 INFO:tasks.workunit.client.0.smithi110.stdout:1/882: dread d0/d14/d5f/d7d/d72/f7e [0,4194304] 0 2024-04-25T15:49:40.299 INFO:tasks.workunit.client.1.smithi193.stdout:0/478: creat d0/d49/d5/d7/dd/d46/d48/fa2 x:0 0 0 2024-04-25T15:49:40.300 INFO:tasks.workunit.client.1.smithi193.stdout:6/468: write dd/d12/d19/d1c/f20 [9195015,24628] 0 2024-04-25T15:49:40.301 INFO:tasks.workunit.client.0.smithi110.stdout:1/883: write d0/d51/f99 [654355,82505] 0 2024-04-25T15:49:40.302 INFO:tasks.workunit.client.1.smithi193.stdout:4/547: dread db/d32/f7f [0,4194304] 0 2024-04-25T15:49:40.304 INFO:tasks.workunit.client.1.smithi193.stdout:1/413: dwrite dc/f2d [4194304,4194304] 0 2024-04-25T15:49:40.304 INFO:tasks.workunit.client.1.smithi193.stdout:3/513: mkdir d8/d13/d14/da0 0 2024-04-25T15:49:40.305 INFO:tasks.workunit.client.1.smithi193.stdout:4/548: write db/f38 [3440510,93182] 0 2024-04-25T15:49:40.308 INFO:tasks.workunit.client.1.smithi193.stdout:1/414: truncate dc/d4a/d50/f53 5585473 0 2024-04-25T15:49:40.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/467: unlink d17/d18/d30/d38/d71/c87 0 2024-04-25T15:49:40.311 INFO:tasks.workunit.client.1.smithi193.stdout:7/446: read d12/d1c/f8a [3014195,127095] 0 2024-04-25T15:49:40.312 INFO:tasks.workunit.client.1.smithi193.stdout:8/454: mkdir d2/d3/d20/d8c 0 2024-04-25T15:49:40.315 INFO:tasks.workunit.client.1.smithi193.stdout:6/469: rmdir dd/d12 39 2024-04-25T15:49:40.316 INFO:tasks.workunit.client.1.smithi193.stdout:4/549: dwrite db/f8d [0,4194304] 0 2024-04-25T15:49:40.317 INFO:tasks.workunit.client.1.smithi193.stdout:4/550: fdatasync db/d13/d1e/d1f/f98 0 2024-04-25T15:49:40.321 INFO:tasks.workunit.client.1.smithi193.stdout:0/479: creat d0/d49/d5/d7/dd/d16/d26/d5e/d99/fa3 x:0 0 0 2024-04-25T15:49:40.324 INFO:tasks.workunit.client.1.smithi193.stdout:3/514: rmdir d8/d13/d4b/d83 39 2024-04-25T15:49:40.324 INFO:tasks.workunit.client.1.smithi193.stdout:9/472: dread df/d2c/d34/f75 [0,4194304] 0 2024-04-25T15:49:40.324 INFO:tasks.workunit.client.1.smithi193.stdout:9/473: chown df/d2c/d34/d3e 44597833 1 2024-04-25T15:49:40.328 INFO:tasks.workunit.client.1.smithi193.stdout:5/468: symlink d17/d18/d30/d34/l9f 0 2024-04-25T15:49:40.331 INFO:tasks.workunit.client.1.smithi193.stdout:3/515: dwrite d8/d30/d72/f9c [0,4194304] 0 2024-04-25T15:49:40.332 INFO:tasks.workunit.client.1.smithi193.stdout:2/469: getdents d0/d37/d4d 0 2024-04-25T15:49:40.333 INFO:tasks.workunit.client.1.smithi193.stdout:9/474: dwrite df/d2c/d34/d3e/d4d/d5e/f6c [0,4194304] 0 2024-04-25T15:49:40.334 INFO:tasks.workunit.client.1.smithi193.stdout:8/455: stat d2/d3/d27/d6f/c78 0 2024-04-25T15:49:40.335 INFO:tasks.workunit.client.1.smithi193.stdout:2/470: dread d0/d37/d25/d5f/d3a/d6d/d6e/f96 [0,4194304] 0 2024-04-25T15:49:40.338 INFO:tasks.workunit.client.1.smithi193.stdout:6/470: readlink dd/l14 0 2024-04-25T15:49:40.343 INFO:tasks.workunit.client.1.smithi193.stdout:7/447: dwrite d12/d1c/f99 [0,4194304] 0 2024-04-25T15:49:40.345 INFO:tasks.workunit.client.1.smithi193.stdout:0/480: unlink d0/d49/d5/f6d 0 2024-04-25T15:49:40.347 INFO:tasks.workunit.client.1.smithi193.stdout:1/415: symlink dc/d4a/d50/d7c/d82/l89 0 2024-04-25T15:49:40.348 INFO:tasks.workunit.client.1.smithi193.stdout:6/471: dwrite dd/d2f/d57/f8b [0,4194304] 0 2024-04-25T15:49:40.354 INFO:tasks.workunit.client.1.smithi193.stdout:5/469: creat d17/d18/d30/d38/d48/fa0 x:0 0 0 2024-04-25T15:49:40.358 INFO:tasks.workunit.client.1.smithi193.stdout:3/516: creat d8/d13/d14/d23/fa1 x:0 0 0 2024-04-25T15:49:40.358 INFO:tasks.workunit.client.1.smithi193.stdout:5/470: dread d17/d18/d30/d34/f46 [0,4194304] 0 2024-04-25T15:49:40.358 INFO:tasks.workunit.client.1.smithi193.stdout:3/517: fdatasync d8/d13/d4b/f57 0 2024-04-25T15:49:40.358 INFO:tasks.workunit.client.1.smithi193.stdout:5/471: chown d17/d18/d75/d85/d5d/l67 48311 1 2024-04-25T15:49:40.359 INFO:tasks.workunit.client.1.smithi193.stdout:8/456: symlink d2/d3/d27/l8d 0 2024-04-25T15:49:40.362 INFO:tasks.workunit.client.1.smithi193.stdout:2/471: rename d0/d37/d25/d5f/d3a/l6c to d0/d9/d15/d31/d90/lbd 0 2024-04-25T15:49:40.368 INFO:tasks.workunit.client.1.smithi193.stdout:6/472: dread - dd/f31 zero size 2024-04-25T15:49:40.370 INFO:tasks.workunit.client.1.smithi193.stdout:5/472: creat d17/d18/d30/d3b/fa1 x:0 0 0 2024-04-25T15:49:40.373 INFO:tasks.workunit.client.1.smithi193.stdout:9/475: link df/d2c/d2f/d4e/d1c/d30/d60/f85 df/d2c/d2f/d33/f8d 0 2024-04-25T15:49:40.375 INFO:tasks.workunit.client.1.smithi193.stdout:2/472: mkdir d0/d37/d4d/dbe 0 2024-04-25T15:49:40.378 INFO:tasks.workunit.client.1.smithi193.stdout:7/448: creat d12/d31/d37/d62/d6e/da5/fa9 x:0 0 0 2024-04-25T15:49:40.379 INFO:tasks.workunit.client.1.smithi193.stdout:7/449: read d12/f52 [3453372,82714] 0 2024-04-25T15:49:40.380 INFO:tasks.workunit.client.1.smithi193.stdout:6/473: dread dd/d12/f5d [0,4194304] 0 2024-04-25T15:49:40.383 INFO:tasks.workunit.client.1.smithi193.stdout:8/457: symlink d2/d3/l8e 0 2024-04-25T15:49:40.384 INFO:tasks.workunit.client.1.smithi193.stdout:5/473: mknod d17/d18/d75/ca2 0 2024-04-25T15:49:40.385 INFO:tasks.workunit.client.1.smithi193.stdout:5/474: chown d17/d18/d75 122 1 2024-04-25T15:49:40.388 INFO:tasks.workunit.client.1.smithi193.stdout:8/458: dwrite d2/d3/d3a/f7c [0,4194304] 0 2024-04-25T15:49:40.388 INFO:tasks.workunit.client.1.smithi193.stdout:4/551: getdents db/d32/d45 0 2024-04-25T15:49:40.389 INFO:tasks.workunit.client.1.smithi193.stdout:4/552: chown db/d13/f21 8 1 2024-04-25T15:49:40.391 INFO:tasks.workunit.client.1.smithi193.stdout:4/553: chown db/d13/d1e/d1f/d56/f74 11 1 2024-04-25T15:49:40.394 INFO:tasks.workunit.client.0.smithi110.stdout:7/755: mknod d1b/dde/cf8 0 2024-04-25T15:49:40.394 INFO:tasks.workunit.client.0.smithi110.stdout:7/756: chown d1b/d29/d46/l66 4659460 1 2024-04-25T15:49:40.395 INFO:tasks.workunit.client.0.smithi110.stdout:7/757: dread - d1b/d29/d38/f5e zero size 2024-04-25T15:49:40.396 INFO:tasks.workunit.client.1.smithi193.stdout:7/450: rename d12/d31/d37/d62/d6e/da5/fa9 to d12/d1c/d63/d70/faa 0 2024-04-25T15:49:40.398 INFO:tasks.workunit.client.0.smithi110.stdout:5/738: rmdir d4/d2c/d2a 39 2024-04-25T15:49:40.399 INFO:tasks.workunit.client.1.smithi193.stdout:6/474: symlink dd/d12/d19/d1c/d40/d3c/l8d 0 2024-04-25T15:49:40.403 INFO:tasks.workunit.client.0.smithi110.stdout:2/675: rename de/d40/d6e/d50/db7/cb8 to de/d40/d5b/d5c/d5f/db2/dad/cd6 0 2024-04-25T15:49:40.407 INFO:tasks.workunit.client.1.smithi193.stdout:5/475: symlink d17/d18/d30/d38/d48/la3 0 2024-04-25T15:49:40.408 INFO:tasks.workunit.client.0.smithi110.stdout:1/884: creat d0/d51/dd8/df0/f11f x:0 0 0 2024-04-25T15:49:40.410 INFO:tasks.workunit.client.0.smithi110.stdout:1/885: write d0/d14/d81/df8/d33/d8e/fd6 [757144,88619] 0 2024-04-25T15:49:40.411 INFO:tasks.workunit.client.0.smithi110.stdout:1/886: truncate d0/d14/f2c 9225293 0 2024-04-25T15:49:40.411 INFO:tasks.workunit.client.0.smithi110.stdout:6/656: unlink d1/d12/fb4 0 2024-04-25T15:49:40.411 INFO:tasks.workunit.client.1.smithi193.stdout:7/451: dread f2 [0,4194304] 0 2024-04-25T15:49:40.412 INFO:tasks.workunit.client.0.smithi110.stdout:1/887: readlink d0/d14/d81/df8/d33/d94/l9e 0 2024-04-25T15:49:40.413 INFO:tasks.workunit.client.0.smithi110.stdout:0/673: creat d1/d47/ff2 x:0 0 0 2024-04-25T15:49:40.417 INFO:tasks.workunit.client.1.smithi193.stdout:8/459: unlink d2/d3/d20/d45/d29/f85 0 2024-04-25T15:49:40.417 INFO:tasks.workunit.client.0.smithi110.stdout:1/888: dwrite d0/d14/d5f/d7d/d9c/fa0 [0,4194304] 0 2024-04-25T15:49:40.419 INFO:tasks.workunit.client.0.smithi110.stdout:1/889: chown d0/d14/d81/df8/d33/ff3 826419912 1 2024-04-25T15:49:40.421 INFO:tasks.workunit.client.0.smithi110.stdout:5/739: symlink d4/d1d/d1f/d35/d7f/lf0 0 2024-04-25T15:49:40.422 INFO:tasks.workunit.client.0.smithi110.stdout:5/740: chown d4/d1d/d1f/d35/d3d/d93/dc4/fd7 4707 1 2024-04-25T15:49:40.422 INFO:tasks.workunit.client.0.smithi110.stdout:4/722: getdents d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/df4 0 2024-04-25T15:49:40.423 INFO:tasks.workunit.client.0.smithi110.stdout:4/723: fdatasync d13/d24/d2d/f62 0 2024-04-25T15:49:40.424 INFO:tasks.workunit.client.0.smithi110.stdout:4/724: chown d13/d24/d2d/d44/d65/d99/daf 16803104 1 2024-04-25T15:49:40.424 INFO:tasks.workunit.client.1.smithi193.stdout:0/481: dread d0/d49/d5/d7/dd/d16/f6f [0,4194304] 0 2024-04-25T15:49:40.427 INFO:tasks.workunit.client.1.smithi193.stdout:0/482: truncate d0/d49/d5/d7/dd/d46/f6b 705392 0 2024-04-25T15:49:40.427 INFO:tasks.workunit.client.0.smithi110.stdout:6/657: fsync d1/db/d24/d69/d75/fd5 0 2024-04-25T15:49:40.427 INFO:tasks.workunit.client.0.smithi110.stdout:6/658: chown d1/d48 175 1 2024-04-25T15:49:40.428 INFO:tasks.workunit.client.1.smithi193.stdout:6/475: rename dd/d2f/c5e to dd/d12/d2d/c8e 0 2024-04-25T15:49:40.430 INFO:tasks.workunit.client.1.smithi193.stdout:5/476: mknod d17/d18/d30/d38/ca4 0 2024-04-25T15:49:40.430 INFO:tasks.workunit.client.1.smithi193.stdout:5/477: fsync d17/d18/d45/f9a 0 2024-04-25T15:49:40.431 INFO:tasks.workunit.client.0.smithi110.stdout:9/687: creat d9/d44/d5e/db5/fda x:0 0 0 2024-04-25T15:49:40.433 INFO:tasks.workunit.client.1.smithi193.stdout:7/452: dread d12/d31/d39/d60/fa8 [0,4194304] 0 2024-04-25T15:49:40.435 INFO:tasks.workunit.client.0.smithi110.stdout:1/890: creat d0/d51/dc8/f120 x:0 0 0 2024-04-25T15:49:40.435 INFO:tasks.workunit.client.0.smithi110.stdout:1/891: stat d0/d14/d81/df8/d33/da9 0 2024-04-25T15:49:40.437 INFO:tasks.workunit.client.1.smithi193.stdout:8/460: symlink d2/d3/d3a/d6d/l8f 0 2024-04-25T15:49:40.439 INFO:tasks.workunit.client.1.smithi193.stdout:7/453: dwrite d12/d13/f71 [0,4194304] 0 2024-04-25T15:49:40.442 INFO:tasks.workunit.client.0.smithi110.stdout:5/741: creat d4/d1d/d1f/d35/d3d/d93/ff1 x:0 0 0 2024-04-25T15:49:40.442 INFO:tasks.workunit.client.1.smithi193.stdout:7/454: dread - d12/d1c/d63/d70/faa zero size 2024-04-25T15:49:40.442 INFO:tasks.workunit.client.1.smithi193.stdout:9/476: truncate df/d11/d8b/f4f 3619844 0 2024-04-25T15:49:40.444 INFO:tasks.workunit.client.0.smithi110.stdout:2/676: mkdir de/d40/dd7 0 2024-04-25T15:49:40.445 INFO:tasks.workunit.client.1.smithi193.stdout:8/461: dwrite d2/f48 [0,4194304] 0 2024-04-25T15:49:40.449 INFO:tasks.workunit.client.0.smithi110.stdout:3/737: dwrite d0/d16/dcb/db0/feb [0,4194304] 0 2024-04-25T15:49:40.449 INFO:tasks.workunit.client.1.smithi193.stdout:2/473: truncate d0/df/f26 479491 0 2024-04-25T15:49:40.451 INFO:tasks.workunit.client.0.smithi110.stdout:6/659: fsync d1/dd/d3c/d91/d8d/dce/f78 0 2024-04-25T15:49:40.452 INFO:tasks.workunit.client.1.smithi193.stdout:5/478: truncate d17/d18/d30/d34/f46 537070 0 2024-04-25T15:49:40.453 INFO:tasks.workunit.client.1.smithi193.stdout:5/479: write d17/d18/f20 [3685598,101396] 0 2024-04-25T15:49:40.455 INFO:tasks.workunit.client.0.smithi110.stdout:1/892: symlink d0/d51/l121 0 2024-04-25T15:49:40.458 INFO:tasks.workunit.client.1.smithi193.stdout:7/455: readlink d12/l35 0 2024-04-25T15:49:40.459 INFO:tasks.workunit.client.1.smithi193.stdout:9/477: symlink df/d2c/d2f/d4e/d1c/d30/d60/l8e 0 2024-04-25T15:49:40.462 INFO:tasks.workunit.client.1.smithi193.stdout:8/462: readlink d2/d18/l4b 0 2024-04-25T15:49:40.464 INFO:tasks.workunit.client.0.smithi110.stdout:3/738: creat d0/d16/dcb/d2f/d4c/ffd x:0 0 0 2024-04-25T15:49:40.465 INFO:tasks.workunit.client.0.smithi110.stdout:3/739: chown d0/d2e/d88/dbf/d99 15370269 1 2024-04-25T15:49:40.465 INFO:tasks.workunit.client.1.smithi193.stdout:6/476: unlink dd/f24 0 2024-04-25T15:49:40.467 INFO:tasks.workunit.client.0.smithi110.stdout:9/688: getdents d9/d13/d5b/d61/db3/db4/dc5 0 2024-04-25T15:49:40.469 INFO:tasks.workunit.client.0.smithi110.stdout:6/660: rename d1/db/d24/d2d/f96 to d1/dd/fe6 0 2024-04-25T15:49:40.471 INFO:tasks.workunit.client.0.smithi110.stdout:7/758: getdents d1b/d29/d38/d55/dab/db2/dcd 0 2024-04-25T15:49:40.472 INFO:tasks.workunit.client.0.smithi110.stdout:1/893: fsync d0/d14/f62 0 2024-04-25T15:49:40.473 INFO:tasks.workunit.client.0.smithi110.stdout:1/894: chown d0/d14/d5f/f113 477 1 2024-04-25T15:49:40.474 INFO:tasks.workunit.client.0.smithi110.stdout:5/742: mknod d4/d1d/d1f/d35/cf2 0 2024-04-25T15:49:40.474 INFO:tasks.workunit.client.0.smithi110.stdout:1/895: stat d0/d14/d81/df8/d33/l93 0 2024-04-25T15:49:40.475 INFO:tasks.workunit.client.1.smithi193.stdout:4/554: dwrite db/d13/d1e/d1f/d22/f2a [0,4194304] 0 2024-04-25T15:49:40.475 INFO:tasks.workunit.client.0.smithi110.stdout:3/740: rmdir d0/d2e/d88/dbf/d99 39 2024-04-25T15:49:40.476 INFO:tasks.workunit.client.1.smithi193.stdout:0/483: creat d0/d49/d5/d7/fa4 x:0 0 0 2024-04-25T15:49:40.476 INFO:tasks.workunit.client.0.smithi110.stdout:3/741: dread - d0/d2e/d88/dbf/d92/f89 zero size 2024-04-25T15:49:40.480 INFO:tasks.workunit.client.0.smithi110.stdout:6/661: rename d1/f26 to d1/db/d22/d33/de4/fe7 0 2024-04-25T15:49:40.481 INFO:tasks.workunit.client.1.smithi193.stdout:0/484: truncate d0/d49/d5/d7/dd/d46/d48/fa2 735486 0 2024-04-25T15:49:40.483 INFO:tasks.workunit.client.0.smithi110.stdout:7/759: mkdir d1b/d29/db5/dca/df9 0 2024-04-25T15:49:40.490 INFO:tasks.workunit.client.1.smithi193.stdout:0/485: dread d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f3f [0,4194304] 0 2024-04-25T15:49:40.490 INFO:tasks.workunit.client.0.smithi110.stdout:9/689: dread d9/db/df/d2c/d3c/fa4 [0,4194304] 0 2024-04-25T15:49:40.490 INFO:tasks.workunit.client.0.smithi110.stdout:4/725: dwrite d13/d24/d2d/d36/d51/f8c [0,4194304] 0 2024-04-25T15:49:40.492 INFO:tasks.workunit.client.1.smithi193.stdout:9/478: dread df/d2c/d2f/d4e/f25 [4194304,4194304] 0 2024-04-25T15:49:40.493 INFO:tasks.workunit.client.0.smithi110.stdout:7/760: dwrite d1b/d29/d87/d8e/f94 [0,4194304] 0 2024-04-25T15:49:40.499 INFO:tasks.workunit.client.0.smithi110.stdout:6/662: unlink d1/db/d22/d33/d63/f9a 0 2024-04-25T15:49:40.504 INFO:tasks.workunit.client.1.smithi193.stdout:7/456: dread d12/d1c/d45/f6a [0,4194304] 0 2024-04-25T15:49:40.504 INFO:tasks.workunit.client.0.smithi110.stdout:2/677: dread de/d40/f72 [0,4194304] 0 2024-04-25T15:49:40.505 INFO:tasks.workunit.client.0.smithi110.stdout:2/678: fdatasync de/d40/d70/f7f 0 2024-04-25T15:49:40.510 INFO:tasks.workunit.client.1.smithi193.stdout:5/480: creat d17/d18/d3e/fa5 x:0 0 0 2024-04-25T15:49:40.510 INFO:tasks.workunit.client.1.smithi193.stdout:8/463: write d2/d3/d3a/f4c [1851573,36148] 0 2024-04-25T15:49:40.511 INFO:tasks.workunit.client.0.smithi110.stdout:4/726: mkdir d13/d24/d2d/dd9/dbb/dd2/dfd 0 2024-04-25T15:49:40.514 INFO:tasks.workunit.client.1.smithi193.stdout:0/486: rename d0/l29 to d0/d49/d5/d7/dd/d16/d26/d5e/d99/la5 0 2024-04-25T15:49:40.515 INFO:tasks.workunit.client.0.smithi110.stdout:7/761: rename d1b/c71 to d1b/dde/cfa 0 2024-04-25T15:49:40.519 INFO:tasks.workunit.client.0.smithi110.stdout:6/663: chown d1/db/d24/cc4 808141589 1 2024-04-25T15:49:40.520 INFO:tasks.workunit.client.1.smithi193.stdout:6/477: truncate dd/d12/d19/f51 1481996 0 2024-04-25T15:49:40.521 INFO:tasks.workunit.client.1.smithi193.stdout:7/457: write d12/d1c/d63/d70/faa [670518,113640] 0 2024-04-25T15:49:40.522 INFO:tasks.workunit.client.0.smithi110.stdout:3/742: dwrite d0/d16/f24 [0,4194304] 0 2024-04-25T15:49:40.522 INFO:tasks.workunit.client.1.smithi193.stdout:4/555: dwrite db/d13/d1e/d1f/d56/d57/fa4 [0,4194304] 0 2024-04-25T15:49:40.522 INFO:tasks.workunit.client.1.smithi193.stdout:6/478: chown dd/d12/d19/f43 1343 1 2024-04-25T15:49:40.524 INFO:tasks.workunit.client.1.smithi193.stdout:6/479: write dd/d2f/f5b [839411,16055] 0 2024-04-25T15:49:40.524 INFO:tasks.workunit.client.1.smithi193.stdout:2/474: getdents d0/d37/d25/d5f 0 2024-04-25T15:49:40.525 INFO:tasks.workunit.client.0.smithi110.stdout:1/896: truncate d0/d14/d81/df8/d33/d94/f115 1695787 0 2024-04-25T15:49:40.525 INFO:tasks.workunit.client.1.smithi193.stdout:2/475: chown d0/d42/c89 883 1 2024-04-25T15:49:40.526 INFO:tasks.workunit.client.1.smithi193.stdout:6/480: write dd/d2f/d57/f7d [128032,100306] 0 2024-04-25T15:49:40.528 INFO:tasks.workunit.client.0.smithi110.stdout:1/897: read d0/d14/d81/df8/d33/d8e/fd6 [634258,97728] 0 2024-04-25T15:49:40.529 INFO:tasks.workunit.client.0.smithi110.stdout:9/690: mkdir d9/d13/d5b/d61/db3/dd3/ddb 0 2024-04-25T15:49:40.530 INFO:tasks.workunit.client.1.smithi193.stdout:1/416: sync 2024-04-25T15:49:40.530 INFO:tasks.workunit.client.1.smithi193.stdout:3/518: sync 2024-04-25T15:49:40.533 INFO:tasks.workunit.client.0.smithi110.stdout:4/727: symlink d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/lfe 0 2024-04-25T15:49:40.534 INFO:tasks.workunit.client.0.smithi110.stdout:4/728: write d13/d24/d2d/d44/d65/d99/fee [940904,63959] 0 2024-04-25T15:49:40.535 INFO:tasks.workunit.client.1.smithi193.stdout:9/479: dread df/d11/f2e [0,4194304] 0 2024-04-25T15:49:40.535 INFO:tasks.workunit.client.1.smithi193.stdout:8/464: rename d2/f75 to d2/d3/d3a/d6d/f90 0 2024-04-25T15:49:40.536 INFO:tasks.workunit.client.1.smithi193.stdout:8/465: dread - d2/d3/d27/f7a zero size 2024-04-25T15:49:40.537 INFO:tasks.workunit.client.0.smithi110.stdout:8/807: sync 2024-04-25T15:49:40.538 INFO:tasks.workunit.client.0.smithi110.stdout:7/762: unlink d1b/d29/d38/d55/dab/db2/dcd/ced 0 2024-04-25T15:49:40.544 INFO:tasks.workunit.client.0.smithi110.stdout:4/729: dread fd [0,4194304] 0 2024-04-25T15:49:40.545 INFO:tasks.workunit.client.1.smithi193.stdout:7/458: rmdir d12/d31/d39/d60 39 2024-04-25T15:49:40.548 INFO:tasks.workunit.client.0.smithi110.stdout:2/679: mknod de/d40/dd7/cd8 0 2024-04-25T15:49:40.549 INFO:tasks.workunit.client.0.smithi110.stdout:2/680: write de/d40/d86/fb6 [279216,112000] 0 2024-04-25T15:49:40.551 INFO:tasks.workunit.client.0.smithi110.stdout:2/681: write de/d40/d6e/d50/f7b [859271,4033] 0 2024-04-25T15:49:40.552 INFO:tasks.workunit.client.1.smithi193.stdout:3/519: mkdir d8/d13/d4b/d8b/da2 0 2024-04-25T15:49:40.553 INFO:tasks.workunit.client.0.smithi110.stdout:1/898: readlink d0/d14/d81/df8/l36 0 2024-04-25T15:49:40.555 INFO:tasks.workunit.client.1.smithi193.stdout:1/417: mkdir dc/d4a/d50/d5c/d5d/d3d/d55/d8a 0 2024-04-25T15:49:40.555 INFO:tasks.workunit.client.1.smithi193.stdout:1/418: stat dc/d3c/c66 0 2024-04-25T15:49:40.555 INFO:tasks.workunit.client.0.smithi110.stdout:9/691: fdatasync f8 0 2024-04-25T15:49:40.556 INFO:tasks.workunit.client.0.smithi110.stdout:5/743: truncate d4/d2c/d2a/f5c 2921716 0 2024-04-25T15:49:40.558 INFO:tasks.workunit.client.0.smithi110.stdout:8/808: creat d14/d32/d36/f115 x:0 0 0 2024-04-25T15:49:40.559 INFO:tasks.workunit.client.0.smithi110.stdout:8/809: chown d14/d32/d36/d48/ca0 8 1 2024-04-25T15:49:40.559 INFO:tasks.workunit.client.0.smithi110.stdout:7/763: symlink d1b/dac/lfb 0 2024-04-25T15:49:40.560 INFO:tasks.workunit.client.0.smithi110.stdout:9/692: dread d9/db/df/d2c/f36 [0,4194304] 0 2024-04-25T15:49:40.562 INFO:tasks.workunit.client.1.smithi193.stdout:2/476: rename d0/d44/d8b to d0/df/dbf 0 2024-04-25T15:49:40.562 INFO:tasks.workunit.client.0.smithi110.stdout:0/674: sync 2024-04-25T15:49:40.563 INFO:tasks.workunit.client.1.smithi193.stdout:2/477: dread - d0/d42/d51/d4f/fa7 zero size 2024-04-25T15:49:40.563 INFO:tasks.workunit.client.1.smithi193.stdout:2/478: readlink d0/d9/d15/d31/l5e 0 2024-04-25T15:49:40.563 INFO:tasks.workunit.client.1.smithi193.stdout:2/479: chown d0/d44 1185512404 1 2024-04-25T15:49:40.564 INFO:tasks.workunit.client.0.smithi110.stdout:0/675: chown d1/d23/d30/d33/d5b/d72/d77/ced 2181 1 2024-04-25T15:49:40.565 INFO:tasks.workunit.client.1.smithi193.stdout:0/487: creat d0/d49/d5/d7/dd/d78/fa6 x:0 0 0 2024-04-25T15:49:40.567 INFO:tasks.workunit.client.0.smithi110.stdout:3/743: dread d0/d2e/d88/dbf/d99/fb5 [0,4194304] 0 2024-04-25T15:49:40.568 INFO:tasks.workunit.client.1.smithi193.stdout:7/459: unlink d12/c32 0 2024-04-25T15:49:40.576 INFO:tasks.workunit.client.0.smithi110.stdout:1/899: chown d0/d51/lff 23604792 1 2024-04-25T15:49:40.576 INFO:tasks.workunit.client.0.smithi110.stdout:4/730: dwrite d13/d24/d2d/d36/d51/f68 [0,4194304] 0 2024-04-25T15:49:40.583 INFO:tasks.workunit.client.1.smithi193.stdout:6/481: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f87 [1143698,9567] 0 2024-04-25T15:49:40.585 INFO:tasks.workunit.client.0.smithi110.stdout:9/693: unlink d9/d13/d5b/d61/f63 0 2024-04-25T15:49:40.586 INFO:tasks.workunit.client.0.smithi110.stdout:9/694: fsync d9/db/df/f68 0 2024-04-25T15:49:40.586 INFO:tasks.workunit.client.1.smithi193.stdout:8/466: mknod d2/c91 0 2024-04-25T15:49:40.588 INFO:tasks.workunit.client.0.smithi110.stdout:7/764: stat d1b/d29/d38/d55/dab/db2/dcd/de2/lef 0 2024-04-25T15:49:40.590 INFO:tasks.workunit.client.0.smithi110.stdout:6/664: creat d1/db/d24/fe8 x:0 0 0 2024-04-25T15:49:40.592 INFO:tasks.workunit.client.1.smithi193.stdout:7/460: fdatasync d12/d1c/d63/d70/d68/d8d/f9c 0 2024-04-25T15:49:40.592 INFO:tasks.workunit.client.0.smithi110.stdout:5/744: dwrite d4/d1d/f70 [0,4194304] 0 2024-04-25T15:49:40.593 INFO:tasks.workunit.client.0.smithi110.stdout:7/765: dread d1b/d29/db5/dca/fd5 [0,4194304] 0 2024-04-25T15:49:40.593 INFO:tasks.workunit.client.1.smithi193.stdout:5/481: link l9 d17/d18/d3e/la6 0 2024-04-25T15:49:40.594 INFO:tasks.workunit.client.0.smithi110.stdout:7/766: readlink d1b/d29/d38/d55/l90 0 2024-04-25T15:49:40.595 INFO:tasks.workunit.client.0.smithi110.stdout:3/744: mknod d0/d16/dcb/d2f/d33/cfe 0 2024-04-25T15:49:40.599 INFO:tasks.workunit.client.1.smithi193.stdout:9/480: creat df/d2c/d2f/f8f x:0 0 0 2024-04-25T15:49:40.603 INFO:tasks.workunit.client.1.smithi193.stdout:3/520: rename d8/d13/d14/f17 to d8/d30/d72/fa3 0 2024-04-25T15:49:40.603 INFO:tasks.workunit.client.0.smithi110.stdout:2/682: dread de/d40/d4f/f49 [0,4194304] 0 2024-04-25T15:49:40.604 INFO:tasks.workunit.client.1.smithi193.stdout:3/521: fdatasync d8/d4d/f87 0 2024-04-25T15:49:40.604 INFO:tasks.workunit.client.0.smithi110.stdout:2/683: chown de/d40/d4f/f4c 24900638 1 2024-04-25T15:49:40.605 INFO:tasks.workunit.client.1.smithi193.stdout:6/482: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f 0 2024-04-25T15:49:40.606 INFO:tasks.workunit.client.1.smithi193.stdout:8/467: symlink d2/d3/d20/d45/d4a/d63/l92 0 2024-04-25T15:49:40.607 INFO:tasks.workunit.client.0.smithi110.stdout:4/731: read d13/d24/d26/f9b [1335972,107570] 0 2024-04-25T15:49:40.608 INFO:tasks.workunit.client.0.smithi110.stdout:1/900: creat d0/d14/d5f/d7d/d9c/f122 x:0 0 0 2024-04-25T15:49:40.608 INFO:tasks.workunit.client.0.smithi110.stdout:4/732: write d13/d24/d2d/d36/ff2 [332821,58276] 0 2024-04-25T15:49:40.609 INFO:tasks.workunit.client.1.smithi193.stdout:2/480: symlink d0/d37/d25/d5f/d3a/db8/lc0 0 2024-04-25T15:49:40.609 INFO:tasks.workunit.client.0.smithi110.stdout:1/901: chown d0/d14/d81/c90 3 1 2024-04-25T15:49:40.610 INFO:tasks.workunit.client.1.smithi193.stdout:0/488: symlink d0/d49/d5/d7/dd/d16/d18/la7 0 2024-04-25T15:49:40.611 INFO:tasks.workunit.client.1.smithi193.stdout:4/556: getdents db/d32/d40 0 2024-04-25T15:49:40.613 INFO:tasks.workunit.client.1.smithi193.stdout:5/482: mkdir d17/d18/d45/da7 0 2024-04-25T15:49:40.615 INFO:tasks.workunit.client.1.smithi193.stdout:9/481: mknod df/d11/d8b/c90 0 2024-04-25T15:49:40.620 INFO:tasks.workunit.client.1.smithi193.stdout:2/481: creat d0/d9/fc1 x:0 0 0 2024-04-25T15:49:40.622 INFO:tasks.workunit.client.1.smithi193.stdout:9/482: dwrite f9 [0,4194304] 0 2024-04-25T15:49:40.623 INFO:tasks.workunit.client.1.smithi193.stdout:9/483: stat df/d11/d8b/l36 0 2024-04-25T15:49:40.624 INFO:tasks.workunit.client.0.smithi110.stdout:1/902: creat d0/d14/d5f/d7d/d72/d61/f123 x:0 0 0 2024-04-25T15:49:40.625 INFO:tasks.workunit.client.0.smithi110.stdout:1/903: write d0/d14/fc0 [3970452,1745] 0 2024-04-25T15:49:40.627 INFO:tasks.workunit.client.1.smithi193.stdout:0/489: creat d0/d49/d5/d7/dd/d16/d26/d5e/fa8 x:0 0 0 2024-04-25T15:49:40.628 INFO:tasks.workunit.client.0.smithi110.stdout:8/810: rename ff to d14/f116 0 2024-04-25T15:49:40.628 INFO:tasks.workunit.client.0.smithi110.stdout:8/811: fsync d14/f16 0 2024-04-25T15:49:40.629 INFO:tasks.workunit.client.1.smithi193.stdout:7/461: creat d12/d1c/d63/d70/d68/d8f/fab x:0 0 0 2024-04-25T15:49:40.630 INFO:tasks.workunit.client.0.smithi110.stdout:2/684: dread de/d40/d4f/d9f/fa3 [0,4194304] 0 2024-04-25T15:49:40.630 INFO:tasks.workunit.client.1.smithi193.stdout:8/468: dread d2/d3/f68 [0,4194304] 0 2024-04-25T15:49:40.631 INFO:tasks.workunit.client.1.smithi193.stdout:8/469: write d2/d3/d3a/f4c [2458590,69002] 0 2024-04-25T15:49:40.633 INFO:tasks.workunit.client.1.smithi193.stdout:6/483: dread dd/d12/d19/d1c/f29 [0,4194304] 0 2024-04-25T15:49:40.635 INFO:tasks.workunit.client.0.smithi110.stdout:6/665: write d1/dd/d3c/d91/d8d/dce/f78 [2442924,68718] 0 2024-04-25T15:49:40.636 INFO:tasks.workunit.client.0.smithi110.stdout:6/666: chown d1/db/d24 637405 1 2024-04-25T15:49:40.639 INFO:tasks.workunit.client.1.smithi193.stdout:1/419: sync 2024-04-25T15:49:40.639 INFO:tasks.workunit.client.0.smithi110.stdout:3/745: mknod d0/d7/d40/cff 0 2024-04-25T15:49:40.639 INFO:tasks.workunit.client.0.smithi110.stdout:4/733: write f6 [501568,19319] 0 2024-04-25T15:49:40.639 INFO:tasks.workunit.client.0.smithi110.stdout:0/676: dwrite d1/f38 [0,4194304] 0 2024-04-25T15:49:40.640 INFO:tasks.workunit.client.1.smithi193.stdout:2/482: creat d0/df/da5/fc2 x:0 0 0 2024-04-25T15:49:40.641 INFO:tasks.workunit.client.1.smithi193.stdout:9/484: rmdir df/d11 39 2024-04-25T15:49:40.642 INFO:tasks.workunit.client.0.smithi110.stdout:4/734: truncate d13/d24/d2d/d36/ff6 836553 0 2024-04-25T15:49:40.643 INFO:tasks.workunit.client.0.smithi110.stdout:4/735: readlink d13/d24/d2d/d44/d65/d99/daf/d56/d88/l30 0 2024-04-25T15:49:40.644 INFO:tasks.workunit.client.1.smithi193.stdout:7/462: creat d12/d13/fac x:0 0 0 2024-04-25T15:49:40.645 INFO:tasks.workunit.client.0.smithi110.stdout:1/904: creat d0/d14/d81/df8/d33/d8e/f124 x:0 0 0 2024-04-25T15:49:40.646 INFO:tasks.workunit.client.0.smithi110.stdout:1/905: fdatasync d0/d14/d81/df8/f6 0 2024-04-25T15:49:40.648 INFO:tasks.workunit.client.1.smithi193.stdout:2/483: dwrite d0/d42/d51/f94 [0,4194304] 0 2024-04-25T15:49:40.648 INFO:tasks.workunit.client.0.smithi110.stdout:2/685: mkdir de/d6f/dd9 0 2024-04-25T15:49:40.649 INFO:tasks.workunit.client.0.smithi110.stdout:2/686: fdatasync de/d40/d6e/f9d 0 2024-04-25T15:49:40.650 INFO:tasks.workunit.client.1.smithi193.stdout:4/557: mkdir db/d32/d88/da8 0 2024-04-25T15:49:40.651 INFO:tasks.workunit.client.0.smithi110.stdout:2/687: chown de/d40/d4f/f8b 0 1 2024-04-25T15:49:40.651 INFO:tasks.workunit.client.0.smithi110.stdout:8/812: symlink d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/l117 0 2024-04-25T15:49:40.655 INFO:tasks.workunit.client.0.smithi110.stdout:1/906: dwrite d0/d14/d81/df8/db4/f106 [0,4194304] 0 2024-04-25T15:49:40.656 INFO:tasks.workunit.client.1.smithi193.stdout:8/470: dread d2/d3/d20/d45/f49 [0,4194304] 0 2024-04-25T15:49:40.659 INFO:tasks.workunit.client.0.smithi110.stdout:7/767: creat d1b/d29/d38/d55/ffc x:0 0 0 2024-04-25T15:49:40.660 INFO:tasks.workunit.client.0.smithi110.stdout:6/667: creat d1/d79/fe9 x:0 0 0 2024-04-25T15:49:40.660 INFO:tasks.workunit.client.0.smithi110.stdout:3/746: symlink d0/d16/dcb/d2f/d4c/l100 0 2024-04-25T15:49:40.661 INFO:tasks.workunit.client.1.smithi193.stdout:1/420: creat dc/d3c/f8b x:0 0 0 2024-04-25T15:49:40.664 INFO:tasks.workunit.client.0.smithi110.stdout:7/768: dread d1b/d29/f5d [0,4194304] 0 2024-04-25T15:49:40.665 INFO:tasks.workunit.client.0.smithi110.stdout:7/769: write d1b/d29/fa4 [2605365,52628] 0 2024-04-25T15:49:40.670 INFO:tasks.workunit.client.0.smithi110.stdout:8/813: chown d14/d21/d26/d6a/d7e/da6/de8/d55/d8d/fad 87320 1 2024-04-25T15:49:40.745 INFO:tasks.workunit.client.0.smithi110.stdout:1/907: mknod d0/d14/d81/df8/db4/dcd/c125 0 2024-04-25T15:49:40.749 INFO:tasks.workunit.client.0.smithi110.stdout:3/747: rmdir d0/d7/dd0 39 2024-04-25T15:49:40.750 INFO:tasks.workunit.client.1.smithi193.stdout:0/490: truncate d0/d49/d5/d7/dd/d16/d18/d28/f94 2051189 0 2024-04-25T15:49:40.751 INFO:tasks.workunit.client.0.smithi110.stdout:3/748: read d0/d16/dcb/d2f/d33/f44 [1557014,11984] 0 2024-04-25T15:49:40.752 INFO:tasks.workunit.client.0.smithi110.stdout:6/668: mkdir d1/db/da0/da5/dea 0 2024-04-25T15:49:40.754 INFO:tasks.workunit.client.1.smithi193.stdout:5/483: creat d17/d18/fa8 x:0 0 0 2024-04-25T15:49:40.756 INFO:tasks.workunit.client.1.smithi193.stdout:8/471: unlink d2/d3/d3a/d6d/l8f 0 2024-04-25T15:49:40.757 INFO:tasks.workunit.client.1.smithi193.stdout:6/484: write dd/f2b [193062,109482] 0 2024-04-25T15:49:40.758 INFO:tasks.workunit.client.1.smithi193.stdout:3/522: getdents d8/d30/d45 0 2024-04-25T15:49:40.759 INFO:tasks.workunit.client.1.smithi193.stdout:3/523: truncate d8/d30/f98 685391 0 2024-04-25T15:49:40.761 INFO:tasks.workunit.client.0.smithi110.stdout:8/814: mkdir d14/d21/d26/d6a/d118 0 2024-04-25T15:49:40.762 INFO:tasks.workunit.client.0.smithi110.stdout:8/815: chown d14/d21/d26/f29 120200 1 2024-04-25T15:49:40.764 INFO:tasks.workunit.client.0.smithi110.stdout:1/908: truncate d0/d14/d5f/d7d/f64 1728678 0 2024-04-25T15:49:40.764 INFO:tasks.workunit.client.1.smithi193.stdout:9/485: symlink df/d2c/d2f/d4e/l91 0 2024-04-25T15:49:40.764 INFO:tasks.workunit.client.1.smithi193.stdout:7/463: mknod d12/d1c/da3/cad 0 2024-04-25T15:49:40.764 INFO:tasks.workunit.client.1.smithi193.stdout:9/486: fdatasync df/d2c/d2f/f6e 0 2024-04-25T15:49:40.766 INFO:tasks.workunit.client.1.smithi193.stdout:9/487: write df/d2c/d2f/d4e/d1c/f23 [592012,44201] 0 2024-04-25T15:49:40.768 INFO:tasks.workunit.client.0.smithi110.stdout:6/669: mknod d1/dd/db5/ceb 0 2024-04-25T15:49:40.771 INFO:tasks.workunit.client.1.smithi193.stdout:8/472: mknod d2/d3/d20/d45/c93 0 2024-04-25T15:49:40.772 INFO:tasks.workunit.client.1.smithi193.stdout:7/464: dwrite d12/d1c/d63/d70/d68/d8f/fab [0,4194304] 0 2024-04-25T15:49:40.773 INFO:tasks.workunit.client.1.smithi193.stdout:7/465: chown d12/l4b 3610926 1 2024-04-25T15:49:40.773 INFO:tasks.workunit.client.1.smithi193.stdout:6/485: creat dd/d2f/d57/f90 x:0 0 0 2024-04-25T15:49:40.773 INFO:tasks.workunit.client.0.smithi110.stdout:8/816: creat d14/d32/d36/db7/dc9/f119 x:0 0 0 2024-04-25T15:49:40.774 INFO:tasks.workunit.client.1.smithi193.stdout:5/484: write d17/d18/d75/f88 [1074230,69509] 0 2024-04-25T15:49:40.776 INFO:tasks.workunit.client.1.smithi193.stdout:5/485: fdatasync d17/d18/d30/d38/f69 0 2024-04-25T15:49:40.780 INFO:tasks.workunit.client.0.smithi110.stdout:6/670: mkdir d1/db/d22/d33/d63/dc2/dc7/dec 0 2024-04-25T15:49:40.782 INFO:tasks.workunit.client.0.smithi110.stdout:0/677: rename d1/d23/c29 to d1/d23/d30/cf3 0 2024-04-25T15:49:40.784 INFO:tasks.workunit.client.0.smithi110.stdout:1/909: dwrite d0/f3f [0,4194304] 0 2024-04-25T15:49:40.784 INFO:tasks.workunit.client.0.smithi110.stdout:0/678: chown d1/d23/d30/d33/d5b/d83/f53 23794629 1 2024-04-25T15:49:40.786 INFO:tasks.workunit.client.0.smithi110.stdout:2/688: getdents de/d40 0 2024-04-25T15:49:40.787 INFO:tasks.workunit.client.0.smithi110.stdout:2/689: chown de/d40/d6e/d52 0 1 2024-04-25T15:49:40.788 INFO:tasks.workunit.client.0.smithi110.stdout:8/817: symlink d14/d21/d26/d6a/l11a 0 2024-04-25T15:49:40.791 INFO:tasks.workunit.client.0.smithi110.stdout:3/749: creat d0/f101 x:0 0 0 2024-04-25T15:49:40.793 INFO:tasks.workunit.client.0.smithi110.stdout:6/671: rmdir d1/db/d24 39 2024-04-25T15:49:40.795 INFO:tasks.workunit.client.0.smithi110.stdout:0/679: stat d1/d20/l21 0 2024-04-25T15:49:40.796 INFO:tasks.workunit.client.0.smithi110.stdout:2/690: rmdir de/d40/d6e/d52 39 2024-04-25T15:49:40.799 INFO:tasks.workunit.client.0.smithi110.stdout:8/818: mknod d14/d21/d26/dab/dac/dbb/c11b 0 2024-04-25T15:49:40.801 INFO:tasks.workunit.client.0.smithi110.stdout:3/750: mknod d0/d2e/d88/c102 0 2024-04-25T15:49:40.803 INFO:tasks.workunit.client.0.smithi110.stdout:6/672: creat d1/db/d22/d33/de4/fed x:0 0 0 2024-04-25T15:49:40.803 INFO:tasks.workunit.client.0.smithi110.stdout:6/673: stat d1/dd/d3c/d91/d8d/dce/fde 0 2024-04-25T15:49:40.805 INFO:tasks.workunit.client.0.smithi110.stdout:7/770: rename d1b/c33 to d1b/d29/d38/d55/dab/db2/db8/de3/cfd 0 2024-04-25T15:49:40.808 INFO:tasks.workunit.client.1.smithi193.stdout:3/524: symlink d8/d13/d1f/d69/d77/la4 0 2024-04-25T15:49:40.812 INFO:tasks.workunit.client.1.smithi193.stdout:4/558: rename db/d4b/l4d to db/d32/la9 0 2024-04-25T15:49:40.813 INFO:tasks.workunit.client.0.smithi110.stdout:2/691: dwrite de/d40/d4f/d9f/fa3 [0,4194304] 0 2024-04-25T15:49:40.822 INFO:tasks.workunit.client.0.smithi110.stdout:7/771: mknod d1b/d29/d38/d55/d95/cfe 0 2024-04-25T15:49:40.825 INFO:tasks.workunit.client.1.smithi193.stdout:3/525: dread d8/d13/d1f/d69/d77/f53 [4194304,4194304] 0 2024-04-25T15:49:40.825 INFO:tasks.workunit.client.1.smithi193.stdout:6/486: rmdir dd/d2f/d57 39 2024-04-25T15:49:40.826 INFO:tasks.workunit.client.0.smithi110.stdout:1/910: truncate d0/d14/d5f/d7d/d72/d88/db6/fd4 1529629 0 2024-04-25T15:49:40.827 INFO:tasks.workunit.client.0.smithi110.stdout:1/911: chown d0/d23 2 1 2024-04-25T15:49:40.829 INFO:tasks.workunit.client.0.smithi110.stdout:2/692: truncate de/d40/d5b/d5c/d5f/fa8 753210 0 2024-04-25T15:49:40.831 INFO:tasks.workunit.client.1.smithi193.stdout:5/486: unlink fe 0 2024-04-25T15:49:40.831 INFO:tasks.workunit.client.0.smithi110.stdout:8/819: write d14/d21/d26/d6a/d7e/da6/de8/d71/fd5 [1746673,81667] 0 2024-04-25T15:49:40.832 INFO:tasks.workunit.client.0.smithi110.stdout:3/751: symlink d0/d2e/dfc/l103 0 2024-04-25T15:49:40.835 INFO:tasks.workunit.client.1.smithi193.stdout:1/421: rename dc/fe to dc/d4a/d50/d7c/d7e/f8c 0 2024-04-25T15:49:40.836 INFO:tasks.workunit.client.1.smithi193.stdout:4/559: rmdir db/d4b/d51 39 2024-04-25T15:49:40.839 INFO:tasks.workunit.client.1.smithi193.stdout:9/488: mknod df/d2c/d2f/d4e/d69/d6b/c92 0 2024-04-25T15:49:40.840 INFO:tasks.workunit.client.1.smithi193.stdout:9/489: stat df/l10 0 2024-04-25T15:49:40.842 INFO:tasks.workunit.client.0.smithi110.stdout:8/820: mknod d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/c11c 0 2024-04-25T15:49:40.842 INFO:tasks.workunit.client.1.smithi193.stdout:4/560: dwrite db/d4b/fa3 [0,4194304] 0 2024-04-25T15:49:40.844 INFO:tasks.workunit.client.0.smithi110.stdout:6/674: write d1/db/da0/fc8 [2912087,3677] 0 2024-04-25T15:49:40.844 INFO:tasks.workunit.client.1.smithi193.stdout:4/561: dread db/f61 [0,4194304] 0 2024-04-25T15:49:40.847 INFO:tasks.workunit.client.0.smithi110.stdout:7/772: creat d1b/d29/d38/d55/d95/dcb/fff x:0 0 0 2024-04-25T15:49:40.848 INFO:tasks.workunit.client.0.smithi110.stdout:0/680: getdents d1/d23/d30/d33/d5b/dbb 0 2024-04-25T15:49:40.850 INFO:tasks.workunit.client.0.smithi110.stdout:8/821: dwrite f6 [4194304,4194304] 0 2024-04-25T15:49:40.851 INFO:tasks.workunit.client.0.smithi110.stdout:0/681: readlink d1/d23/d30/d33/d5b/dbb/da8/laa 0 2024-04-25T15:49:40.856 INFO:tasks.workunit.client.0.smithi110.stdout:6/675: mkdir d1/db/d22/d90/d76/dee 0 2024-04-25T15:49:40.857 INFO:tasks.workunit.client.1.smithi193.stdout:7/466: rename d12/d13/c8b to d12/d1c/da3/cae 0 2024-04-25T15:49:40.859 INFO:tasks.workunit.client.0.smithi110.stdout:0/682: mkdir d1/d23/d81/df4 0 2024-04-25T15:49:40.861 INFO:tasks.workunit.client.0.smithi110.stdout:0/683: truncate d1/d6c/ff1 110977 0 2024-04-25T15:49:40.862 INFO:tasks.workunit.client.0.smithi110.stdout:0/684: readlink d1/d23/d30/d33/d5b/l63 0 2024-04-25T15:49:40.865 INFO:tasks.workunit.client.0.smithi110.stdout:3/752: creat d0/d7/f104 x:0 0 0 2024-04-25T15:49:40.866 INFO:tasks.workunit.client.1.smithi193.stdout:1/422: dwrite dc/f17 [4194304,4194304] 0 2024-04-25T15:49:40.867 INFO:tasks.workunit.client.0.smithi110.stdout:3/753: write d0/d2e/d88/dbf/d92/fd2 [1728181,3553] 0 2024-04-25T15:49:40.869 INFO:tasks.workunit.client.0.smithi110.stdout:3/754: chown d0/d16/dcb/d2f/d33/f36 36 1 2024-04-25T15:49:40.873 INFO:tasks.workunit.client.0.smithi110.stdout:0/685: dwrite d1/d47/fa9 [4194304,4194304] 0 2024-04-25T15:49:40.884 INFO:tasks.workunit.client.0.smithi110.stdout:7/773: dwrite d1b/d29/d38/f3d [4194304,4194304] 0 2024-04-25T15:49:40.884 INFO:tasks.workunit.client.1.smithi193.stdout:5/487: creat d17/d18/d9b/fa9 x:0 0 0 2024-04-25T15:49:40.886 INFO:tasks.workunit.client.1.smithi193.stdout:2/484: rename d0/d9/d15/d7b to d0/d9/d3d/d7d/dc3 0 2024-04-25T15:49:40.887 INFO:tasks.workunit.client.1.smithi193.stdout:0/491: link d0/d49/d5/d7/dd/d16/d26/d5e/d99/la5 d0/d49/d5/d61/la9 0 2024-04-25T15:49:40.888 INFO:tasks.workunit.client.0.smithi110.stdout:0/686: symlink d1/d6c/d7f/lf5 0 2024-04-25T15:49:40.890 INFO:tasks.workunit.client.1.smithi193.stdout:8/473: getdents d2/d3/d27/d69/d86 0 2024-04-25T15:49:40.890 INFO:tasks.workunit.client.1.smithi193.stdout:7/467: dread d12/f2e [0,4194304] 0 2024-04-25T15:49:40.891 INFO:tasks.workunit.client.1.smithi193.stdout:1/423: mknod dc/d4a/d50/d5c/d5d/d3d/c8d 0 2024-04-25T15:49:40.895 INFO:tasks.workunit.client.1.smithi193.stdout:4/562: creat db/d4b/d51/d54/d77/faa x:0 0 0 2024-04-25T15:49:40.897 INFO:tasks.workunit.client.1.smithi193.stdout:5/488: dread d17/d18/d30/d38/d48/f8f [0,4194304] 0 2024-04-25T15:49:40.898 INFO:tasks.workunit.client.1.smithi193.stdout:0/492: read d0/d49/d5/d7/dd/d16/d1a/f8e [2114592,130250] 0 2024-04-25T15:49:40.898 INFO:tasks.workunit.client.1.smithi193.stdout:4/563: write db/fc [1509046,59217] 0 2024-04-25T15:49:40.899 INFO:tasks.workunit.client.0.smithi110.stdout:8/822: dwrite d14/d21/d26/d6a/d7e/da6/de8/f57 [4194304,4194304] 0 2024-04-25T15:49:40.899 INFO:tasks.workunit.client.1.smithi193.stdout:2/485: dwrite d0/d42/d51/d4f/fac [0,4194304] 0 2024-04-25T15:49:40.900 INFO:tasks.workunit.client.0.smithi110.stdout:3/755: dread d0/d16/f24 [4194304,4194304] 0 2024-04-25T15:49:40.903 INFO:tasks.workunit.client.1.smithi193.stdout:6/487: creat dd/f91 x:0 0 0 2024-04-25T15:49:40.904 INFO:tasks.workunit.client.1.smithi193.stdout:2/486: write d0/d37/d25/d5f/d3a/d6d/d6e/f3e [2539974,7913] 0 2024-04-25T15:49:40.905 INFO:tasks.workunit.client.1.smithi193.stdout:4/564: truncate db/d13/d1e/d1f/f98 963610 0 2024-04-25T15:49:40.909 INFO:tasks.workunit.client.1.smithi193.stdout:7/468: readlink d12/d1c/d45/l66 0 2024-04-25T15:49:40.910 INFO:tasks.workunit.client.1.smithi193.stdout:8/474: dwrite d2/d3/d20/d45/f88 [0,4194304] 0 2024-04-25T15:49:40.911 INFO:tasks.workunit.client.1.smithi193.stdout:1/424: creat dc/d59/f8e x:0 0 0 2024-04-25T15:49:40.912 INFO:tasks.workunit.client.0.smithi110.stdout:3/756: truncate d0/d2e/f30 7641066 0 2024-04-25T15:49:40.913 INFO:tasks.workunit.client.0.smithi110.stdout:6/676: dwrite d1/dd/d3c/d91/d8d/f99 [0,4194304] 0 2024-04-25T15:49:40.914 INFO:tasks.workunit.client.1.smithi193.stdout:5/489: dwrite d17/d18/f21 [8388608,4194304] 0 2024-04-25T15:49:40.914 INFO:tasks.workunit.client.0.smithi110.stdout:6/677: chown d1/dd/d3c/d91/d8d/dce/da1 81174984 1 2024-04-25T15:49:40.915 INFO:tasks.workunit.client.1.smithi193.stdout:0/493: fsync d0/f35 0 2024-04-25T15:49:40.918 INFO:tasks.workunit.client.0.smithi110.stdout:8/823: rename d14/d32/d36/d48/d8f/ccd to d14/d21/d26/d6a/d7e/da6/de8/c11d 0 2024-04-25T15:49:40.921 INFO:tasks.workunit.client.0.smithi110.stdout:3/757: creat d0/d16/dcb/db0/f105 x:0 0 0 2024-04-25T15:49:40.924 INFO:tasks.workunit.client.1.smithi193.stdout:4/565: mkdir db/d32/d45/dab 0 2024-04-25T15:49:40.924 INFO:tasks.workunit.client.1.smithi193.stdout:4/566: write db/f61 [1343375,77371] 0 2024-04-25T15:49:40.925 INFO:tasks.workunit.client.1.smithi193.stdout:3/526: truncate d8/d30/d72/f7e 1662563 0 2024-04-25T15:49:40.926 INFO:tasks.workunit.client.1.smithi193.stdout:3/527: write d8/d30/d45/f70 [6281087,111052] 0 2024-04-25T15:49:40.929 INFO:tasks.workunit.client.0.smithi110.stdout:6/678: creat d1/db/d24/d69/d75/fef x:0 0 0 2024-04-25T15:49:40.931 INFO:tasks.workunit.client.1.smithi193.stdout:7/469: creat d12/d1c/faf x:0 0 0 2024-04-25T15:49:40.935 INFO:tasks.workunit.client.1.smithi193.stdout:8/475: write d2/d3/d3a/d6d/f90 [88464,20026] 0 2024-04-25T15:49:40.936 INFO:tasks.workunit.client.0.smithi110.stdout:6/679: truncate d1/db/f1a 4594137 0 2024-04-25T15:49:40.938 INFO:tasks.workunit.client.1.smithi193.stdout:8/476: readlink d2/d3/d20/d45/d4a/l82 0 2024-04-25T15:49:40.938 INFO:tasks.workunit.client.0.smithi110.stdout:8/824: rmdir d14/d21/d26/d6a/d7e/da6/de8/d55/de5 0 2024-04-25T15:49:40.939 INFO:tasks.workunit.client.0.smithi110.stdout:7/774: write d1b/d29/d38/f61 [646027,79805] 0 2024-04-25T15:49:40.939 INFO:tasks.workunit.client.1.smithi193.stdout:7/470: dwrite d12/f2e [0,4194304] 0 2024-04-25T15:49:40.939 INFO:tasks.workunit.client.1.smithi193.stdout:8/477: stat d2/d3/d3a/f5e 0 2024-04-25T15:49:40.944 INFO:tasks.workunit.client.0.smithi110.stdout:8/825: mkdir d14/d32/d36/dde/d11e 0 2024-04-25T15:49:40.945 INFO:tasks.workunit.client.0.smithi110.stdout:6/680: dwrite d1/db/d24/d69/fb3 [0,4194304] 0 2024-04-25T15:49:40.947 INFO:tasks.workunit.client.1.smithi193.stdout:9/490: sync 2024-04-25T15:49:40.948 INFO:tasks.workunit.client.1.smithi193.stdout:4/567: unlink db/d13/d1e/d1f/d56/f74 0 2024-04-25T15:49:40.948 INFO:tasks.workunit.client.1.smithi193.stdout:9/491: write df/d2c/d2f/f6e [218785,96794] 0 2024-04-25T15:49:40.948 INFO:tasks.workunit.client.1.smithi193.stdout:4/568: write f2 [5143651,43760] 0 2024-04-25T15:49:40.950 INFO:tasks.workunit.client.0.smithi110.stdout:8/826: chown d14/d21/d26/d6a/d7e/da6/de8/d71/le6 402965 1 2024-04-25T15:49:40.950 INFO:tasks.workunit.client.0.smithi110.stdout:0/687: dwrite d1/d23/d30/d33/d35/da1/db4/db9/fd3 [0,4194304] 0 2024-04-25T15:49:40.951 INFO:tasks.workunit.client.1.smithi193.stdout:9/492: write df/d2c/d34/d3e/d4d/d5e/d81/d41/f70 [1626449,26279] 0 2024-04-25T15:49:40.958 INFO:tasks.workunit.client.0.smithi110.stdout:9/695: sync 2024-04-25T15:49:40.962 INFO:tasks.workunit.client.0.smithi110.stdout:6/681: unlink d1/db/d22/f55 0 2024-04-25T15:49:40.964 INFO:tasks.workunit.client.1.smithi193.stdout:3/528: rename d8/d13/f41 to d8/d13/d1f/d9f/fa5 0 2024-04-25T15:49:40.965 INFO:tasks.workunit.client.1.smithi193.stdout:3/529: chown d8/d13/d14/f80 181629292 1 2024-04-25T15:49:40.966 INFO:tasks.workunit.client.1.smithi193.stdout:1/425: symlink dc/l8f 0 2024-04-25T15:49:40.969 INFO:tasks.workunit.client.0.smithi110.stdout:8/827: readlink d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/l10c 0 2024-04-25T15:49:40.970 INFO:tasks.workunit.client.1.smithi193.stdout:5/490: chown d17/d18/d75/d85/d5d 14036 1 2024-04-25T15:49:40.972 INFO:tasks.workunit.client.0.smithi110.stdout:9/696: mknod d9/db/df/d2c/d3c/d56/da0/cdc 0 2024-04-25T15:49:40.973 INFO:tasks.workunit.client.0.smithi110.stdout:6/682: mknod d1/db/d22/d33/de4/cf0 0 2024-04-25T15:49:40.975 INFO:tasks.workunit.client.0.smithi110.stdout:0/688: mknod d1/d23/d30/cf6 0 2024-04-25T15:49:40.977 INFO:tasks.workunit.client.0.smithi110.stdout:8/828: mkdir d14/d32/dd6/d11f 0 2024-04-25T15:49:40.982 INFO:tasks.workunit.client.0.smithi110.stdout:7/775: write d1b/d29/f98 [660398,116376] 0 2024-04-25T15:49:40.987 INFO:tasks.workunit.client.1.smithi193.stdout:8/478: rename d2/d3/d3a/d6d to d2/d3/d27/d6f/d94 0 2024-04-25T15:49:40.989 INFO:tasks.workunit.client.0.smithi110.stdout:0/689: unlink d1/f2a 0 2024-04-25T15:49:40.990 INFO:tasks.workunit.client.1.smithi193.stdout:2/487: truncate d0/d37/f67 949704 0 2024-04-25T15:49:40.991 INFO:tasks.workunit.client.1.smithi193.stdout:3/530: mknod d8/d13/d1f/d69/d77/d71/ca6 0 2024-04-25T15:49:40.992 INFO:tasks.workunit.client.1.smithi193.stdout:3/531: write d8/d4d/f87 [338596,104212] 0 2024-04-25T15:49:40.998 INFO:tasks.workunit.client.0.smithi110.stdout:6/683: dread d1/db/f19 [0,4194304] 0 2024-04-25T15:49:40.999 INFO:tasks.workunit.client.1.smithi193.stdout:5/491: fsync d17/d18/d30/d38/d48/d4f/f5f 0 2024-04-25T15:49:41.001 INFO:tasks.workunit.client.1.smithi193.stdout:7/471: mkdir d12/d31/d39/db0 0 2024-04-25T15:49:41.005 INFO:tasks.workunit.client.0.smithi110.stdout:0/690: dread d1/d20/f96 [0,4194304] 0 2024-04-25T15:49:41.005 INFO:tasks.workunit.client.1.smithi193.stdout:0/494: creat d0/d49/faa x:0 0 0 2024-04-25T15:49:41.006 INFO:tasks.workunit.client.0.smithi110.stdout:0/691: dread - d1/d47/d54/d99/fc1 zero size 2024-04-25T15:49:41.007 INFO:tasks.workunit.client.1.smithi193.stdout:6/488: getdents dd/d12/d2d 0 2024-04-25T15:49:41.010 INFO:tasks.workunit.client.1.smithi193.stdout:8/479: mkdir d2/d3/d3a/d95 0 2024-04-25T15:49:41.012 INFO:tasks.workunit.client.0.smithi110.stdout:9/697: rename d9/d13/f29 to d9/d13/d5b/d61/db3/dd3/ddb/fdd 0 2024-04-25T15:49:41.013 INFO:tasks.workunit.client.1.smithi193.stdout:2/488: fdatasync d0/d9/d15/f78 0 2024-04-25T15:49:41.015 INFO:tasks.workunit.client.0.smithi110.stdout:7/776: symlink d1b/d29/d38/d55/l100 0 2024-04-25T15:49:41.017 INFO:tasks.workunit.client.1.smithi193.stdout:8/480: dwrite d2/d3/d3a/f7c [0,4194304] 0 2024-04-25T15:49:41.017 INFO:tasks.workunit.client.0.smithi110.stdout:6/684: creat d1/db/d22/d90/d76/dad/ff1 x:0 0 0 2024-04-25T15:49:41.025 INFO:tasks.workunit.client.0.smithi110.stdout:6/685: dread d1/f5e [0,4194304] 0 2024-04-25T15:49:41.025 INFO:tasks.workunit.client.1.smithi193.stdout:5/492: symlink d17/d18/d30/d34/laa 0 2024-04-25T15:49:41.025 INFO:tasks.workunit.client.0.smithi110.stdout:8/829: rmdir d14/d32/dd6/d11f 0 2024-04-25T15:49:41.031 INFO:tasks.workunit.client.1.smithi193.stdout:2/489: dread d0/df/dbf/f9e [0,4194304] 0 2024-04-25T15:49:41.032 INFO:tasks.workunit.client.0.smithi110.stdout:7/777: mknod d1b/d29/d38/d55/dab/db2/db8/c101 0 2024-04-25T15:49:41.033 INFO:tasks.workunit.client.1.smithi193.stdout:2/490: write d0/d9/d15/f78 [374900,81832] 0 2024-04-25T15:49:41.034 INFO:tasks.workunit.client.1.smithi193.stdout:2/491: truncate d0/d9/d15/f79 4962440 0 2024-04-25T15:49:41.037 INFO:tasks.workunit.client.0.smithi110.stdout:6/686: mknod d1/db/d22/d33/d43/cf2 0 2024-04-25T15:49:41.038 INFO:tasks.workunit.client.0.smithi110.stdout:8/830: symlink d14/d32/d36/l120 0 2024-04-25T15:49:41.038 INFO:tasks.workunit.client.1.smithi193.stdout:3/532: mknod d8/d13/d4b/d8b/da2/ca7 0 2024-04-25T15:49:41.039 INFO:tasks.workunit.client.0.smithi110.stdout:6/687: write d1/dd/db5/fe3 [51734,18365] 0 2024-04-25T15:49:41.039 INFO:tasks.workunit.client.0.smithi110.stdout:0/692: dwrite d1/d47/d54/f71 [0,4194304] 0 2024-04-25T15:49:41.040 INFO:tasks.workunit.client.1.smithi193.stdout:7/472: dread d12/f36 [0,4194304] 0 2024-04-25T15:49:41.040 INFO:tasks.workunit.client.1.smithi193.stdout:8/481: creat d2/d3/d27/f96 x:0 0 0 2024-04-25T15:49:41.041 INFO:tasks.workunit.client.0.smithi110.stdout:9/698: mkdir d9/d13/d5b/d61/d67/dde 0 2024-04-25T15:49:41.041 INFO:tasks.workunit.client.0.smithi110.stdout:6/688: fsync d1/db/d22/d33/de4/fed 0 2024-04-25T15:49:41.042 INFO:tasks.workunit.client.1.smithi193.stdout:7/473: stat d12/d31/d37/d62/d6e/d79/d9d 0 2024-04-25T15:49:41.043 INFO:tasks.workunit.client.0.smithi110.stdout:7/778: creat d1b/d29/d38/d55/dab/db2/db8/f102 x:0 0 0 2024-04-25T15:49:41.044 INFO:tasks.workunit.client.0.smithi110.stdout:7/779: dread - d1b/d29/d87/fe7 zero size 2024-04-25T15:49:41.046 INFO:tasks.workunit.client.1.smithi193.stdout:5/493: mkdir d17/d18/d30/d38/dab 0 2024-04-25T15:49:41.049 INFO:tasks.workunit.client.0.smithi110.stdout:0/693: rmdir d1 39 2024-04-25T15:49:41.051 INFO:tasks.workunit.client.1.smithi193.stdout:0/495: fsync d0/fc 0 2024-04-25T15:49:41.052 INFO:tasks.workunit.client.0.smithi110.stdout:9/699: stat d9/d13/l18 0 2024-04-25T15:49:41.052 INFO:tasks.workunit.client.0.smithi110.stdout:7/780: dwrite d1b/d29/d38/d8b/fd7 [0,4194304] 0 2024-04-25T15:49:41.053 INFO:tasks.workunit.client.0.smithi110.stdout:9/700: write d9/d13/fca [815449,73862] 0 2024-04-25T15:49:41.054 INFO:tasks.workunit.client.1.smithi193.stdout:5/494: dwrite d17/d18/d30/d38/f7b [0,4194304] 0 2024-04-25T15:49:41.054 INFO:tasks.workunit.client.0.smithi110.stdout:7/781: fdatasync d1b/d29/d38/f54 0 2024-04-25T15:49:41.055 INFO:tasks.workunit.client.1.smithi193.stdout:5/495: fsync d17/d18/f24 0 2024-04-25T15:49:41.056 INFO:tasks.workunit.client.1.smithi193.stdout:6/489: unlink dd/d12/d19/c48 0 2024-04-25T15:49:41.057 INFO:tasks.workunit.client.0.smithi110.stdout:4/736: sync 2024-04-25T15:49:41.058 INFO:tasks.workunit.client.0.smithi110.stdout:5/745: sync 2024-04-25T15:49:41.058 INFO:tasks.workunit.client.1.smithi193.stdout:4/569: rename db/d13/d1e/d1f/c24 to db/cac 0 2024-04-25T15:49:41.059 INFO:tasks.workunit.client.0.smithi110.stdout:4/737: readlink d13/d24/d2d/dd9/dbb/dd2/ld1 0 2024-04-25T15:49:41.061 INFO:tasks.workunit.client.1.smithi193.stdout:2/492: creat d0/d42/d51/d4f/fc4 x:0 0 0 2024-04-25T15:49:41.069 INFO:tasks.workunit.client.1.smithi193.stdout:3/533: mknod d8/d30/d45/ca8 0 2024-04-25T15:49:41.069 INFO:tasks.workunit.client.1.smithi193.stdout:3/534: chown d8/d30 21552 1 2024-04-25T15:49:41.071 INFO:tasks.workunit.client.1.smithi193.stdout:8/482: write d2/d3/f68 [3839481,69575] 0 2024-04-25T15:49:41.073 INFO:tasks.workunit.client.0.smithi110.stdout:7/782: unlink d1b/d29/d46/f5c 0 2024-04-25T15:49:41.074 INFO:tasks.workunit.client.1.smithi193.stdout:1/426: link dc/c22 dc/d4a/d50/d5c/d5d/d3d/d55/d8a/c90 0 2024-04-25T15:49:41.075 INFO:tasks.workunit.client.0.smithi110.stdout:5/746: mknod d4/d1d/d85/cf3 0 2024-04-25T15:49:41.079 INFO:tasks.workunit.client.0.smithi110.stdout:4/738: read d13/d24/d2d/d44/d90/fc1 [1279809,964] 0 2024-04-25T15:49:41.080 INFO:tasks.workunit.client.0.smithi110.stdout:0/694: mknod d1/d23/d30/d33/d35/cf7 0 2024-04-25T15:49:41.081 INFO:tasks.workunit.client.0.smithi110.stdout:4/739: readlink d13/d24/d26/l3e 0 2024-04-25T15:49:41.081 INFO:tasks.workunit.client.0.smithi110.stdout:8/831: dwrite d14/f2d [0,4194304] 0 2024-04-25T15:49:41.083 INFO:tasks.workunit.client.1.smithi193.stdout:0/496: mknod d0/d49/d5/d7/dd/d46/cab 0 2024-04-25T15:49:41.083 INFO:tasks.workunit.client.0.smithi110.stdout:8/832: chown d14/d32/d36/db7/dc9/df9/d10e 720 1 2024-04-25T15:49:41.084 INFO:tasks.workunit.client.1.smithi193.stdout:0/497: chown d0/d49/d5/d7/dd/d16/f6f 1005 1 2024-04-25T15:49:41.084 INFO:tasks.workunit.client.1.smithi193.stdout:0/498: chown d0/d49/f2c 4079038 1 2024-04-25T15:49:41.088 INFO:tasks.workunit.client.1.smithi193.stdout:5/496: fsync d17/d18/d30/d38/d48/f8f 0 2024-04-25T15:49:41.089 INFO:tasks.workunit.client.0.smithi110.stdout:6/689: write d1/db/d22/d33/d43/f60 [1919375,64250] 0 2024-04-25T15:49:41.090 INFO:tasks.workunit.client.0.smithi110.stdout:9/701: dread d9/db/df/d25/f37 [0,4194304] 0 2024-04-25T15:49:41.092 INFO:tasks.workunit.client.0.smithi110.stdout:7/783: creat d1b/d29/d38/d55/dab/db2/f103 x:0 0 0 2024-04-25T15:49:41.093 INFO:tasks.workunit.client.1.smithi193.stdout:6/490: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f92 x:0 0 0 2024-04-25T15:49:41.093 INFO:tasks.workunit.client.1.smithi193.stdout:6/491: dread - dd/f91 zero size 2024-04-25T15:49:41.094 INFO:tasks.workunit.client.0.smithi110.stdout:0/695: fdatasync d1/d47/f8e 0 2024-04-25T15:49:41.095 INFO:tasks.workunit.client.0.smithi110.stdout:5/747: dread d4/d1d/d1f/d35/d3d/d52/f55 [0,4194304] 0 2024-04-25T15:49:41.095 INFO:tasks.workunit.client.1.smithi193.stdout:4/570: mkdir db/d13/d1e/d1f/d56/d69/dad 0 2024-04-25T15:49:41.097 INFO:tasks.workunit.client.1.smithi193.stdout:2/493: mkdir d0/d9/d15/d31/d90/d9f/dc5 0 2024-04-25T15:49:41.098 INFO:tasks.workunit.client.0.smithi110.stdout:8/833: readlink d14/d32/lbd 0 2024-04-25T15:49:41.100 INFO:tasks.workunit.client.1.smithi193.stdout:3/535: mknod d8/d13/d1f/d9f/ca9 0 2024-04-25T15:49:41.101 INFO:tasks.workunit.client.1.smithi193.stdout:0/499: dread d0/d49/d5/d7/dd/d16/f30 [0,4194304] 0 2024-04-25T15:49:41.103 INFO:tasks.workunit.client.1.smithi193.stdout:8/483: mknod d2/d3/d27/d69/d86/d72/c97 0 2024-04-25T15:49:41.106 INFO:tasks.workunit.client.1.smithi193.stdout:1/427: write dc/f2b [386795,73246] 0 2024-04-25T15:49:41.106 INFO:tasks.workunit.client.1.smithi193.stdout:9/493: sync 2024-04-25T15:49:41.106 INFO:tasks.workunit.client.0.smithi110.stdout:5/748: symlink d4/d1d/lf4 0 2024-04-25T15:49:41.107 INFO:tasks.workunit.client.1.smithi193.stdout:0/500: dwrite d0/d49/d5/d7/dd/d46/d48/f85 [0,4194304] 0 2024-04-25T15:49:41.107 INFO:tasks.workunit.client.0.smithi110.stdout:6/690: dread d1/db/d24/d2d/f7b [0,4194304] 0 2024-04-25T15:49:41.108 INFO:tasks.workunit.client.0.smithi110.stdout:5/749: dread - d4/d1d/d1f/d35/d3d/d52/dd2/fdd zero size 2024-04-25T15:49:41.109 INFO:tasks.workunit.client.0.smithi110.stdout:7/784: mknod d1b/d29/d38/d55/dab/db2/db8/c104 0 2024-04-25T15:49:41.113 INFO:tasks.workunit.client.0.smithi110.stdout:0/696: creat d1/d23/d30/d33/d5b/d78/ddb/ff8 x:0 0 0 2024-04-25T15:49:41.114 INFO:tasks.workunit.client.1.smithi193.stdout:5/497: chown d17/d18/d75/ca2 10748 1 2024-04-25T15:49:41.115 INFO:tasks.workunit.client.1.smithi193.stdout:5/498: write d17/d18/d30/d3b/fa1 [461228,31441] 0 2024-04-25T15:49:41.115 INFO:tasks.workunit.client.1.smithi193.stdout:5/499: chown d17/d18/d30/d38/l3c 7574 1 2024-04-25T15:49:41.116 INFO:tasks.workunit.client.1.smithi193.stdout:5/500: fsync d17/d18/d30/f8a 0 2024-04-25T15:49:41.116 INFO:tasks.workunit.client.0.smithi110.stdout:8/834: rmdir d14/d21/d26/d6a/d7e/da6/de8 39 2024-04-25T15:49:41.118 INFO:tasks.workunit.client.0.smithi110.stdout:6/691: symlink d1/db/d22/d90/d76/dad/lf3 0 2024-04-25T15:49:41.120 INFO:tasks.workunit.client.1.smithi193.stdout:7/474: sync 2024-04-25T15:49:41.120 INFO:tasks.workunit.client.0.smithi110.stdout:5/750: creat d4/d1d/d1f/d35/d7f/ff5 x:0 0 0 2024-04-25T15:49:41.121 INFO:tasks.workunit.client.1.smithi193.stdout:7/475: chown d12/d31/d37/d62/d6e/d79 636224 1 2024-04-25T15:49:41.121 INFO:tasks.workunit.client.0.smithi110.stdout:7/785: mkdir d1b/d29/d38/d55/d95/dcb/d105 0 2024-04-25T15:49:41.123 INFO:tasks.workunit.client.1.smithi193.stdout:2/494: dwrite d0/d42/d51/f40 [0,4194304] 0 2024-04-25T15:49:41.124 INFO:tasks.workunit.client.0.smithi110.stdout:0/697: rmdir d1/d23/d30/d33/d5b/d72/d77 39 2024-04-25T15:49:41.124 INFO:tasks.workunit.client.0.smithi110.stdout:0/698: fsync d1/d47/f95 0 2024-04-25T15:49:41.126 INFO:tasks.workunit.client.1.smithi193.stdout:9/494: mknod df/d2c/d2f/d4e/d69/d6b/c93 0 2024-04-25T15:49:41.129 INFO:tasks.workunit.client.0.smithi110.stdout:9/702: rename d9/db/f91 to d9/db/df/fdf 0 2024-04-25T15:49:41.133 INFO:tasks.workunit.client.0.smithi110.stdout:5/751: creat d4/d15/d51/d90/ded/ff6 x:0 0 0 2024-04-25T15:49:41.135 INFO:tasks.workunit.client.0.smithi110.stdout:7/786: unlink d1b/d29/d38/d55/dab/db2/db8/c101 0 2024-04-25T15:49:41.137 INFO:tasks.workunit.client.1.smithi193.stdout:4/571: rename db/d13/d1e/d1f/d22/c42 to db/d4b/d51/cae 0 2024-04-25T15:49:41.138 INFO:tasks.workunit.client.0.smithi110.stdout:0/699: fdatasync d1/d23/f24 0 2024-04-25T15:49:41.140 INFO:tasks.workunit.client.0.smithi110.stdout:0/700: write d1/d47/d54/f71 [5167393,27591] 0 2024-04-25T15:49:41.140 INFO:tasks.workunit.client.1.smithi193.stdout:7/476: unlink d12/d13/l2f 0 2024-04-25T15:49:41.141 INFO:tasks.workunit.client.0.smithi110.stdout:8/835: truncate d14/f80 969934 0 2024-04-25T15:49:41.141 INFO:tasks.workunit.client.0.smithi110.stdout:8/836: chown d14/d21/d26/l46 940263151 1 2024-04-25T15:49:41.143 INFO:tasks.workunit.client.0.smithi110.stdout:9/703: creat d9/db/df/d2c/d3c/fe0 x:0 0 0 2024-04-25T15:49:41.143 INFO:tasks.workunit.client.1.smithi193.stdout:4/572: dwrite db/d4b/f71 [0,4194304] 0 2024-04-25T15:49:41.145 INFO:tasks.workunit.client.1.smithi193.stdout:4/573: write db/d13/d1e/d1f/d22/f5a [1754248,91340] 0 2024-04-25T15:49:41.145 INFO:tasks.workunit.client.0.smithi110.stdout:5/752: mknod d4/d1d/d1f/d35/d3d/d52/dd2/dd4/cf7 0 2024-04-25T15:49:41.146 INFO:tasks.workunit.client.0.smithi110.stdout:5/753: write d4/d1d/d1f/d35/d7f/ff5 [565585,53175] 0 2024-04-25T15:49:41.150 INFO:tasks.workunit.client.1.smithi193.stdout:2/495: dread d0/d37/d25/fa9 [0,4194304] 0 2024-04-25T15:49:41.151 INFO:tasks.workunit.client.1.smithi193.stdout:9/495: creat df/d2c/d34/d3e/d4d/d5e/d81/d41/d5d/f94 x:0 0 0 2024-04-25T15:49:41.151 INFO:tasks.workunit.client.1.smithi193.stdout:2/496: readlink d0/d9/d3d/l4b 0 2024-04-25T15:49:41.153 INFO:tasks.workunit.client.1.smithi193.stdout:2/497: read d0/d37/d25/f77 [866936,119986] 0 2024-04-25T15:49:41.155 INFO:tasks.workunit.client.1.smithi193.stdout:6/492: link dd/d12/d19/d1c/d40/d3c/l8d dd/d12/d2d/l93 0 2024-04-25T15:49:41.157 INFO:tasks.workunit.client.0.smithi110.stdout:0/701: creat d1/d23/d9e/ff9 x:0 0 0 2024-04-25T15:49:41.157 INFO:tasks.workunit.client.0.smithi110.stdout:0/702: write d1/f16 [1049446,70872] 0 2024-04-25T15:49:41.159 INFO:tasks.workunit.client.1.smithi193.stdout:3/536: rename d8/l58 to d8/d13/d1f/laa 0 2024-04-25T15:49:41.161 INFO:tasks.workunit.client.0.smithi110.stdout:8/837: mknod d14/d32/d36/d48/c121 0 2024-04-25T15:49:41.161 INFO:tasks.workunit.client.1.smithi193.stdout:1/428: truncate dc/f2d 8159825 0 2024-04-25T15:49:41.162 INFO:tasks.workunit.client.1.smithi193.stdout:1/429: write dc/d59/f65 [651622,86268] 0 2024-04-25T15:49:41.163 INFO:tasks.workunit.client.1.smithi193.stdout:7/477: mknod d12/d1c/d63/d70/d68/d8f/cb1 0 2024-04-25T15:49:41.167 INFO:tasks.workunit.client.0.smithi110.stdout:0/703: creat d1/d23/d30/d33/d5b/dbb/da8/ffa x:0 0 0 2024-04-25T15:49:41.168 INFO:tasks.workunit.client.0.smithi110.stdout:8/838: mknod d14/d21/d23/c122 0 2024-04-25T15:49:41.171 INFO:tasks.workunit.client.1.smithi193.stdout:5/501: sync 2024-04-25T15:49:41.171 INFO:tasks.workunit.client.1.smithi193.stdout:0/501: creat d0/d49/d5/d7/dd/d16/d18/fac x:0 0 0 2024-04-25T15:49:41.173 INFO:tasks.workunit.client.0.smithi110.stdout:5/754: creat d4/d2c/d83/ff8 x:0 0 0 2024-04-25T15:49:41.174 INFO:tasks.workunit.client.0.smithi110.stdout:5/755: write d4/f8 [4713202,130222] 0 2024-04-25T15:49:41.181 INFO:tasks.workunit.client.1.smithi193.stdout:3/537: creat d8/d13/d4b/d8b/fab x:0 0 0 2024-04-25T15:49:41.182 INFO:tasks.workunit.client.1.smithi193.stdout:3/538: fsync d8/d30/d72/f76 0 2024-04-25T15:49:41.184 INFO:tasks.workunit.client.0.smithi110.stdout:5/756: fsync d4/d1d/d1f/f32 0 2024-04-25T15:49:41.186 INFO:tasks.workunit.client.1.smithi193.stdout:7/478: creat d12/d31/d37/d62/fb2 x:0 0 0 2024-04-25T15:49:41.188 INFO:tasks.workunit.client.1.smithi193.stdout:0/502: dread d0/d49/d5/d7/dd/d16/d18/d28/d56/f70 [0,4194304] 0 2024-04-25T15:49:41.189 INFO:tasks.workunit.client.0.smithi110.stdout:7/787: getdents d1b/d29/d38/d55/dab/db2/db8 0 2024-04-25T15:49:41.190 INFO:tasks.workunit.client.0.smithi110.stdout:9/704: truncate d9/db/df/d2c/d3c/f89 1212050 0 2024-04-25T15:49:41.192 INFO:tasks.workunit.client.1.smithi193.stdout:9/496: mkdir df/d2c/d95 0 2024-04-25T15:49:41.196 INFO:tasks.workunit.client.1.smithi193.stdout:5/502: mknod d17/d18/d75/d85/cac 0 2024-04-25T15:49:41.197 INFO:tasks.workunit.client.0.smithi110.stdout:8/839: mkdir d14/d32/d36/dde/d11e/d123 0 2024-04-25T15:49:41.198 INFO:tasks.workunit.client.0.smithi110.stdout:9/705: dwrite d9/db/d58/fc2 [0,4194304] 0 2024-04-25T15:49:41.199 INFO:tasks.workunit.client.0.smithi110.stdout:9/706: fdatasync d9/db/f1b 0 2024-04-25T15:49:41.201 INFO:tasks.workunit.client.0.smithi110.stdout:5/757: truncate d4/d1d/d1f/d35/f6a 1736940 0 2024-04-25T15:49:41.201 INFO:tasks.workunit.client.0.smithi110.stdout:9/707: write d9/d44/d5e/d80/dbb/fcd [4147,76336] 0 2024-04-25T15:49:41.203 INFO:tasks.workunit.client.1.smithi193.stdout:6/493: mknod dd/c94 0 2024-04-25T15:49:41.203 INFO:tasks.workunit.client.0.smithi110.stdout:9/708: chown d9/d13/fd2 1187558 1 2024-04-25T15:49:41.203 INFO:tasks.workunit.client.0.smithi110.stdout:9/709: stat d9/d44/d5e/d80/c99 0 2024-04-25T15:49:41.205 INFO:tasks.workunit.client.0.smithi110.stdout:7/788: rename d1b/dac to d1b/d29/d46/d106 0 2024-04-25T15:49:41.207 INFO:tasks.workunit.client.1.smithi193.stdout:8/484: rename d2/l2e to d2/d3/d27/d69/d86/d72/l98 0 2024-04-25T15:49:41.210 INFO:tasks.workunit.client.0.smithi110.stdout:8/840: truncate d14/f5c 2959521 0 2024-04-25T15:49:41.211 INFO:tasks.workunit.client.1.smithi193.stdout:7/479: mkdir d12/d31/d37/d62/db3 0 2024-04-25T15:49:41.212 INFO:tasks.workunit.client.1.smithi193.stdout:7/480: dread - d12/d1c/d45/f7a zero size 2024-04-25T15:49:41.214 INFO:tasks.workunit.client.1.smithi193.stdout:0/503: mkdir d0/d49/d6a/dad 0 2024-04-25T15:49:41.215 INFO:tasks.workunit.client.0.smithi110.stdout:7/789: fsync d1b/d29/d38/d8b/d89/f96 0 2024-04-25T15:49:41.216 INFO:tasks.workunit.client.1.smithi193.stdout:9/497: fsync df/d2c/d2f/d4e/d1c/d30/d60/f85 0 2024-04-25T15:49:41.220 INFO:tasks.workunit.client.1.smithi193.stdout:5/503: mkdir d17/d18/d75/d85/d5d/dad 0 2024-04-25T15:49:41.220 INFO:tasks.workunit.client.0.smithi110.stdout:0/704: write d1/d23/d30/d33/d5b/dbb/da8/ff0 [1428748,42614] 0 2024-04-25T15:49:41.220 INFO:tasks.workunit.client.0.smithi110.stdout:5/758: creat d4/d15/d51/d90/ded/def/ff9 x:0 0 0 2024-04-25T15:49:41.223 INFO:tasks.workunit.client.0.smithi110.stdout:9/710: creat d9/d2d/dc7/fe1 x:0 0 0 2024-04-25T15:49:41.224 INFO:tasks.workunit.client.0.smithi110.stdout:9/711: chown d9/db/df/d2c/d3c/d56/da0 7019121 1 2024-04-25T15:49:41.225 INFO:tasks.workunit.client.0.smithi110.stdout:0/705: dread d1/d23/d30/d33/d35/da1/db4/db9/fd3 [0,4194304] 0 2024-04-25T15:49:41.226 INFO:tasks.workunit.client.0.smithi110.stdout:0/706: chown d1/c8f 118503539 1 2024-04-25T15:49:41.230 INFO:tasks.workunit.client.0.smithi110.stdout:0/707: dread d1/f38 [0,4194304] 0 2024-04-25T15:49:41.230 INFO:tasks.workunit.client.1.smithi193.stdout:0/504: dread d0/d49/d5/d7/dd/d16/d26/d2e/f4b [0,4194304] 0 2024-04-25T15:49:41.233 INFO:tasks.workunit.client.0.smithi110.stdout:5/759: fsync d4/d1d/f34 0 2024-04-25T15:49:41.237 INFO:tasks.workunit.client.0.smithi110.stdout:7/790: dread d1b/d29/d38/d55/d7f/f72 [0,4194304] 0 2024-04-25T15:49:41.238 INFO:tasks.workunit.client.0.smithi110.stdout:9/712: mkdir d9/db/df/d4d/de2 0 2024-04-25T15:49:41.239 INFO:tasks.workunit.client.1.smithi193.stdout:1/430: creat dc/d4a/d50/d5c/f91 x:0 0 0 2024-04-25T15:49:41.240 INFO:tasks.workunit.client.1.smithi193.stdout:4/574: getdents db/d13/d1e/d1f/d56/d69/d70/d8a 0 2024-04-25T15:49:41.244 INFO:tasks.workunit.client.1.smithi193.stdout:7/481: creat d12/d1c/d63/d70/d68/fb4 x:0 0 0 2024-04-25T15:49:41.245 INFO:tasks.workunit.client.1.smithi193.stdout:6/494: dread dd/d12/d19/d1c/d40/f23 [0,4194304] 0 2024-04-25T15:49:41.247 INFO:tasks.workunit.client.0.smithi110.stdout:0/708: dread d1/d23/d30/d33/d5b/f98 [0,4194304] 0 2024-04-25T15:49:41.248 INFO:tasks.workunit.client.0.smithi110.stdout:0/709: stat d1/d23/d30/d33/d35 0 2024-04-25T15:49:41.251 INFO:tasks.workunit.client.1.smithi193.stdout:3/539: truncate d8/d13/f32 2924540 0 2024-04-25T15:49:41.253 INFO:tasks.workunit.client.0.smithi110.stdout:8/841: dwrite d14/fb2 [0,4194304] 0 2024-04-25T15:49:41.254 INFO:tasks.workunit.client.1.smithi193.stdout:2/498: getdents d0 0 2024-04-25T15:49:41.254 INFO:tasks.workunit.client.1.smithi193.stdout:8/485: dwrite d2/d3/d3a/f61 [0,4194304] 0 2024-04-25T15:49:41.255 INFO:tasks.workunit.client.0.smithi110.stdout:7/791: dread - d1b/d29/d38/d55/d95/fa9 zero size 2024-04-25T15:49:41.255 INFO:tasks.workunit.client.1.smithi193.stdout:4/575: dread db/d13/d1e/f48 [0,4194304] 0 2024-04-25T15:49:41.256 INFO:tasks.workunit.client.0.smithi110.stdout:9/713: readlink d9/l28 0 2024-04-25T15:49:41.260 INFO:tasks.workunit.client.1.smithi193.stdout:0/505: dread d0/d49/d5/d7/dd/d38/f4c [0,4194304] 0 2024-04-25T15:49:41.262 INFO:tasks.workunit.client.1.smithi193.stdout:9/498: write df/d2c/d2f/d4e/f37 [1247589,61870] 0 2024-04-25T15:49:41.263 INFO:tasks.workunit.client.0.smithi110.stdout:0/710: rename d1/f8b to d1/d20/d69/ffb 0 2024-04-25T15:49:41.265 INFO:tasks.workunit.client.1.smithi193.stdout:5/504: write d17/d18/d30/d34/f6c [615879,100192] 0 2024-04-25T15:49:41.269 INFO:tasks.workunit.client.1.smithi193.stdout:6/495: rename dd/d12/d2d/l5f to dd/d12/d19/d45/l95 0 2024-04-25T15:49:41.269 INFO:tasks.workunit.client.0.smithi110.stdout:7/792: dread d1b/d29/d38/d55/d95/ff4 [0,4194304] 0 2024-04-25T15:49:41.270 INFO:tasks.workunit.client.1.smithi193.stdout:8/486: dwrite d2/d3/d20/d45/d29/f60 [0,4194304] 0 2024-04-25T15:49:41.270 INFO:tasks.workunit.client.0.smithi110.stdout:9/714: truncate d9/d44/fab 3780198 0 2024-04-25T15:49:41.271 INFO:tasks.workunit.client.1.smithi193.stdout:6/496: dread - dd/f59 zero size 2024-04-25T15:49:41.272 INFO:tasks.workunit.client.0.smithi110.stdout:0/711: truncate d1/d23/d30/d33/d35/f36 653283 0 2024-04-25T15:49:41.272 INFO:tasks.workunit.client.1.smithi193.stdout:5/505: write d17/d18/d30/d38/d48/d4f/f98 [1513959,50683] 0 2024-04-25T15:49:41.273 INFO:tasks.workunit.client.0.smithi110.stdout:8/842: mknod d14/d32/d36/dde/d11e/d123/c124 0 2024-04-25T15:49:41.273 INFO:tasks.workunit.client.0.smithi110.stdout:5/760: dread d4/d1d/d1f/d35/d3d/f58 [0,4194304] 0 2024-04-25T15:49:41.274 INFO:tasks.workunit.client.1.smithi193.stdout:0/506: truncate d0/d49/d5/d7/dd/d16/d18/f7c 376754 0 2024-04-25T15:49:41.275 INFO:tasks.workunit.client.1.smithi193.stdout:1/431: dwrite dc/f4b [0,4194304] 0 2024-04-25T15:49:41.275 INFO:tasks.workunit.client.1.smithi193.stdout:8/487: readlink d2/d3/d20/d45/l70 0 2024-04-25T15:49:41.275 INFO:tasks.workunit.client.1.smithi193.stdout:3/540: dread - d8/d4d/f67 zero size 2024-04-25T15:49:41.276 INFO:tasks.workunit.client.0.smithi110.stdout:7/793: fdatasync d1b/d29/d38/d55/fae 0 2024-04-25T15:49:41.276 INFO:tasks.workunit.client.1.smithi193.stdout:1/432: stat l3 0 2024-04-25T15:49:41.278 INFO:tasks.workunit.client.0.smithi110.stdout:0/712: readlink d1/lb 0 2024-04-25T15:49:41.278 INFO:tasks.workunit.client.1.smithi193.stdout:8/488: fsync d2/d3/d3a/f76 0 2024-04-25T15:49:41.278 INFO:tasks.workunit.client.1.smithi193.stdout:3/541: chown d8/d13/d14/d27 21945 1 2024-04-25T15:49:41.280 INFO:tasks.workunit.client.1.smithi193.stdout:7/482: creat d12/d31/d39/db0/fb5 x:0 0 0 2024-04-25T15:49:41.281 INFO:tasks.workunit.client.1.smithi193.stdout:3/542: truncate d8/d30/f47 4568188 0 2024-04-25T15:49:41.284 INFO:tasks.workunit.client.1.smithi193.stdout:9/499: symlink df/d2c/d34/d3e/d4d/d5e/d81/d41/l96 0 2024-04-25T15:49:41.284 INFO:tasks.workunit.client.1.smithi193.stdout:7/483: write d12/d31/d37/d62/f9b [673975,1511] 0 2024-04-25T15:49:41.287 INFO:tasks.workunit.client.0.smithi110.stdout:5/761: mkdir d4/d1d/d1f/d35/d3d/d52/d75/dfa 0 2024-04-25T15:49:41.287 INFO:tasks.workunit.client.1.smithi193.stdout:5/506: chown d17/c28 2 1 2024-04-25T15:49:41.288 INFO:tasks.workunit.client.0.smithi110.stdout:0/713: readlink d1/l4f 0 2024-04-25T15:49:41.288 INFO:tasks.workunit.client.1.smithi193.stdout:6/497: dwrite dd/d2f/f56 [0,4194304] 0 2024-04-25T15:49:41.303 INFO:tasks.workunit.client.1.smithi193.stdout:9/500: dread df/d2c/d2f/d4e/f37 [0,4194304] 0 2024-04-25T15:49:41.304 INFO:tasks.workunit.client.1.smithi193.stdout:9/501: write df/d2c/d2f/d4e/f3a [4300831,65126] 0 2024-04-25T15:49:41.305 INFO:tasks.workunit.client.1.smithi193.stdout:9/502: write df/d2c/d34/d3e/f7d [307153,112560] 0 2024-04-25T15:49:41.334 INFO:tasks.workunit.client.0.smithi110.stdout:2/693: sync 2024-04-25T15:49:41.334 INFO:tasks.workunit.client.0.smithi110.stdout:1/912: sync 2024-04-25T15:49:41.336 INFO:tasks.workunit.client.1.smithi193.stdout:4/576: link db/d4b/fa3 db/d32/d88/da8/faf 0 2024-04-25T15:49:41.337 INFO:tasks.workunit.client.1.smithi193.stdout:4/577: write db/f8d [3807205,113019] 0 2024-04-25T15:49:41.337 INFO:tasks.workunit.client.0.smithi110.stdout:1/913: chown d0/d14/d5f/d7d/d72/le8 54409055 1 2024-04-25T15:49:41.338 INFO:tasks.workunit.client.1.smithi193.stdout:4/578: chown db/d13/c7e 0 1 2024-04-25T15:49:41.338 INFO:tasks.workunit.client.1.smithi193.stdout:2/499: sync 2024-04-25T15:49:41.340 INFO:tasks.workunit.client.1.smithi193.stdout:2/500: read d0/d9/d15/d31/d50/f6b [1584967,5850] 0 2024-04-25T15:49:41.342 INFO:tasks.workunit.client.1.smithi193.stdout:0/507: rmdir d0/d49/d5/d7/dd/d16 39 2024-04-25T15:49:41.344 INFO:tasks.workunit.client.1.smithi193.stdout:3/543: symlink d8/d13/d14/d23/lac 0 2024-04-25T15:49:41.344 INFO:tasks.workunit.client.0.smithi110.stdout:1/914: dwrite d0/d14/d5f/d7d/f67 [0,4194304] 0 2024-04-25T15:49:41.349 INFO:tasks.workunit.client.1.smithi193.stdout:3/544: dwrite d8/d13/f56 [0,4194304] 0 2024-04-25T15:49:41.352 INFO:tasks.workunit.client.1.smithi193.stdout:7/484: dread d12/d31/d39/d60/f64 [0,4194304] 0 2024-04-25T15:49:41.356 INFO:tasks.workunit.client.1.smithi193.stdout:6/498: unlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/c63 0 2024-04-25T15:49:41.357 INFO:tasks.workunit.client.1.smithi193.stdout:6/499: fsync dd/d12/d19/f43 0 2024-04-25T15:49:41.359 INFO:tasks.workunit.client.1.smithi193.stdout:9/503: truncate df/d2c/d34/d3e/d4d/d5e/d81/f4c 246627 0 2024-04-25T15:49:41.361 INFO:tasks.workunit.client.0.smithi110.stdout:9/715: dwrite d9/db/df/f21 [0,4194304] 0 2024-04-25T15:49:41.363 INFO:tasks.workunit.client.1.smithi193.stdout:4/579: creat db/d13/d1e/d1f/fb0 x:0 0 0 2024-04-25T15:49:41.365 INFO:tasks.workunit.client.1.smithi193.stdout:4/580: write db/d13/d1e/f41 [2024104,12792] 0 2024-04-25T15:49:41.367 INFO:tasks.workunit.client.1.smithi193.stdout:1/433: write dc/d4a/f54 [1776749,82175] 0 2024-04-25T15:49:41.368 INFO:tasks.workunit.client.1.smithi193.stdout:2/501: mknod d0/d42/d51/d4f/d65/cc6 0 2024-04-25T15:49:41.373 INFO:tasks.workunit.client.1.smithi193.stdout:0/508: dread - d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/f95 zero size 2024-04-25T15:49:41.375 INFO:tasks.workunit.client.1.smithi193.stdout:1/434: dwrite dc/d4a/d50/f53 [0,4194304] 0 2024-04-25T15:49:41.382 INFO:tasks.workunit.client.1.smithi193.stdout:3/545: truncate d8/d13/f2d 1107219 0 2024-04-25T15:49:41.386 INFO:tasks.workunit.client.1.smithi193.stdout:7/485: creat d12/d31/d37/d62/d6e/da5/fb6 x:0 0 0 2024-04-25T15:49:41.390 INFO:tasks.workunit.client.1.smithi193.stdout:6/500: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d96 0 2024-04-25T15:49:41.391 INFO:tasks.workunit.client.1.smithi193.stdout:6/501: fdatasync dd/d12/d19/d1c/d40/d3c/d3f/d4f/f8a 0 2024-04-25T15:49:41.394 INFO:tasks.workunit.client.1.smithi193.stdout:4/581: symlink db/d32/d40/lb1 0 2024-04-25T15:49:41.399 INFO:tasks.workunit.client.1.smithi193.stdout:2/502: symlink d0/d9/d15/d31/d90/lc7 0 2024-04-25T15:49:41.401 INFO:tasks.workunit.client.1.smithi193.stdout:0/509: creat d0/d49/d5/d61/fae x:0 0 0 2024-04-25T15:49:41.402 INFO:tasks.workunit.client.1.smithi193.stdout:0/510: write d0/d49/d5/d7/dd/d16/d26/d5e/f80 [2409090,103793] 0 2024-04-25T15:49:41.403 INFO:tasks.workunit.client.1.smithi193.stdout:0/511: chown d0/f35 210239 1 2024-04-25T15:49:41.404 INFO:tasks.workunit.client.1.smithi193.stdout:8/489: truncate d2/d3/d20/d45/f5a 281903 0 2024-04-25T15:49:41.405 INFO:tasks.workunit.client.1.smithi193.stdout:8/490: fdatasync d2/d3/d27/f96 0 2024-04-25T15:49:41.409 INFO:tasks.workunit.client.0.smithi110.stdout:7/794: mknod d1b/c107 0 2024-04-25T15:49:41.410 INFO:tasks.workunit.client.0.smithi110.stdout:7/795: dread - d1b/d29/d38/d55/d95/f9c zero size 2024-04-25T15:49:41.410 INFO:tasks.workunit.client.1.smithi193.stdout:7/486: mkdir d12/d1c/d63/db7 0 2024-04-25T15:49:41.411 INFO:tasks.workunit.client.0.smithi110.stdout:8/843: rename d14/d21/d26/d6a/d7e/da6/de8/d55/c6c to d14/d32/d36/d48/d8f/c125 0 2024-04-25T15:49:41.413 INFO:tasks.workunit.client.0.smithi110.stdout:5/762: fdatasync d4/f6d 0 2024-04-25T15:49:41.415 INFO:tasks.workunit.client.0.smithi110.stdout:0/714: creat d1/d23/d30/d33/d5b/d78/ddb/ffc x:0 0 0 2024-04-25T15:49:41.415 INFO:tasks.workunit.client.1.smithi193.stdout:9/504: write df/f38 [112771,6216] 0 2024-04-25T15:49:41.415 INFO:tasks.workunit.client.1.smithi193.stdout:4/582: rmdir db/d13/d1e 39 2024-04-25T15:49:41.420 INFO:tasks.workunit.client.1.smithi193.stdout:0/512: symlink d0/d49/d5/d7/dd/d46/d48/laf 0 2024-04-25T15:49:41.421 INFO:tasks.workunit.client.0.smithi110.stdout:9/716: mknod d9/d44/d5e/db5/ce3 0 2024-04-25T15:49:41.423 INFO:tasks.workunit.client.1.smithi193.stdout:3/546: mknod d8/d13/d4b/d8b/d9a/cad 0 2024-04-25T15:49:41.425 INFO:tasks.workunit.client.1.smithi193.stdout:5/507: rename d17/d18/d3e/c6a to d17/d18/d3e/cae 0 2024-04-25T15:49:41.426 INFO:tasks.workunit.client.0.smithi110.stdout:8/844: symlink d14/d21/d23/l126 0 2024-04-25T15:49:41.428 INFO:tasks.workunit.client.0.smithi110.stdout:8/845: read - d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/ddd/f100 zero size 2024-04-25T15:49:41.429 INFO:tasks.workunit.client.1.smithi193.stdout:6/502: mknod dd/d2f/d57/c97 0 2024-04-25T15:49:41.429 INFO:tasks.workunit.client.0.smithi110.stdout:7/796: dwrite d1b/f88 [8388608,4194304] 0 2024-04-25T15:49:41.429 INFO:tasks.workunit.client.0.smithi110.stdout:8/846: chown d14/fb2 295336 1 2024-04-25T15:49:41.431 INFO:tasks.workunit.client.0.smithi110.stdout:5/763: fsync d4/d15/d51/fa4 0 2024-04-25T15:49:41.432 INFO:tasks.workunit.client.0.smithi110.stdout:5/764: chown d4/d2c/d2a/f46 1886031 1 2024-04-25T15:49:41.436 INFO:tasks.workunit.client.0.smithi110.stdout:2/694: mkdir de/d40/d6e/d52/dda 0 2024-04-25T15:49:41.439 INFO:tasks.workunit.client.0.smithi110.stdout:0/715: dread d1/d20/f96 [0,4194304] 0 2024-04-25T15:49:41.439 INFO:tasks.workunit.client.0.smithi110.stdout:7/797: dwrite d1b/d29/d38/d55/d95/dcb/fe4 [0,4194304] 0 2024-04-25T15:49:41.441 INFO:tasks.workunit.client.1.smithi193.stdout:2/503: symlink d0/d9/d15/d31/lc8 0 2024-04-25T15:49:41.442 INFO:tasks.workunit.client.0.smithi110.stdout:7/798: chown d1b/d29/d38/d55/d7f/lf7 7 1 2024-04-25T15:49:41.444 INFO:tasks.workunit.client.0.smithi110.stdout:3/758: sync 2024-04-25T15:49:41.445 INFO:tasks.workunit.client.1.smithi193.stdout:1/435: mkdir dc/d4a/d4d/d7a/d85/d92 0 2024-04-25T15:49:41.450 INFO:tasks.workunit.client.0.smithi110.stdout:2/695: dread de/d40/f3e [0,4194304] 0 2024-04-25T15:49:41.451 INFO:tasks.workunit.client.0.smithi110.stdout:5/765: fsync d4/d1d/d1f/d35/d7f/fac 0 2024-04-25T15:49:41.454 INFO:tasks.workunit.client.0.smithi110.stdout:1/915: creat d0/d14/d81/df8/f126 x:0 0 0 2024-04-25T15:49:41.455 INFO:tasks.workunit.client.1.smithi193.stdout:5/508: read f16 [3910326,93308] 0 2024-04-25T15:49:41.456 INFO:tasks.workunit.client.1.smithi193.stdout:6/503: mknod dd/d12/d19/d45/c98 0 2024-04-25T15:49:41.457 INFO:tasks.workunit.client.0.smithi110.stdout:7/799: unlink d1b/d29/f98 0 2024-04-25T15:49:41.458 INFO:tasks.workunit.client.0.smithi110.stdout:7/800: chown d1b/f79 287267 1 2024-04-25T15:49:41.459 INFO:tasks.workunit.client.0.smithi110.stdout:9/717: mkdir d9/d13/d5b/d61/db3/db4/dc5/de4 0 2024-04-25T15:49:41.460 INFO:tasks.workunit.client.0.smithi110.stdout:7/801: chown d1b/d29/d38/d55/dab/db2/db8/cd2 27739518 1 2024-04-25T15:49:41.460 INFO:tasks.workunit.client.0.smithi110.stdout:3/759: rmdir d0/d16/dcb/d2f/d4c/db2/df2 39 2024-04-25T15:49:41.460 INFO:tasks.workunit.client.1.smithi193.stdout:9/505: unlink df/d11/c56 0 2024-04-25T15:49:41.461 INFO:tasks.workunit.client.1.smithi193.stdout:6/504: dwrite dd/d12/d19/f43 [8388608,4194304] 0 2024-04-25T15:49:41.465 INFO:tasks.workunit.client.0.smithi110.stdout:2/696: creat de/d40/d5b/d5c/d5f/db2/dad/fdb x:0 0 0 2024-04-25T15:49:41.466 INFO:tasks.workunit.client.1.smithi193.stdout:1/436: creat dc/d4a/d50/d5c/d5d/d3d/d55/f93 x:0 0 0 2024-04-25T15:49:41.467 INFO:tasks.workunit.client.0.smithi110.stdout:0/716: mkdir d1/d23/d81/df4/dfd 0 2024-04-25T15:49:41.470 INFO:tasks.workunit.client.0.smithi110.stdout:1/916: rename d0/d14/f4b to d0/d51/dc8/f127 0 2024-04-25T15:49:41.474 INFO:tasks.workunit.client.0.smithi110.stdout:9/718: unlink d9/db/c12 0 2024-04-25T15:49:41.476 INFO:tasks.workunit.client.1.smithi193.stdout:7/487: rename d12/d1c/d63/d70/d68/d8d/f89 to d12/fb8 0 2024-04-25T15:49:41.477 INFO:tasks.workunit.client.1.smithi193.stdout:7/488: dread - d12/d1c/faf zero size 2024-04-25T15:49:41.480 INFO:tasks.workunit.client.0.smithi110.stdout:3/760: creat d0/d2e/d88/dbf/da5/f106 x:0 0 0 2024-04-25T15:49:41.482 INFO:tasks.workunit.client.1.smithi193.stdout:9/506: creat df/d2c/d2f/d4e/d1c/f97 x:0 0 0 2024-04-25T15:49:41.483 INFO:tasks.workunit.client.0.smithi110.stdout:8/847: write d14/f116 [7234701,74353] 0 2024-04-25T15:49:41.483 INFO:tasks.workunit.client.0.smithi110.stdout:0/717: readlink d1/d20/l5e 0 2024-04-25T15:49:41.484 INFO:tasks.workunit.client.1.smithi193.stdout:2/504: write d0/d37/f43 [490539,115065] 0 2024-04-25T15:49:41.485 INFO:tasks.workunit.client.1.smithi193.stdout:6/505: creat dd/d12/d19/d1c/f99 x:0 0 0 2024-04-25T15:49:41.487 INFO:tasks.workunit.client.0.smithi110.stdout:2/697: rename de/d40/d4f/f90 to de/d6f/fdc 0 2024-04-25T15:49:41.487 INFO:tasks.workunit.client.1.smithi193.stdout:4/583: dwrite db/d13/d1e/d1f/fa5 [0,4194304] 0 2024-04-25T15:49:41.488 INFO:tasks.workunit.client.1.smithi193.stdout:4/584: chown db/d13/d1e/d1f/d56/d57/fa4 4420690 1 2024-04-25T15:49:41.492 INFO:tasks.workunit.client.0.smithi110.stdout:7/802: getdents d1b/d29/db5/dca/df9 0 2024-04-25T15:49:41.493 INFO:tasks.workunit.client.1.smithi193.stdout:8/491: getdents d2/d3/d27 0 2024-04-25T15:49:41.495 INFO:tasks.workunit.client.1.smithi193.stdout:0/513: link d0/d49/d5/d7/dd/d46/d48/d62/f8f d0/d49/d5/d7/dd/d16/d26/fb0 0 2024-04-25T15:49:41.496 INFO:tasks.workunit.client.1.smithi193.stdout:5/509: dwrite d17/d18/d30/f35 [0,4194304] 0 2024-04-25T15:49:41.498 INFO:tasks.workunit.client.1.smithi193.stdout:0/514: dread d0/d49/d5/d7/dd/d46/f6b [0,4194304] 0 2024-04-25T15:49:41.499 INFO:tasks.workunit.client.1.smithi193.stdout:7/489: symlink d12/d31/d37/d62/lb9 0 2024-04-25T15:49:41.500 INFO:tasks.workunit.client.0.smithi110.stdout:9/719: symlink d9/d13/d7c/le5 0 2024-04-25T15:49:41.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/507: rename df/f38 to df/d2c/d34/d3e/d4d/d5e/d81/d41/d5d/f98 0 2024-04-25T15:49:41.503 INFO:tasks.workunit.client.1.smithi193.stdout:5/510: dread d17/d18/d30/d38/d48/f8f [0,4194304] 0 2024-04-25T15:49:41.504 INFO:tasks.workunit.client.1.smithi193.stdout:2/505: symlink d0/d44/lc9 0 2024-04-25T15:49:41.504 INFO:tasks.workunit.client.0.smithi110.stdout:3/761: symlink d0/d7/dd0/ddf/l107 0 2024-04-25T15:49:41.506 INFO:tasks.workunit.client.1.smithi193.stdout:2/506: read d0/d37/d25/fa9 [391909,60094] 0 2024-04-25T15:49:41.506 INFO:tasks.workunit.client.1.smithi193.stdout:3/547: dwrite d8/d13/f2d [0,4194304] 0 2024-04-25T15:49:41.508 INFO:tasks.workunit.client.1.smithi193.stdout:1/437: truncate dc/f47 625020 0 2024-04-25T15:49:41.508 INFO:tasks.workunit.client.1.smithi193.stdout:1/438: fdatasync dc/d4a/d4d/f5e 0 2024-04-25T15:49:41.512 INFO:tasks.workunit.client.0.smithi110.stdout:9/720: mknod d9/d44/d5e/d80/ce6 0 2024-04-25T15:49:41.514 INFO:tasks.workunit.client.0.smithi110.stdout:9/721: read d9/db/df/d2c/f70 [1028982,60490] 0 2024-04-25T15:49:41.515 INFO:tasks.workunit.client.0.smithi110.stdout:3/762: write d0/d16/dcb/d2f/d33/f44 [3738856,6119] 0 2024-04-25T15:49:41.516 INFO:tasks.workunit.client.1.smithi193.stdout:7/490: mknod d12/d1c/d63/d70/cba 0 2024-04-25T15:49:41.517 INFO:tasks.workunit.client.1.smithi193.stdout:7/491: write d12/d31/d39/f53 [471332,54194] 0 2024-04-25T15:49:41.518 INFO:tasks.workunit.client.1.smithi193.stdout:7/492: chown d12/d1c/d63/f65 17587461 1 2024-04-25T15:49:41.519 INFO:tasks.workunit.client.1.smithi193.stdout:9/508: readlink l0 0 2024-04-25T15:49:41.521 INFO:tasks.workunit.client.0.smithi110.stdout:2/698: creat de/d40/d5b/d5c/d5f/db2/fdd x:0 0 0 2024-04-25T15:49:41.522 INFO:tasks.workunit.client.0.smithi110.stdout:1/917: getdents d0/d14/d81/df8/d33/d94/ddb 0 2024-04-25T15:49:41.524 INFO:tasks.workunit.client.0.smithi110.stdout:7/803: write d1b/d29/d87/fa2 [896172,28438] 0 2024-04-25T15:49:41.524 INFO:tasks.workunit.client.1.smithi193.stdout:0/515: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/lb1 0 2024-04-25T15:49:41.529 INFO:tasks.workunit.client.1.smithi193.stdout:2/507: dread d0/df/f10 [4194304,4194304] 0 2024-04-25T15:49:41.531 INFO:tasks.workunit.client.1.smithi193.stdout:0/516: dwrite d0/d49/d5/d7/dd/d46/d48/f57 [0,4194304] 0 2024-04-25T15:49:41.531 INFO:tasks.workunit.client.1.smithi193.stdout:7/493: dread d12/d31/d37/d62/d6e/d79/f3f [0,4194304] 0 2024-04-25T15:49:41.532 INFO:tasks.workunit.client.1.smithi193.stdout:3/548: dread f2 [0,4194304] 0 2024-04-25T15:49:41.533 INFO:tasks.workunit.client.1.smithi193.stdout:3/549: read - d8/d13/d14/d27/f96 zero size 2024-04-25T15:49:41.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:41 smithi110 ceph-mon[26344]: pgmap v37: 65 pgs: 65 active+clean; 2.6 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 70 MiB/s rd, 236 MiB/s wr, 433 op/s 2024-04-25T15:49:41.674 INFO:tasks.workunit.client.0.smithi110.stdout:0/718: write d1/d23/fc2 [289297,90706] 0 2024-04-25T15:49:41.674 INFO:tasks.workunit.client.1.smithi193.stdout:6/506: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/c9a 0 2024-04-25T15:49:41.676 INFO:tasks.workunit.client.0.smithi110.stdout:8/848: getdents d14/d21/d26/d6a/d7e 0 2024-04-25T15:49:41.678 INFO:tasks.workunit.client.0.smithi110.stdout:2/699: mkdir de/d40/dde 0 2024-04-25T15:49:41.679 INFO:tasks.workunit.client.0.smithi110.stdout:1/918: mknod d0/d23/c128 0 2024-04-25T15:49:41.682 INFO:tasks.workunit.client.1.smithi193.stdout:5/511: write f16 [2429084,65292] 0 2024-04-25T15:49:41.682 INFO:tasks.workunit.client.0.smithi110.stdout:4/740: sync 2024-04-25T15:49:41.682 INFO:tasks.workunit.client.0.smithi110.stdout:6/692: sync 2024-04-25T15:49:41.683 INFO:tasks.workunit.client.1.smithi193.stdout:8/492: getdents d2/d3/d3a/d95 0 2024-04-25T15:49:41.683 INFO:tasks.workunit.client.1.smithi193.stdout:8/493: chown d2/d3/d27/d6f/d94/l83 66 1 2024-04-25T15:49:41.684 INFO:tasks.workunit.client.1.smithi193.stdout:8/494: fdatasync d2/d3/d27/d69/d86/f84 0 2024-04-25T15:49:41.687 INFO:tasks.workunit.client.1.smithi193.stdout:5/512: dwrite d17/d18/d75/d85/f73 [0,4194304] 0 2024-04-25T15:49:41.688 INFO:tasks.workunit.client.0.smithi110.stdout:3/763: dwrite d0/d2e/d88/dbf/d92/f89 [0,4194304] 0 2024-04-25T15:49:41.691 INFO:tasks.workunit.client.1.smithi193.stdout:2/508: creat d0/d42/d51/d4f/d65/d7e/fca x:0 0 0 2024-04-25T15:49:41.692 INFO:tasks.workunit.client.0.smithi110.stdout:1/919: symlink d0/d14/d81/l129 0 2024-04-25T15:49:41.692 INFO:tasks.workunit.client.0.smithi110.stdout:4/741: dwrite d13/d24/d2d/d5e/fa9 [0,4194304] 0 2024-04-25T15:49:41.693 INFO:tasks.workunit.client.1.smithi193.stdout:7/494: read d12/d1c/d63/d70/d68/d8d/f9c [3514,57532] 0 2024-04-25T15:49:41.693 INFO:tasks.workunit.client.1.smithi193.stdout:3/550: mknod d8/d13/d14/d23/cae 0 2024-04-25T15:49:41.696 INFO:tasks.workunit.client.0.smithi110.stdout:9/722: rmdir d9/d97 0 2024-04-25T15:49:41.696 INFO:tasks.workunit.client.1.smithi193.stdout:3/551: write d8/d13/f66 [442978,90126] 0 2024-04-25T15:49:41.701 INFO:tasks.workunit.client.0.smithi110.stdout:8/849: dwrite d14/d21/d26/f29 [0,4194304] 0 2024-04-25T15:49:41.702 INFO:tasks.workunit.client.0.smithi110.stdout:8/850: fsync d14/fb2 0 2024-04-25T15:49:41.704 INFO:tasks.workunit.client.0.smithi110.stdout:7/804: truncate d1b/d29/d46/fe5 92515 0 2024-04-25T15:49:41.708 INFO:tasks.workunit.client.0.smithi110.stdout:0/719: mkdir d1/d23/d30/d33/dfe 0 2024-04-25T15:49:41.709 INFO:tasks.workunit.client.1.smithi193.stdout:4/585: creat db/d4b/fb2 x:0 0 0 2024-04-25T15:49:41.712 INFO:tasks.workunit.client.1.smithi193.stdout:1/439: truncate dc/f2d 5515292 0 2024-04-25T15:49:41.714 INFO:tasks.workunit.client.0.smithi110.stdout:1/920: read d0/d14/d5f/d7d/f64 [11772,110551] 0 2024-04-25T15:49:41.715 INFO:tasks.workunit.client.0.smithi110.stdout:2/700: write de/d40/d5b/d5c/d5f/db2/fa0 [236888,96083] 0 2024-04-25T15:49:41.716 INFO:tasks.workunit.client.1.smithi193.stdout:8/495: mknod d2/d18/c99 0 2024-04-25T15:49:41.717 INFO:tasks.workunit.client.0.smithi110.stdout:9/723: mkdir d9/d44/d5e/db5/de7 0 2024-04-25T15:49:41.718 INFO:tasks.workunit.client.0.smithi110.stdout:0/720: dwrite d1/d23/d30/d33/d5b/dbb/da8/fe6 [0,4194304] 0 2024-04-25T15:49:41.719 INFO:tasks.workunit.client.0.smithi110.stdout:0/721: chown d1 12 1 2024-04-25T15:49:41.720 INFO:tasks.workunit.client.1.smithi193.stdout:9/509: link df/d11/d8b/f52 df/d2d/f99 0 2024-04-25T15:49:41.723 INFO:tasks.workunit.client.1.smithi193.stdout:5/513: unlink d17/f1d 0 2024-04-25T15:49:41.724 INFO:tasks.workunit.client.1.smithi193.stdout:5/514: dread - d17/f89 zero size 2024-04-25T15:49:41.724 INFO:tasks.workunit.client.1.smithi193.stdout:5/515: stat d17/d18/d75/d85/c94 0 2024-04-25T15:49:41.725 INFO:tasks.workunit.client.0.smithi110.stdout:8/851: rename d14/d32/d105/d108 to d14/d21/d26/d6a/de9/d127 0 2024-04-25T15:49:41.725 INFO:tasks.workunit.client.0.smithi110.stdout:9/724: dwrite d9/db/df/f21 [0,4194304] 0 2024-04-25T15:49:41.725 INFO:tasks.workunit.client.0.smithi110.stdout:7/805: rename d1b/d29 to d1b/d29/d38/d8b/d89/dc6/d108 22 2024-04-25T15:49:41.727 INFO:tasks.workunit.client.0.smithi110.stdout:7/806: readlink d1b/d29/d46/l9d 0 2024-04-25T15:49:41.727 INFO:tasks.workunit.client.0.smithi110.stdout:8/852: write d14/f116 [7009335,87652] 0 2024-04-25T15:49:41.729 INFO:tasks.workunit.client.0.smithi110.stdout:3/764: truncate d0/d16/dcb/d50/fca 254225 0 2024-04-25T15:49:41.731 INFO:tasks.workunit.client.0.smithi110.stdout:1/921: dread d0/d14/f53 [0,4194304] 0 2024-04-25T15:49:41.732 INFO:tasks.workunit.client.1.smithi193.stdout:0/517: rename d0/d49/d5/d7/dd/d16/d26/l66 to d0/d49/d5/d7/dd/d46/d48/lb2 0 2024-04-25T15:49:41.732 INFO:tasks.workunit.client.1.smithi193.stdout:3/552: dread d8/d13/f61 [0,4194304] 0 2024-04-25T15:49:41.732 INFO:tasks.workunit.client.0.smithi110.stdout:6/693: write d1/db/d22/d33/d63/f97 [514621,20452] 0 2024-04-25T15:49:41.734 INFO:tasks.workunit.client.0.smithi110.stdout:1/922: write d0/d14/d5f/d7d/fc1 [2951401,43817] 0 2024-04-25T15:49:41.735 INFO:tasks.workunit.client.0.smithi110.stdout:1/923: stat d0/d14/d5f/d6f/f82 0 2024-04-25T15:49:41.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:41 smithi193 ceph-mon[33747]: pgmap v37: 65 pgs: 65 active+clean; 2.6 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 70 MiB/s rd, 236 MiB/s wr, 433 op/s 2024-04-25T15:49:41.741 INFO:tasks.workunit.client.1.smithi193.stdout:4/586: creat db/d13/d1e/d1f/fb3 x:0 0 0 2024-04-25T15:49:41.745 INFO:tasks.workunit.client.1.smithi193.stdout:8/496: creat d2/d3/d27/d69/f9a x:0 0 0 2024-04-25T15:49:41.746 INFO:tasks.workunit.client.1.smithi193.stdout:8/497: fdatasync d2/d3/f5f 0 2024-04-25T15:49:41.749 INFO:tasks.workunit.client.0.smithi110.stdout:4/742: dwrite d13/d24/d2d/d5e/f71 [0,4194304] 0 2024-04-25T15:49:41.753 INFO:tasks.workunit.client.1.smithi193.stdout:7/495: getdents d12/d1c/d63/db7 0 2024-04-25T15:49:41.753 INFO:tasks.workunit.client.0.smithi110.stdout:0/722: rename d1/d23/d30/d33/d5b/dbb/lc0 to d1/d23/d81/lff 0 2024-04-25T15:49:41.755 INFO:tasks.workunit.client.0.smithi110.stdout:9/725: dread - d9/fba zero size 2024-04-25T15:49:41.756 INFO:tasks.workunit.client.1.smithi193.stdout:0/518: read d0/d49/d5/d7/dd/d16/d26/d2e/f4b [1273605,22190] 0 2024-04-25T15:49:41.756 INFO:tasks.workunit.client.0.smithi110.stdout:4/743: dwrite d13/d24/d2d/d55/faa [0,4194304] 0 2024-04-25T15:49:41.756 INFO:tasks.workunit.client.0.smithi110.stdout:0/723: dread d1/d23/d30/d33/d5b/f98 [0,4194304] 0 2024-04-25T15:49:41.757 INFO:tasks.workunit.client.1.smithi193.stdout:4/587: dread f2 [0,4194304] 0 2024-04-25T15:49:41.758 INFO:tasks.workunit.client.0.smithi110.stdout:8/853: creat d14/d32/d36/dde/d11e/d123/f128 x:0 0 0 2024-04-25T15:49:41.762 INFO:tasks.workunit.client.0.smithi110.stdout:0/724: read d1/d23/d30/d33/d35/f12 [7158899,44831] 0 2024-04-25T15:49:41.767 INFO:tasks.workunit.client.1.smithi193.stdout:6/507: link dd/d12/d19/d1c/d40/d3c/l8d dd/d12/d2d/l9b 0 2024-04-25T15:49:41.769 INFO:tasks.workunit.client.0.smithi110.stdout:6/694: mkdir d1/dd/db5/df4 0 2024-04-25T15:49:41.770 INFO:tasks.workunit.client.0.smithi110.stdout:0/725: dwrite d1/f19 [0,4194304] 0 2024-04-25T15:49:41.770 INFO:tasks.workunit.client.0.smithi110.stdout:2/701: fdatasync de/d40/d6e/d52/d6d/d7d/f85 0 2024-04-25T15:49:41.773 INFO:tasks.workunit.client.1.smithi193.stdout:8/498: creat d2/d3/d27/f9b x:0 0 0 2024-04-25T15:49:41.774 INFO:tasks.workunit.client.1.smithi193.stdout:8/499: dread - d2/f6e zero size 2024-04-25T15:49:41.779 INFO:tasks.workunit.client.0.smithi110.stdout:0/726: dwrite d1/d47/f95 [0,4194304] 0 2024-04-25T15:49:41.781 INFO:tasks.workunit.client.1.smithi193.stdout:7/496: symlink d12/d31/d37/d62/d6e/da5/lbb 0 2024-04-25T15:49:41.782 INFO:tasks.workunit.client.1.smithi193.stdout:7/497: chown d12/d1c/d63/d70/f42 91 1 2024-04-25T15:49:41.789 INFO:tasks.workunit.client.0.smithi110.stdout:6/695: dread d1/dd/d3c/d91/d8d/dce/f73 [0,4194304] 0 2024-04-25T15:49:41.794 INFO:tasks.workunit.client.0.smithi110.stdout:8/854: fsync d14/d21/d26/d6a/d7e/da6/fc5 0 2024-04-25T15:49:41.795 INFO:tasks.workunit.client.0.smithi110.stdout:8/855: write d14/d32/d36/f115 [636546,108025] 0 2024-04-25T15:49:41.796 INFO:tasks.workunit.client.1.smithi193.stdout:5/516: truncate f16 4640123 0 2024-04-25T15:49:41.798 INFO:tasks.workunit.client.1.smithi193.stdout:0/519: write d0/d49/d5/d7/dd/d16/d26/d2e/f76 [1046380,47634] 0 2024-04-25T15:49:41.799 INFO:tasks.workunit.client.1.smithi193.stdout:0/520: readlink d0/d49/d5/d7/dd/l3b 0 2024-04-25T15:49:41.800 INFO:tasks.workunit.client.0.smithi110.stdout:7/807: truncate d1b/d29/d87/d8e/f94 2795343 0 2024-04-25T15:49:41.801 INFO:tasks.workunit.client.0.smithi110.stdout:7/808: fdatasync d1b/d29/d38/d55/dab/db2/f103 0 2024-04-25T15:49:41.802 INFO:tasks.workunit.client.0.smithi110.stdout:9/726: truncate f6 4180516 0 2024-04-25T15:49:41.804 INFO:tasks.workunit.client.1.smithi193.stdout:0/521: dwrite d0/d49/d5/d7/dd/f58 [0,4194304] 0 2024-04-25T15:49:41.804 INFO:tasks.workunit.client.0.smithi110.stdout:4/744: symlink d13/d24/d2d/lff 0 2024-04-25T15:49:41.807 INFO:tasks.workunit.client.1.smithi193.stdout:2/509: sync 2024-04-25T15:49:41.808 INFO:tasks.workunit.client.1.smithi193.stdout:2/510: readlink d0/d9/d15/d31/lc8 0 2024-04-25T15:49:41.812 INFO:tasks.workunit.client.0.smithi110.stdout:9/727: rmdir d9/db/d7d 39 2024-04-25T15:49:41.814 INFO:tasks.workunit.client.0.smithi110.stdout:9/728: write d9/d13/fca [876827,25059] 0 2024-04-25T15:49:41.816 INFO:tasks.workunit.client.1.smithi193.stdout:8/500: read d2/d3/d20/d45/f5a [32781,60188] 0 2024-04-25T15:49:41.816 INFO:tasks.workunit.client.0.smithi110.stdout:7/809: dwrite d1b/d29/f5d [0,4194304] 0 2024-04-25T15:49:41.817 INFO:tasks.workunit.client.0.smithi110.stdout:4/745: mknod d13/d24/d2d/dd9/c100 0 2024-04-25T15:49:41.820 INFO:tasks.workunit.client.1.smithi193.stdout:3/553: link d8/ce d8/d13/d14/d23/caf 0 2024-04-25T15:49:41.821 INFO:tasks.workunit.client.1.smithi193.stdout:3/554: chown d8/d13/d1f/d69/d77/f53 194122517 1 2024-04-25T15:49:41.821 INFO:tasks.workunit.client.1.smithi193.stdout:3/555: dread - d8/d13/d14/d27/f96 zero size 2024-04-25T15:49:41.826 INFO:tasks.workunit.client.1.smithi193.stdout:0/522: creat d0/d49/d5/d7/dd/d78/fb3 x:0 0 0 2024-04-25T15:49:41.828 INFO:tasks.workunit.client.1.smithi193.stdout:2/511: symlink d0/d37/d25/d5f/d3a/d6d/d85/lcb 0 2024-04-25T15:49:41.829 INFO:tasks.workunit.client.1.smithi193.stdout:6/508: symlink dd/d12/d19/l9c 0 2024-04-25T15:49:41.830 INFO:tasks.workunit.client.1.smithi193.stdout:1/440: link dc/d4a/c7f dc/d3c/c94 0 2024-04-25T15:49:41.830 INFO:tasks.workunit.client.1.smithi193.stdout:1/441: readlink dc/d3c/l45 0 2024-04-25T15:49:41.832 INFO:tasks.workunit.client.1.smithi193.stdout:9/510: getdents df/d11 0 2024-04-25T15:49:41.833 INFO:tasks.workunit.client.0.smithi110.stdout:7/810: mknod d1b/d29/d38/d55/dab/db2/db8/c109 0 2024-04-25T15:49:41.835 INFO:tasks.workunit.client.0.smithi110.stdout:2/702: write de/d40/f96 [258185,20492] 0 2024-04-25T15:49:41.836 INFO:tasks.workunit.client.0.smithi110.stdout:9/729: dread d9/f96 [0,4194304] 0 2024-04-25T15:49:41.837 INFO:tasks.workunit.client.0.smithi110.stdout:9/730: readlink d9/db/df/l41 0 2024-04-25T15:49:41.837 INFO:tasks.workunit.client.0.smithi110.stdout:4/746: mknod d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/d92/c101 0 2024-04-25T15:49:41.838 INFO:tasks.workunit.client.0.smithi110.stdout:4/747: chown d13/d24/d2d/d44/d65/d99/daf/d56/d88/f2c 416555 1 2024-04-25T15:49:41.839 INFO:tasks.workunit.client.1.smithi193.stdout:8/501: creat d2/d3/d27/d69/f9c x:0 0 0 2024-04-25T15:49:41.839 INFO:tasks.workunit.client.0.smithi110.stdout:4/748: chown d13/d24/d2d/d44/da4/la7 919285 1 2024-04-25T15:49:41.840 INFO:tasks.workunit.client.0.smithi110.stdout:4/749: readlink d13/d24/d2d/lf0 0 2024-04-25T15:49:41.841 INFO:tasks.workunit.client.1.smithi193.stdout:3/556: rmdir d8/d13/d4b 39 2024-04-25T15:49:41.842 INFO:tasks.workunit.client.0.smithi110.stdout:3/765: getdents d0/d16/dcb/d2f/d4c/db2/df2 0 2024-04-25T15:49:41.843 INFO:tasks.workunit.client.1.smithi193.stdout:6/509: dread dd/d12/d19/f79 [4194304,4194304] 0 2024-04-25T15:49:41.843 INFO:tasks.workunit.client.0.smithi110.stdout:3/766: readlink d0/d16/dcb/d2f/d4c/l54 0 2024-04-25T15:49:41.844 INFO:tasks.workunit.client.1.smithi193.stdout:5/517: write d17/d83/f95 [1335505,116914] 0 2024-04-25T15:49:41.845 INFO:tasks.workunit.client.1.smithi193.stdout:0/523: creat d0/d49/d5/d7/dd/d46/d48/fb4 x:0 0 0 2024-04-25T15:49:41.846 INFO:tasks.workunit.client.0.smithi110.stdout:0/727: getdents d1/d23/d30 0 2024-04-25T15:49:41.846 INFO:tasks.workunit.client.1.smithi193.stdout:0/524: write d0/d49/d5/d7/dd/d78/fb3 [738055,30127] 0 2024-04-25T15:49:41.847 INFO:tasks.workunit.client.1.smithi193.stdout:2/512: mknod d0/d37/d25/d5f/d3a/d6d/d85/ccc 0 2024-04-25T15:49:41.848 INFO:tasks.workunit.client.0.smithi110.stdout:6/696: getdents d1/db/da0/da5 0 2024-04-25T15:49:41.848 INFO:tasks.workunit.client.1.smithi193.stdout:0/525: dread - d0/d49/d5/d7/dd/d16/d26/d5e/fa8 zero size 2024-04-25T15:49:41.849 INFO:tasks.workunit.client.0.smithi110.stdout:6/697: chown d1/db/d24/d69/d75/fd5 1422491593 1 2024-04-25T15:49:41.850 INFO:tasks.workunit.client.0.smithi110.stdout:7/811: unlink d1b/d29/d46/f6a 0 2024-04-25T15:49:41.852 INFO:tasks.workunit.client.0.smithi110.stdout:2/703: fsync de/f56 0 2024-04-25T15:49:41.854 INFO:tasks.workunit.client.1.smithi193.stdout:7/498: link d12/d31/d37/d62/lb9 d12/d1c/d63/d70/d68/d8f/lbc 0 2024-04-25T15:49:41.855 INFO:tasks.workunit.client.1.smithi193.stdout:4/588: sync 2024-04-25T15:49:41.856 INFO:tasks.workunit.client.0.smithi110.stdout:8/856: getdents d14/d21/d26/dab 0 2024-04-25T15:49:41.857 INFO:tasks.workunit.client.1.smithi193.stdout:4/589: write db/f61 [1124485,94078] 0 2024-04-25T15:49:41.857 INFO:tasks.workunit.client.1.smithi193.stdout:5/518: dread d17/f86 [0,4194304] 0 2024-04-25T15:49:41.858 INFO:tasks.workunit.client.1.smithi193.stdout:8/502: symlink d2/d3/d27/l9d 0 2024-04-25T15:49:41.861 INFO:tasks.workunit.client.0.smithi110.stdout:3/767: mkdir d0/d16/dcb/d2f/d4c/d108 0 2024-04-25T15:49:41.862 INFO:tasks.workunit.client.1.smithi193.stdout:1/442: dread dc/f2d [0,4194304] 0 2024-04-25T15:49:41.864 INFO:tasks.workunit.client.1.smithi193.stdout:6/510: mknod dd/d2f/d57/c9d 0 2024-04-25T15:49:41.865 INFO:tasks.workunit.client.0.smithi110.stdout:0/728: rename d1/d23/d30/d33/d5b/fc7 to d1/d23/d81/f100 0 2024-04-25T15:49:41.866 INFO:tasks.workunit.client.0.smithi110.stdout:6/698: mknod d1/db/da0/cf5 0 2024-04-25T15:49:41.867 INFO:tasks.workunit.client.0.smithi110.stdout:0/729: write d1/d23/d30/d33/d5b/d78/da4/feb [271802,124173] 0 2024-04-25T15:49:41.868 INFO:tasks.workunit.client.1.smithi193.stdout:1/443: dwrite dc/d4a/d50/d7c/d7e/f88 [0,4194304] 0 2024-04-25T15:49:41.868 INFO:tasks.workunit.client.1.smithi193.stdout:2/513: creat d0/d37/d25/d5f/d3a/d6d/fcd x:0 0 0 2024-04-25T15:49:41.871 INFO:tasks.workunit.client.0.smithi110.stdout:7/812: symlink d1b/d29/db5/l10a 0 2024-04-25T15:49:41.881 INFO:tasks.workunit.client.0.smithi110.stdout:4/750: unlink d13/d24/d2d/d44/d65/d99/daf/d56/c75 0 2024-04-25T15:49:41.882 INFO:tasks.workunit.client.1.smithi193.stdout:5/519: mkdir d17/d18/d30/d34/d52/daf 0 2024-04-25T15:49:41.882 INFO:tasks.workunit.client.0.smithi110.stdout:4/751: stat d13/d24/d2d/d44/d65/d99/daf/d56/f76 0 2024-04-25T15:49:41.883 INFO:tasks.workunit.client.0.smithi110.stdout:4/752: write d13/d24/d2d/d36/d51/fb6 [2551699,96869] 0 2024-04-25T15:49:41.884 INFO:tasks.workunit.client.1.smithi193.stdout:8/503: truncate d2/d3/d20/d45/f5a 930931 0 2024-04-25T15:49:41.884 INFO:tasks.workunit.client.0.smithi110.stdout:8/857: write d14/d21/d26/d6a/d7e/da6/de8/d55/d8d/f95 [3942115,127652] 0 2024-04-25T15:49:41.885 INFO:tasks.workunit.client.0.smithi110.stdout:6/699: dread - d1/db/d24/d69/d75/d82/f85 zero size 2024-04-25T15:49:41.886 INFO:tasks.workunit.client.1.smithi193.stdout:0/526: dread d0/fc [0,4194304] 0 2024-04-25T15:49:41.889 INFO:tasks.workunit.client.1.smithi193.stdout:6/511: rmdir dd/d12/d19/d1c/d40/d3c/d3f/d4f 39 2024-04-25T15:49:41.891 INFO:tasks.workunit.client.1.smithi193.stdout:6/512: write dd/d2f/f56 [1503269,62313] 0 2024-04-25T15:49:41.891 INFO:tasks.workunit.client.0.smithi110.stdout:7/813: mkdir d1b/d29/db5/dca/d10b 0 2024-04-25T15:49:41.891 INFO:tasks.workunit.client.0.smithi110.stdout:3/768: write d0/d16/dcb/db0/fe8 [285084,88168] 0 2024-04-25T15:49:41.893 INFO:tasks.workunit.client.1.smithi193.stdout:0/527: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/f76 [0,4194304] 0 2024-04-25T15:49:41.893 INFO:tasks.workunit.client.1.smithi193.stdout:6/513: write dd/d2f/d57/f8b [2669670,40528] 0 2024-04-25T15:49:41.896 INFO:tasks.workunit.client.1.smithi193.stdout:6/514: truncate dd/d2f/d57/f5c 889296 0 2024-04-25T15:49:41.896 INFO:tasks.workunit.client.1.smithi193.stdout:9/511: rename df/d2c/d2f/f8f to df/d2c/d2f/f9a 0 2024-04-25T15:49:41.896 INFO:tasks.workunit.client.1.smithi193.stdout:7/499: getdents d12/d31/d37/d62/d6e/d79/d9d 0 2024-04-25T15:49:41.897 INFO:tasks.workunit.client.1.smithi193.stdout:6/515: chown dd/c94 329 1 2024-04-25T15:49:41.901 INFO:tasks.workunit.client.0.smithi110.stdout:8/858: truncate d14/d21/d26/d6a/d7e/da6/de8/d55/f7d 2758771 0 2024-04-25T15:49:41.903 INFO:tasks.workunit.client.0.smithi110.stdout:2/704: mkdir de/d40/dcd/ddf 0 2024-04-25T15:49:41.904 INFO:tasks.workunit.client.1.smithi193.stdout:9/512: dwrite df/d2c/d2f/d4e/d1c/f97 [0,4194304] 0 2024-04-25T15:49:41.912 INFO:tasks.workunit.client.1.smithi193.stdout:6/516: dread dd/d12/d19/d1c/f71 [0,4194304] 0 2024-04-25T15:49:41.942 INFO:tasks.workunit.client.1.smithi193.stdout:2/514: symlink d0/d37/lce 0 2024-04-25T15:49:41.942 INFO:tasks.workunit.client.1.smithi193.stdout:2/515: chown d0/d42/c89 7328 1 2024-04-25T15:49:41.946 INFO:tasks.workunit.client.0.smithi110.stdout:8/859: dread - d14/d21/d26/dab/dac/fd4 zero size 2024-04-25T15:49:41.948 INFO:tasks.workunit.client.1.smithi193.stdout:2/516: dwrite d0/d42/d51/d4f/f97 [0,4194304] 0 2024-04-25T15:49:41.948 INFO:tasks.workunit.client.0.smithi110.stdout:0/730: dwrite d1/d23/d30/d33/d5b/d72/d77/fcc [0,4194304] 0 2024-04-25T15:49:41.948 INFO:tasks.workunit.client.0.smithi110.stdout:7/814: fsync d1b/d29/d38/f5b 0 2024-04-25T15:49:41.949 INFO:tasks.workunit.client.0.smithi110.stdout:7/815: fdatasync d1b/f88 0 2024-04-25T15:49:41.951 INFO:tasks.workunit.client.1.smithi193.stdout:0/528: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/cb5 0 2024-04-25T15:49:41.951 INFO:tasks.workunit.client.1.smithi193.stdout:1/444: truncate dc/d4a/f75 356960 0 2024-04-25T15:49:41.952 INFO:tasks.workunit.client.1.smithi193.stdout:0/529: dread - d0/d49/d5/d7/dd/d16/d26/fb0 zero size 2024-04-25T15:49:41.953 INFO:tasks.workunit.client.0.smithi110.stdout:3/769: mknod d0/d7/d40/c109 0 2024-04-25T15:49:41.953 INFO:tasks.workunit.client.0.smithi110.stdout:3/770: chown d0/d16/l51 6 1 2024-04-25T15:49:41.954 INFO:tasks.workunit.client.0.smithi110.stdout:3/771: write d0/d2e/f76 [3601788,31668] 0 2024-04-25T15:49:41.955 INFO:tasks.workunit.client.1.smithi193.stdout:7/500: unlink d12/d1c/d45/l66 0 2024-04-25T15:49:41.955 INFO:tasks.workunit.client.1.smithi193.stdout:7/501: dread - d12/d1c/f7e zero size 2024-04-25T15:49:41.958 INFO:tasks.workunit.client.1.smithi193.stdout:5/520: sync 2024-04-25T15:49:41.958 INFO:tasks.workunit.client.1.smithi193.stdout:7/502: chown d12/d1c/d63/d70/f25 17270029 1 2024-04-25T15:49:41.959 INFO:tasks.workunit.client.1.smithi193.stdout:5/521: readlink d17/d18/d30/d38/d71/l72 0 2024-04-25T15:49:41.959 INFO:tasks.workunit.client.0.smithi110.stdout:4/753: rename d13/d24/d26/dcd/fdc to d13/d24/d2d/f102 0 2024-04-25T15:49:41.961 INFO:tasks.workunit.client.1.smithi193.stdout:3/557: getdents d8/d13 0 2024-04-25T15:49:41.962 INFO:tasks.workunit.client.1.smithi193.stdout:3/558: chown d8/d13/d14/d27/f95 26514 1 2024-04-25T15:49:41.964 INFO:tasks.workunit.client.0.smithi110.stdout:8/860: creat d14/d21/d26/d6a/de9/d127/f129 x:0 0 0 2024-04-25T15:49:41.965 INFO:tasks.workunit.client.0.smithi110.stdout:8/861: readlink d14/d21/d26/d6a/d7e/da6/de8/d71/l90 0 2024-04-25T15:49:41.967 INFO:tasks.workunit.client.0.smithi110.stdout:0/731: truncate d1/d23/d30/d33/d5b/d72/fd1 804532 0 2024-04-25T15:49:41.969 INFO:tasks.workunit.client.1.smithi193.stdout:2/517: mknod d0/df/ccf 0 2024-04-25T15:49:41.970 INFO:tasks.workunit.client.0.smithi110.stdout:2/705: creat de/d40/d4f/d9f/fe0 x:0 0 0 2024-04-25T15:49:41.975 INFO:tasks.workunit.client.0.smithi110.stdout:4/754: creat d13/d24/d2d/d5e/f103 x:0 0 0 2024-04-25T15:49:41.976 INFO:tasks.workunit.client.1.smithi193.stdout:1/445: creat dc/d4a/d50/d5c/d5d/d3d/f95 x:0 0 0 2024-04-25T15:49:41.977 INFO:tasks.workunit.client.1.smithi193.stdout:8/504: truncate d2/d3/d20/f22 1997498 0 2024-04-25T15:49:41.977 INFO:tasks.workunit.client.0.smithi110.stdout:6/700: getdents d1/db/d22/d90/d76/dad 0 2024-04-25T15:49:41.980 INFO:tasks.workunit.client.1.smithi193.stdout:0/530: creat d0/d49/d5/d7/dd/d16/d1a/fb6 x:0 0 0 2024-04-25T15:49:41.982 INFO:tasks.workunit.client.1.smithi193.stdout:4/590: rename db/d13/d1e/d1f/d22/f5d to db/d32/fb4 0 2024-04-25T15:49:41.983 INFO:tasks.workunit.client.1.smithi193.stdout:8/505: dwrite d2/d3/d27/f96 [0,4194304] 0 2024-04-25T15:49:41.983 INFO:tasks.workunit.client.0.smithi110.stdout:6/701: dwrite d1/d12/db8/fe1 [0,4194304] 0 2024-04-25T15:49:41.984 INFO:tasks.workunit.client.0.smithi110.stdout:8/862: mknod d14/d21/d26/dab/c12a 0 2024-04-25T15:49:41.986 INFO:tasks.workunit.client.1.smithi193.stdout:5/522: creat d17/d18/d75/d85/d5d/fb0 x:0 0 0 2024-04-25T15:49:41.987 INFO:tasks.workunit.client.0.smithi110.stdout:8/863: truncate d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/fb0 5165629 0 2024-04-25T15:49:41.987 INFO:tasks.workunit.client.0.smithi110.stdout:0/732: fsync d1/d20/f96 0 2024-04-25T15:49:41.990 INFO:tasks.workunit.client.0.smithi110.stdout:3/772: mkdir d0/d16/dcb/d2f/d10a 0 2024-04-25T15:49:41.994 INFO:tasks.workunit.client.1.smithi193.stdout:3/559: unlink d8/d13/d14/d27/d54/c8a 0 2024-04-25T15:49:41.999 INFO:tasks.workunit.client.0.smithi110.stdout:6/702: rmdir d1/d12/db8 39 2024-04-25T15:49:42.000 INFO:tasks.workunit.client.1.smithi193.stdout:2/518: mknod d0/d42/d51/cd0 0 2024-04-25T15:49:42.005 INFO:tasks.workunit.client.1.smithi193.stdout:1/446: creat dc/d4a/d4d/d7a/f96 x:0 0 0 2024-04-25T15:49:42.007 INFO:tasks.workunit.client.0.smithi110.stdout:8/864: fdatasync d14/f94 0 2024-04-25T15:49:42.008 INFO:tasks.workunit.client.1.smithi193.stdout:1/447: dread dc/d59/f65 [0,4194304] 0 2024-04-25T15:49:42.009 INFO:tasks.workunit.client.0.smithi110.stdout:0/733: truncate d1/d23/d30/d33/d5b/dbb/da8/fb6 5885483 0 2024-04-25T15:49:42.010 INFO:tasks.workunit.client.1.smithi193.stdout:0/531: chown d0/d49/d5/d61/la9 117701 1 2024-04-25T15:49:42.011 INFO:tasks.workunit.client.1.smithi193.stdout:0/532: write d0/d49/d5/d7/dd/f6e [813256,107850] 0 2024-04-25T15:49:42.011 INFO:tasks.workunit.client.0.smithi110.stdout:3/773: mkdir d0/d7/dd0/d10b 0 2024-04-25T15:49:42.012 INFO:tasks.workunit.client.0.smithi110.stdout:3/774: chown d0/c86 0 1 2024-04-25T15:49:42.015 INFO:tasks.workunit.client.0.smithi110.stdout:0/734: dwrite d1/d23/d30/fca [0,4194304] 0 2024-04-25T15:49:42.016 INFO:tasks.workunit.client.1.smithi193.stdout:9/513: truncate df/d2c/d2f/d4e/f1a 4952578 0 2024-04-25T15:49:42.017 INFO:tasks.workunit.client.1.smithi193.stdout:0/533: dwrite d0/d49/d5/d7/dd/d46/d48/f57 [0,4194304] 0 2024-04-25T15:49:42.020 INFO:tasks.workunit.client.1.smithi193.stdout:8/506: mknod d2/d3/d27/d69/d86/d72/c9e 0 2024-04-25T15:49:42.024 INFO:tasks.workunit.client.1.smithi193.stdout:5/523: rmdir d17/d18/d30/d3b 39 2024-04-25T15:49:42.024 INFO:tasks.workunit.client.1.smithi193.stdout:5/524: stat d17/d18 0 2024-04-25T15:49:42.027 INFO:tasks.workunit.client.0.smithi110.stdout:8/865: creat d14/d21/d26/d6a/d7e/f12b x:0 0 0 2024-04-25T15:49:42.028 INFO:tasks.workunit.client.1.smithi193.stdout:5/525: dread d17/d18/d30/f35 [0,4194304] 0 2024-04-25T15:49:42.028 INFO:tasks.workunit.client.1.smithi193.stdout:3/560: truncate d8/d13/d14/d51/f6a 502783 0 2024-04-25T15:49:42.029 INFO:tasks.workunit.client.1.smithi193.stdout:5/526: stat d17/d18/d3e 0 2024-04-25T15:49:42.030 INFO:tasks.workunit.client.1.smithi193.stdout:5/527: read - d17/d18/d3e/fa5 zero size 2024-04-25T15:49:42.030 INFO:tasks.workunit.client.1.smithi193.stdout:3/561: write d8/d13/d14/d27/d54/f65 [227473,27444] 0 2024-04-25T15:49:42.030 INFO:tasks.workunit.client.0.smithi110.stdout:3/775: truncate d0/d16/dcb/f35 4420663 0 2024-04-25T15:49:42.031 INFO:tasks.workunit.client.0.smithi110.stdout:3/776: chown d0/f101 63592 1 2024-04-25T15:49:42.031 INFO:tasks.workunit.client.1.smithi193.stdout:5/528: write d17/d18/f21 [554115,120651] 0 2024-04-25T15:49:42.031 INFO:tasks.workunit.client.1.smithi193.stdout:3/562: write d8/d4d/f87 [867832,1963] 0 2024-04-25T15:49:42.033 INFO:tasks.workunit.client.1.smithi193.stdout:1/448: creat dc/d4a/d50/d5c/d5d/d3d/f97 x:0 0 0 2024-04-25T15:49:42.033 INFO:tasks.workunit.client.0.smithi110.stdout:0/735: fdatasync d1/d23/d30/d33/d5b/f98 0 2024-04-25T15:49:42.035 INFO:tasks.workunit.client.0.smithi110.stdout:6/703: symlink d1/db/d22/d33/d63/dc2/dc7/dec/lf6 0 2024-04-25T15:49:42.036 INFO:tasks.workunit.client.0.smithi110.stdout:6/704: readlink d1/db/d24/d69/d75/d82/l83 0 2024-04-25T15:49:42.038 INFO:tasks.workunit.client.0.smithi110.stdout:3/777: creat d0/d7/dd0/ddf/f10c x:0 0 0 2024-04-25T15:49:42.039 INFO:tasks.workunit.client.0.smithi110.stdout:3/778: write d0/d16/dcb/db0/fe8 [1237087,44170] 0 2024-04-25T15:49:42.040 INFO:tasks.workunit.client.1.smithi193.stdout:7/503: rename d12/d1c/d63/d70/c2b to d12/d31/d37/d62/d6e/d79/cbd 0 2024-04-25T15:49:42.041 INFO:tasks.workunit.client.0.smithi110.stdout:4/755: write d13/d24/d2d/d36/f74 [470408,122853] 0 2024-04-25T15:49:42.042 INFO:tasks.workunit.client.1.smithi193.stdout:7/504: read d12/d13/f43 [302404,103427] 0 2024-04-25T15:49:42.042 INFO:tasks.workunit.client.0.smithi110.stdout:4/756: chown d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa 823 1 2024-04-25T15:49:42.042 INFO:tasks.workunit.client.1.smithi193.stdout:4/591: symlink db/d32/d88/da8/lb5 0 2024-04-25T15:49:42.043 INFO:tasks.workunit.client.1.smithi193.stdout:4/592: chown db/d4b/d51/d54/d77 0 1 2024-04-25T15:49:42.046 INFO:tasks.workunit.client.1.smithi193.stdout:9/514: creat df/d2c/d2f/d4e/d1c/f9b x:0 0 0 2024-04-25T15:49:42.048 INFO:tasks.workunit.client.0.smithi110.stdout:3/779: mknod d0/d2e/d88/c10d 0 2024-04-25T15:49:42.051 INFO:tasks.workunit.client.0.smithi110.stdout:4/757: symlink d13/d24/d2d/d55/l104 0 2024-04-25T15:49:42.054 INFO:tasks.workunit.client.0.smithi110.stdout:6/705: truncate d1/db/d22/d33/f56 1083789 0 2024-04-25T15:49:42.055 INFO:tasks.workunit.client.1.smithi193.stdout:6/517: getdents dd/d12 0 2024-04-25T15:49:42.055 INFO:tasks.workunit.client.0.smithi110.stdout:3/780: dwrite d0/d7/dd0/ddf/f10c [0,4194304] 0 2024-04-25T15:49:42.056 INFO:tasks.workunit.client.1.smithi193.stdout:4/593: dread db/fc [0,4194304] 0 2024-04-25T15:49:42.056 INFO:tasks.workunit.client.0.smithi110.stdout:4/758: creat d13/d24/d2d/d55/f105 x:0 0 0 2024-04-25T15:49:42.057 INFO:tasks.workunit.client.0.smithi110.stdout:4/759: stat d13/f22 0 2024-04-25T15:49:42.062 INFO:tasks.workunit.client.1.smithi193.stdout:6/518: dwrite dd/d12/d19/d1c/f99 [0,4194304] 0 2024-04-25T15:49:42.062 INFO:tasks.workunit.client.0.smithi110.stdout:6/706: mkdir d1/db/d22/d90/df7 0 2024-04-25T15:49:42.064 INFO:tasks.workunit.client.1.smithi193.stdout:5/529: creat d17/d18/d30/d38/fb1 x:0 0 0 2024-04-25T15:49:42.068 INFO:tasks.workunit.client.0.smithi110.stdout:4/760: dread d13/d24/f9f [0,4194304] 0 2024-04-25T15:49:42.069 INFO:tasks.workunit.client.0.smithi110.stdout:4/761: dread - d13/d24/d2d/d55/f105 zero size 2024-04-25T15:49:42.072 INFO:tasks.workunit.client.1.smithi193.stdout:8/507: rename d2/d3/d27/d6f/d94/l83 to d2/d3/d20/d45/d4a/d63/l9f 0 2024-04-25T15:49:42.074 INFO:tasks.workunit.client.0.smithi110.stdout:6/707: mkdir d1/dd/d3c/df8 0 2024-04-25T15:49:42.074 INFO:tasks.workunit.client.1.smithi193.stdout:0/534: write d0/d49/d5/d7/fb [33039,94596] 0 2024-04-25T15:49:42.076 INFO:tasks.workunit.client.0.smithi110.stdout:3/781: mknod d0/d7/c10e 0 2024-04-25T15:49:42.077 INFO:tasks.workunit.client.1.smithi193.stdout:0/535: fdatasync d0/d49/d5/d7/dd/d16/d26/d5e/d99/f9a 0 2024-04-25T15:49:42.078 INFO:tasks.workunit.client.0.smithi110.stdout:4/762: mknod d13/d24/da6/c106 0 2024-04-25T15:49:42.078 INFO:tasks.workunit.client.1.smithi193.stdout:0/536: dread - d0/d49/d5/d61/fae zero size 2024-04-25T15:49:42.079 INFO:tasks.workunit.client.0.smithi110.stdout:5/766: sync 2024-04-25T15:49:42.079 INFO:tasks.workunit.client.1.smithi193.stdout:8/508: dwrite d2/d3/d20/f33 [0,4194304] 0 2024-04-25T15:49:42.081 INFO:tasks.workunit.client.0.smithi110.stdout:6/708: mkdir d1/db/d22/d90/df9 0 2024-04-25T15:49:42.081 INFO:tasks.workunit.client.1.smithi193.stdout:8/509: write d2/d3/d27/d69/d86/f40 [9009647,122881] 0 2024-04-25T15:49:42.082 INFO:tasks.workunit.client.0.smithi110.stdout:6/709: chown d1/dd/d3c/f51 504315 1 2024-04-25T15:49:42.082 INFO:tasks.workunit.client.1.smithi193.stdout:9/515: fdatasync df/d11/f2e 0 2024-04-25T15:49:42.082 INFO:tasks.workunit.client.1.smithi193.stdout:2/519: sync 2024-04-25T15:49:42.083 INFO:tasks.workunit.client.1.smithi193.stdout:8/510: read d2/d3/d20/f33 [3271171,87035] 0 2024-04-25T15:49:42.083 INFO:tasks.workunit.client.0.smithi110.stdout:3/782: rmdir d0/d2e/d88/dbf/d99/dad/daf 39 2024-04-25T15:49:42.083 INFO:tasks.workunit.client.1.smithi193.stdout:9/516: stat df/d2d/f99 0 2024-04-25T15:49:42.085 INFO:tasks.workunit.client.0.smithi110.stdout:4/763: mknod d13/d24/d2d/d44/da4/c107 0 2024-04-25T15:49:42.089 INFO:tasks.workunit.client.1.smithi193.stdout:4/594: mknod db/d13/d1e/d1f/d56/d57/cb6 0 2024-04-25T15:49:42.092 INFO:tasks.workunit.client.0.smithi110.stdout:6/710: symlink d1/dd/d3c/lfa 0 2024-04-25T15:49:42.094 INFO:tasks.workunit.client.1.smithi193.stdout:4/595: dread db/d32/f94 [0,4194304] 0 2024-04-25T15:49:42.096 INFO:tasks.workunit.client.1.smithi193.stdout:4/596: write db/d32/f94 [990730,38225] 0 2024-04-25T15:49:42.099 INFO:tasks.workunit.client.0.smithi110.stdout:5/767: fdatasync d4/d2c/f27 0 2024-04-25T15:49:42.101 INFO:tasks.workunit.client.1.smithi193.stdout:3/563: link d8/d13/f66 d8/d13/d1f/d69/fb0 0 2024-04-25T15:49:42.102 INFO:tasks.workunit.client.0.smithi110.stdout:4/764: dwrite d13/d24/d2d/f102 [0,4194304] 0 2024-04-25T15:49:42.105 INFO:tasks.workunit.client.0.smithi110.stdout:3/783: dread d0/d7/f69 [0,4194304] 0 2024-04-25T15:49:42.106 INFO:tasks.workunit.client.1.smithi193.stdout:3/564: dwrite d8/d30/f47 [0,4194304] 0 2024-04-25T15:49:42.111 INFO:tasks.workunit.client.0.smithi110.stdout:6/711: write d1/dd/d3c/f51 [1063251,36527] 0 2024-04-25T15:49:42.113 INFO:tasks.workunit.client.0.smithi110.stdout:3/784: getdents d0/d16/dcb/d2f/d4c/d108 0 2024-04-25T15:49:42.117 INFO:tasks.workunit.client.0.smithi110.stdout:4/765: dread d13/d24/d2d/f62 [0,4194304] 0 2024-04-25T15:49:42.118 INFO:tasks.workunit.client.0.smithi110.stdout:5/768: dwrite d4/f7c [0,4194304] 0 2024-04-25T15:49:42.120 INFO:tasks.workunit.client.0.smithi110.stdout:6/712: rename d1/db/d22/d90/d76/fca to d1/dd/d3c/ffb 0 2024-04-25T15:49:42.121 INFO:tasks.workunit.client.0.smithi110.stdout:3/785: mkdir d0/d2e/dfc/d10f 0 2024-04-25T15:49:42.123 INFO:tasks.workunit.client.0.smithi110.stdout:5/769: symlink d4/d1d/d1f/d35/d3d/d52/lfb 0 2024-04-25T15:49:42.124 INFO:tasks.workunit.client.0.smithi110.stdout:5/770: dread - d4/d15/d51/d90/ded/ff6 zero size 2024-04-25T15:49:42.124 INFO:tasks.workunit.client.1.smithi193.stdout:8/511: mkdir d2/d18/da0 0 2024-04-25T15:49:42.125 INFO:tasks.workunit.client.1.smithi193.stdout:8/512: write d2/d3/d27/d69/f9c [315825,114] 0 2024-04-25T15:49:42.126 INFO:tasks.workunit.client.0.smithi110.stdout:6/713: creat d1/dd/d3c/d91/d8d/dce/ffc x:0 0 0 2024-04-25T15:49:42.127 INFO:tasks.workunit.client.1.smithi193.stdout:2/520: mknod d0/d9/d15/d31/d50/cd1 0 2024-04-25T15:49:42.127 INFO:tasks.workunit.client.0.smithi110.stdout:3/786: truncate d0/d16/dcb/f5f 2280304 0 2024-04-25T15:49:42.128 INFO:tasks.workunit.client.1.smithi193.stdout:9/517: fsync df/d2c/d2f/f9a 0 2024-04-25T15:49:42.128 INFO:tasks.workunit.client.0.smithi110.stdout:3/787: readlink d0/d16/la9 0 2024-04-25T15:49:42.129 INFO:tasks.workunit.client.0.smithi110.stdout:5/771: creat d4/d1d/d85/ffc x:0 0 0 2024-04-25T15:49:42.130 INFO:tasks.workunit.client.0.smithi110.stdout:5/772: chown d4/d1d/c63 125390 1 2024-04-25T15:49:42.130 INFO:tasks.workunit.client.0.smithi110.stdout:6/714: dread - d1/db/d24/d69/d75/fa8 zero size 2024-04-25T15:49:42.130 INFO:tasks.workunit.client.1.smithi193.stdout:6/519: rmdir dd/d12/d19/d1c/d40/d3c/d3f/d4f 39 2024-04-25T15:49:42.135 INFO:tasks.workunit.client.0.smithi110.stdout:6/715: creat d1/db/d22/d90/d9e/ffd x:0 0 0 2024-04-25T15:49:42.136 INFO:tasks.workunit.client.0.smithi110.stdout:6/716: chown d1/db/d22/d33/d63/fd4 42331 1 2024-04-25T15:49:42.136 INFO:tasks.workunit.client.1.smithi193.stdout:6/520: dwrite dd/d2f/f56 [4194304,4194304] 0 2024-04-25T15:49:42.138 INFO:tasks.workunit.client.1.smithi193.stdout:6/521: read dd/d12/d19/d1c/d40/f23 [3461752,44382] 0 2024-04-25T15:49:42.138 INFO:tasks.workunit.client.1.smithi193.stdout:3/565: symlink d8/d30/d72/lb1 0 2024-04-25T15:49:42.139 INFO:tasks.workunit.client.1.smithi193.stdout:7/505: creat d12/d31/d37/d62/d6e/fbe x:0 0 0 2024-04-25T15:49:42.140 INFO:tasks.workunit.client.1.smithi193.stdout:6/522: chown dd/d12/d19/d1c/d40/d3c/c58 61 1 2024-04-25T15:49:42.141 INFO:tasks.workunit.client.0.smithi110.stdout:6/717: rename d1/dd/d3c/f51 to d1/db/d22/d33/d63/ffe 0 2024-04-25T15:49:42.143 INFO:tasks.workunit.client.1.smithi193.stdout:0/537: rename d0/d49/d5/d7/dd/d46/d48/lb2 to d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/lb7 0 2024-04-25T15:49:42.148 INFO:tasks.workunit.client.1.smithi193.stdout:8/513: readlink d2/l41 0 2024-04-25T15:49:42.155 INFO:tasks.workunit.client.0.smithi110.stdout:3/788: write d0/d16/dcb/d2f/d4c/fa0 [1054461,20215] 0 2024-04-25T15:49:42.157 INFO:tasks.workunit.client.1.smithi193.stdout:4/597: creat db/d13/d9d/fb7 x:0 0 0 2024-04-25T15:49:42.157 INFO:tasks.workunit.client.1.smithi193.stdout:5/530: truncate d17/d18/d75/d85/f73 2132639 0 2024-04-25T15:49:42.158 INFO:tasks.workunit.client.1.smithi193.stdout:1/449: getdents dc/d4a/d50 0 2024-04-25T15:49:42.159 INFO:tasks.workunit.client.1.smithi193.stdout:5/531: chown d17/d18/d30/d38/l3c 25349 1 2024-04-25T15:49:42.159 INFO:tasks.workunit.client.1.smithi193.stdout:1/450: write dc/d4a/d4d/f57 [4277632,39965] 0 2024-04-25T15:49:42.159 INFO:tasks.workunit.client.0.smithi110.stdout:5/773: dwrite d4/d1d/d1f/d35/d3d/f58 [0,4194304] 0 2024-04-25T15:49:42.160 INFO:tasks.workunit.client.1.smithi193.stdout:5/532: write d17/d18/d3e/f3f [1104880,22119] 0 2024-04-25T15:49:42.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/566: mkdir d8/d30/d72/db2 0 2024-04-25T15:49:42.162 INFO:tasks.workunit.client.1.smithi193.stdout:3/567: chown d8/d13/d4b/d8b 102370822 1 2024-04-25T15:49:42.165 INFO:tasks.workunit.client.1.smithi193.stdout:6/523: rmdir dd/d12/d19/d45 39 2024-04-25T15:49:42.168 INFO:tasks.workunit.client.0.smithi110.stdout:5/774: creat d4/d1d/d1f/d35/d3d/dc8/ffd x:0 0 0 2024-04-25T15:49:42.169 INFO:tasks.workunit.client.1.smithi193.stdout:0/538: dread d0/d49/d5/d7/dd/d16/d26/d5e/f80 [0,4194304] 0 2024-04-25T15:49:42.170 INFO:tasks.workunit.client.1.smithi193.stdout:8/514: unlink d2/d3/d27/f96 0 2024-04-25T15:49:42.170 INFO:tasks.workunit.client.1.smithi193.stdout:1/451: dwrite dc/f2b [0,4194304] 0 2024-04-25T15:49:42.172 INFO:tasks.workunit.client.1.smithi193.stdout:2/521: link d0/d37/d25/d5f/d3a/f58 d0/d9/d3d/d7d/fd2 0 2024-04-25T15:49:42.175 INFO:tasks.workunit.client.1.smithi193.stdout:4/598: rmdir db/d32/d45 39 2024-04-25T15:49:42.175 INFO:tasks.workunit.client.1.smithi193.stdout:4/599: dread - db/d4b/fb2 zero size 2024-04-25T15:49:42.176 INFO:tasks.workunit.client.1.smithi193.stdout:4/600: write db/f6a [971434,32000] 0 2024-04-25T15:49:42.177 INFO:tasks.workunit.client.1.smithi193.stdout:4/601: chown db/d13/d1e/d1f/d22 441921 1 2024-04-25T15:49:42.177 INFO:tasks.workunit.client.1.smithi193.stdout:4/602: chown db/d13/f1d 1597640 1 2024-04-25T15:49:42.178 INFO:tasks.workunit.client.0.smithi110.stdout:6/718: dwrite d1/db/d22/d33/de4/fe7 [0,4194304] 0 2024-04-25T15:49:42.184 INFO:tasks.workunit.client.1.smithi193.stdout:5/533: symlink d17/d18/d30/d38/d48/d4f/lb2 0 2024-04-25T15:49:42.185 INFO:tasks.workunit.client.1.smithi193.stdout:5/534: fdatasync d17/d18/d45/f6d 0 2024-04-25T15:49:42.185 INFO:tasks.workunit.client.1.smithi193.stdout:7/506: write f8 [1823521,61841] 0 2024-04-25T15:49:42.186 INFO:tasks.workunit.client.0.smithi110.stdout:3/789: truncate d0/d7/f6c 3414362 0 2024-04-25T15:49:42.189 INFO:tasks.workunit.client.1.smithi193.stdout:6/524: mknod dd/d12/d2d/c9e 0 2024-04-25T15:49:42.197 INFO:tasks.workunit.client.0.smithi110.stdout:3/790: stat d0/d16/d8c/fea 0 2024-04-25T15:49:42.197 INFO:tasks.workunit.client.1.smithi193.stdout:8/515: mknod d2/d3/d27/d69/ca1 0 2024-04-25T15:49:42.199 INFO:tasks.workunit.client.1.smithi193.stdout:9/518: creat df/d2c/d2f/d4e/f9c x:0 0 0 2024-04-25T15:49:42.203 INFO:tasks.workunit.client.1.smithi193.stdout:4/603: creat db/d13/d1e/d1f/d56/d69/d70/d8a/fb8 x:0 0 0 2024-04-25T15:49:42.204 INFO:tasks.workunit.client.0.smithi110.stdout:5/775: dread d4/d1d/d1f/f42 [0,4194304] 0 2024-04-25T15:49:42.207 INFO:tasks.workunit.client.1.smithi193.stdout:5/535: symlink d17/d18/d30/d38/lb3 0 2024-04-25T15:49:42.207 INFO:tasks.workunit.client.0.smithi110.stdout:5/776: mknod d4/d1d/d1f/d35/d3d/cfe 0 2024-04-25T15:49:42.209 INFO:tasks.workunit.client.0.smithi110.stdout:3/791: link d0/d2e/d88/dbf/l97 d0/d7/d7d/l110 0 2024-04-25T15:49:42.210 INFO:tasks.workunit.client.0.smithi110.stdout:5/777: symlink d4/d15/d51/d90/lff 0 2024-04-25T15:49:42.212 INFO:tasks.workunit.client.0.smithi110.stdout:3/792: creat d0/d16/dcb/db0/f111 x:0 0 0 2024-04-25T15:49:42.213 INFO:tasks.workunit.client.0.smithi110.stdout:6/719: dread d1/db/d24/f29 [0,4194304] 0 2024-04-25T15:49:42.213 INFO:tasks.workunit.client.1.smithi193.stdout:0/539: getdents d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53 0 2024-04-25T15:49:42.214 INFO:tasks.workunit.client.0.smithi110.stdout:6/720: readlink d1/db/d24/d69/lbf 0 2024-04-25T15:49:42.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/452: creat dc/d4a/d4d/d80/f98 x:0 0 0 2024-04-25T15:49:42.216 INFO:tasks.workunit.client.0.smithi110.stdout:5/778: rename d4/d1d/d1f/d35/d3d/d52/dd2/fdd to d4/d1d/d1f/d35/d7f/f100 0 2024-04-25T15:49:42.217 INFO:tasks.workunit.client.1.smithi193.stdout:2/522: mkdir d0/d9/d15/d31/dd3 0 2024-04-25T15:49:42.217 INFO:tasks.workunit.client.1.smithi193.stdout:2/523: fdatasync d0/d37/f43 0 2024-04-25T15:49:42.220 INFO:tasks.workunit.client.1.smithi193.stdout:8/516: mkdir d2/d3/d20/d45/d4a/d63/da2 0 2024-04-25T15:49:42.222 INFO:tasks.workunit.client.0.smithi110.stdout:6/721: rename d1/dd/d3c/d91/d8d/dce/l46 to d1/dd/db5/df4/lff 0 2024-04-25T15:49:42.223 INFO:tasks.workunit.client.0.smithi110.stdout:6/722: chown d1/db/d24/d69/d75/fef 31 1 2024-04-25T15:49:42.225 INFO:tasks.workunit.client.0.smithi110.stdout:5/779: creat d4/d1d/d1f/d35/d7f/d92/d9b/f101 x:0 0 0 2024-04-25T15:49:42.227 INFO:tasks.workunit.client.1.smithi193.stdout:5/536: symlink d17/d53/lb4 0 2024-04-25T15:49:42.227 INFO:tasks.workunit.client.1.smithi193.stdout:7/507: mkdir d12/d31/d37/d92/dbf 0 2024-04-25T15:49:42.230 INFO:tasks.workunit.client.1.smithi193.stdout:3/568: dwrite d8/d30/d72/fa3 [0,4194304] 0 2024-04-25T15:49:42.232 INFO:tasks.workunit.client.0.smithi110.stdout:6/723: rename d1/db/d22/d90/d76/lda to d1/db/l100 0 2024-04-25T15:49:42.232 INFO:tasks.workunit.client.1.smithi193.stdout:2/524: rmdir d0 39 2024-04-25T15:49:42.233 INFO:tasks.workunit.client.1.smithi193.stdout:8/517: mknod d2/d3/d3a/ca3 0 2024-04-25T15:49:42.235 INFO:tasks.workunit.client.0.smithi110.stdout:5/780: getdents d4/d1d/d1f/d56 0 2024-04-25T15:49:42.237 INFO:tasks.workunit.client.0.smithi110.stdout:5/781: truncate d4/d1d/d1f/d35/d7f/f100 843381 0 2024-04-25T15:49:42.237 INFO:tasks.workunit.client.1.smithi193.stdout:5/537: rmdir d17/d18/d75/d85/d5d 39 2024-04-25T15:49:42.243 INFO:tasks.workunit.client.0.smithi110.stdout:6/724: link d1/db/d24/d2d/fba d1/dd/d3c/d91/d8d/f101 0 2024-04-25T15:49:42.244 INFO:tasks.workunit.client.0.smithi110.stdout:5/782: creat d4/d15/f102 x:0 0 0 2024-04-25T15:49:42.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/525: dread d0/d42/d51/d4f/f97 [0,4194304] 0 2024-04-25T15:49:42.248 INFO:tasks.workunit.client.0.smithi110.stdout:6/725: rename d1/db/d24/d69 to d1/dd/d3c/d91/d8d/dce/d102 0 2024-04-25T15:49:42.249 INFO:tasks.workunit.client.1.smithi193.stdout:5/538: fdatasync d17/d18/d30/d38/d48/d4f/f5f 0 2024-04-25T15:49:42.250 INFO:tasks.workunit.client.1.smithi193.stdout:1/453: creat dc/d4a/d4d/f99 x:0 0 0 2024-04-25T15:49:42.251 INFO:tasks.workunit.client.0.smithi110.stdout:6/726: chown d1/db/d24/d2d/fba 0 1 2024-04-25T15:49:42.252 INFO:tasks.workunit.client.1.smithi193.stdout:8/518: creat d2/d3/d20/d45/d4a/d63/da2/fa4 x:0 0 0 2024-04-25T15:49:42.253 INFO:tasks.workunit.client.0.smithi110.stdout:5/783: dread d4/d1d/d1f/f8f [0,4194304] 0 2024-04-25T15:49:42.256 INFO:tasks.workunit.client.1.smithi193.stdout:2/526: mkdir d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4 0 2024-04-25T15:49:42.259 INFO:tasks.workunit.client.0.smithi110.stdout:6/727: mknod d1/db/d24/c103 0 2024-04-25T15:49:42.265 INFO:tasks.workunit.client.1.smithi193.stdout:5/539: dwrite d17/d18/d30/d3b/f60 [0,4194304] 0 2024-04-25T15:49:42.265 INFO:tasks.workunit.client.0.smithi110.stdout:6/728: creat d1/db/f104 x:0 0 0 2024-04-25T15:49:42.266 INFO:tasks.workunit.client.1.smithi193.stdout:1/454: mknod dc/d4a/d50/d5c/c9a 0 2024-04-25T15:49:42.267 INFO:tasks.workunit.client.0.smithi110.stdout:3/793: dwrite d0/d16/d8c/fd6 [0,4194304] 0 2024-04-25T15:49:42.268 INFO:tasks.workunit.client.1.smithi193.stdout:2/527: dread d0/df/dbf/f9e [0,4194304] 0 2024-04-25T15:49:42.268 INFO:tasks.workunit.client.1.smithi193.stdout:8/519: mkdir d2/da5 0 2024-04-25T15:49:42.271 INFO:tasks.workunit.client.1.smithi193.stdout:8/520: write d2/d3/d27/d6f/d94/f2b [957866,124233] 0 2024-04-25T15:49:42.272 INFO:tasks.workunit.client.1.smithi193.stdout:2/528: creat d0/df/dbf/fd5 x:0 0 0 2024-04-25T15:49:42.274 INFO:tasks.workunit.client.1.smithi193.stdout:1/455: dwrite dc/d4a/d50/d5c/d5d/d3d/f95 [0,4194304] 0 2024-04-25T15:49:42.275 INFO:tasks.workunit.client.1.smithi193.stdout:5/540: creat d17/d18/d75/d85/d5d/dad/fb5 x:0 0 0 2024-04-25T15:49:42.279 INFO:tasks.workunit.client.1.smithi193.stdout:8/521: mkdir d2/d3/d20/d8c/da6 0 2024-04-25T15:49:42.280 INFO:tasks.workunit.client.1.smithi193.stdout:2/529: creat d0/d37/d25/d5f/d3a/d6d/d6e/db9/fd6 x:0 0 0 2024-04-25T15:49:42.282 INFO:tasks.workunit.client.1.smithi193.stdout:1/456: mknod dc/d4a/d50/d5c/c9b 0 2024-04-25T15:49:42.284 INFO:tasks.workunit.client.1.smithi193.stdout:5/541: mkdir d17/d18/d3e/db6 0 2024-04-25T15:49:42.285 INFO:tasks.workunit.client.1.smithi193.stdout:2/530: symlink d0/d37/d25/d5f/d3a/d6d/d6e/db9/ld7 0 2024-04-25T15:49:42.285 INFO:tasks.workunit.client.1.smithi193.stdout:8/522: creat d2/da5/fa7 x:0 0 0 2024-04-25T15:49:42.286 INFO:tasks.workunit.client.1.smithi193.stdout:1/457: truncate dc/d59/f65 23790 0 2024-04-25T15:49:42.287 INFO:tasks.workunit.client.1.smithi193.stdout:5/542: mknod d17/d18/d75/cb7 0 2024-04-25T15:49:42.289 INFO:tasks.workunit.client.1.smithi193.stdout:8/523: symlink d2/d3/d3a/la8 0 2024-04-25T15:49:42.292 INFO:tasks.workunit.client.1.smithi193.stdout:5/543: dwrite d17/d18/d30/f76 [0,4194304] 0 2024-04-25T15:49:42.298 INFO:tasks.workunit.client.1.smithi193.stdout:5/544: dwrite d17/d18/d30/d34/f6c [0,4194304] 0 2024-04-25T15:49:42.298 INFO:tasks.workunit.client.1.smithi193.stdout:2/531: link d0/d37/d25/fa9 d0/d37/d4d/dbe/fd8 0 2024-04-25T15:49:42.299 INFO:tasks.workunit.client.1.smithi193.stdout:0/540: sync 2024-04-25T15:49:42.305 INFO:tasks.workunit.client.1.smithi193.stdout:1/458: getdents dc/d4a/d50/d5c/d63 0 2024-04-25T15:49:42.307 INFO:tasks.workunit.client.1.smithi193.stdout:5/545: link d17/d18/d30/f8a d17/d18/d30/d38/d48/d4f/fb8 0 2024-04-25T15:49:42.308 INFO:tasks.workunit.client.1.smithi193.stdout:5/546: dread - d17/d18/d75/d85/d5d/fb0 zero size 2024-04-25T15:49:42.309 INFO:tasks.workunit.client.1.smithi193.stdout:8/524: getdents d2/d18 0 2024-04-25T15:49:42.310 INFO:tasks.workunit.client.1.smithi193.stdout:5/547: chown d17/d18/fa8 225 1 2024-04-25T15:49:42.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/532: rename d0/d9/fc1 to d0/d9/d15/d31/fd9 0 2024-04-25T15:49:42.312 INFO:tasks.workunit.client.1.smithi193.stdout:1/459: dwrite dc/d4a/d50/d5c/d5d/d3d/d55/f73 [0,4194304] 0 2024-04-25T15:49:42.314 INFO:tasks.workunit.client.1.smithi193.stdout:5/548: unlink d17/d83/f8b 0 2024-04-25T15:49:42.316 INFO:tasks.workunit.client.1.smithi193.stdout:2/533: write d0/df/dbf/fbc [1544241,46883] 0 2024-04-25T15:49:42.320 INFO:tasks.workunit.client.1.smithi193.stdout:8/525: dwrite d2/d3/d27/f7a [0,4194304] 0 2024-04-25T15:49:42.320 INFO:tasks.workunit.client.1.smithi193.stdout:1/460: read dc/f1e [107741,61035] 0 2024-04-25T15:49:42.320 INFO:tasks.workunit.client.1.smithi193.stdout:2/534: read d0/d42/d51/f40 [5847626,127747] 0 2024-04-25T15:49:42.323 INFO:tasks.workunit.client.1.smithi193.stdout:5/549: creat d17/d18/d30/d34/d52/daf/fb9 x:0 0 0 2024-04-25T15:49:42.323 INFO:tasks.workunit.client.1.smithi193.stdout:5/550: chown d17 10962 1 2024-04-25T15:49:42.326 INFO:tasks.workunit.client.1.smithi193.stdout:5/551: creat d17/d83/fba x:0 0 0 2024-04-25T15:49:42.327 INFO:tasks.workunit.client.1.smithi193.stdout:8/526: symlink d2/d3/d20/d8c/da6/la9 0 2024-04-25T15:49:42.328 INFO:tasks.workunit.client.1.smithi193.stdout:1/461: dwrite dc/d4a/d50/d5c/f91 [0,4194304] 0 2024-04-25T15:49:42.329 INFO:tasks.workunit.client.1.smithi193.stdout:1/462: stat dc/d4a/d50/f53 0 2024-04-25T15:49:42.329 INFO:tasks.workunit.client.1.smithi193.stdout:8/527: write d2/d3/d3a/f76 [4349996,109955] 0 2024-04-25T15:49:42.332 INFO:tasks.workunit.client.1.smithi193.stdout:8/528: write d2/d3/d27/d69/d86/f84 [881427,123053] 0 2024-04-25T15:49:42.335 INFO:tasks.workunit.client.1.smithi193.stdout:8/529: symlink d2/d3/d27/d6f/laa 0 2024-04-25T15:49:42.339 INFO:tasks.workunit.client.1.smithi193.stdout:8/530: creat d2/fab x:0 0 0 2024-04-25T15:49:42.340 INFO:tasks.workunit.client.1.smithi193.stdout:8/531: rmdir d2/d3/d20 39 2024-04-25T15:49:42.369 INFO:tasks.workunit.client.1.smithi193.stdout:0/541: read d0/d49/d5/d7/dd/d46/f4d [133283,93910] 0 2024-04-25T15:49:42.373 INFO:tasks.workunit.client.1.smithi193.stdout:0/542: truncate d0/d49/d5/d7/dd/d16/d26/fb0 318130 0 2024-04-25T15:49:42.374 INFO:tasks.workunit.client.1.smithi193.stdout:0/543: readlink d0/d49/d5/d7/dd/l7b 0 2024-04-25T15:49:42.376 INFO:tasks.workunit.client.1.smithi193.stdout:6/525: truncate dd/d2f/f5b 600589 0 2024-04-25T15:49:42.376 INFO:tasks.workunit.client.1.smithi193.stdout:9/519: dwrite df/d2c/d2f/d4e/d1c/f76 [4194304,4194304] 0 2024-04-25T15:49:42.380 INFO:tasks.workunit.client.1.smithi193.stdout:6/526: creat dd/d12/d19/d1c/f9f x:0 0 0 2024-04-25T15:49:42.382 INFO:tasks.workunit.client.1.smithi193.stdout:7/508: write d12/d1c/f8a [3548452,78879] 0 2024-04-25T15:49:42.385 INFO:tasks.workunit.client.1.smithi193.stdout:6/527: chown dd/d12/d19/d1c/d40/d3c/d3f/d4f/c82 10 1 2024-04-25T15:49:42.386 INFO:tasks.workunit.client.1.smithi193.stdout:4/604: dwrite db/d13/d1e/d1f/d22/f6b [0,4194304] 0 2024-04-25T15:49:42.386 INFO:tasks.workunit.client.1.smithi193.stdout:6/528: dread - dd/d2f/d57/f90 zero size 2024-04-25T15:49:42.391 INFO:tasks.workunit.client.1.smithi193.stdout:9/520: rename df/l53 to df/d2c/d2f/d4e/l9d 0 2024-04-25T15:49:42.397 INFO:tasks.workunit.client.1.smithi193.stdout:0/544: link d0/d49/d5/d7/dd/d16/d26/f55 d0/d49/d5/d7/dd/d16/fb8 0 2024-04-25T15:49:42.398 INFO:tasks.workunit.client.0.smithi110.stdout:5/784: write d4/d1d/d1f/d35/f39 [5159701,66757] 0 2024-04-25T15:49:42.398 INFO:tasks.workunit.client.1.smithi193.stdout:6/529: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0 0 2024-04-25T15:49:42.398 INFO:tasks.workunit.client.1.smithi193.stdout:9/521: write df/d2c/d2f/d4e/f25 [3034158,126683] 0 2024-04-25T15:49:42.399 INFO:tasks.workunit.client.1.smithi193.stdout:7/509: link d12/d31/d39/d60/fa8 d12/d1c/d63/d70/d68/d8f/fc0 0 2024-04-25T15:49:42.399 INFO:tasks.workunit.client.1.smithi193.stdout:0/545: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d 3 1 2024-04-25T15:49:42.401 INFO:tasks.workunit.client.1.smithi193.stdout:3/569: dwrite d8/d30/d72/f7e [0,4194304] 0 2024-04-25T15:49:42.402 INFO:tasks.workunit.client.0.smithi110.stdout:5/785: truncate d4/d1d/d1f/f32 1053971 0 2024-04-25T15:49:42.402 INFO:tasks.workunit.client.1.smithi193.stdout:9/522: stat df/d2c/d2f/d4e/d69/d6b 0 2024-04-25T15:49:42.402 INFO:tasks.workunit.client.1.smithi193.stdout:4/605: dwrite db/d13/f21 [4194304,4194304] 0 2024-04-25T15:49:42.403 INFO:tasks.workunit.client.1.smithi193.stdout:7/510: fdatasync d12/d1c/f7e 0 2024-04-25T15:49:42.403 INFO:tasks.workunit.client.0.smithi110.stdout:5/786: read d4/d1d/d1f/d56/f7e [443538,22617] 0 2024-04-25T15:49:42.405 INFO:tasks.workunit.client.1.smithi193.stdout:3/570: dread - d8/d13/d4b/d8b/fab zero size 2024-04-25T15:49:42.410 INFO:tasks.workunit.client.1.smithi193.stdout:7/511: creat d12/da1/fc1 x:0 0 0 2024-04-25T15:49:42.412 INFO:tasks.workunit.client.1.smithi193.stdout:7/512: chown d12/d1c/da3 8576538 1 2024-04-25T15:49:42.412 INFO:tasks.workunit.client.0.smithi110.stdout:6/729: dwrite d1/db/d22/f38 [4194304,4194304] 0 2024-04-25T15:49:42.414 INFO:tasks.workunit.client.1.smithi193.stdout:0/546: dwrite d0/d49/d5/d7/dd/d46/d48/d62/f92 [4194304,4194304] 0 2024-04-25T15:49:42.415 INFO:tasks.workunit.client.1.smithi193.stdout:9/523: dwrite df/d2c/d34/d3e/f4a [0,4194304] 0 2024-04-25T15:49:42.417 INFO:tasks.workunit.client.1.smithi193.stdout:3/571: mkdir d8/d13/d14/d27/d54/db3 0 2024-04-25T15:49:42.419 INFO:tasks.workunit.client.0.smithi110.stdout:6/730: truncate d1/dd/d3c/d91/f8b 608267 0 2024-04-25T15:49:42.426 INFO:tasks.workunit.client.1.smithi193.stdout:0/547: creat d0/d49/d5/d7/dd/d16/d26/d2e/d93/fb9 x:0 0 0 2024-04-25T15:49:42.427 INFO:tasks.workunit.client.1.smithi193.stdout:9/524: dwrite df/d2d/f39 [0,4194304] 0 2024-04-25T15:49:42.431 INFO:tasks.workunit.client.1.smithi193.stdout:9/525: truncate df/d2d/f61 2695 0 2024-04-25T15:49:42.431 INFO:tasks.workunit.client.1.smithi193.stdout:7/513: creat d12/d31/d37/d92/dbf/fc2 x:0 0 0 2024-04-25T15:49:42.431 INFO:tasks.workunit.client.1.smithi193.stdout:4/606: dread db/d4b/f7b [0,4194304] 0 2024-04-25T15:49:42.433 INFO:tasks.workunit.client.1.smithi193.stdout:0/548: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/lba 0 2024-04-25T15:49:42.440 INFO:tasks.workunit.client.1.smithi193.stdout:9/526: mknod df/d2c/d2f/d4e/d1c/d30/c9e 0 2024-04-25T15:49:42.441 INFO:tasks.workunit.client.1.smithi193.stdout:9/527: fsync df/d2c/d2f/d33/f3c 0 2024-04-25T15:49:42.444 INFO:tasks.workunit.client.0.smithi110.stdout:5/787: dread d4/d1d/d1f/f47 [0,4194304] 0 2024-04-25T15:49:42.444 INFO:tasks.workunit.client.1.smithi193.stdout:9/528: symlink df/d2c/d2f/d4e/d1c/l9f 0 2024-04-25T15:49:42.445 INFO:tasks.workunit.client.0.smithi110.stdout:1/924: sync 2024-04-25T15:49:42.446 INFO:tasks.workunit.client.1.smithi193.stdout:0/549: rename d0/d49/d5/d7/dd/d46/d48/laf to d0/d49/d5/d7/dd/lbb 0 2024-04-25T15:49:42.447 INFO:tasks.workunit.client.0.smithi110.stdout:5/788: creat d4/d1d/d1f/d35/d3d/d52/dd2/f103 x:0 0 0 2024-04-25T15:49:42.450 INFO:tasks.workunit.client.1.smithi193.stdout:0/550: write d0/d49/d5/d7/dd/d16/d18/d28/d56/f70 [1088234,6376] 0 2024-04-25T15:49:42.451 INFO:tasks.workunit.client.0.smithi110.stdout:5/789: symlink d4/d1d/d1f/d35/d3d/d52/d75/l104 0 2024-04-25T15:49:42.456 INFO:tasks.workunit.client.1.smithi193.stdout:9/529: link df/d2c/d34/d3e/d4d/d5e/d81/d41/f59 df/d2c/d2f/d4e/fa0 0 2024-04-25T15:49:42.456 INFO:tasks.workunit.client.1.smithi193.stdout:0/551: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/fa8 [0,4194304] 0 2024-04-25T15:49:42.459 INFO:tasks.workunit.client.0.smithi110.stdout:1/925: link d0/d14/d81/df8/cbc d0/d51/dd8/df0/c12a 0 2024-04-25T15:49:42.461 INFO:tasks.workunit.client.1.smithi193.stdout:0/552: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/fbc x:0 0 0 2024-04-25T15:49:42.464 INFO:tasks.workunit.client.0.smithi110.stdout:1/926: creat d0/d14/d5f/d7d/d72/d88/db6/f12b x:0 0 0 2024-04-25T15:49:42.465 INFO:tasks.workunit.client.1.smithi193.stdout:2/535: chown d0/d9/d15/c64 88787 1 2024-04-25T15:49:42.467 INFO:tasks.workunit.client.0.smithi110.stdout:5/790: truncate d4/f10 1236414 0 2024-04-25T15:49:42.469 INFO:tasks.workunit.client.1.smithi193.stdout:0/553: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/lbd 0 2024-04-25T15:49:42.470 INFO:tasks.workunit.client.0.smithi110.stdout:5/791: write d4/d15/d51/d90/fa3 [790939,21160] 0 2024-04-25T15:49:42.472 INFO:tasks.workunit.client.0.smithi110.stdout:1/927: dwrite d0/d14/d5f/d7d/fbd [0,4194304] 0 2024-04-25T15:49:42.473 INFO:tasks.workunit.client.1.smithi193.stdout:2/536: dread d0/d9/d15/f78 [0,4194304] 0 2024-04-25T15:49:42.474 INFO:tasks.workunit.client.1.smithi193.stdout:0/554: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/lbe 0 2024-04-25T15:49:42.475 INFO:tasks.workunit.client.1.smithi193.stdout:0/555: chown d0/d49/d5/d7/l44 437972783 1 2024-04-25T15:49:42.476 INFO:tasks.workunit.client.0.smithi110.stdout:1/928: dread d0/d14/d5f/d7d/f67 [0,4194304] 0 2024-04-25T15:49:42.480 INFO:tasks.workunit.client.0.smithi110.stdout:5/792: mkdir d4/d1d/d1f/d35/d3d/d52/d75/dfa/d105 0 2024-04-25T15:49:42.482 INFO:tasks.workunit.client.1.smithi193.stdout:0/556: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf 0 2024-04-25T15:49:42.483 INFO:tasks.workunit.client.0.smithi110.stdout:1/929: creat d0/d51/d6b/df1/f12c x:0 0 0 2024-04-25T15:49:42.483 INFO:tasks.workunit.client.1.smithi193.stdout:0/557: read d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f59 [2136216,21461] 0 2024-04-25T15:49:42.484 INFO:tasks.workunit.client.1.smithi193.stdout:0/558: readlink d0/d49/d5/d7/dd/l3b 0 2024-04-25T15:49:42.484 INFO:tasks.workunit.client.1.smithi193.stdout:2/537: mkdir d0/d37/d25/dda 0 2024-04-25T15:49:42.486 INFO:tasks.workunit.client.0.smithi110.stdout:5/793: mkdir d4/d15/d51/d90/ded/d106 0 2024-04-25T15:49:42.487 INFO:tasks.workunit.client.0.smithi110.stdout:1/930: rmdir d0/d14/d5f/d7d/d72/d61 39 2024-04-25T15:49:42.490 INFO:tasks.workunit.client.1.smithi193.stdout:0/559: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/d99/f9a [0,4194304] 0 2024-04-25T15:49:42.491 INFO:tasks.workunit.client.1.smithi193.stdout:2/538: mknod d0/d42/d51/d4f/d93/cdb 0 2024-04-25T15:49:42.492 INFO:tasks.workunit.client.0.smithi110.stdout:5/794: symlink d4/d1d/d1f/d35/l107 0 2024-04-25T15:49:42.496 INFO:tasks.workunit.client.1.smithi193.stdout:2/539: rmdir d0/d37/d25/d5f/d3a/db8 39 2024-04-25T15:49:42.497 INFO:tasks.workunit.client.1.smithi193.stdout:5/552: write d17/f2a [4284120,70659] 0 2024-04-25T15:49:42.500 INFO:tasks.workunit.client.1.smithi193.stdout:0/560: dwrite d0/d49/d5/d7/dd/d46/d48/f85 [4194304,4194304] 0 2024-04-25T15:49:42.500 INFO:tasks.workunit.client.1.smithi193.stdout:1/463: truncate dc/d4a/d4d/f57 1007074 0 2024-04-25T15:49:42.500 INFO:tasks.workunit.client.1.smithi193.stdout:5/553: mknod d17/d18/d30/d38/d71/cbb 0 2024-04-25T15:49:42.501 INFO:tasks.workunit.client.1.smithi193.stdout:0/561: write d0/d49/faa [22736,5265] 0 2024-04-25T15:49:42.502 INFO:tasks.workunit.client.1.smithi193.stdout:1/464: dread dc/f15 [0,4194304] 0 2024-04-25T15:49:42.503 INFO:tasks.workunit.client.1.smithi193.stdout:8/532: truncate d2/f11 4625811 0 2024-04-25T15:49:42.505 INFO:tasks.workunit.client.1.smithi193.stdout:5/554: creat d17/d18/d30/d38/fbc x:0 0 0 2024-04-25T15:49:42.506 INFO:tasks.workunit.client.1.smithi193.stdout:1/465: rmdir dc/d4a/d50/d5c/d5d 39 2024-04-25T15:49:42.506 INFO:tasks.workunit.client.1.smithi193.stdout:5/555: read - d17/d18/fa8 zero size 2024-04-25T15:49:42.508 INFO:tasks.workunit.client.1.smithi193.stdout:9/530: getdents df 0 2024-04-25T15:49:42.509 INFO:tasks.workunit.client.1.smithi193.stdout:5/556: dread d17/d18/d30/d34/f6c [4194304,4194304] 0 2024-04-25T15:49:42.510 INFO:tasks.workunit.client.1.smithi193.stdout:9/531: chown df/d2c/d34/d3e/d4d/d5e/d81/f40 97190810 1 2024-04-25T15:49:42.514 INFO:tasks.workunit.client.1.smithi193.stdout:1/466: write dc/d4a/d50/d5c/d5d/d3d/f41 [760505,32392] 0 2024-04-25T15:49:42.514 INFO:tasks.workunit.client.1.smithi193.stdout:2/540: link d0/d37/d25/d5f/d3a/d6d/d85/lcb d0/d37/d25/d5f/d3a/ldc 0 2024-04-25T15:49:42.515 INFO:tasks.workunit.client.1.smithi193.stdout:6/530: write dd/d12/d19/d1c/f71 [4592551,105110] 0 2024-04-25T15:49:42.516 INFO:tasks.workunit.client.1.smithi193.stdout:9/532: mknod df/d2c/d2f/d4e/d69/ca1 0 2024-04-25T15:49:42.518 INFO:tasks.workunit.client.1.smithi193.stdout:8/533: dwrite d2/d3/d20/d45/d4a/d63/da2/fa4 [0,4194304] 0 2024-04-25T15:49:42.519 INFO:tasks.workunit.client.1.smithi193.stdout:6/531: write dd/f1b [5188633,101538] 0 2024-04-25T15:49:42.520 INFO:tasks.workunit.client.1.smithi193.stdout:8/534: chown d2/d3/d27/l64 0 1 2024-04-25T15:49:42.522 INFO:tasks.workunit.client.1.smithi193.stdout:2/541: creat d0/d9/d15/d31/d90/d9c/fdd x:0 0 0 2024-04-25T15:49:42.528 INFO:tasks.workunit.client.1.smithi193.stdout:8/535: creat d2/d3/d20/d45/d29/fac x:0 0 0 2024-04-25T15:49:42.530 INFO:tasks.workunit.client.1.smithi193.stdout:5/557: getdents d17/d83 0 2024-04-25T15:49:42.531 INFO:tasks.workunit.client.1.smithi193.stdout:7/514: rmdir d12/d31/d37 39 2024-04-25T15:49:42.532 INFO:tasks.workunit.client.0.smithi110.stdout:6/731: dwrite d1/d12/f49 [0,4194304] 0 2024-04-25T15:49:42.533 INFO:tasks.workunit.client.1.smithi193.stdout:4/607: write db/d32/f94 [658167,97500] 0 2024-04-25T15:49:42.533 INFO:tasks.workunit.client.0.smithi110.stdout:6/732: chown d1/d79/fe9 1027129 1 2024-04-25T15:49:42.534 INFO:tasks.workunit.client.1.smithi193.stdout:3/572: dwrite d8/d4d/f62 [0,4194304] 0 2024-04-25T15:49:42.535 INFO:tasks.workunit.client.1.smithi193.stdout:4/608: chown f1 89 1 2024-04-25T15:49:42.537 INFO:tasks.workunit.client.1.smithi193.stdout:9/533: mkdir df/d2c/d2f/d5c/d7e/da2 0 2024-04-25T15:49:42.540 INFO:tasks.workunit.client.0.smithi110.stdout:6/733: mknod d1/db/d24/d2d/c105 0 2024-04-25T15:49:42.542 INFO:tasks.workunit.client.1.smithi193.stdout:5/558: dwrite d17/d18/d45/f6d [0,4194304] 0 2024-04-25T15:49:42.544 INFO:tasks.workunit.client.1.smithi193.stdout:9/534: write df/d2c/d34/d3e/d4d/d5e/d81/d41/f70 [1319273,95544] 0 2024-04-25T15:49:42.544 INFO:tasks.workunit.client.1.smithi193.stdout:2/542: link d0/df/dbf/fd5 d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4/fde 0 2024-04-25T15:49:42.546 INFO:tasks.workunit.client.1.smithi193.stdout:3/573: dread d8/d4d/f62 [0,4194304] 0 2024-04-25T15:49:42.547 INFO:tasks.workunit.client.1.smithi193.stdout:1/467: getdents dc/d4a 0 2024-04-25T15:49:42.549 INFO:tasks.workunit.client.0.smithi110.stdout:6/734: mknod d1/d12/db8/c106 0 2024-04-25T15:49:42.554 INFO:tasks.workunit.client.1.smithi193.stdout:4/609: dread - db/d32/f6e zero size 2024-04-25T15:49:42.560 INFO:tasks.workunit.client.0.smithi110.stdout:6/735: getdents d1/dd/d3c/df8 0 2024-04-25T15:49:42.561 INFO:tasks.workunit.client.1.smithi193.stdout:1/468: dread dc/f19 [0,4194304] 0 2024-04-25T15:49:42.562 INFO:tasks.workunit.client.1.smithi193.stdout:7/515: mkdir d12/d31/dc3 0 2024-04-25T15:49:42.566 INFO:tasks.workunit.client.1.smithi193.stdout:5/559: mknod d17/d18/d30/d38/d48/d4f/cbd 0 2024-04-25T15:49:42.566 INFO:tasks.workunit.client.1.smithi193.stdout:5/560: truncate d17/d18/d30/d3b/fa1 1509170 0 2024-04-25T15:49:42.566 INFO:tasks.workunit.client.1.smithi193.stdout:5/561: chown d17/d18/d45 275639 1 2024-04-25T15:49:42.567 INFO:tasks.workunit.client.1.smithi193.stdout:8/536: truncate d2/d3/d20/d45/d4a/d53/f58 3593918 0 2024-04-25T15:49:42.567 INFO:tasks.workunit.client.0.smithi110.stdout:6/736: rmdir d1/db/d24/daa 39 2024-04-25T15:49:42.569 INFO:tasks.workunit.client.1.smithi193.stdout:3/574: creat d8/fb4 x:0 0 0 2024-04-25T15:49:42.570 INFO:tasks.workunit.client.1.smithi193.stdout:2/543: symlink d0/d9/d15/d31/d90/d9c/ldf 0 2024-04-25T15:49:42.572 INFO:tasks.workunit.client.0.smithi110.stdout:6/737: rename d1/dd/d3c/d91/d8d/f99 to d1/db/d22/d33/d63/dc2/f107 0 2024-04-25T15:49:42.573 INFO:tasks.workunit.client.1.smithi193.stdout:4/610: symlink db/d4b/d51/d54/d77/lb9 0 2024-04-25T15:49:42.575 INFO:tasks.workunit.client.1.smithi193.stdout:4/611: read db/d13/d1e/f41 [2072816,38020] 0 2024-04-25T15:49:42.577 INFO:tasks.workunit.client.0.smithi110.stdout:5/795: dwrite d4/fb2 [0,4194304] 0 2024-04-25T15:49:42.578 INFO:tasks.workunit.client.1.smithi193.stdout:7/516: read d12/f52 [2282417,61889] 0 2024-04-25T15:49:42.579 INFO:tasks.workunit.client.0.smithi110.stdout:6/738: truncate d1/dd/f21 8651403 0 2024-04-25T15:49:42.579 INFO:tasks.workunit.client.1.smithi193.stdout:7/517: write d12/d31/d39/db0/fb5 [943283,98109] 0 2024-04-25T15:49:42.581 INFO:tasks.workunit.client.1.smithi193.stdout:1/469: mkdir dc/d4a/d50/d5c/d5d/d3d/d55/d9c 0 2024-04-25T15:49:42.583 INFO:tasks.workunit.client.1.smithi193.stdout:5/562: truncate d17/d18/d30/f35 3562815 0 2024-04-25T15:49:42.583 INFO:tasks.workunit.client.1.smithi193.stdout:0/562: write d0/d49/d5/d7/dd/d16/d26/d5e/f69 [2882748,104302] 0 2024-04-25T15:49:42.586 INFO:tasks.workunit.client.1.smithi193.stdout:0/563: write d0/d49/d5/d7/dd/d46/d48/fb4 [647505,6127] 0 2024-04-25T15:49:42.596 INFO:tasks.workunit.client.1.smithi193.stdout:1/470: mknod dc/d4a/d50/d7c/d7e/c9d 0 2024-04-25T15:49:42.596 INFO:tasks.workunit.client.1.smithi193.stdout:7/518: dread d12/d1c/d63/d70/f34 [0,4194304] 0 2024-04-25T15:49:42.597 INFO:tasks.workunit.client.1.smithi193.stdout:5/563: symlink d17/d83/lbe 0 2024-04-25T15:49:42.597 INFO:tasks.workunit.client.1.smithi193.stdout:1/471: dread - dc/d4a/d50/d5c/d5d/d3d/f83 zero size 2024-04-25T15:49:42.597 INFO:tasks.workunit.client.1.smithi193.stdout:7/519: readlink d12/d31/d39/l75 0 2024-04-25T15:49:42.600 INFO:tasks.workunit.client.1.smithi193.stdout:6/532: truncate dd/f3d 10579860 0 2024-04-25T15:49:42.601 INFO:tasks.workunit.client.1.smithi193.stdout:3/575: creat d8/d13/d14/da0/fb5 x:0 0 0 2024-04-25T15:49:42.602 INFO:tasks.workunit.client.1.smithi193.stdout:0/564: rename d0/d49/d6a/c9b to d0/d49/d5/d7/dd/d46/cc0 0 2024-04-25T15:49:42.603 INFO:tasks.workunit.client.1.smithi193.stdout:2/544: creat d0/d37/d25/dda/fe0 x:0 0 0 2024-04-25T15:49:42.606 INFO:tasks.workunit.client.1.smithi193.stdout:3/576: dwrite d8/d30/f98 [0,4194304] 0 2024-04-25T15:49:42.610 INFO:tasks.workunit.client.1.smithi193.stdout:1/472: truncate dc/f2d 2145215 0 2024-04-25T15:49:42.611 INFO:tasks.workunit.client.1.smithi193.stdout:0/565: chown d0/d49/d5/f24 37749 1 2024-04-25T15:49:42.613 INFO:tasks.workunit.client.1.smithi193.stdout:9/535: dwrite df/d2c/d2f/d4e/fa0 [0,4194304] 0 2024-04-25T15:49:42.615 INFO:tasks.workunit.client.1.smithi193.stdout:9/536: fdatasync df/d2c/d2f/d4e/d1c/d30/d60/f73 0 2024-04-25T15:49:42.617 INFO:tasks.workunit.client.0.smithi110.stdout:9/731: sync 2024-04-25T15:49:42.617 INFO:tasks.workunit.client.1.smithi193.stdout:4/612: rmdir db/d4b/d51/d95 0 2024-04-25T15:49:42.620 INFO:tasks.workunit.client.1.smithi193.stdout:3/577: unlink d8/cd 0 2024-04-25T15:49:42.621 INFO:tasks.workunit.client.1.smithi193.stdout:3/578: truncate d8/d13/d4b/f88 1570702 0 2024-04-25T15:49:42.622 INFO:tasks.workunit.client.1.smithi193.stdout:3/579: write d8/d30/f36 [5136633,106620] 0 2024-04-25T15:49:42.623 INFO:tasks.workunit.client.1.smithi193.stdout:3/580: write d8/d13/d14/d27/f95 [281263,30188] 0 2024-04-25T15:49:42.626 INFO:tasks.workunit.client.0.smithi110.stdout:9/732: truncate d9/db/df/d25/f51 1689658 0 2024-04-25T15:49:42.627 INFO:tasks.workunit.client.1.smithi193.stdout:0/566: dread d0/d49/d5/d7/dd/d16/f41 [0,4194304] 0 2024-04-25T15:49:42.629 INFO:tasks.workunit.client.1.smithi193.stdout:0/567: fdatasync d0/d49/d5/d7/dd/d46/d48/fb4 0 2024-04-25T15:49:42.633 INFO:tasks.workunit.client.1.smithi193.stdout:2/545: unlink d0/d9/f88 0 2024-04-25T15:49:42.633 INFO:tasks.workunit.client.0.smithi110.stdout:6/739: dwrite d1/db/d24/daa/fdb [0,4194304] 0 2024-04-25T15:49:42.635 INFO:tasks.workunit.client.0.smithi110.stdout:9/733: creat d9/d13/d5b/d61/db3/db4/dc5/de4/fe8 x:0 0 0 2024-04-25T15:49:42.636 INFO:tasks.workunit.client.1.smithi193.stdout:0/568: dread d0/d49/d5/d7/dd/d16/d1a/f8e [4194304,4194304] 0 2024-04-25T15:49:42.637 INFO:tasks.workunit.client.1.smithi193.stdout:8/537: truncate d2/d3/d3a/f61 1017597 0 2024-04-25T15:49:42.640 INFO:tasks.workunit.client.1.smithi193.stdout:9/537: mkdir df/d2c/d2f/d5c/da3 0 2024-04-25T15:49:42.641 INFO:tasks.workunit.client.0.smithi110.stdout:6/740: read d1/dd/d3c/d91/f8b [133942,46306] 0 2024-04-25T15:49:42.642 INFO:tasks.workunit.client.0.smithi110.stdout:2/706: sync 2024-04-25T15:49:42.642 INFO:tasks.workunit.client.0.smithi110.stdout:3/794: sync 2024-04-25T15:49:42.642 INFO:tasks.workunit.client.0.smithi110.stdout:0/736: sync 2024-04-25T15:49:42.642 INFO:tasks.workunit.client.0.smithi110.stdout:7/816: sync 2024-04-25T15:49:42.642 INFO:tasks.workunit.client.0.smithi110.stdout:8/866: sync 2024-04-25T15:49:42.642 INFO:tasks.workunit.client.0.smithi110.stdout:4/766: sync 2024-04-25T15:49:42.642 INFO:tasks.workunit.client.0.smithi110.stdout:1/931: sync 2024-04-25T15:49:42.645 INFO:tasks.workunit.client.0.smithi110.stdout:9/734: rmdir d9/d44/d5e 39 2024-04-25T15:49:42.646 INFO:tasks.workunit.client.0.smithi110.stdout:6/741: chown d1/dd/d3c/d91/d8d/f101 0 1 2024-04-25T15:49:42.648 INFO:tasks.workunit.client.1.smithi193.stdout:5/564: creat d17/d18/d75/fbf x:0 0 0 2024-04-25T15:49:42.651 INFO:tasks.workunit.client.0.smithi110.stdout:4/767: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/f108 x:0 0 0 2024-04-25T15:49:42.653 INFO:tasks.workunit.client.1.smithi193.stdout:5/565: dwrite d17/d18/d30/d38/fbc [0,4194304] 0 2024-04-25T15:49:42.658 INFO:tasks.workunit.client.0.smithi110.stdout:4/768: dwrite d13/d24/d2d/d44/fb4 [0,4194304] 0 2024-04-25T15:49:42.658 INFO:tasks.workunit.client.0.smithi110.stdout:3/795: unlink d0/d16/dcb/d50/f6d 0 2024-04-25T15:49:42.660 INFO:tasks.workunit.client.1.smithi193.stdout:6/533: creat dd/d2f/fa1 x:0 0 0 2024-04-25T15:49:42.663 INFO:tasks.workunit.client.1.smithi193.stdout:3/581: dwrite d8/d13/d1f/d69/d77/f53 [8388608,4194304] 0 2024-04-25T15:49:42.664 INFO:tasks.workunit.client.0.smithi110.stdout:7/817: unlink d1b/d29/d38/d8b/fd7 0 2024-04-25T15:49:42.665 INFO:tasks.workunit.client.1.smithi193.stdout:1/473: write dc/d4a/d50/d5c/d63/f67 [571912,29414] 0 2024-04-25T15:49:42.667 INFO:tasks.workunit.client.0.smithi110.stdout:2/707: creat de/d40/d5b/d5c/d5f/db2/da9/dac/fe1 x:0 0 0 2024-04-25T15:49:42.670 INFO:tasks.workunit.client.0.smithi110.stdout:5/796: truncate d4/d2c/d2a/f46 1767307 0 2024-04-25T15:49:42.674 INFO:tasks.workunit.client.0.smithi110.stdout:9/735: mknod d9/d13/d5b/d61/db3/db4/dc5/de4/ce9 0 2024-04-25T15:49:42.680 INFO:tasks.workunit.client.0.smithi110.stdout:0/737: mknod d1/d20/c101 0 2024-04-25T15:49:42.681 INFO:tasks.workunit.client.0.smithi110.stdout:0/738: fsync d1/d23/d30/d33/d5b/d72/d77/fcc 0 2024-04-25T15:49:42.681 INFO:tasks.workunit.client.1.smithi193.stdout:0/569: dread d0/d49/d5/d7/dd/d46/d48/d62/f8f [0,4194304] 0 2024-04-25T15:49:42.684 INFO:tasks.workunit.client.1.smithi193.stdout:9/538: rename df/d2c/d2f/d5c/f6d to df/d2c/d2f/d4e/d1c/d30/d60/fa4 0 2024-04-25T15:49:42.688 INFO:tasks.workunit.client.0.smithi110.stdout:3/796: rename d0/d16/dcb/f5f to d0/d16/dcb/d2f/d4c/f112 0 2024-04-25T15:49:42.688 INFO:tasks.workunit.client.0.smithi110.stdout:1/932: write d0/d14/f62 [2890651,78691] 0 2024-04-25T15:49:42.689 INFO:tasks.workunit.client.1.smithi193.stdout:7/520: getdents d12/d1c/d63 0 2024-04-25T15:49:42.689 INFO:tasks.workunit.client.0.smithi110.stdout:1/933: chown d0/d14/d5f/d7d/d72/c96 586869488 1 2024-04-25T15:49:42.694 INFO:tasks.workunit.client.1.smithi193.stdout:6/534: unlink dd/d12/d2d/l89 0 2024-04-25T15:49:42.695 INFO:tasks.workunit.client.0.smithi110.stdout:9/736: rmdir d9/db/df/d2c 39 2024-04-25T15:49:42.696 INFO:tasks.workunit.client.1.smithi193.stdout:5/566: dwrite d17/d18/d75/f9c [0,4194304] 0 2024-04-25T15:49:42.700 INFO:tasks.workunit.client.1.smithi193.stdout:1/474: creat dc/d4a/d50/d7c/f9e x:0 0 0 2024-04-25T15:49:42.700 INFO:tasks.workunit.client.1.smithi193.stdout:2/546: write d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4/fde [463805,13649] 0 2024-04-25T15:49:42.702 INFO:tasks.workunit.client.0.smithi110.stdout:0/739: mknod d1/d47/d54/c102 0 2024-04-25T15:49:42.702 INFO:tasks.workunit.client.0.smithi110.stdout:0/740: read - d1/d6c/f94 zero size 2024-04-25T15:49:42.704 INFO:tasks.workunit.client.0.smithi110.stdout:4/769: creat d13/d24/d2d/dd9/de1/f109 x:0 0 0 2024-04-25T15:49:42.708 INFO:tasks.workunit.client.0.smithi110.stdout:1/934: mkdir d0/d51/dc8/d12d 0 2024-04-25T15:49:42.709 INFO:tasks.workunit.client.1.smithi193.stdout:0/570: creat d0/d49/d5/d7/dd/d38/fc1 x:0 0 0 2024-04-25T15:49:42.710 INFO:tasks.workunit.client.0.smithi110.stdout:7/818: symlink d1b/d29/db5/dca/d10b/l10c 0 2024-04-25T15:49:42.711 INFO:tasks.workunit.client.1.smithi193.stdout:3/582: write d8/d30/f42 [2330538,28702] 0 2024-04-25T15:49:42.715 INFO:tasks.workunit.client.1.smithi193.stdout:4/613: creat db/d4b/fba x:0 0 0 2024-04-25T15:49:42.716 INFO:tasks.workunit.client.0.smithi110.stdout:6/742: dwrite d1/db/d24/f9c [0,4194304] 0 2024-04-25T15:49:42.717 INFO:tasks.workunit.client.1.smithi193.stdout:7/521: mkdir d12/d13/dc4 0 2024-04-25T15:49:42.719 INFO:tasks.workunit.client.0.smithi110.stdout:0/741: mknod d1/d23/d30/d33/d5b/dbb/c103 0 2024-04-25T15:49:42.720 INFO:tasks.workunit.client.0.smithi110.stdout:1/935: dread d0/d14/d5f/d7d/fc1 [0,4194304] 0 2024-04-25T15:49:42.722 INFO:tasks.workunit.client.0.smithi110.stdout:4/770: truncate ff 4473612 0 2024-04-25T15:49:42.726 INFO:tasks.workunit.client.0.smithi110.stdout:3/797: unlink d0/d2e/d88/dbf/d99/dad/daf/lcc 0 2024-04-25T15:49:42.729 INFO:tasks.workunit.client.1.smithi193.stdout:5/567: mknod d17/d18/d30/d38/cc0 0 2024-04-25T15:49:42.729 INFO:tasks.workunit.client.0.smithi110.stdout:4/771: dwrite d13/d24/d2d/d44/d65/f72 [0,4194304] 0 2024-04-25T15:49:42.731 INFO:tasks.workunit.client.0.smithi110.stdout:7/819: mknod d1b/d29/d38/d55/dab/db2/c10d 0 2024-04-25T15:49:42.735 INFO:tasks.workunit.client.1.smithi193.stdout:1/475: mknod dc/d4a/d4d/d7a/d85/c9f 0 2024-04-25T15:49:42.738 INFO:tasks.workunit.client.1.smithi193.stdout:8/538: rmdir d2/d3/d3a/d95 0 2024-04-25T15:49:42.739 INFO:tasks.workunit.client.0.smithi110.stdout:9/737: truncate d9/d13/f8e 3930345 0 2024-04-25T15:49:42.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:42 smithi193 ceph-mon[33747]: pgmap v38: 65 pgs: 65 active+clean; 2.6 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 63 MiB/s rd, 210 MiB/s wr, 374 op/s 2024-04-25T15:49:42.739 INFO:tasks.workunit.client.1.smithi193.stdout:0/571: symlink d0/d49/d5/d7/dd/d16/d26/d5e/d99/lc2 0 2024-04-25T15:49:42.743 INFO:tasks.workunit.client.0.smithi110.stdout:0/742: creat d1/d47/d54/d99/f104 x:0 0 0 2024-04-25T15:49:42.743 INFO:tasks.workunit.client.0.smithi110.stdout:6/743: read d1/db/d24/d2d/f34 [871408,101539] 0 2024-04-25T15:49:42.747 INFO:tasks.workunit.client.1.smithi193.stdout:9/539: rename df/d2c/d2f/d4e/d1c/f21 to df/d2c/d34/d3e/fa5 0 2024-04-25T15:49:42.750 INFO:tasks.workunit.client.1.smithi193.stdout:6/535: truncate dd/d12/d19/d1c/d40/d3c/d3f/f83 265457 0 2024-04-25T15:49:42.750 INFO:tasks.workunit.client.1.smithi193.stdout:6/536: readlink dd/d2f/l36 0 2024-04-25T15:49:42.752 INFO:tasks.workunit.client.1.smithi193.stdout:5/568: creat d17/d53/fc1 x:0 0 0 2024-04-25T15:49:42.756 INFO:tasks.workunit.client.1.smithi193.stdout:6/537: dwrite dd/d12/d19/d1c/f71 [0,4194304] 0 2024-04-25T15:49:42.756 INFO:tasks.workunit.client.1.smithi193.stdout:6/538: dread - dd/f31 zero size 2024-04-25T15:49:42.756 INFO:tasks.workunit.client.0.smithi110.stdout:7/820: rmdir d1b/d29/d46 39 2024-04-25T15:49:42.758 INFO:tasks.workunit.client.0.smithi110.stdout:7/821: write d1b/d29/fa4 [313297,58590] 0 2024-04-25T15:49:42.760 INFO:tasks.workunit.client.0.smithi110.stdout:5/797: getdents d4/d1d/d1f/d35/d3d/d52 0 2024-04-25T15:49:42.762 INFO:tasks.workunit.client.1.smithi193.stdout:4/614: write db/f3b [176892,30847] 0 2024-04-25T15:49:42.767 INFO:tasks.workunit.client.0.smithi110.stdout:0/743: dread - d1/d23/d9e/fda zero size 2024-04-25T15:49:42.768 INFO:tasks.workunit.client.0.smithi110.stdout:5/798: dwrite d4/d1d/d1f/d35/fc6 [0,4194304] 0 2024-04-25T15:49:42.771 INFO:tasks.workunit.client.0.smithi110.stdout:5/799: dread - d4/d1d/d1f/d35/d7f/d92/d9b/f101 zero size 2024-04-25T15:49:42.771 INFO:tasks.workunit.client.0.smithi110.stdout:6/744: creat d1/d79/f108 x:0 0 0 2024-04-25T15:49:42.771 INFO:tasks.workunit.client.0.smithi110.stdout:4/772: write d13/d24/d2d/d44/d65/d99/f9c [1853950,92841] 0 2024-04-25T15:49:42.773 INFO:tasks.workunit.client.0.smithi110.stdout:4/773: truncate d13/d24/d2d/d36/ff6 1025487 0 2024-04-25T15:49:42.773 INFO:tasks.workunit.client.0.smithi110.stdout:3/798: rename d0/d7/d7d/fda to d0/d2e/d88/dbf/da5/f113 0 2024-04-25T15:49:42.775 INFO:tasks.workunit.client.1.smithi193.stdout:5/569: mknod d17/d18/d30/d38/d48/d4f/cc2 0 2024-04-25T15:49:42.777 INFO:tasks.workunit.client.1.smithi193.stdout:6/539: symlink dd/d12/d2d/la2 0 2024-04-25T15:49:42.779 INFO:tasks.workunit.client.1.smithi193.stdout:0/572: unlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/l54 0 2024-04-25T15:49:42.779 INFO:tasks.workunit.client.0.smithi110.stdout:7/822: mknod d1b/d29/d38/d55/d7f/c10e 0 2024-04-25T15:49:42.781 INFO:tasks.workunit.client.0.smithi110.stdout:9/738: symlink d9/lea 0 2024-04-25T15:49:42.783 INFO:tasks.workunit.client.1.smithi193.stdout:3/583: dwrite d8/d13/d1f/d9f/fa5 [0,4194304] 0 2024-04-25T15:49:42.786 INFO:tasks.workunit.client.0.smithi110.stdout:7/823: dwrite d1b/d29/d38/d55/dab/db2/db8/ff5 [0,4194304] 0 2024-04-25T15:49:42.787 INFO:tasks.workunit.client.0.smithi110.stdout:9/739: write d9/db/df/d2c/f70 [578088,20127] 0 2024-04-25T15:49:42.788 INFO:tasks.workunit.client.0.smithi110.stdout:5/800: mkdir d4/d1d/d1f/d108 0 2024-04-25T15:49:42.790 INFO:tasks.workunit.client.1.smithi193.stdout:0/573: dwrite d0/d49/d5/d7/dd/d46/d48/d62/f92 [0,4194304] 0 2024-04-25T15:49:42.790 INFO:tasks.workunit.client.1.smithi193.stdout:4/615: dread db/d13/d1e/f48 [0,4194304] 0 2024-04-25T15:49:42.792 INFO:tasks.workunit.client.0.smithi110.stdout:7/824: truncate d1b/d29/d38/d55/dab/db2/db8/f102 764237 0 2024-04-25T15:49:42.793 INFO:tasks.workunit.client.1.smithi193.stdout:3/584: dwrite d8/d13/d4b/f57 [0,4194304] 0 2024-04-25T15:49:42.795 INFO:tasks.workunit.client.0.smithi110.stdout:3/799: unlink d0/cdb 0 2024-04-25T15:49:42.795 INFO:tasks.workunit.client.0.smithi110.stdout:4/774: mkdir d13/d24/d2d/db5/dd7/d10a 0 2024-04-25T15:49:42.797 INFO:tasks.workunit.client.0.smithi110.stdout:3/800: dread - d0/d2e/d88/dbf/da5/f106 zero size 2024-04-25T15:49:42.799 INFO:tasks.workunit.client.0.smithi110.stdout:3/801: readlink d0/d2e/d88/dbf/d99/dad/daf/lf4 0 2024-04-25T15:49:42.799 INFO:tasks.workunit.client.0.smithi110.stdout:0/744: dread d1/d6c/d7f/fb5 [0,4194304] 0 2024-04-25T15:49:42.801 INFO:tasks.workunit.client.0.smithi110.stdout:7/825: creat d1b/d29/d38/d8b/d89/f10f x:0 0 0 2024-04-25T15:49:42.803 INFO:tasks.workunit.client.0.smithi110.stdout:5/801: mknod d4/d15/c109 0 2024-04-25T15:49:42.805 INFO:tasks.workunit.client.0.smithi110.stdout:4/775: dwrite d13/d24/d2d/f37 [4194304,4194304] 0 2024-04-25T15:49:42.805 INFO:tasks.workunit.client.0.smithi110.stdout:5/802: write d4/d1d/d1f/d35/d3d/dc8/ffd [903641,59453] 0 2024-04-25T15:49:42.808 INFO:tasks.workunit.client.0.smithi110.stdout:6/745: getdents d1/db/d22/d33/d63/dc2/dc7/dec 0 2024-04-25T15:49:42.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/802: symlink d0/d16/l114 0 2024-04-25T15:49:42.814 INFO:tasks.workunit.client.0.smithi110.stdout:3/803: dread - d0/d7/d40/ff1 zero size 2024-04-25T15:49:42.815 INFO:tasks.workunit.client.1.smithi193.stdout:5/570: creat d17/d18/d9b/fc3 x:0 0 0 2024-04-25T15:49:42.815 INFO:tasks.workunit.client.1.smithi193.stdout:5/571: chown d17/d18/d30/d34 11262609 1 2024-04-25T15:49:42.817 INFO:tasks.workunit.client.1.smithi193.stdout:1/476: creat dc/d4a/d50/d5c/d5d/fa0 x:0 0 0 2024-04-25T15:49:42.818 INFO:tasks.workunit.client.0.smithi110.stdout:5/803: rmdir d4/d1d/d1f/d35/d7f/d92 39 2024-04-25T15:49:42.820 INFO:tasks.workunit.client.0.smithi110.stdout:6/746: creat d1/dd/f109 x:0 0 0 2024-04-25T15:49:42.824 INFO:tasks.workunit.client.0.smithi110.stdout:0/745: rename d1/d23/d30/d33/d35/c1f to d1/d23/dc5/c105 0 2024-04-25T15:49:42.827 INFO:tasks.workunit.client.0.smithi110.stdout:7/826: symlink d1b/d29/d38/d55/l110 0 2024-04-25T15:49:42.827 INFO:tasks.workunit.client.1.smithi193.stdout:2/547: link d0/d37/d4d/dbe/fd8 d0/d9/d3d/d7d/fe1 0 2024-04-25T15:49:42.828 INFO:tasks.workunit.client.1.smithi193.stdout:2/548: dread - d0/d42/d51/d4f/d65/d7e/fca zero size 2024-04-25T15:49:42.829 INFO:tasks.workunit.client.0.smithi110.stdout:5/804: dread d4/d1d/d1f/d35/f39 [0,4194304] 0 2024-04-25T15:49:42.830 INFO:tasks.workunit.client.1.smithi193.stdout:5/572: rename d17/d18/d45 to d17/d18/d30/d38/d71/d96/dc4 0 2024-04-25T15:49:42.832 INFO:tasks.workunit.client.1.smithi193.stdout:1/477: unlink dc/d4a/f54 0 2024-04-25T15:49:42.833 INFO:tasks.workunit.client.0.smithi110.stdout:6/747: mknod d1/dd/d3c/d91/d8d/dce/d102/d75/c10a 0 2024-04-25T15:49:42.835 INFO:tasks.workunit.client.1.smithi193.stdout:9/540: getdents df/d2c/d34/d3e 0 2024-04-25T15:49:42.839 INFO:tasks.workunit.client.0.smithi110.stdout:4/776: link d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/f81 d13/d24/d2d/d36/d51/f10b 0 2024-04-25T15:49:42.842 INFO:tasks.workunit.client.0.smithi110.stdout:5/805: rename d4/d1d/d1f/d35/d3d/d52/dd2 to d4/d1d/d1f/d35/d3d/d52/d75/dfa/d105/d10a 0 2024-04-25T15:49:42.847 INFO:tasks.workunit.client.0.smithi110.stdout:6/748: creat d1/dd/d3c/d91/d8d/dce/d102/d75/d82/f10b x:0 0 0 2024-04-25T15:49:42.847 INFO:tasks.workunit.client.0.smithi110.stdout:3/804: link d0/d7/d40/l42 d0/d2e/d88/dbf/d92/l115 0 2024-04-25T15:49:42.850 INFO:tasks.workunit.client.0.smithi110.stdout:6/749: chown d1/dd/d3c/d91/d8d/dce/ffc 77127075 1 2024-04-25T15:49:42.850 INFO:tasks.workunit.client.0.smithi110.stdout:5/806: dwrite d4/d15/d51/d90/ded/ff6 [0,4194304] 0 2024-04-25T15:49:42.851 INFO:tasks.workunit.client.0.smithi110.stdout:7/827: mknod d1b/c111 0 2024-04-25T15:49:42.855 INFO:tasks.workunit.client.0.smithi110.stdout:6/750: symlink d1/dd/db5/l10c 0 2024-04-25T15:49:42.857 INFO:tasks.workunit.client.0.smithi110.stdout:5/807: unlink d4/d1d/f70 0 2024-04-25T15:49:42.858 INFO:tasks.workunit.client.0.smithi110.stdout:7/828: mkdir d1b/d29/db5/dca/d10b/d112 0 2024-04-25T15:49:42.861 INFO:tasks.workunit.client.1.smithi193.stdout:2/549: creat d0/d37/d25/d5f/d3a/d6d/d6e/fe2 x:0 0 0 2024-04-25T15:49:42.861 INFO:tasks.workunit.client.0.smithi110.stdout:4/777: rename d13/d24/d2d/d44/d65/d99/daf/d56/f66 to d13/d24/d2d/dd9/dbb/dd2/dfd/f10c 0 2024-04-25T15:49:42.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:42 smithi110 ceph-mon[26344]: pgmap v38: 65 pgs: 65 active+clean; 2.6 GiB data, 8.9 GiB used, 527 GiB / 536 GiB avail; 63 MiB/s rd, 210 MiB/s wr, 374 op/s 2024-04-25T15:49:42.863 INFO:tasks.workunit.client.0.smithi110.stdout:3/805: mkdir d0/d16/d8c/dce/d116 0 2024-04-25T15:49:42.863 INFO:tasks.workunit.client.0.smithi110.stdout:3/806: readlink d0/d7/dd0/ddf/l107 0 2024-04-25T15:49:42.864 INFO:tasks.workunit.client.0.smithi110.stdout:3/807: chown d0/f3 983455 1 2024-04-25T15:49:42.865 INFO:tasks.workunit.client.1.smithi193.stdout:4/616: getdents db/d13/d9d 0 2024-04-25T15:49:42.866 INFO:tasks.workunit.client.0.smithi110.stdout:6/751: mkdir d1/dd/d3c/d91/d10d 0 2024-04-25T15:49:42.867 INFO:tasks.workunit.client.0.smithi110.stdout:6/752: stat d1/dd/d3c/d91/lab 0 2024-04-25T15:49:42.871 INFO:tasks.workunit.client.0.smithi110.stdout:9/740: write d9/db/df/d25/f79 [2459210,126827] 0 2024-04-25T15:49:42.875 INFO:tasks.workunit.client.1.smithi193.stdout:9/541: fdatasync df/d2c/d34/d3e/d4d/d5e/d81/d41/d5d/f98 0 2024-04-25T15:49:42.876 INFO:tasks.workunit.client.0.smithi110.stdout:4/778: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d10d 0 2024-04-25T15:49:42.876 INFO:tasks.workunit.client.0.smithi110.stdout:5/808: dwrite d4/d1d/d1f/d35/d7f/d92/d9b/dcb/fce [0,4194304] 0 2024-04-25T15:49:42.877 INFO:tasks.workunit.client.1.smithi193.stdout:4/617: rmdir db/d32/d88 39 2024-04-25T15:49:42.882 INFO:tasks.workunit.client.0.smithi110.stdout:6/753: dread d1/db/d22/d90/d9e/f9f [0,4194304] 0 2024-04-25T15:49:42.884 INFO:tasks.workunit.client.1.smithi193.stdout:9/542: mkdir df/d2c/d2f/d33/da6 0 2024-04-25T15:49:42.885 INFO:tasks.workunit.client.1.smithi193.stdout:2/550: dread d0/d9/d15/f1b [0,4194304] 0 2024-04-25T15:49:42.886 INFO:tasks.workunit.client.1.smithi193.stdout:9/543: write df/d2d/f61 [765760,1335] 0 2024-04-25T15:49:42.886 INFO:tasks.workunit.client.1.smithi193.stdout:4/618: dread fa [0,4194304] 0 2024-04-25T15:49:42.887 INFO:tasks.workunit.client.1.smithi193.stdout:6/540: fdatasync dd/d12/d19/d1c/d40/d3c/d3f/f83 0 2024-04-25T15:49:42.888 INFO:tasks.workunit.client.1.smithi193.stdout:7/522: sync 2024-04-25T15:49:42.888 INFO:tasks.workunit.client.1.smithi193.stdout:2/551: readlink d0/d37/d25/d5f/d3a/d6d/d6e/d8e/la0 0 2024-04-25T15:49:42.889 INFO:tasks.workunit.client.1.smithi193.stdout:2/552: fsync d0/d42/d51/d4f/d65/d7e/f95 0 2024-04-25T15:49:42.889 INFO:tasks.workunit.client.1.smithi193.stdout:3/585: write d8/d13/d14/f59 [1170821,7201] 0 2024-04-25T15:49:42.890 INFO:tasks.workunit.client.1.smithi193.stdout:6/541: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f87 [2005246,12856] 0 2024-04-25T15:49:42.894 INFO:tasks.workunit.client.0.smithi110.stdout:3/808: chown d0/d7/d40/l42 651491024 1 2024-04-25T15:49:42.896 INFO:tasks.workunit.client.1.smithi193.stdout:6/542: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f87 [0,4194304] 0 2024-04-25T15:49:42.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/746: write d1/d47/f61 [547978,8270] 0 2024-04-25T15:49:42.913 INFO:tasks.workunit.client.0.smithi110.stdout:7/829: getdents d1b/d29/db5/dca 0 2024-04-25T15:49:42.917 INFO:tasks.workunit.client.0.smithi110.stdout:5/809: rmdir d4/d1d/d1f/d35/d3d/d52/d97/dd9 0 2024-04-25T15:49:42.919 INFO:tasks.workunit.client.0.smithi110.stdout:7/830: readlink d1b/d29/d46/l66 0 2024-04-25T15:49:42.921 INFO:tasks.workunit.client.0.smithi110.stdout:4/779: getdents d13/d24/d2d/d44/d65 0 2024-04-25T15:49:42.922 INFO:tasks.workunit.client.0.smithi110.stdout:5/810: mknod d4/d1d/d1f/d35/d3d/dc8/c10b 0 2024-04-25T15:49:42.926 INFO:tasks.workunit.client.0.smithi110.stdout:6/754: dwrite d1/dd/d3c/d91/d8d/dce/f73 [0,4194304] 0 2024-04-25T15:49:42.930 INFO:tasks.workunit.client.0.smithi110.stdout:7/831: getdents d1b/d29/d38/d55/dab 0 2024-04-25T15:49:42.930 INFO:tasks.workunit.client.0.smithi110.stdout:9/741: dwrite d9/d44/fab [0,4194304] 0 2024-04-25T15:49:42.933 INFO:tasks.workunit.client.0.smithi110.stdout:7/832: chown d1b/d29/d38/f61 225 1 2024-04-25T15:49:42.935 INFO:tasks.workunit.client.1.smithi193.stdout:4/619: truncate db/d13/d1e/f48 5123472 0 2024-04-25T15:49:42.936 INFO:tasks.workunit.client.1.smithi193.stdout:2/553: rename d0 to d0/d9/d15/d31/de3 22 2024-04-25T15:49:42.936 INFO:tasks.workunit.client.1.smithi193.stdout:1/478: link dc/c1f dc/d4a/d4d/ca1 0 2024-04-25T15:49:42.937 INFO:tasks.workunit.client.0.smithi110.stdout:9/742: rename d9/db/df/d4d/c50 to d9/d13/d5b/d61/db3/ceb 0 2024-04-25T15:49:42.941 INFO:tasks.workunit.client.0.smithi110.stdout:6/755: creat d1/dd/d3c/d91/d8d/dce/d102/f10e x:0 0 0 2024-04-25T15:49:42.941 INFO:tasks.workunit.client.0.smithi110.stdout:4/780: dwrite d13/d24/d2d/d44/d65/d99/daf/f7a [0,4194304] 0 2024-04-25T15:49:42.942 INFO:tasks.workunit.client.0.smithi110.stdout:9/743: chown d9/c1d 16768 1 2024-04-25T15:49:42.943 INFO:tasks.workunit.client.0.smithi110.stdout:4/781: chown d13/d24/d2d/d44/d65/d99/daf/d56 293120960 1 2024-04-25T15:49:42.943 INFO:tasks.workunit.client.1.smithi193.stdout:9/544: truncate df/d2c/d2f/d4e/f1a 1706972 0 2024-04-25T15:49:42.945 INFO:tasks.workunit.client.0.smithi110.stdout:7/833: rename d1b/d29/d38/c65 to d1b/d29/d38/d55/dab/c113 0 2024-04-25T15:49:42.945 INFO:tasks.workunit.client.1.smithi193.stdout:9/545: write df/d2c/d2f/d4e/f25 [7449453,19997] 0 2024-04-25T15:49:42.946 INFO:tasks.workunit.client.0.smithi110.stdout:5/811: dread d4/d1d/d1f/d35/d3d/f53 [4194304,4194304] 0 2024-04-25T15:49:42.947 INFO:tasks.workunit.client.0.smithi110.stdout:6/756: mknod d1/db/d22/d33/d63/dc2/dc7/dec/c10f 0 2024-04-25T15:49:42.948 INFO:tasks.workunit.client.1.smithi193.stdout:7/523: dread d12/d1c/f8a [0,4194304] 0 2024-04-25T15:49:42.949 INFO:tasks.workunit.client.1.smithi193.stdout:2/554: creat d0/d42/d51/d4f/d93/fe4 x:0 0 0 2024-04-25T15:49:42.951 INFO:tasks.workunit.client.0.smithi110.stdout:4/782: mknod d13/d24/d2d/d5e/def/c10e 0 2024-04-25T15:49:42.953 INFO:tasks.workunit.client.0.smithi110.stdout:5/812: mknod d4/d1d/d1f/d35/d7f/d92/d9b/dcb/c10c 0 2024-04-25T15:49:42.953 INFO:tasks.workunit.client.1.smithi193.stdout:6/543: link dd/d12/d19/d1c/d40/d3c/d3f/d4f/l65 dd/d2f/d57/la3 0 2024-04-25T15:49:42.955 INFO:tasks.workunit.client.0.smithi110.stdout:4/783: creat d13/d24/d2d/d36/d51/f10f x:0 0 0 2024-04-25T15:49:42.956 INFO:tasks.workunit.client.1.smithi193.stdout:9/546: dwrite df/d2c/d2f/d4e/d1c/f9b [0,4194304] 0 2024-04-25T15:49:42.956 INFO:tasks.workunit.client.1.smithi193.stdout:2/555: mknod d0/d9/d15/d31/d90/ce5 0 2024-04-25T15:49:42.958 INFO:tasks.workunit.client.0.smithi110.stdout:5/813: creat d4/d1d/d1f/d9e/f10d x:0 0 0 2024-04-25T15:49:42.961 INFO:tasks.workunit.client.1.smithi193.stdout:1/479: rename dc/d4a/f75 to dc/d4a/d50/d5c/fa2 0 2024-04-25T15:49:42.962 INFO:tasks.workunit.client.1.smithi193.stdout:9/547: symlink df/d11/d8b/la7 0 2024-04-25T15:49:42.963 INFO:tasks.workunit.client.1.smithi193.stdout:8/539: sync 2024-04-25T15:49:42.963 INFO:tasks.workunit.client.0.smithi110.stdout:5/814: dread d4/f7c [0,4194304] 0 2024-04-25T15:49:42.964 INFO:tasks.workunit.client.0.smithi110.stdout:6/757: getdents d1/db/d22/d33/d63/dc2 0 2024-04-25T15:49:42.964 INFO:tasks.workunit.client.1.smithi193.stdout:2/556: creat d0/d37/d25/d5f/d3a/d6d/d6e/d8e/fe6 x:0 0 0 2024-04-25T15:49:42.965 INFO:tasks.workunit.client.1.smithi193.stdout:4/620: dread db/d13/d1e/f41 [0,4194304] 0 2024-04-25T15:49:42.967 INFO:tasks.workunit.client.0.smithi110.stdout:4/784: getdents d13/d24/d2d/dd9 0 2024-04-25T15:49:42.967 INFO:tasks.workunit.client.1.smithi193.stdout:1/480: chown dc/d4a/d50/d5c/d5d/f51 7774 1 2024-04-25T15:49:42.968 INFO:tasks.workunit.client.0.smithi110.stdout:6/758: creat d1/dd/d3c/d91/d8d/f110 x:0 0 0 2024-04-25T15:49:42.968 INFO:tasks.workunit.client.1.smithi193.stdout:1/481: fsync dc/f17 0 2024-04-25T15:49:42.969 INFO:tasks.workunit.client.1.smithi193.stdout:6/544: dwrite dd/d12/d19/d1c/d40/d3c/d3f/f61 [0,4194304] 0 2024-04-25T15:49:42.969 INFO:tasks.workunit.client.1.smithi193.stdout:1/482: stat dc 0 2024-04-25T15:49:42.970 INFO:tasks.workunit.client.1.smithi193.stdout:4/621: dread db/d32/f94 [0,4194304] 0 2024-04-25T15:49:42.970 INFO:tasks.workunit.client.1.smithi193.stdout:8/540: mkdir d2/d3/d20/d45/d4a/d63/da2/dad 0 2024-04-25T15:49:42.978 INFO:tasks.workunit.client.0.smithi110.stdout:6/759: rmdir d1/db 39 2024-04-25T15:49:42.980 INFO:tasks.workunit.client.0.smithi110.stdout:7/834: write d1b/d29/d38/d55/d95/fa9 [681692,120975] 0 2024-04-25T15:49:42.981 INFO:tasks.workunit.client.0.smithi110.stdout:7/835: chown d1b/d29/d38/d55/fa0 260798 1 2024-04-25T15:49:42.982 INFO:tasks.workunit.client.0.smithi110.stdout:4/785: symlink d13/d24/d2d/d44/d65/d99/daf/l110 0 2024-04-25T15:49:42.982 INFO:tasks.workunit.client.1.smithi193.stdout:7/524: write f2 [2229153,82053] 0 2024-04-25T15:49:42.983 INFO:tasks.workunit.client.1.smithi193.stdout:0/574: sync 2024-04-25T15:49:42.983 INFO:tasks.workunit.client.1.smithi193.stdout:5/573: sync 2024-04-25T15:49:42.986 INFO:tasks.workunit.client.1.smithi193.stdout:1/483: dwrite dc/f3f [0,4194304] 0 2024-04-25T15:49:42.986 INFO:tasks.workunit.client.0.smithi110.stdout:6/760: creat d1/db/da0/da5/f111 x:0 0 0 2024-04-25T15:49:42.987 INFO:tasks.workunit.client.1.smithi193.stdout:1/484: write dc/d4a/d50/d5c/f91 [2757357,72982] 0 2024-04-25T15:49:42.988 INFO:tasks.workunit.client.0.smithi110.stdout:4/786: link d13/d24/d2d/d36/fde d13/d24/d26/f111 0 2024-04-25T15:49:42.988 INFO:tasks.workunit.client.1.smithi193.stdout:2/557: creat d0/d9/d15/d31/d90/d9f/dc5/fe7 x:0 0 0 2024-04-25T15:49:42.996 INFO:tasks.workunit.client.1.smithi193.stdout:1/485: dwrite dc/d4a/d4d/d7a/f96 [0,4194304] 0 2024-04-25T15:49:43.009 INFO:tasks.workunit.client.1.smithi193.stdout:7/525: read d12/d13/f14 [1396213,15191] 0 2024-04-25T15:49:43.010 INFO:tasks.workunit.client.1.smithi193.stdout:0/575: mknod d0/d49/d5/d61/cc3 0 2024-04-25T15:49:43.011 INFO:tasks.workunit.client.0.smithi110.stdout:7/836: dread d1b/d29/d38/d55/d7f/f72 [4194304,4194304] 0 2024-04-25T15:49:43.013 INFO:tasks.workunit.client.0.smithi110.stdout:4/787: rename d13/d24/d2d/d36/fde to d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/f112 0 2024-04-25T15:49:43.015 INFO:tasks.workunit.client.1.smithi193.stdout:3/586: sync 2024-04-25T15:49:43.015 INFO:tasks.workunit.client.1.smithi193.stdout:7/526: dwrite d12/d1c/d45/f5d [0,4194304] 0 2024-04-25T15:49:43.016 INFO:tasks.workunit.client.1.smithi193.stdout:4/622: creat db/fbb x:0 0 0 2024-04-25T15:49:43.017 INFO:tasks.workunit.client.1.smithi193.stdout:7/527: chown d12/l4b 136333616 1 2024-04-25T15:49:43.018 INFO:tasks.workunit.client.1.smithi193.stdout:5/574: mknod d17/d18/cc5 0 2024-04-25T15:49:43.024 INFO:tasks.workunit.client.1.smithi193.stdout:3/587: dwrite d8/d30/f98 [0,4194304] 0 2024-04-25T15:49:43.025 INFO:tasks.workunit.client.1.smithi193.stdout:9/548: dread df/d11/d8b/f4f [0,4194304] 0 2024-04-25T15:49:43.025 INFO:tasks.workunit.client.0.smithi110.stdout:7/837: link d1b/cc3 d1b/d29/db5/dca/d10b/c114 0 2024-04-25T15:49:43.027 INFO:tasks.workunit.client.1.smithi193.stdout:1/486: chown dc/c2e 7 1 2024-04-25T15:49:43.028 INFO:tasks.workunit.client.1.smithi193.stdout:0/576: mknod d0/d49/d5/d7/dd/d16/d1a/cc4 0 2024-04-25T15:49:43.029 INFO:tasks.workunit.client.1.smithi193.stdout:0/577: write d0/d49/d5/d7/fa4 [252780,91642] 0 2024-04-25T15:49:43.029 INFO:tasks.workunit.client.0.smithi110.stdout:7/838: symlink d1b/d29/d46/l115 0 2024-04-25T15:49:43.034 INFO:tasks.workunit.client.0.smithi110.stdout:5/815: dwrite d4/d1d/d1f/d35/d7f/d92/fcd [0,4194304] 0 2024-04-25T15:49:43.037 INFO:tasks.workunit.client.0.smithi110.stdout:7/839: fdatasync d1b/d29/d38/d8b/d89/f96 0 2024-04-25T15:49:43.042 INFO:tasks.workunit.client.1.smithi193.stdout:8/541: getdents d2/d3/d20/d45 0 2024-04-25T15:49:43.043 INFO:tasks.workunit.client.0.smithi110.stdout:5/816: fsync d4/d1d/d1f/d35/d3d/d52/d97/fa1 0 2024-04-25T15:49:43.044 INFO:tasks.workunit.client.1.smithi193.stdout:6/545: truncate dd/d12/d19/d1c/f20 2614148 0 2024-04-25T15:49:43.046 INFO:tasks.workunit.client.1.smithi193.stdout:2/558: link d0/d9/d15/d31/d90/d9c/fba d0/d37/d25/d5f/d3a/d6d/d6e/db9/fe8 0 2024-04-25T15:49:43.048 INFO:tasks.workunit.client.1.smithi193.stdout:2/559: truncate d0/d37/d25/d5f/d3a/d6d/d6e/fe2 186842 0 2024-04-25T15:49:43.049 INFO:tasks.workunit.client.0.smithi110.stdout:7/840: symlink d1b/d29/d38/d55/d7f/l116 0 2024-04-25T15:49:43.051 INFO:tasks.workunit.client.1.smithi193.stdout:9/549: rmdir df/d2c/d2f/d33 39 2024-04-25T15:49:43.054 INFO:tasks.workunit.client.0.smithi110.stdout:5/817: write d4/f7c [2398922,44302] 0 2024-04-25T15:49:43.055 INFO:tasks.workunit.client.1.smithi193.stdout:1/487: symlink dc/d59/la3 0 2024-04-25T15:49:43.056 INFO:tasks.workunit.client.1.smithi193.stdout:1/488: write dc/f3f [1036228,76369] 0 2024-04-25T15:49:43.057 INFO:tasks.workunit.client.0.smithi110.stdout:7/841: fdatasync d1b/d29/d38/d55/f8d 0 2024-04-25T15:49:43.057 INFO:tasks.workunit.client.0.smithi110.stdout:4/788: dwrite d13/d24/d2d/d36/fb0 [0,4194304] 0 2024-04-25T15:49:43.058 INFO:tasks.workunit.client.1.smithi193.stdout:5/575: dread d17/d18/f2e [0,4194304] 0 2024-04-25T15:49:43.059 INFO:tasks.workunit.client.1.smithi193.stdout:0/578: chown d0/d49/d5/d7/dd/d46/d48 188497717 1 2024-04-25T15:49:43.062 INFO:tasks.workunit.client.0.smithi110.stdout:5/818: mknod d4/d1d/d1f/d35/d3d/d93/c10e 0 2024-04-25T15:49:43.063 INFO:tasks.workunit.client.1.smithi193.stdout:7/528: write d12/d1c/d63/d70/d68/d8d/f9c [646548,51587] 0 2024-04-25T15:49:43.066 INFO:tasks.workunit.client.1.smithi193.stdout:7/529: read d12/d31/d39/f53 [79263,111685] 0 2024-04-25T15:49:43.066 INFO:tasks.workunit.client.0.smithi110.stdout:7/842: creat d1b/d29/db5/f117 x:0 0 0 2024-04-25T15:49:43.066 INFO:tasks.workunit.client.1.smithi193.stdout:4/623: dwrite fa [0,4194304] 0 2024-04-25T15:49:43.070 INFO:tasks.workunit.client.0.smithi110.stdout:4/789: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d113 0 2024-04-25T15:49:43.072 INFO:tasks.workunit.client.0.smithi110.stdout:4/790: write d13/d24/d2d/d44/d65/d99/daf/d56/d88/f108 [869129,124993] 0 2024-04-25T15:49:43.076 INFO:tasks.workunit.client.1.smithi193.stdout:9/550: write df/d2c/d34/f75 [3167046,56423] 0 2024-04-25T15:49:43.078 INFO:tasks.workunit.client.1.smithi193.stdout:9/551: stat df/d2c/d34/d3e/d4d/d5e/d81/d41/d5d/f98 0 2024-04-25T15:49:43.078 INFO:tasks.workunit.client.1.smithi193.stdout:9/552: chown df/d11/d8b/l48 12000 1 2024-04-25T15:49:43.081 INFO:tasks.workunit.client.0.smithi110.stdout:4/791: rmdir d13/d24/d2d/d44/d65/d99/dbc 39 2024-04-25T15:49:43.081 INFO:tasks.workunit.client.0.smithi110.stdout:5/819: dwrite d4/d1d/d1f/f49 [4194304,4194304] 0 2024-04-25T15:49:43.082 INFO:tasks.workunit.client.0.smithi110.stdout:5/820: stat d4/d1d/d1f/d35/d3d/d52/fc9 0 2024-04-25T15:49:43.083 INFO:tasks.workunit.client.0.smithi110.stdout:4/792: chown d13/d24/d2d/d44/da4/ff9 1745607551 1 2024-04-25T15:49:43.088 INFO:tasks.workunit.client.1.smithi193.stdout:6/546: write dd/d12/d19/d1c/d40/d3c/d3f/f54 [2117893,109826] 0 2024-04-25T15:49:43.092 INFO:tasks.workunit.client.1.smithi193.stdout:3/588: write d8/d13/d1f/f79 [1001386,79090] 0 2024-04-25T15:49:43.093 INFO:tasks.workunit.client.1.smithi193.stdout:3/589: stat d8/d13/d14/da0 0 2024-04-25T15:49:43.094 INFO:tasks.workunit.client.1.smithi193.stdout:1/489: write f6 [774652,60298] 0 2024-04-25T15:49:43.096 INFO:tasks.workunit.client.0.smithi110.stdout:7/843: write d1b/d29/d87/d8e/f94 [807774,115500] 0 2024-04-25T15:49:43.099 INFO:tasks.workunit.client.1.smithi193.stdout:8/542: truncate d2/d3/f68 1616015 0 2024-04-25T15:49:43.101 INFO:tasks.workunit.client.1.smithi193.stdout:0/579: dwrite d0/d49/d5/d7/dd/d16/fb8 [0,4194304] 0 2024-04-25T15:49:43.102 INFO:tasks.workunit.client.0.smithi110.stdout:5/821: link d4/d1d/d1f/d35/d3d/d52/d75/dfa/d105/d10a/dd4/fb7 d4/d15/d51/d90/ded/def/f10f 0 2024-04-25T15:49:43.105 INFO:tasks.workunit.client.0.smithi110.stdout:4/793: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/f81 [0,4194304] 0 2024-04-25T15:49:43.106 INFO:tasks.workunit.client.1.smithi193.stdout:4/624: dwrite db/d13/d1e/d1f/d22/f7a [4194304,4194304] 0 2024-04-25T15:49:43.107 INFO:tasks.workunit.client.1.smithi193.stdout:4/625: chown db/d13/d1e/d1f/c8e 162352 1 2024-04-25T15:49:43.107 INFO:tasks.workunit.client.0.smithi110.stdout:7/844: creat d1b/d29/d87/f118 x:0 0 0 2024-04-25T15:49:43.110 INFO:tasks.workunit.client.0.smithi110.stdout:5/822: mknod d4/d15/d51/d99/c110 0 2024-04-25T15:49:43.115 INFO:tasks.workunit.client.0.smithi110.stdout:7/845: mkdir d1b/d29/d38/d55/dab/d119 0 2024-04-25T15:49:43.116 INFO:tasks.workunit.client.1.smithi193.stdout:4/626: dwrite fa [0,4194304] 0 2024-04-25T15:49:43.117 INFO:tasks.workunit.client.1.smithi193.stdout:3/590: rmdir d8/d13/d14/d23 39 2024-04-25T15:49:43.118 INFO:tasks.workunit.client.1.smithi193.stdout:2/560: sync 2024-04-25T15:49:43.118 INFO:tasks.workunit.client.0.smithi110.stdout:5/823: mknod d4/d15/d51/d99/c111 0 2024-04-25T15:49:43.122 INFO:tasks.workunit.client.1.smithi193.stdout:7/530: rename d12/f52 to d12/d31/d37/fc5 0 2024-04-25T15:49:43.123 INFO:tasks.workunit.client.0.smithi110.stdout:7/846: creat d1b/d29/d46/d106/f11a x:0 0 0 2024-04-25T15:49:43.124 INFO:tasks.workunit.client.0.smithi110.stdout:7/847: fsync d1b/f88 0 2024-04-25T15:49:43.127 INFO:tasks.workunit.client.1.smithi193.stdout:1/490: dread dc/f40 [0,4194304] 0 2024-04-25T15:49:43.127 INFO:tasks.workunit.client.1.smithi193.stdout:2/561: dwrite d0/df/da5/fc2 [0,4194304] 0 2024-04-25T15:49:43.129 INFO:tasks.workunit.client.1.smithi193.stdout:8/543: rmdir d2 39 2024-04-25T15:49:43.131 INFO:tasks.workunit.client.0.smithi110.stdout:5/824: rename d4/d1d/c8b to d4/d1d/d1f/d35/d3d/d52/d75/c112 0 2024-04-25T15:49:43.132 INFO:tasks.workunit.client.0.smithi110.stdout:5/825: chown d4/d1d/d1f/cd6 200469 1 2024-04-25T15:49:43.134 INFO:tasks.workunit.client.1.smithi193.stdout:6/547: write dd/f13 [2341613,128968] 0 2024-04-25T15:49:43.135 INFO:tasks.workunit.client.1.smithi193.stdout:5/576: link d17/l1c d17/d18/d75/d85/d5d/dad/lc6 0 2024-04-25T15:49:43.136 INFO:tasks.workunit.client.0.smithi110.stdout:5/826: mkdir d4/d1d/d1f/d35/d3d/d52/d75/d113 0 2024-04-25T15:49:43.140 INFO:tasks.workunit.client.1.smithi193.stdout:0/580: read d0/d49/d5/d7/dd/d16/d18/f2a [898110,7549] 0 2024-04-25T15:49:43.140 INFO:tasks.workunit.client.1.smithi193.stdout:3/591: creat d8/d13/d14/d27/fb6 x:0 0 0 2024-04-25T15:49:43.141 INFO:tasks.workunit.client.0.smithi110.stdout:4/794: dwrite d13/d24/d2d/d36/f4d [0,4194304] 0 2024-04-25T15:49:43.142 INFO:tasks.workunit.client.1.smithi193.stdout:7/531: unlink d12/d13/f40 0 2024-04-25T15:49:43.143 INFO:tasks.workunit.client.1.smithi193.stdout:7/532: write d12/d31/d37/d62/fb2 [541463,15137] 0 2024-04-25T15:49:43.144 INFO:tasks.workunit.client.1.smithi193.stdout:2/562: rmdir d0/d37/d25 39 2024-04-25T15:49:43.145 INFO:tasks.workunit.client.1.smithi193.stdout:9/553: link df/d2c/d34/d3e/d4d/l7f df/la8 0 2024-04-25T15:49:43.145 INFO:tasks.workunit.client.1.smithi193.stdout:2/563: stat d0/d42/d51/d4f/f97 0 2024-04-25T15:49:43.146 INFO:tasks.workunit.client.1.smithi193.stdout:5/577: readlink d17/d18/d30/d38/d48/l6e 0 2024-04-25T15:49:43.148 INFO:tasks.workunit.client.0.smithi110.stdout:4/795: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d10d/d114 0 2024-04-25T15:49:43.148 INFO:tasks.workunit.client.1.smithi193.stdout:5/578: chown d17/d18/d30/d38/d48/d4f/f5f 517 1 2024-04-25T15:49:43.149 INFO:tasks.workunit.client.1.smithi193.stdout:6/548: rmdir dd/d12/d2d 39 2024-04-25T15:49:43.150 INFO:tasks.workunit.client.1.smithi193.stdout:4/627: mknod db/d32/d45/cbc 0 2024-04-25T15:49:43.151 INFO:tasks.workunit.client.0.smithi110.stdout:4/796: symlink d13/d24/d26/dcd/l115 0 2024-04-25T15:49:43.152 INFO:tasks.workunit.client.1.smithi193.stdout:8/544: dwrite d2/d3/d27/d69/d86/f84 [0,4194304] 0 2024-04-25T15:49:43.154 INFO:tasks.workunit.client.1.smithi193.stdout:1/491: fdatasync dc/f18 0 2024-04-25T15:49:43.154 INFO:tasks.workunit.client.1.smithi193.stdout:8/545: truncate d2/da5/fa7 905408 0 2024-04-25T15:49:43.155 INFO:tasks.workunit.client.1.smithi193.stdout:7/533: mkdir d12/d1c/d63/d70/dc6 0 2024-04-25T15:49:43.156 INFO:tasks.workunit.client.1.smithi193.stdout:1/492: write dc/d4a/d50/d5c/f91 [3631285,10192] 0 2024-04-25T15:49:43.158 INFO:tasks.workunit.client.1.smithi193.stdout:7/534: read d12/fb8 [3299679,58377] 0 2024-04-25T15:49:43.159 INFO:tasks.workunit.client.1.smithi193.stdout:2/564: creat d0/d9/d3d/d7d/dc3/fe9 x:0 0 0 2024-04-25T15:49:43.159 INFO:tasks.workunit.client.1.smithi193.stdout:9/554: fdatasync df/d2c/d34/d3e/d4d/d5e/d81/d41/f59 0 2024-04-25T15:49:43.163 INFO:tasks.workunit.client.1.smithi193.stdout:5/579: creat d17/d18/d30/d38/d71/d96/fc7 x:0 0 0 2024-04-25T15:49:43.164 INFO:tasks.workunit.client.1.smithi193.stdout:1/493: dwrite dc/d4a/d50/d7c/d7e/f88 [0,4194304] 0 2024-04-25T15:49:43.164 INFO:tasks.workunit.client.1.smithi193.stdout:5/580: dread - d17/d18/d30/d38/d48/d4f/f5f zero size 2024-04-25T15:49:43.166 INFO:tasks.workunit.client.1.smithi193.stdout:1/494: dread - dc/d4a/d4d/d80/f98 zero size 2024-04-25T15:49:43.166 INFO:tasks.workunit.client.1.smithi193.stdout:5/581: truncate d17/d18/d75/fbf 966635 0 2024-04-25T15:49:43.171 INFO:tasks.workunit.client.0.smithi110.stdout:5/827: dwrite d4/f6 [0,4194304] 0 2024-04-25T15:49:43.174 INFO:tasks.workunit.client.1.smithi193.stdout:2/565: dread - d0/d9/f73 zero size 2024-04-25T15:49:43.176 INFO:tasks.workunit.client.1.smithi193.stdout:3/592: link d8/d13/d4b/d8b/d9a/cad d8/d13/d1f/d9f/cb7 0 2024-04-25T15:49:43.177 INFO:tasks.workunit.client.1.smithi193.stdout:6/549: truncate dd/d12/d19/d1c/f22 4716409 0 2024-04-25T15:49:43.181 INFO:tasks.workunit.client.1.smithi193.stdout:9/555: rename df/d2c/d34/d3e/d4d/d5e/d81 to df/d2c/d2f/d4e/d1c/d30/d3b/da9 0 2024-04-25T15:49:43.183 INFO:tasks.workunit.client.1.smithi193.stdout:4/628: creat db/d32/d45/dab/fbd x:0 0 0 2024-04-25T15:49:43.183 INFO:tasks.workunit.client.1.smithi193.stdout:9/556: dread - df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/f94 zero size 2024-04-25T15:49:43.184 INFO:tasks.workunit.client.1.smithi193.stdout:4/629: chown db/d13/d1e/d1f/l90 2 1 2024-04-25T15:49:43.187 INFO:tasks.workunit.client.1.smithi193.stdout:1/495: symlink dc/d4a/d77/la4 0 2024-04-25T15:49:43.187 INFO:tasks.workunit.client.1.smithi193.stdout:1/496: chown dc/c22 172378363 1 2024-04-25T15:49:43.188 INFO:tasks.workunit.client.1.smithi193.stdout:1/497: dread - dc/d4a/d4d/f5e zero size 2024-04-25T15:49:43.188 INFO:tasks.workunit.client.0.smithi110.stdout:5/828: link d4/d1d/d1f/l5b d4/d1d/d1f/d35/d3d/d52/d75/dfa/d105/l114 0 2024-04-25T15:49:43.189 INFO:tasks.workunit.client.1.smithi193.stdout:0/581: truncate d0/d49/d5/d7/dd/d16/f41 752146 0 2024-04-25T15:49:43.191 INFO:tasks.workunit.client.1.smithi193.stdout:2/566: fdatasync d0/d42/d51/f46 0 2024-04-25T15:49:43.192 INFO:tasks.workunit.client.1.smithi193.stdout:3/593: mknod d8/d13/d1f/d69/cb8 0 2024-04-25T15:49:43.194 INFO:tasks.workunit.client.1.smithi193.stdout:4/630: creat db/d13/d1e/d1f/fbe x:0 0 0 2024-04-25T15:49:43.196 INFO:tasks.workunit.client.1.smithi193.stdout:0/582: dwrite d0/d49/d5/d7/fb [4194304,4194304] 0 2024-04-25T15:49:43.198 INFO:tasks.workunit.client.1.smithi193.stdout:7/535: creat d12/d31/fc7 x:0 0 0 2024-04-25T15:49:43.202 INFO:tasks.workunit.client.1.smithi193.stdout:2/567: creat d0/d9/d15/d31/d90/fea x:0 0 0 2024-04-25T15:49:43.206 INFO:tasks.workunit.client.1.smithi193.stdout:9/557: link df/d2c/d34/d3e/d4d/d5e/f6c df/d2c/d34/faa 0 2024-04-25T15:49:43.209 INFO:tasks.workunit.client.1.smithi193.stdout:8/546: getdents d2/d3/d27 0 2024-04-25T15:49:43.209 INFO:tasks.workunit.client.1.smithi193.stdout:2/568: dwrite d0/d42/d51/d4f/f69 [0,4194304] 0 2024-04-25T15:49:43.210 INFO:tasks.workunit.client.1.smithi193.stdout:1/498: unlink dc/cd 0 2024-04-25T15:49:43.211 INFO:tasks.workunit.client.1.smithi193.stdout:2/569: write d0/df/dbf/fd5 [539424,91888] 0 2024-04-25T15:49:43.216 INFO:tasks.workunit.client.1.smithi193.stdout:4/631: creat db/d32/d88/fbf x:0 0 0 2024-04-25T15:49:43.218 INFO:tasks.workunit.client.1.smithi193.stdout:5/582: link d17/d18/d30/d34/d52/l62 d17/lc8 0 2024-04-25T15:49:43.223 INFO:tasks.workunit.client.1.smithi193.stdout:4/632: fsync f9 0 2024-04-25T15:49:43.226 INFO:tasks.workunit.client.1.smithi193.stdout:1/499: link dc/f3f dc/d4a/d50/d5c/d5d/d3d/d55/d8a/fa5 0 2024-04-25T15:49:43.228 INFO:tasks.workunit.client.1.smithi193.stdout:0/583: rename d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/f95 to d0/d49/d5/d7/dd/d16/d18/fc5 0 2024-04-25T15:49:43.229 INFO:tasks.workunit.client.1.smithi193.stdout:6/550: getdents dd/d12/d19/d45 0 2024-04-25T15:49:43.232 INFO:tasks.workunit.client.1.smithi193.stdout:7/536: getdents d12/d1c 0 2024-04-25T15:49:43.232 INFO:tasks.workunit.client.1.smithi193.stdout:2/570: creat d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4/feb x:0 0 0 2024-04-25T15:49:43.233 INFO:tasks.workunit.client.1.smithi193.stdout:7/537: read f8 [200880,14642] 0 2024-04-25T15:49:43.235 INFO:tasks.workunit.client.1.smithi193.stdout:3/594: rename d8/d13/d1f/d69/d77/l4a to d8/d13/d14/d23/lb9 0 2024-04-25T15:49:43.236 INFO:tasks.workunit.client.1.smithi193.stdout:6/551: write dd/d12/f35 [3761949,26610] 0 2024-04-25T15:49:43.237 INFO:tasks.workunit.client.1.smithi193.stdout:4/633: mkdir db/d13/d1e/d1f/d56/d69/dad/dc0 0 2024-04-25T15:49:43.238 INFO:tasks.workunit.client.1.smithi193.stdout:4/634: write db/d13/d1e/d1f/f98 [1859129,8833] 0 2024-04-25T15:49:43.243 INFO:tasks.workunit.client.1.smithi193.stdout:9/558: rename df/d2c/d2f/d4e/d69/ca1 to df/d2c/d2f/d4e/d1c/cab 0 2024-04-25T15:49:43.243 INFO:tasks.workunit.client.1.smithi193.stdout:9/559: chown l7 101 1 2024-04-25T15:49:43.248 INFO:tasks.workunit.client.1.smithi193.stdout:1/500: truncate dc/d4a/d50/d5c/d5d/d3d/d55/f73 1369941 0 2024-04-25T15:49:43.249 INFO:tasks.workunit.client.1.smithi193.stdout:9/560: dwrite df/d2c/d34/f75 [0,4194304] 0 2024-04-25T15:49:43.251 INFO:tasks.workunit.client.1.smithi193.stdout:8/547: rename d2/d3/d20/d45 to d2/d3/d27/d69/dae 0 2024-04-25T15:49:43.252 INFO:tasks.workunit.client.1.smithi193.stdout:2/571: dwrite d0/d42/d51/f46 [0,4194304] 0 2024-04-25T15:49:43.252 INFO:tasks.workunit.client.1.smithi193.stdout:7/538: truncate d12/d31/d37/d62/d6e/f8c 1715532 0 2024-04-25T15:49:43.254 INFO:tasks.workunit.client.1.smithi193.stdout:4/635: symlink db/d84/lc1 0 2024-04-25T15:49:43.256 INFO:tasks.workunit.client.1.smithi193.stdout:4/636: dread - db/d13/d1e/d1f/f28 zero size 2024-04-25T15:49:43.257 INFO:tasks.workunit.client.1.smithi193.stdout:1/501: dread dc/d4a/d50/d5c/d63/f67 [0,4194304] 0 2024-04-25T15:49:43.260 INFO:tasks.workunit.client.1.smithi193.stdout:0/584: rename d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/l64 to d0/lc6 0 2024-04-25T15:49:43.260 INFO:tasks.workunit.client.1.smithi193.stdout:6/552: write dd/d2f/f5b [573473,50835] 0 2024-04-25T15:49:43.264 INFO:tasks.workunit.client.1.smithi193.stdout:8/548: mknod d2/caf 0 2024-04-25T15:49:43.266 INFO:tasks.workunit.client.1.smithi193.stdout:1/502: mkdir dc/d4a/d4d/d7a/da6 0 2024-04-25T15:49:43.268 INFO:tasks.workunit.client.1.smithi193.stdout:4/637: mknod db/d13/d1e/d1f/d56/d69/dad/cc2 0 2024-04-25T15:49:43.270 INFO:tasks.workunit.client.1.smithi193.stdout:3/595: rename d8/d13/f66 to d8/d13/d4b/d83/fba 0 2024-04-25T15:49:43.271 INFO:tasks.workunit.client.1.smithi193.stdout:2/572: creat d0/d37/d25/d5f/d98/fec x:0 0 0 2024-04-25T15:49:43.272 INFO:tasks.workunit.client.1.smithi193.stdout:6/553: creat dd/d12/d19/d1c/fa4 x:0 0 0 2024-04-25T15:49:43.275 INFO:tasks.workunit.client.1.smithi193.stdout:8/549: symlink d2/d3/d27/d69/dae/d4a/lb0 0 2024-04-25T15:49:43.275 INFO:tasks.workunit.client.1.smithi193.stdout:9/561: dread df/d11/f2a [0,4194304] 0 2024-04-25T15:49:43.276 INFO:tasks.workunit.client.1.smithi193.stdout:1/503: creat dc/d4a/d50/fa7 x:0 0 0 2024-04-25T15:49:43.276 INFO:tasks.workunit.client.1.smithi193.stdout:0/585: read d0/d49/d5/d7/dd/d46/d48/f85 [727178,1636] 0 2024-04-25T15:49:43.277 INFO:tasks.workunit.client.1.smithi193.stdout:4/638: rmdir db/d4b/d51/d54/d77 39 2024-04-25T15:49:43.279 INFO:tasks.workunit.client.1.smithi193.stdout:7/539: link d12/d1c/d45/l50 d12/d31/d37/d62/d6e/lc8 0 2024-04-25T15:49:43.281 INFO:tasks.workunit.client.1.smithi193.stdout:2/573: dwrite d0/d9/d15/d31/d50/f5c [0,4194304] 0 2024-04-25T15:49:43.281 INFO:tasks.workunit.client.1.smithi193.stdout:7/540: write f2 [2327105,9116] 0 2024-04-25T15:49:43.282 INFO:tasks.workunit.client.1.smithi193.stdout:4/639: write db/d13/d1e/d1f/fbe [855905,42676] 0 2024-04-25T15:49:43.285 INFO:tasks.workunit.client.1.smithi193.stdout:8/550: dread d2/d3/d27/d69/dae/f49 [0,4194304] 0 2024-04-25T15:49:43.286 INFO:tasks.workunit.client.1.smithi193.stdout:2/574: write d0/d37/d25/dda/fe0 [464332,119568] 0 2024-04-25T15:49:43.286 INFO:tasks.workunit.client.1.smithi193.stdout:4/640: dread - db/d13/d1e/d1f/f28 zero size 2024-04-25T15:49:43.286 INFO:tasks.workunit.client.1.smithi193.stdout:9/562: mkdir df/d2c/d2f/d5c/da3/dac 0 2024-04-25T15:49:43.288 INFO:tasks.workunit.client.1.smithi193.stdout:2/575: chown d0/d42/d51/d4f/f69 5562389 1 2024-04-25T15:49:43.289 INFO:tasks.workunit.client.1.smithi193.stdout:9/563: readlink df/d2c/d34/d3e/d4d/l82 0 2024-04-25T15:49:43.290 INFO:tasks.workunit.client.1.smithi193.stdout:1/504: fsync dc/d4a/d50/d7c/d7e/f8c 0 2024-04-25T15:49:43.290 INFO:tasks.workunit.client.1.smithi193.stdout:4/641: truncate db/d13/d1e/d1f/fbe 1585045 0 2024-04-25T15:49:43.292 INFO:tasks.workunit.client.1.smithi193.stdout:3/596: write f2 [3987018,41027] 0 2024-04-25T15:49:43.294 INFO:tasks.workunit.client.1.smithi193.stdout:3/597: stat d8/d30/c7c 0 2024-04-25T15:49:43.295 INFO:tasks.workunit.client.1.smithi193.stdout:6/554: getdents dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f 0 2024-04-25T15:49:43.297 INFO:tasks.workunit.client.1.smithi193.stdout:8/551: dread d2/d3/d27/d69/dae/d4a/d63/da2/fa4 [0,4194304] 0 2024-04-25T15:49:43.298 INFO:tasks.workunit.client.1.smithi193.stdout:4/642: chown db/d13/d1e/f48 361 1 2024-04-25T15:49:43.300 INFO:tasks.workunit.client.1.smithi193.stdout:6/555: dread - dd/d12/d19/d1c/d40/d3c/d3f/f85 zero size 2024-04-25T15:49:43.300 INFO:tasks.workunit.client.1.smithi193.stdout:9/564: dwrite df/d2c/d2f/d4e/fa0 [0,4194304] 0 2024-04-25T15:49:43.301 INFO:tasks.workunit.client.1.smithi193.stdout:1/505: unlink dc/d4a/d50/d5c/d5d/d3d/f41 0 2024-04-25T15:49:43.303 INFO:tasks.workunit.client.1.smithi193.stdout:1/506: truncate dc/f30 9434066 0 2024-04-25T15:49:43.305 INFO:tasks.workunit.client.1.smithi193.stdout:8/552: rename d2/d3/d27/d34 to d2/d18/db1 0 2024-04-25T15:49:43.307 INFO:tasks.workunit.client.1.smithi193.stdout:0/586: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/f4b [4194304,4194304] 0 2024-04-25T15:49:43.308 INFO:tasks.workunit.client.1.smithi193.stdout:8/553: read d2/d3/d3a/f54 [205217,58020] 0 2024-04-25T15:49:43.309 INFO:tasks.workunit.client.1.smithi193.stdout:5/583: sync 2024-04-25T15:49:43.309 INFO:tasks.workunit.client.1.smithi193.stdout:4/643: symlink db/d13/d1e/d1f/d56/d69/dad/lc3 0 2024-04-25T15:49:43.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/576: dread d0/d37/f48 [0,4194304] 0 2024-04-25T15:49:43.313 INFO:tasks.workunit.client.1.smithi193.stdout:5/584: truncate d17/d18/d75/d85/d5d/fb0 667485 0 2024-04-25T15:49:43.314 INFO:tasks.workunit.client.1.smithi193.stdout:1/507: symlink dc/d4a/d50/d5c/d63/la8 0 2024-04-25T15:49:43.314 INFO:tasks.workunit.client.1.smithi193.stdout:3/598: fsync d8/d13/d4b/f57 0 2024-04-25T15:49:43.317 INFO:tasks.workunit.client.1.smithi193.stdout:7/541: dread d12/f2c [0,4194304] 0 2024-04-25T15:49:43.318 INFO:tasks.workunit.client.1.smithi193.stdout:0/587: mknod d0/d49/d6a/cc7 0 2024-04-25T15:49:43.318 INFO:tasks.workunit.client.1.smithi193.stdout:2/577: unlink d0/d9/d15/d31/d50/cd1 0 2024-04-25T15:49:43.318 INFO:tasks.workunit.client.1.smithi193.stdout:3/599: dread d8/d30/f47 [4194304,4194304] 0 2024-04-25T15:49:43.320 INFO:tasks.workunit.client.1.smithi193.stdout:0/588: fsync d0/d49/d5/d7/dd/d16/d26/f55 0 2024-04-25T15:49:43.322 INFO:tasks.workunit.client.1.smithi193.stdout:9/565: fdatasync df/d2d/f31 0 2024-04-25T15:49:43.324 INFO:tasks.workunit.client.1.smithi193.stdout:6/556: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5 0 2024-04-25T15:49:43.330 INFO:tasks.workunit.client.1.smithi193.stdout:1/508: rmdir dc/d4a/d50/d5c/d5d/d3d/d55/d8a 39 2024-04-25T15:49:43.335 INFO:tasks.workunit.client.1.smithi193.stdout:4/644: mknod db/d13/d1e/d1f/d56/d69/dad/dc0/cc4 0 2024-04-25T15:49:43.337 INFO:tasks.workunit.client.1.smithi193.stdout:2/578: creat d0/fed x:0 0 0 2024-04-25T15:49:43.342 INFO:tasks.workunit.client.1.smithi193.stdout:8/554: rename d2/d3/d27/d69/d86/d72/f87 to d2/d18/da0/fb2 0 2024-04-25T15:49:43.344 INFO:tasks.workunit.client.1.smithi193.stdout:8/555: truncate d2/d3/d27/d69/dae/d29/fac 290037 0 2024-04-25T15:49:43.349 INFO:tasks.workunit.client.1.smithi193.stdout:5/585: creat d17/d18/d30/d38/dab/fc9 x:0 0 0 2024-04-25T15:49:43.351 INFO:tasks.workunit.client.1.smithi193.stdout:1/509: mknod dc/d4a/ca9 0 2024-04-25T15:49:43.354 INFO:tasks.workunit.client.1.smithi193.stdout:4/645: mknod db/d13/d1e/d1f/d22/cc5 0 2024-04-25T15:49:43.355 INFO:tasks.workunit.client.1.smithi193.stdout:2/579: creat d0/d37/d25/d5f/d3a/d6d/d6e/db9/fee x:0 0 0 2024-04-25T15:49:43.360 INFO:tasks.workunit.client.1.smithi193.stdout:9/566: rename df/d2c/d2f/d4e/c68 to df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/cad 0 2024-04-25T15:49:43.361 INFO:tasks.workunit.client.1.smithi193.stdout:0/589: getdents d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53 0 2024-04-25T15:49:43.361 INFO:tasks.workunit.client.1.smithi193.stdout:9/567: chown df/d2c/d2f/d4e/c22 192104 1 2024-04-25T15:49:43.362 INFO:tasks.workunit.client.1.smithi193.stdout:7/542: dwrite d12/d13/f43 [0,4194304] 0 2024-04-25T15:49:43.363 INFO:tasks.workunit.client.1.smithi193.stdout:8/556: fdatasync d2/d3/d20/f33 0 2024-04-25T15:49:43.364 INFO:tasks.workunit.client.1.smithi193.stdout:0/590: write d0/d49/d5/d7/dd/d16/d18/fac [655143,73127] 0 2024-04-25T15:49:43.365 INFO:tasks.workunit.client.1.smithi193.stdout:8/557: stat d2/d3/d27/l6a 0 2024-04-25T15:49:43.366 INFO:tasks.workunit.client.1.smithi193.stdout:5/586: rmdir d17/d18/d75/d85/d5d 39 2024-04-25T15:49:43.368 INFO:tasks.workunit.client.1.smithi193.stdout:5/587: chown d17/d18/c78 40642122 1 2024-04-25T15:49:43.368 INFO:tasks.workunit.client.1.smithi193.stdout:9/568: dread df/d2c/d2f/f6e [0,4194304] 0 2024-04-25T15:49:43.371 INFO:tasks.workunit.client.1.smithi193.stdout:4/646: creat db/d13/d1e/fc6 x:0 0 0 2024-04-25T15:49:43.372 INFO:tasks.workunit.client.1.smithi193.stdout:2/580: creat d0/d42/fef x:0 0 0 2024-04-25T15:49:43.381 INFO:tasks.workunit.client.1.smithi193.stdout:6/557: creat dd/d12/d19/d1c/d40/fa6 x:0 0 0 2024-04-25T15:49:43.384 INFO:tasks.workunit.client.1.smithi193.stdout:0/591: rename d0/d49/d5/d7/dd/d16/f3a to d0/d49/d5/d7/dd/d16/d26/d5e/fc8 0 2024-04-25T15:49:43.384 INFO:tasks.workunit.client.1.smithi193.stdout:3/600: write d8/d13/d14/f80 [559188,82573] 0 2024-04-25T15:49:43.385 INFO:tasks.workunit.client.1.smithi193.stdout:5/588: mkdir d17/d18/d30/d38/dca 0 2024-04-25T15:49:43.389 INFO:tasks.workunit.client.1.smithi193.stdout:4/647: readlink db/lf 0 2024-04-25T15:49:43.394 INFO:tasks.workunit.client.1.smithi193.stdout:5/589: truncate d17/d18/d75/d85/d5d/dad/fb5 794676 0 2024-04-25T15:49:43.395 INFO:tasks.workunit.client.1.smithi193.stdout:7/543: rmdir d12/d1c/d63/d70/dc6 0 2024-04-25T15:49:43.396 INFO:tasks.workunit.client.1.smithi193.stdout:7/544: write d12/d1c/d63/d70/d68/fb4 [159532,18440] 0 2024-04-25T15:49:43.397 INFO:tasks.workunit.client.1.smithi193.stdout:4/648: dread db/d13/d1e/d1f/d56/d69/d70/f76 [0,4194304] 0 2024-04-25T15:49:43.398 INFO:tasks.workunit.client.1.smithi193.stdout:7/545: read d12/d1c/d63/d70/d68/d8d/f9c [87590,69689] 0 2024-04-25T15:49:43.398 INFO:tasks.workunit.client.1.smithi193.stdout:1/510: rename dc/d4a/d50/d5c/d63/f76 to dc/d4a/d4d/d80/faa 0 2024-04-25T15:49:43.400 INFO:tasks.workunit.client.1.smithi193.stdout:3/601: creat d8/d13/d1f/d69/d77/fbb x:0 0 0 2024-04-25T15:49:43.400 INFO:tasks.workunit.client.1.smithi193.stdout:3/602: chown d8/d13/d1f/d69/d77/f3a 106 1 2024-04-25T15:49:43.401 INFO:tasks.workunit.client.1.smithi193.stdout:0/592: mknod d0/d49/d5/d7/cc9 0 2024-04-25T15:49:43.406 INFO:tasks.workunit.client.1.smithi193.stdout:2/581: write d0/d42/d51/f40 [2853269,110101] 0 2024-04-25T15:49:43.406 INFO:tasks.workunit.client.1.smithi193.stdout:1/511: mkdir dc/d4a/d50/dab 0 2024-04-25T15:49:43.407 INFO:tasks.workunit.client.1.smithi193.stdout:1/512: fsync dc/f17 0 2024-04-25T15:49:43.408 INFO:tasks.workunit.client.1.smithi193.stdout:8/558: write d2/f48 [936125,1673] 0 2024-04-25T15:49:43.408 INFO:tasks.workunit.client.1.smithi193.stdout:2/582: readlink d0/d42/d51/d4f/d65/la1 0 2024-04-25T15:49:43.410 INFO:tasks.workunit.client.1.smithi193.stdout:2/583: write d0/d42/d51/d4f/fc4 [601506,57569] 0 2024-04-25T15:49:43.414 INFO:tasks.workunit.client.1.smithi193.stdout:7/546: dwrite d12/d31/d37/d62/d6e/d79/f74 [0,4194304] 0 2024-04-25T15:49:43.415 INFO:tasks.workunit.client.1.smithi193.stdout:3/603: dread d8/d30/d45/f70 [0,4194304] 0 2024-04-25T15:49:43.415 INFO:tasks.workunit.client.1.smithi193.stdout:5/590: mkdir d17/d18/dcb 0 2024-04-25T15:49:43.416 INFO:tasks.workunit.client.1.smithi193.stdout:7/547: truncate d12/d1c/d63/d70/faa 1085867 0 2024-04-25T15:49:43.418 INFO:tasks.workunit.client.1.smithi193.stdout:5/591: truncate d17/d18/d30/d38/dab/fc9 244842 0 2024-04-25T15:49:43.418 INFO:tasks.workunit.client.1.smithi193.stdout:4/649: mknod db/d32/cc7 0 2024-04-25T15:49:43.418 INFO:tasks.workunit.client.1.smithi193.stdout:0/593: rmdir d0/d49 39 2024-04-25T15:49:43.419 INFO:tasks.workunit.client.1.smithi193.stdout:5/592: readlink d17/d18/d75/d85/l54 0 2024-04-25T15:49:43.422 INFO:tasks.workunit.client.1.smithi193.stdout:6/558: rename dd/d12/d19/d1c/d40/d3c/d3f/f41 to dd/d12/d19/d1c/d40/d3c/fa7 0 2024-04-25T15:49:43.423 INFO:tasks.workunit.client.1.smithi193.stdout:6/559: chown dd/d12/d19/d1c/d40/d3c 0 1 2024-04-25T15:49:43.489 INFO:tasks.workunit.client.1.smithi193.stdout:2/584: mknod d0/d37/dbb/cf0 0 2024-04-25T15:49:43.489 INFO:tasks.workunit.client.1.smithi193.stdout:6/560: stat dd/d2f/d57/la3 0 2024-04-25T15:49:43.491 INFO:tasks.workunit.client.1.smithi193.stdout:4/650: link db/f61 db/d32/d45/dab/fc8 0 2024-04-25T15:49:43.495 INFO:tasks.workunit.client.1.smithi193.stdout:1/513: rename dc/c6b to dc/d4a/d4d/d7a/cac 0 2024-04-25T15:49:43.496 INFO:tasks.workunit.client.1.smithi193.stdout:2/585: write d0/d9/d15/d31/f91 [4028416,62034] 0 2024-04-25T15:49:43.497 INFO:tasks.workunit.client.1.smithi193.stdout:6/561: truncate dd/d12/d19/d45/f55 537706 0 2024-04-25T15:49:43.498 INFO:tasks.workunit.client.1.smithi193.stdout:7/548: truncate f2 4489951 0 2024-04-25T15:49:43.498 INFO:tasks.workunit.client.1.smithi193.stdout:4/651: fsync f1 0 2024-04-25T15:49:43.499 INFO:tasks.workunit.client.1.smithi193.stdout:6/562: write dd/d12/d19/d1c/fa4 [260814,47253] 0 2024-04-25T15:49:43.501 INFO:tasks.workunit.client.1.smithi193.stdout:3/604: getdents d8/d13/d4b 0 2024-04-25T15:49:43.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/569: sync 2024-04-25T15:49:43.505 INFO:tasks.workunit.client.1.smithi193.stdout:5/593: getdents d17/d18/d30/d34/d52/daf 0 2024-04-25T15:49:43.506 INFO:tasks.workunit.client.1.smithi193.stdout:9/570: chown df/d2c/d2f/d4e/d1c/d30/f7a 3089 1 2024-04-25T15:49:43.507 INFO:tasks.workunit.client.1.smithi193.stdout:0/594: dwrite d0/d49/d5/d7/dd/d38/f4c [0,4194304] 0 2024-04-25T15:49:43.508 INFO:tasks.workunit.client.1.smithi193.stdout:3/605: stat d8/d13/l2a 0 2024-04-25T15:49:43.509 INFO:tasks.workunit.client.1.smithi193.stdout:5/594: write d17/d18/fa8 [235014,77627] 0 2024-04-25T15:49:43.510 INFO:tasks.workunit.client.1.smithi193.stdout:7/549: creat d12/d31/dc3/fc9 x:0 0 0 2024-04-25T15:49:43.513 INFO:tasks.workunit.client.1.smithi193.stdout:1/514: dwrite dc/d4a/d4d/d80/faa [0,4194304] 0 2024-04-25T15:49:43.513 INFO:tasks.workunit.client.1.smithi193.stdout:6/563: link dd/d2f/d57/f7d dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/fa8 0 2024-04-25T15:49:43.514 INFO:tasks.workunit.client.1.smithi193.stdout:5/595: truncate d17/d18/d75/f9c 4321028 0 2024-04-25T15:49:43.515 INFO:tasks.workunit.client.1.smithi193.stdout:9/571: mknod df/d2c/d34/d3e/cae 0 2024-04-25T15:49:43.517 INFO:tasks.workunit.client.1.smithi193.stdout:4/652: link db/f3b db/d4b/d51/d54/d77/fc9 0 2024-04-25T15:49:43.518 INFO:tasks.workunit.client.1.smithi193.stdout:2/586: getdents d0/d37/d25/d5f/d3a/d6d/d6e/db9 0 2024-04-25T15:49:43.519 INFO:tasks.workunit.client.1.smithi193.stdout:8/559: sync 2024-04-25T15:49:43.530 INFO:tasks.workunit.client.1.smithi193.stdout:0/595: fdatasync d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f3f 0 2024-04-25T15:49:43.530 INFO:tasks.workunit.client.1.smithi193.stdout:2/587: stat d0/d9/d15/d31/d90/d9c/ldf 0 2024-04-25T15:49:43.531 INFO:tasks.workunit.client.1.smithi193.stdout:4/653: creat db/d13/d1e/d1f/d22/fca x:0 0 0 2024-04-25T15:49:43.532 INFO:tasks.workunit.client.1.smithi193.stdout:3/606: write d8/ff [311211,65146] 0 2024-04-25T15:49:43.532 INFO:tasks.workunit.client.1.smithi193.stdout:1/515: symlink dc/lad 0 2024-04-25T15:49:43.533 INFO:tasks.workunit.client.1.smithi193.stdout:6/564: mknod dd/d2f/ca9 0 2024-04-25T15:49:43.533 INFO:tasks.workunit.client.1.smithi193.stdout:9/572: write df/d2c/d2f/f6e [236994,93780] 0 2024-04-25T15:49:43.536 INFO:tasks.workunit.client.1.smithi193.stdout:2/588: rmdir d0/d37/d25/d5f 39 2024-04-25T15:49:43.536 INFO:tasks.workunit.client.1.smithi193.stdout:8/560: creat d2/d18/db1/fb3 x:0 0 0 2024-04-25T15:49:43.539 INFO:tasks.workunit.client.1.smithi193.stdout:0/596: creat d0/d49/d5/fca x:0 0 0 2024-04-25T15:49:43.542 INFO:tasks.workunit.client.1.smithi193.stdout:2/589: unlink d0/d9/d15/f79 0 2024-04-25T15:49:43.543 INFO:tasks.workunit.client.1.smithi193.stdout:5/596: dwrite d17/d18/d30/d38/d48/d4f/f5f [0,4194304] 0 2024-04-25T15:49:43.543 INFO:tasks.workunit.client.1.smithi193.stdout:3/607: rmdir d8/d13/d14/d27/d54/db3 0 2024-04-25T15:49:43.545 INFO:tasks.workunit.client.1.smithi193.stdout:5/597: fsync f11 0 2024-04-25T15:49:43.547 INFO:tasks.workunit.client.1.smithi193.stdout:4/654: dwrite db/d13/d1e/d1f/fa5 [0,4194304] 0 2024-04-25T15:49:43.547 INFO:tasks.workunit.client.1.smithi193.stdout:1/516: rename dc/l28 to dc/d4a/d50/d5c/d5d/d3d/d55/d8a/lae 0 2024-04-25T15:49:43.548 INFO:tasks.workunit.client.1.smithi193.stdout:8/561: creat d2/fb4 x:0 0 0 2024-04-25T15:49:43.552 INFO:tasks.workunit.client.1.smithi193.stdout:8/562: chown d2/d18/l4b 2423860 1 2024-04-25T15:49:43.555 INFO:tasks.workunit.client.1.smithi193.stdout:5/598: mknod d17/d18/d30/d38/d71/d96/dc4/ccc 0 2024-04-25T15:49:43.555 INFO:tasks.workunit.client.1.smithi193.stdout:0/597: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/ccb 0 2024-04-25T15:49:43.557 INFO:tasks.workunit.client.1.smithi193.stdout:9/573: rename df/d2c/d2f/d33 to df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf 0 2024-04-25T15:49:43.558 INFO:tasks.workunit.client.1.smithi193.stdout:8/563: symlink d2/d3/d27/d6f/lb5 0 2024-04-25T15:49:43.558 INFO:tasks.workunit.client.1.smithi193.stdout:4/655: mkdir db/d13/dcb 0 2024-04-25T15:49:43.559 INFO:tasks.workunit.client.1.smithi193.stdout:1/517: symlink dc/d4a/d50/d5c/d5d/d3d/d55/d8a/laf 0 2024-04-25T15:49:43.559 INFO:tasks.workunit.client.1.smithi193.stdout:5/599: readlink d17/d18/d30/d38/d71/l90 0 2024-04-25T15:49:43.559 INFO:tasks.workunit.client.1.smithi193.stdout:6/565: dread dd/d2f/f56 [0,4194304] 0 2024-04-25T15:49:43.562 INFO:tasks.workunit.client.1.smithi193.stdout:3/608: dwrite d8/d13/f7b [0,4194304] 0 2024-04-25T15:49:43.562 INFO:tasks.workunit.client.1.smithi193.stdout:0/598: mknod d0/d49/d5/d7/dd/d46/ccc 0 2024-04-25T15:49:43.566 INFO:tasks.workunit.client.1.smithi193.stdout:9/574: mknod df/d2c/d2f/d4e/d1c/d30/d3b/cb0 0 2024-04-25T15:49:43.566 INFO:tasks.workunit.client.1.smithi193.stdout:0/599: chown d0/fc 2940 1 2024-04-25T15:49:43.566 INFO:tasks.workunit.client.1.smithi193.stdout:5/600: mknod d17/d18/d30/d38/dab/ccd 0 2024-04-25T15:49:43.567 INFO:tasks.workunit.client.1.smithi193.stdout:1/518: mkdir dc/d4a/d50/d5c/d5d/d3d/db0 0 2024-04-25T15:49:43.567 INFO:tasks.workunit.client.1.smithi193.stdout:3/609: chown d8/d13/d1f/d69/d77/la4 948 1 2024-04-25T15:49:43.568 INFO:tasks.workunit.client.1.smithi193.stdout:5/601: fdatasync d17/d18/d30/d38/d71/d96/dc4/f47 0 2024-04-25T15:49:43.568 INFO:tasks.workunit.client.1.smithi193.stdout:1/519: dread - dc/d3c/f8b zero size 2024-04-25T15:49:43.569 INFO:tasks.workunit.client.1.smithi193.stdout:3/610: write d8/d30/d72/fa3 [4606693,6581] 0 2024-04-25T15:49:43.571 INFO:tasks.workunit.client.1.smithi193.stdout:0/600: dread d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f59 [4194304,4194304] 0 2024-04-25T15:49:43.576 INFO:tasks.workunit.client.1.smithi193.stdout:3/611: write d8/d13/d14/d27/d6d/f91 [2248137,79266] 0 2024-04-25T15:49:43.579 INFO:tasks.workunit.client.1.smithi193.stdout:3/612: write d8/d13/d1f/d69/d77/d71/f86 [1087061,28956] 0 2024-04-25T15:49:43.582 INFO:tasks.workunit.client.1.smithi193.stdout:2/590: dread d0/d42/d51/f40 [0,4194304] 0 2024-04-25T15:49:43.583 INFO:tasks.workunit.client.1.smithi193.stdout:2/591: write d0/df/dbf/fd5 [161072,41149] 0 2024-04-25T15:49:43.584 INFO:tasks.workunit.client.1.smithi193.stdout:2/592: chown d0/d9/d15/d31/l5e 31 1 2024-04-25T15:49:43.585 INFO:tasks.workunit.client.1.smithi193.stdout:2/593: dread - d0/d9/d15/d31/d90/d9c/fdd zero size 2024-04-25T15:49:43.590 INFO:tasks.workunit.client.1.smithi193.stdout:4/656: dread db/d13/d1e/d1f/d22/f6b [0,4194304] 0 2024-04-25T15:49:43.599 INFO:tasks.workunit.client.0.smithi110.stdout:8/867: sync 2024-04-25T15:49:43.642 INFO:tasks.workunit.client.1.smithi193.stdout:7/550: sync 2024-04-25T15:49:43.656 INFO:tasks.workunit.client.1.smithi193.stdout:6/566: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa x:0 0 0 2024-04-25T15:49:43.660 INFO:tasks.workunit.client.0.smithi110.stdout:8/868: symlink d14/d32/d36/l12c 0 2024-04-25T15:49:43.661 INFO:tasks.workunit.client.0.smithi110.stdout:8/869: chown d14/d21/d26/d6a/d7e/f12b 63346814 1 2024-04-25T15:49:43.664 INFO:tasks.workunit.client.1.smithi193.stdout:1/520: mknod dc/d59/cb1 0 2024-04-25T15:49:43.665 INFO:tasks.workunit.client.1.smithi193.stdout:1/521: dread - dc/d4a/d4d/f5e zero size 2024-04-25T15:49:43.666 INFO:tasks.workunit.client.1.smithi193.stdout:1/522: write dc/f36 [27612,62316] 0 2024-04-25T15:49:43.666 INFO:tasks.workunit.client.1.smithi193.stdout:9/575: dread df/d2c/d2f/d4e/f37 [0,4194304] 0 2024-04-25T15:49:43.667 INFO:tasks.workunit.client.0.smithi110.stdout:8/870: truncate d14/d21/d26/d6a/d7e/da6/de8/d55/d8d/feb 1060898 0 2024-04-25T15:49:43.669 INFO:tasks.workunit.client.1.smithi193.stdout:2/594: sync 2024-04-25T15:49:43.669 INFO:tasks.workunit.client.1.smithi193.stdout:0/601: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/lb7 54629826 1 2024-04-25T15:49:43.672 INFO:tasks.workunit.client.1.smithi193.stdout:3/613: mknod d8/d13/d14/d51/cbc 0 2024-04-25T15:49:43.676 INFO:tasks.workunit.client.1.smithi193.stdout:4/657: creat db/d13/d1e/d1f/d22/fcc x:0 0 0 2024-04-25T15:49:43.680 INFO:tasks.workunit.client.0.smithi110.stdout:8/871: link d14/d21/d26/d6a/l11a d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/ddd/l12d 0 2024-04-25T15:49:43.680 INFO:tasks.workunit.client.0.smithi110.stdout:8/872: chown d14/d32/d36/db7 6469222 1 2024-04-25T15:49:43.681 INFO:tasks.workunit.client.1.smithi193.stdout:8/564: getdents d2/d3/d27/d69/dae 0 2024-04-25T15:49:43.687 INFO:tasks.workunit.client.1.smithi193.stdout:8/565: dwrite d2/d18/db1/fb3 [0,4194304] 0 2024-04-25T15:49:43.688 INFO:tasks.workunit.client.1.smithi193.stdout:8/566: write d2/d18/db1/fb3 [1481405,35951] 0 2024-04-25T15:49:43.689 INFO:tasks.workunit.client.1.smithi193.stdout:9/576: creat df/d2d/fb1 x:0 0 0 2024-04-25T15:49:43.692 INFO:tasks.workunit.client.1.smithi193.stdout:2/595: write d0/d9/d3d/d7d/fab [2499368,25503] 0 2024-04-25T15:49:43.694 INFO:tasks.workunit.client.1.smithi193.stdout:0/602: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/ccd 0 2024-04-25T15:49:43.695 INFO:tasks.workunit.client.1.smithi193.stdout:0/603: readlink d0/d49/d5/d7/dd/d46/l72 0 2024-04-25T15:49:43.696 INFO:tasks.workunit.client.1.smithi193.stdout:3/614: symlink d8/d30/d72/lbd 0 2024-04-25T15:49:43.698 INFO:tasks.workunit.client.1.smithi193.stdout:4/658: creat db/d32/d45/fcd x:0 0 0 2024-04-25T15:49:43.701 INFO:tasks.workunit.client.1.smithi193.stdout:7/551: unlink d12/d31/d37/d62/d6e/d79/cbd 0 2024-04-25T15:49:43.701 INFO:tasks.workunit.client.1.smithi193.stdout:7/552: fdatasync d12/f2e 0 2024-04-25T15:49:43.703 INFO:tasks.workunit.client.1.smithi193.stdout:6/567: symlink dd/d12/d19/d1c/d40/lab 0 2024-04-25T15:49:43.714 INFO:tasks.workunit.client.1.smithi193.stdout:2/596: rename d0/d9/d15/d31/d90 to d0/d42/d51/df1 0 2024-04-25T15:49:43.715 INFO:tasks.workunit.client.1.smithi193.stdout:2/597: fsync d0/d42/d51/f94 0 2024-04-25T15:49:43.716 INFO:tasks.workunit.client.1.smithi193.stdout:0/604: fdatasync d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f3f 0 2024-04-25T15:49:43.716 INFO:tasks.workunit.client.1.smithi193.stdout:3/615: dread d8/d13/d14/d27/f5a [0,4194304] 0 2024-04-25T15:49:43.723 INFO:tasks.workunit.client.1.smithi193.stdout:7/553: dread d12/d31/d39/f53 [0,4194304] 0 2024-04-25T15:49:43.732 INFO:tasks.workunit.client.1.smithi193.stdout:9/577: link df/d11/d8b/f4f df/d2c/d2f/d5c/fb2 0 2024-04-25T15:49:43.735 INFO:tasks.workunit.client.1.smithi193.stdout:7/554: symlink d12/da1/lca 0 2024-04-25T15:49:43.738 INFO:tasks.workunit.client.1.smithi193.stdout:0/605: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf/cce 0 2024-04-25T15:49:43.740 INFO:tasks.workunit.client.1.smithi193.stdout:7/555: dwrite d12/d31/fc7 [0,4194304] 0 2024-04-25T15:49:43.741 INFO:tasks.workunit.client.1.smithi193.stdout:7/556: stat d12/d1c/d63/d70/f42 0 2024-04-25T15:49:43.742 INFO:tasks.workunit.client.1.smithi193.stdout:0/606: mkdir d0/d49/d5/d7/dd/d16/d18/d28/dcf 0 2024-04-25T15:49:43.743 INFO:tasks.workunit.client.1.smithi193.stdout:7/557: write d12/d1c/d45/f7a [86752,32749] 0 2024-04-25T15:49:43.745 INFO:tasks.workunit.client.1.smithi193.stdout:7/558: mknod d12/d31/d37/d92/ccb 0 2024-04-25T15:49:43.746 INFO:tasks.workunit.client.1.smithi193.stdout:7/559: dread - d12/d1c/f6d zero size 2024-04-25T15:49:43.747 INFO:tasks.workunit.client.1.smithi193.stdout:8/567: symlink d2/d3/d27/d69/dae/d4a/lb6 0 2024-04-25T15:49:43.747 INFO:tasks.workunit.client.1.smithi193.stdout:8/568: readlink d2/d3/d27/d6f/laa 0 2024-04-25T15:49:43.753 INFO:tasks.workunit.client.1.smithi193.stdout:0/607: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f59 [0,4194304] 0 2024-04-25T15:49:43.755 INFO:tasks.workunit.client.1.smithi193.stdout:7/560: getdents d12/d1c/d63/d70/d68/d8d 0 2024-04-25T15:49:43.758 INFO:tasks.workunit.client.0.smithi110.stdout:2/708: sync 2024-04-25T15:49:43.758 INFO:tasks.workunit.client.0.smithi110.stdout:1/936: sync 2024-04-25T15:49:43.762 INFO:tasks.workunit.client.1.smithi193.stdout:7/561: dwrite d12/d31/dc3/fc9 [0,4194304] 0 2024-04-25T15:49:43.764 INFO:tasks.workunit.client.1.smithi193.stdout:7/562: chown d12/d31/d37/d62/d6e/d79/c86 878 1 2024-04-25T15:49:43.771 INFO:tasks.workunit.client.1.smithi193.stdout:0/608: dread d0/d49/d5/d7/dd/f58 [0,4194304] 0 2024-04-25T15:49:43.772 INFO:tasks.workunit.client.0.smithi110.stdout:2/709: getdents de/d40/d70/dc6 0 2024-04-25T15:49:43.777 INFO:tasks.workunit.client.1.smithi193.stdout:0/609: readlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/lbe 0 2024-04-25T15:49:43.826 INFO:tasks.workunit.client.1.smithi193.stdout:5/602: write d17/d18/d30/f35 [880833,102772] 0 2024-04-25T15:49:43.826 INFO:tasks.workunit.client.1.smithi193.stdout:1/523: write dc/f3f [3270297,97433] 0 2024-04-25T15:49:43.826 INFO:tasks.workunit.client.0.smithi110.stdout:8/873: write d14/d32/d36/db7/dc9/ff0 [745193,101978] 0 2024-04-25T15:49:43.828 INFO:tasks.workunit.client.1.smithi193.stdout:1/524: chown dc/d4a/d50/d5c/d5d/f3a 17029 1 2024-04-25T15:49:43.830 INFO:tasks.workunit.client.0.smithi110.stdout:9/744: sync 2024-04-25T15:49:43.830 INFO:tasks.workunit.client.0.smithi110.stdout:3/809: sync 2024-04-25T15:49:43.831 INFO:tasks.workunit.client.0.smithi110.stdout:0/747: sync 2024-04-25T15:49:43.833 INFO:tasks.workunit.client.0.smithi110.stdout:0/748: read d1/d23/d30/d33/d35/f36 [223191,11914] 0 2024-04-25T15:49:43.835 INFO:tasks.workunit.client.1.smithi193.stdout:1/525: dwrite dc/d4a/d50/d5c/d5d/fa0 [0,4194304] 0 2024-04-25T15:49:43.838 INFO:tasks.workunit.client.0.smithi110.stdout:9/745: read d9/d13/d7c/fb7 [743277,101210] 0 2024-04-25T15:49:43.844 INFO:tasks.workunit.client.0.smithi110.stdout:0/749: creat d1/d23/d30/d33/d5b/d83/f106 x:0 0 0 2024-04-25T15:49:43.844 INFO:tasks.workunit.client.1.smithi193.stdout:5/603: dread d17/d18/d30/d38/d71/f74 [0,4194304] 0 2024-04-25T15:49:43.845 INFO:tasks.workunit.client.1.smithi193.stdout:1/526: fsync dc/d4a/d4d/f57 0 2024-04-25T15:49:43.858 INFO:tasks.workunit.client.0.smithi110.stdout:0/750: getdents d1/d23/d30/d33/d35 0 2024-04-25T15:49:43.858 INFO:tasks.workunit.client.0.smithi110.stdout:0/751: write d1/d23/d81/fc3 [711756,8610] 0 2024-04-25T15:49:43.861 INFO:tasks.workunit.client.1.smithi193.stdout:4/659: dwrite db/d32/f6e [0,4194304] 0 2024-04-25T15:49:43.865 INFO:tasks.workunit.client.1.smithi193.stdout:4/660: mkdir db/d13/dce 0 2024-04-25T15:49:43.867 INFO:tasks.workunit.client.1.smithi193.stdout:4/661: creat db/d13/d1e/d1f/fcf x:0 0 0 2024-04-25T15:49:43.876 INFO:tasks.workunit.client.1.smithi193.stdout:8/569: write d2/d3/d27/d69/dae/d4a/d53/f58 [2122287,90370] 0 2024-04-25T15:49:43.877 INFO:tasks.workunit.client.0.smithi110.stdout:0/752: dread d1/d47/d54/f71 [0,4194304] 0 2024-04-25T15:49:43.878 INFO:tasks.workunit.client.1.smithi193.stdout:8/570: chown d2/d3/d27/d69/dae/d4a/lb0 205808022 1 2024-04-25T15:49:43.879 INFO:tasks.workunit.client.1.smithi193.stdout:6/568: creat dd/d12/d19/d1c/d40/d3c/d3f/fac x:0 0 0 2024-04-25T15:49:43.882 INFO:tasks.workunit.client.0.smithi110.stdout:0/753: rename d1/d47/d54/d99 to d1/d23/d30/d33/d5b/d78/d107 0 2024-04-25T15:49:43.884 INFO:tasks.workunit.client.0.smithi110.stdout:1/937: dwrite d0/d14/d81/df8/ff4 [0,4194304] 0 2024-04-25T15:49:43.886 INFO:tasks.workunit.client.0.smithi110.stdout:0/754: truncate d1/d6c/f94 1002966 0 2024-04-25T15:49:43.887 INFO:tasks.workunit.client.0.smithi110.stdout:0/755: stat d1/d23/d30/d33/d5b/dbb/da8/laa 0 2024-04-25T15:49:43.888 INFO:tasks.workunit.client.1.smithi193.stdout:7/563: dwrite d12/d31/d37/fc5 [0,4194304] 0 2024-04-25T15:49:43.889 INFO:tasks.workunit.client.1.smithi193.stdout:7/564: chown d12/d31/dc3/fc9 7554 1 2024-04-25T15:49:43.890 INFO:tasks.workunit.client.0.smithi110.stdout:1/938: mknod d0/d14/d81/df8/d33/d94/ddb/c12e 0 2024-04-25T15:49:43.892 INFO:tasks.workunit.client.0.smithi110.stdout:0/756: mkdir d1/d23/dc5/d108 0 2024-04-25T15:49:43.893 INFO:tasks.workunit.client.0.smithi110.stdout:2/710: dwrite de/d40/d6e/d50/db7/faa [0,4194304] 0 2024-04-25T15:49:43.893 INFO:tasks.workunit.client.1.smithi193.stdout:8/571: link d2/d3/d27/d69/d86/d72/l98 d2/d3/d27/d6f/lb7 0 2024-04-25T15:49:43.896 INFO:tasks.workunit.client.1.smithi193.stdout:6/569: link dd/d12/d19/d1c/d40/l33 dd/d12/lad 0 2024-04-25T15:49:43.901 INFO:tasks.workunit.client.1.smithi193.stdout:8/572: unlink d2/d3/d20/f33 0 2024-04-25T15:49:43.902 INFO:tasks.workunit.client.0.smithi110.stdout:2/711: rmdir de/d40/d6e/d50 39 2024-04-25T15:49:43.907 INFO:tasks.workunit.client.0.smithi110.stdout:1/939: link d0/d51/fdf d0/d14/d81/f12f 0 2024-04-25T15:49:43.910 INFO:tasks.workunit.client.0.smithi110.stdout:1/940: creat d0/d14/d5f/d6f/f130 x:0 0 0 2024-04-25T15:49:43.911 INFO:tasks.workunit.client.1.smithi193.stdout:7/565: getdents d12/d1c/d63/d70 0 2024-04-25T15:49:43.912 INFO:tasks.workunit.client.1.smithi193.stdout:8/573: getdents d2/d3/d27/d69/dae/d29 0 2024-04-25T15:49:43.913 INFO:tasks.workunit.client.0.smithi110.stdout:1/941: mkdir d0/d14/d81/df8/d33/d8e/d109/d131 0 2024-04-25T15:49:43.914 INFO:tasks.workunit.client.1.smithi193.stdout:8/574: mkdir d2/d3/d20/db8 0 2024-04-25T15:49:43.916 INFO:tasks.workunit.client.1.smithi193.stdout:7/566: truncate d12/d13/f41 161527 0 2024-04-25T15:49:43.916 INFO:tasks.workunit.client.1.smithi193.stdout:7/567: stat d12/d31/d37 0 2024-04-25T15:49:43.917 INFO:tasks.workunit.client.0.smithi110.stdout:1/942: symlink d0/d14/d5f/d7d/d72/d61/l132 0 2024-04-25T15:49:43.918 INFO:tasks.workunit.client.1.smithi193.stdout:8/575: creat d2/d3/d27/d69/dae/d4a/d63/da2/fb9 x:0 0 0 2024-04-25T15:49:43.920 INFO:tasks.workunit.client.1.smithi193.stdout:7/568: mkdir d12/d13/dcc 0 2024-04-25T15:49:43.923 INFO:tasks.workunit.client.1.smithi193.stdout:6/570: dread dd/d12/f35 [0,4194304] 0 2024-04-25T15:49:43.924 INFO:tasks.workunit.client.1.smithi193.stdout:8/576: mknod d2/d3/d20/db8/cba 0 2024-04-25T15:49:43.925 INFO:tasks.workunit.client.1.smithi193.stdout:8/577: rmdir d2/d18/da0 39 2024-04-25T15:49:43.925 INFO:tasks.workunit.client.1.smithi193.stdout:6/571: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa [568422,123474] 0 2024-04-25T15:49:43.926 INFO:tasks.workunit.client.1.smithi193.stdout:8/578: readlink d2/d3/d27/d69/dae/d4a/lb0 0 2024-04-25T15:49:43.927 INFO:tasks.workunit.client.1.smithi193.stdout:3/616: rename d8/d13/d1f/f79 to d8/d30/d45/fbe 0 2024-04-25T15:49:43.927 INFO:tasks.workunit.client.1.smithi193.stdout:3/617: fsync f2 0 2024-04-25T15:49:43.929 INFO:tasks.workunit.client.1.smithi193.stdout:2/598: unlink d0/d37/d25/d5f/d3a/d6d/la2 0 2024-04-25T15:49:43.931 INFO:tasks.workunit.client.1.smithi193.stdout:3/618: rename d8/d13/d4b/d83 to d8/d13/d1f/dbf 0 2024-04-25T15:49:43.932 INFO:tasks.workunit.client.1.smithi193.stdout:2/599: truncate d0/d42/d51/d4f/d65/d6a/f3b 3442759 0 2024-04-25T15:49:43.935 INFO:tasks.workunit.client.1.smithi193.stdout:9/578: mkdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/db3 0 2024-04-25T15:49:43.935 INFO:tasks.workunit.client.1.smithi193.stdout:3/619: dread d8/d13/d1f/f16 [4194304,4194304] 0 2024-04-25T15:49:43.936 INFO:tasks.workunit.client.1.smithi193.stdout:0/610: rename d0/d49/d5/d7/dd/lbb to d0/d49/d5/d7/dd/ld0 0 2024-04-25T15:49:43.936 INFO:tasks.workunit.client.1.smithi193.stdout:2/600: truncate d0/d42/d51/df1/d9f/dc5/fe7 912068 0 2024-04-25T15:49:43.939 INFO:tasks.workunit.client.1.smithi193.stdout:6/572: dread dd/d2f/d57/f8b [0,4194304] 0 2024-04-25T15:49:43.939 INFO:tasks.workunit.client.1.smithi193.stdout:8/579: dread d2/f48 [0,4194304] 0 2024-04-25T15:49:43.942 INFO:tasks.workunit.client.1.smithi193.stdout:2/601: dread d0/d37/fb6 [0,4194304] 0 2024-04-25T15:49:43.942 INFO:tasks.workunit.client.0.smithi110.stdout:3/810: dwrite d0/d2e/f4f [0,4194304] 0 2024-04-25T15:49:43.943 INFO:tasks.workunit.client.1.smithi193.stdout:4/662: creat db/fd0 x:0 0 0 2024-04-25T15:49:43.943 INFO:tasks.workunit.client.1.smithi193.stdout:2/602: chown d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4/feb 1602242930 1 2024-04-25T15:49:43.944 INFO:tasks.workunit.client.0.smithi110.stdout:8/874: dwrite d14/d32/d36/dde/fff [0,4194304] 0 2024-04-25T15:49:43.944 INFO:tasks.workunit.client.1.smithi193.stdout:2/603: chown d0/d37/d25/d5f/d3a/d6d/d6e/db9 12615 1 2024-04-25T15:49:43.945 INFO:tasks.workunit.client.1.smithi193.stdout:2/604: chown d0/d42/f61 206 1 2024-04-25T15:49:43.950 INFO:tasks.workunit.client.1.smithi193.stdout:1/527: truncate dc/d4a/d50/d7c/d7e/f88 2691978 0 2024-04-25T15:49:43.951 INFO:tasks.workunit.client.1.smithi193.stdout:1/528: write dc/f19 [775446,93257] 0 2024-04-25T15:49:43.952 INFO:tasks.workunit.client.1.smithi193.stdout:5/604: truncate d17/d18/d3e/f4c 4150366 0 2024-04-25T15:49:43.953 INFO:tasks.workunit.client.1.smithi193.stdout:1/529: dread - dc/d4a/d50/d5c/d5d/d3d/d55/f93 zero size 2024-04-25T15:49:43.954 INFO:tasks.workunit.client.1.smithi193.stdout:5/605: write d17/d18/d3e/f3f [5023292,5358] 0 2024-04-25T15:49:43.960 INFO:tasks.workunit.client.1.smithi193.stdout:2/605: mknod d0/df/da5/cf2 0 2024-04-25T15:49:43.961 INFO:tasks.workunit.client.0.smithi110.stdout:3/811: link d0/d16/f79 d0/d16/f117 0 2024-04-25T15:49:43.962 INFO:tasks.workunit.client.1.smithi193.stdout:2/606: truncate d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4/fde 1409419 0 2024-04-25T15:49:43.963 INFO:tasks.workunit.client.1.smithi193.stdout:1/530: mknod dc/d4a/d77/cb2 0 2024-04-25T15:49:43.964 INFO:tasks.workunit.client.1.smithi193.stdout:1/531: dread - dc/d4a/d50/d5c/d5d/d3d/f97 zero size 2024-04-25T15:49:43.965 INFO:tasks.workunit.client.0.smithi110.stdout:3/812: chown d0/d16/dcb/d2f/d4c/db2/lcd 158 1 2024-04-25T15:49:43.966 INFO:tasks.workunit.client.1.smithi193.stdout:5/606: creat d17/d53/fce x:0 0 0 2024-04-25T15:49:43.971 INFO:tasks.workunit.client.1.smithi193.stdout:2/607: mkdir d0/d9/d15/df3 0 2024-04-25T15:49:43.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/757: write d1/d23/d30/d33/f2d [3055690,121685] 0 2024-04-25T15:49:43.972 INFO:tasks.workunit.client.1.smithi193.stdout:5/607: symlink d17/d53/lcf 0 2024-04-25T15:49:43.974 INFO:tasks.workunit.client.1.smithi193.stdout:2/608: rmdir d0/d9/d15/d31 39 2024-04-25T15:49:43.974 INFO:tasks.workunit.client.1.smithi193.stdout:2/609: dread - d0/d42/d51/df1/fea zero size 2024-04-25T15:49:43.975 INFO:tasks.workunit.client.1.smithi193.stdout:2/610: stat d0/d37/d25/d5f/d3a/d6d/d85/f99 0 2024-04-25T15:49:43.975 INFO:tasks.workunit.client.0.smithi110.stdout:0/758: write d1/d20/d69/ffb [7429142,97792] 0 2024-04-25T15:49:43.976 INFO:tasks.workunit.client.1.smithi193.stdout:5/608: symlink d17/d18/d30/d38/d48/d4f/ld0 0 2024-04-25T15:49:43.977 INFO:tasks.workunit.client.1.smithi193.stdout:2/611: symlink d0/d44/lf4 0 2024-04-25T15:49:43.978 INFO:tasks.workunit.client.0.smithi110.stdout:0/759: mknod d1/d23/dc5/c109 0 2024-04-25T15:49:43.979 INFO:tasks.workunit.client.1.smithi193.stdout:1/532: dread dc/d4a/d50/d7c/d7e/f8c [0,4194304] 0 2024-04-25T15:49:43.981 INFO:tasks.workunit.client.1.smithi193.stdout:2/612: stat d0/d42/d51/df1/lbd 0 2024-04-25T15:49:43.984 INFO:tasks.workunit.client.1.smithi193.stdout:1/533: mknod dc/d4a/d50/cb3 0 2024-04-25T15:49:43.986 INFO:tasks.workunit.client.0.smithi110.stdout:0/760: fdatasync d1/d23/d30/d33/f88 0 2024-04-25T15:49:43.988 INFO:tasks.workunit.client.1.smithi193.stdout:2/613: creat d0/d42/d51/df1/d9f/dc5/ff5 x:0 0 0 2024-04-25T15:49:43.988 INFO:tasks.workunit.client.0.smithi110.stdout:0/761: write d1/d20/d69/ffb [8627106,68031] 0 2024-04-25T15:49:43.990 INFO:tasks.workunit.client.0.smithi110.stdout:1/943: dwrite d0/d23/f5a [0,4194304] 0 2024-04-25T15:49:43.991 INFO:tasks.workunit.client.1.smithi193.stdout:1/534: dwrite dc/d4a/d4d/f99 [0,4194304] 0 2024-04-25T15:49:43.992 INFO:tasks.workunit.client.1.smithi193.stdout:7/569: dwrite d12/d31/d39/f53 [0,4194304] 0 2024-04-25T15:49:43.997 INFO:tasks.workunit.client.1.smithi193.stdout:2/614: mkdir d0/d37/d25/d5f/d3a/d6d/d6e/df6 0 2024-04-25T15:49:43.999 INFO:tasks.workunit.client.0.smithi110.stdout:1/944: mknod d0/d14/d81/df8/db4/c133 0 2024-04-25T15:49:44.002 INFO:tasks.workunit.client.0.smithi110.stdout:1/945: read d0/d14/d81/df8/d33/d8e/fa6 [346668,66595] 0 2024-04-25T15:49:44.006 INFO:tasks.workunit.client.1.smithi193.stdout:7/570: mkdir d12/d31/d37/dcd 0 2024-04-25T15:49:44.006 INFO:tasks.workunit.client.1.smithi193.stdout:9/579: rmdir df/d2c/d2f/d4e/d1c 39 2024-04-25T15:49:44.006 INFO:tasks.workunit.client.1.smithi193.stdout:7/571: chown d12/d1c/d45/f5d 643 1 2024-04-25T15:49:44.008 INFO:tasks.workunit.client.1.smithi193.stdout:2/615: write d0/d42/d51/f40 [7114370,44401] 0 2024-04-25T15:49:44.009 INFO:tasks.workunit.client.0.smithi110.stdout:1/946: truncate d0/d14/d5f/d7d/f67 4606984 0 2024-04-25T15:49:44.009 INFO:tasks.workunit.client.1.smithi193.stdout:8/580: write d2/d3/d27/d69/dae/d4a/f4e [2430188,112385] 0 2024-04-25T15:49:44.010 INFO:tasks.workunit.client.1.smithi193.stdout:8/581: readlink d2/d3/d27/d69/dae/d4a/l82 0 2024-04-25T15:49:44.011 INFO:tasks.workunit.client.0.smithi110.stdout:0/762: link d1/d20/d69/l9d d1/d6c/d7f/d79/l10a 0 2024-04-25T15:49:44.012 INFO:tasks.workunit.client.1.smithi193.stdout:0/611: write d0/d49/d5/d7/dd/d46/d48/d62/f8f [1216566,7173] 0 2024-04-25T15:49:44.017 INFO:tasks.workunit.client.1.smithi193.stdout:6/573: write dd/d12/f5d [241396,84378] 0 2024-04-25T15:49:44.017 INFO:tasks.workunit.client.0.smithi110.stdout:1/947: truncate d0/d14/d5f/faa 311599 0 2024-04-25T15:49:44.019 INFO:tasks.workunit.client.0.smithi110.stdout:1/948: read d0/d14/d81/df8/db4/f106 [1359813,114934] 0 2024-04-25T15:49:44.020 INFO:tasks.workunit.client.1.smithi193.stdout:9/580: truncate df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/f94 132188 0 2024-04-25T15:49:44.021 INFO:tasks.workunit.client.0.smithi110.stdout:1/949: write d0/d23/f13 [447074,21581] 0 2024-04-25T15:49:44.022 INFO:tasks.workunit.client.1.smithi193.stdout:7/572: creat d12/d1c/d63/fce x:0 0 0 2024-04-25T15:49:44.022 INFO:tasks.workunit.client.0.smithi110.stdout:8/875: write d14/d21/f2a [2741156,9303] 0 2024-04-25T15:49:44.023 INFO:tasks.workunit.client.1.smithi193.stdout:4/663: write db/d4b/d51/d54/d77/fc9 [57460,54826] 0 2024-04-25T15:49:44.024 INFO:tasks.workunit.client.1.smithi193.stdout:2/616: rmdir d0/d42/d51/df1 39 2024-04-25T15:49:44.025 INFO:tasks.workunit.client.1.smithi193.stdout:4/664: write db/d32/d40/f93 [4808685,99185] 0 2024-04-25T15:49:44.026 INFO:tasks.workunit.client.1.smithi193.stdout:3/620: symlink d8/d13/d4b/d8b/d9a/lc0 0 2024-04-25T15:49:44.027 INFO:tasks.workunit.client.1.smithi193.stdout:2/617: truncate d0/d37/d25/d5f/d3a/d6d/fcd 991483 0 2024-04-25T15:49:44.028 INFO:tasks.workunit.client.1.smithi193.stdout:8/582: mkdir d2/d3/d27/d6f/dbb 0 2024-04-25T15:49:44.029 INFO:tasks.workunit.client.1.smithi193.stdout:8/583: readlink d2/d3/d27/d6f/l81 0 2024-04-25T15:49:44.031 INFO:tasks.workunit.client.1.smithi193.stdout:4/665: dwrite db/d32/d45/f60 [0,4194304] 0 2024-04-25T15:49:44.032 INFO:tasks.workunit.client.1.smithi193.stdout:4/666: fdatasync db/d13/f1d 0 2024-04-25T15:49:44.033 INFO:tasks.workunit.client.0.smithi110.stdout:3/813: dwrite d0/d7/f17 [0,4194304] 0 2024-04-25T15:49:44.037 INFO:tasks.workunit.client.1.smithi193.stdout:9/581: creat df/d2c/d2f/d5c/fb4 x:0 0 0 2024-04-25T15:49:44.043 INFO:tasks.workunit.client.1.smithi193.stdout:5/609: dwrite d17/d18/d30/d38/d71/d96/dc4/f56 [0,4194304] 0 2024-04-25T15:49:44.044 INFO:tasks.workunit.client.0.smithi110.stdout:1/950: getdents d0/d14/d5f/d7d/d72/d88/db6 0 2024-04-25T15:49:44.045 INFO:tasks.workunit.client.0.smithi110.stdout:1/951: stat d0/d14/d5f/d7d/d9c/c103 0 2024-04-25T15:49:44.045 INFO:tasks.workunit.client.0.smithi110.stdout:1/952: chown d0/d51/dd8/fef 52685 1 2024-04-25T15:49:44.047 INFO:tasks.workunit.client.1.smithi193.stdout:1/535: getdents dc/d4a/d50/d7c/d7e 0 2024-04-25T15:49:44.047 INFO:tasks.workunit.client.1.smithi193.stdout:1/536: write f6 [3904682,58950] 0 2024-04-25T15:49:44.049 INFO:tasks.workunit.client.1.smithi193.stdout:2/618: rename d0/d42/d51/d4f/d93 to d0/d37/d4d/dbe/df7 0 2024-04-25T15:49:44.053 INFO:tasks.workunit.client.0.smithi110.stdout:1/953: rename d0/d14/d81/df8/db4/dcd to d0/d14/d81/df8/d33/d8e/d134 0 2024-04-25T15:49:44.058 INFO:tasks.workunit.client.1.smithi193.stdout:4/667: truncate f4 457298 0 2024-04-25T15:49:44.059 INFO:tasks.workunit.client.1.smithi193.stdout:6/574: mknod dd/d12/d2d/cae 0 2024-04-25T15:49:44.064 INFO:tasks.workunit.client.0.smithi110.stdout:0/763: write d1/d6c/fc6 [131236,130687] 0 2024-04-25T15:49:44.068 INFO:tasks.workunit.client.1.smithi193.stdout:9/582: dread df/d2d/f31 [0,4194304] 0 2024-04-25T15:49:44.070 INFO:tasks.workunit.client.1.smithi193.stdout:2/619: unlink d0/l12 0 2024-04-25T15:49:44.071 INFO:tasks.workunit.client.1.smithi193.stdout:2/620: write d0/d42/d51/d4f/fac [1998698,110792] 0 2024-04-25T15:49:44.073 INFO:tasks.workunit.client.1.smithi193.stdout:8/584: truncate d2/d3/d27/d69/dae/f1a 1092955 0 2024-04-25T15:49:44.074 INFO:tasks.workunit.client.0.smithi110.stdout:8/876: truncate d14/d21/d26/d6a/d7e/da6/fa8 3234949 0 2024-04-25T15:49:44.076 INFO:tasks.workunit.client.1.smithi193.stdout:4/668: creat db/d13/d1e/d1f/d56/fd1 x:0 0 0 2024-04-25T15:49:44.076 INFO:tasks.workunit.client.1.smithi193.stdout:6/575: mkdir dd/d2f/d57/daf 0 2024-04-25T15:49:44.077 INFO:tasks.workunit.client.1.smithi193.stdout:7/573: link d12/d13/c82 d12/d31/d37/d62/d6e/ccf 0 2024-04-25T15:49:44.078 INFO:tasks.workunit.client.0.smithi110.stdout:1/954: dread d0/d14/d81/df8/d33/f111 [0,4194304] 0 2024-04-25T15:49:44.079 INFO:tasks.workunit.client.0.smithi110.stdout:1/955: chown d0/d14/d5f/d7d/d72/l9f 444667 1 2024-04-25T15:49:44.080 INFO:tasks.workunit.client.1.smithi193.stdout:0/612: truncate d0/d49/d5/d7/dd/d16/d26/d5e/fa8 1987666 0 2024-04-25T15:49:44.082 INFO:tasks.workunit.client.0.smithi110.stdout:8/877: mknod d14/d21/d26/d6a/d7e/da6/de8/c12e 0 2024-04-25T15:49:44.083 INFO:tasks.workunit.client.1.smithi193.stdout:1/537: truncate dc/f5b 5214523 0 2024-04-25T15:49:44.084 INFO:tasks.workunit.client.1.smithi193.stdout:0/613: dread d0/d49/d5/d7/dd/d16/d18/fac [0,4194304] 0 2024-04-25T15:49:44.085 INFO:tasks.workunit.client.1.smithi193.stdout:0/614: chown d0/d49/d5/d7/dd/d16/d18/d28/l5a 24 1 2024-04-25T15:49:44.086 INFO:tasks.workunit.client.1.smithi193.stdout:9/583: creat df/d2c/d34/d3e/d4d/d5e/fb5 x:0 0 0 2024-04-25T15:49:44.091 INFO:tasks.workunit.client.1.smithi193.stdout:5/610: write d17/d18/d30/d38/d71/d96/dc4/f70 [515582,71004] 0 2024-04-25T15:49:44.092 INFO:tasks.workunit.client.1.smithi193.stdout:9/584: dwrite df/d2c/d2f/d4e/d1c/f76 [0,4194304] 0 2024-04-25T15:49:44.094 INFO:tasks.workunit.client.1.smithi193.stdout:9/585: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f3c [2537385,77026] 0 2024-04-25T15:49:44.095 INFO:tasks.workunit.client.0.smithi110.stdout:8/878: rename d14/d32/d36/d48/lf5 to d14/d21/d26/d6a/d118/l12f 0 2024-04-25T15:49:44.096 INFO:tasks.workunit.client.0.smithi110.stdout:0/764: dwrite d1/d23/d30/d33/d5b/d78/d107/fb7 [4194304,4194304] 0 2024-04-25T15:49:44.101 INFO:tasks.workunit.client.1.smithi193.stdout:6/576: fdatasync dd/d12/d19/d1c/f26 0 2024-04-25T15:49:44.105 INFO:tasks.workunit.client.0.smithi110.stdout:5/829: sync 2024-04-25T15:49:44.105 INFO:tasks.workunit.client.0.smithi110.stdout:7/848: sync 2024-04-25T15:49:44.105 INFO:tasks.workunit.client.0.smithi110.stdout:4/797: sync 2024-04-25T15:49:44.105 INFO:tasks.workunit.client.0.smithi110.stdout:6/761: sync 2024-04-25T15:49:44.106 INFO:tasks.workunit.client.1.smithi193.stdout:7/574: mknod d12/d1c/d63/d70/d68/d8f/cd0 0 2024-04-25T15:49:44.107 INFO:tasks.workunit.client.0.smithi110.stdout:7/849: chown d1b/d29/c86 7289252 1 2024-04-25T15:49:44.110 INFO:tasks.workunit.client.1.smithi193.stdout:3/621: getdents d8/d13/d14/d51 0 2024-04-25T15:49:44.111 INFO:tasks.workunit.client.1.smithi193.stdout:3/622: truncate d8/d30/d45/f70 8472492 0 2024-04-25T15:49:44.111 INFO:tasks.workunit.client.0.smithi110.stdout:0/765: mkdir d1/d23/d30/d33/d5b/d78/ddb/d10b 0 2024-04-25T15:49:44.116 INFO:tasks.workunit.client.1.smithi193.stdout:1/538: mknod dc/d4a/d50/cb4 0 2024-04-25T15:49:44.118 INFO:tasks.workunit.client.1.smithi193.stdout:0/615: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/ld1 0 2024-04-25T15:49:44.122 INFO:tasks.workunit.client.1.smithi193.stdout:2/621: chown d0/d42/d51/d4f/d65/la1 1017774 1 2024-04-25T15:49:44.128 INFO:tasks.workunit.client.1.smithi193.stdout:5/611: unlink d17/f86 0 2024-04-25T15:49:44.130 INFO:tasks.workunit.client.1.smithi193.stdout:4/669: mkdir db/d13/dd2 0 2024-04-25T15:49:44.132 INFO:tasks.workunit.client.1.smithi193.stdout:6/577: creat dd/d12/d19/d1c/fb0 x:0 0 0 2024-04-25T15:49:44.133 INFO:tasks.workunit.client.1.smithi193.stdout:7/575: mkdir d12/d31/d37/d92/dd1 0 2024-04-25T15:49:44.133 INFO:tasks.workunit.client.1.smithi193.stdout:6/578: chown dd/d12/d19/d1c/fa4 29 1 2024-04-25T15:49:44.138 INFO:tasks.workunit.client.1.smithi193.stdout:0/616: chown d0/d49/l7d 245346777 1 2024-04-25T15:49:44.138 INFO:tasks.workunit.client.1.smithi193.stdout:7/576: dwrite d12/d1c/d63/d70/f34 [0,4194304] 0 2024-04-25T15:49:44.138 INFO:tasks.workunit.client.1.smithi193.stdout:0/617: readlink d0/d49/l7d 0 2024-04-25T15:49:44.141 INFO:tasks.workunit.client.1.smithi193.stdout:3/623: mknod d8/d30/cc1 0 2024-04-25T15:49:44.142 INFO:tasks.workunit.client.1.smithi193.stdout:5/612: mkdir d17/d18/d3e/dd1 0 2024-04-25T15:49:44.142 INFO:tasks.workunit.client.1.smithi193.stdout:7/577: symlink d12/d31/d39/d60/ld2 0 2024-04-25T15:49:44.143 INFO:tasks.workunit.client.1.smithi193.stdout:5/613: fsync d17/f2a 0 2024-04-25T15:49:44.144 INFO:tasks.workunit.client.1.smithi193.stdout:7/578: fdatasync d12/d1c/d63/d70/faa 0 2024-04-25T15:49:44.146 INFO:tasks.workunit.client.1.smithi193.stdout:3/624: rename d8/d30/d72/f76 to d8/d13/d1f/d69/fc2 0 2024-04-25T15:49:44.146 INFO:tasks.workunit.client.1.smithi193.stdout:0/618: dwrite d0/d49/d5/d7/dd/d16/d1a/f8e [0,4194304] 0 2024-04-25T15:49:44.148 INFO:tasks.workunit.client.1.smithi193.stdout:0/619: write d0/d49/d5/d7/dd/d16/fb8 [861890,111035] 0 2024-04-25T15:49:44.149 INFO:tasks.workunit.client.1.smithi193.stdout:5/614: fdatasync d17/d18/d3e/f4c 0 2024-04-25T15:49:44.150 INFO:tasks.workunit.client.1.smithi193.stdout:3/625: dread d8/d13/d14/f59 [0,4194304] 0 2024-04-25T15:49:44.167 INFO:tasks.workunit.client.0.smithi110.stdout:5/830: dread d4/d1d/d1f/d35/d3d/d52/f7a [0,4194304] 0 2024-04-25T15:49:44.171 INFO:tasks.workunit.client.0.smithi110.stdout:7/850: symlink d1b/d29/d38/d55/dab/db2/dcd/l11b 0 2024-04-25T15:49:44.172 INFO:tasks.workunit.client.1.smithi193.stdout:7/579: read d12/d13/f41 [88721,43238] 0 2024-04-25T15:49:44.172 INFO:tasks.workunit.client.0.smithi110.stdout:8/879: truncate d14/d21/d26/d6a/d7e/da6/de8/d55/f7d 3373667 0 2024-04-25T15:49:44.172 INFO:tasks.workunit.client.0.smithi110.stdout:0/766: dwrite d1/d23/d30/d33/d5b/d83/fe9 [0,4194304] 0 2024-04-25T15:49:44.177 INFO:tasks.workunit.client.0.smithi110.stdout:6/762: getdents d1/db/d22/d90/d76/dee 0 2024-04-25T15:49:44.180 INFO:tasks.workunit.client.1.smithi193.stdout:0/620: rename d0/l2 to d0/d49/d5/d7/dd/d16/d26/d2e/ld2 0 2024-04-25T15:49:44.184 INFO:tasks.workunit.client.0.smithi110.stdout:1/956: dwrite d0/d14/d5f/f11c [0,4194304] 0 2024-04-25T15:49:44.188 INFO:tasks.workunit.client.0.smithi110.stdout:1/957: stat d0/d51/dd8/df0/f116 0 2024-04-25T15:49:44.191 INFO:tasks.workunit.client.0.smithi110.stdout:0/767: rmdir d1/d6c 39 2024-04-25T15:49:44.193 INFO:tasks.workunit.client.1.smithi193.stdout:1/539: rmdir dc 39 2024-04-25T15:49:44.194 INFO:tasks.workunit.client.0.smithi110.stdout:4/798: dread d13/d24/d2d/d5e/f7b [0,4194304] 0 2024-04-25T15:49:44.195 INFO:tasks.workunit.client.1.smithi193.stdout:6/579: link dd/d2f/l4a dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/lb1 0 2024-04-25T15:49:44.197 INFO:tasks.workunit.client.0.smithi110.stdout:6/763: mkdir d1/d79/d112 0 2024-04-25T15:49:44.200 INFO:tasks.workunit.client.1.smithi193.stdout:0/621: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/cd3 0 2024-04-25T15:49:44.202 INFO:tasks.workunit.client.1.smithi193.stdout:0/622: write d0/d49/d5/d7/dd/d16/d26/d2e/f76 [1629001,49329] 0 2024-04-25T15:49:44.202 INFO:tasks.workunit.client.1.smithi193.stdout:8/585: truncate d2/d3/d27/d69/dae/d4a/d63/da2/fa4 1346970 0 2024-04-25T15:49:44.203 INFO:tasks.workunit.client.0.smithi110.stdout:6/764: dwrite d1/db/da0/fc8 [0,4194304] 0 2024-04-25T15:49:44.203 INFO:tasks.workunit.client.1.smithi193.stdout:8/586: write d2/da5/fa7 [406816,91199] 0 2024-04-25T15:49:44.203 INFO:tasks.workunit.client.1.smithi193.stdout:3/626: unlink d8/d13/d1f/d9f/cb7 0 2024-04-25T15:49:44.204 INFO:tasks.workunit.client.1.smithi193.stdout:0/623: write d0/d49/d5/d7/dd/f3c [7104669,114276] 0 2024-04-25T15:49:44.204 INFO:tasks.workunit.client.0.smithi110.stdout:6/765: stat d1/db/d22/d33/d63/dc2/dc7/dec 0 2024-04-25T15:49:44.205 INFO:tasks.workunit.client.0.smithi110.stdout:1/958: creat d0/d14/d81/df8/d33/d8e/d109/f135 x:0 0 0 2024-04-25T15:49:44.210 INFO:tasks.workunit.client.1.smithi193.stdout:3/627: dwrite d8/d4d/f87 [0,4194304] 0 2024-04-25T15:49:44.215 INFO:tasks.workunit.client.0.smithi110.stdout:4/799: unlink d13/d24/d2d/d5e/f103 0 2024-04-25T15:49:44.216 INFO:tasks.workunit.client.1.smithi193.stdout:1/540: dwrite dc/d4a/d50/d5c/d5d/d3d/f97 [0,4194304] 0 2024-04-25T15:49:44.219 INFO:tasks.workunit.client.1.smithi193.stdout:1/541: read dc/f2b [3809723,97553] 0 2024-04-25T15:49:44.219 INFO:tasks.workunit.client.1.smithi193.stdout:1/542: fdatasync dc/d4a/d4d/d7a/f96 0 2024-04-25T15:49:44.233 INFO:tasks.workunit.client.0.smithi110.stdout:7/851: creat d1b/d29/d38/d55/f11c x:0 0 0 2024-04-25T15:49:44.234 INFO:tasks.workunit.client.0.smithi110.stdout:7/852: fsync d1b/d29/d38/d8b/d89/f10f 0 2024-04-25T15:49:44.235 INFO:tasks.workunit.client.1.smithi193.stdout:8/587: fdatasync d2/d3/d27/d69/dae/f5a 0 2024-04-25T15:49:44.238 INFO:tasks.workunit.client.1.smithi193.stdout:0/624: dread - d0/d49/f2c zero size 2024-04-25T15:49:44.238 INFO:tasks.workunit.client.0.smithi110.stdout:1/959: rename d0/d14/d5f/d6f/f82 to d0/d14/d5f/d7d/d9c/dcc/f136 0 2024-04-25T15:49:44.239 INFO:tasks.workunit.client.0.smithi110.stdout:6/766: truncate d1/dd/d3c/d91/d8d/dce/d102/d75/fa8 962781 0 2024-04-25T15:49:44.241 INFO:tasks.workunit.client.1.smithi193.stdout:7/580: link d12/d31/d37/d62/d6e/d79/f4a d12/d31/d39/db0/fd3 0 2024-04-25T15:49:44.243 INFO:tasks.workunit.client.1.smithi193.stdout:6/580: mkdir dd/d12/d19/db2 0 2024-04-25T15:49:44.244 INFO:tasks.workunit.client.1.smithi193.stdout:6/581: write dd/d12/d19/d1c/fa4 [1076308,70919] 0 2024-04-25T15:49:44.247 INFO:tasks.workunit.client.0.smithi110.stdout:4/800: fdatasync d13/d24/d2d/dd9/dbb/dd2/fe3 0 2024-04-25T15:49:44.248 INFO:tasks.workunit.client.1.smithi193.stdout:7/581: dread d12/d1c/d63/d70/faa [0,4194304] 0 2024-04-25T15:49:44.248 INFO:tasks.workunit.client.1.smithi193.stdout:5/615: getdents d17/d18/d75 0 2024-04-25T15:49:44.250 INFO:tasks.workunit.client.0.smithi110.stdout:7/853: unlink d1b/d29/d38/d55/dab/db2/db8/c104 0 2024-04-25T15:49:44.251 INFO:tasks.workunit.client.0.smithi110.stdout:7/854: write d1b/d29/d38/d55/d7f/f7a [153654,104281] 0 2024-04-25T15:49:44.252 INFO:tasks.workunit.client.0.smithi110.stdout:7/855: chown d1b/d29/d38/d55/d95/ff4 132 1 2024-04-25T15:49:44.254 INFO:tasks.workunit.client.0.smithi110.stdout:1/960: creat d0/d14/d81/df8/d33/da9/f137 x:0 0 0 2024-04-25T15:49:44.258 INFO:tasks.workunit.client.0.smithi110.stdout:0/768: dwrite d1/d23/d30/d33/d5b/d72/f82 [0,4194304] 0 2024-04-25T15:49:44.260 INFO:tasks.workunit.client.1.smithi193.stdout:1/543: truncate dc/f15 164256 0 2024-04-25T15:49:44.260 INFO:tasks.workunit.client.0.smithi110.stdout:5/831: getdents d4/d15/d51/d99 0 2024-04-25T15:49:44.261 INFO:tasks.workunit.client.0.smithi110.stdout:5/832: readlink d4/d1d/la2 0 2024-04-25T15:49:44.261 INFO:tasks.workunit.client.1.smithi193.stdout:1/544: chown dc/f36 8 1 2024-04-25T15:49:44.264 INFO:tasks.workunit.client.0.smithi110.stdout:8/880: truncate d14/d21/d26/d6a/d7e/da6/de8/d71/fd5 413014 0 2024-04-25T15:49:44.269 INFO:tasks.workunit.client.1.smithi193.stdout:8/588: dwrite d2/d3/d3a/f54 [0,4194304] 0 2024-04-25T15:49:44.272 INFO:tasks.workunit.client.0.smithi110.stdout:0/769: mkdir d1/d23/d30/d33/d5b/d78/da4/d10c 0 2024-04-25T15:49:44.273 INFO:tasks.workunit.client.1.smithi193.stdout:3/628: write d8/d13/d14/d23/f48 [1879085,88032] 0 2024-04-25T15:49:44.276 INFO:tasks.workunit.client.1.smithi193.stdout:8/589: dwrite d2/d3/d27/d6f/d94/f90 [0,4194304] 0 2024-04-25T15:49:44.278 INFO:tasks.workunit.client.1.smithi193.stdout:8/590: write d2/d3/ff [409637,19496] 0 2024-04-25T15:49:44.280 INFO:tasks.workunit.client.1.smithi193.stdout:2/622: rename d0/d42/d51/d4f/d65/d7e to d0/d37/dbb/df8 0 2024-04-25T15:49:44.282 INFO:tasks.workunit.client.0.smithi110.stdout:8/881: creat d14/d21/d26/d6a/d7e/da6/de8/d55/f130 x:0 0 0 2024-04-25T15:49:44.289 INFO:tasks.workunit.client.1.smithi193.stdout:6/582: truncate dd/d12/d19/d1c/d40/d3c/fa7 119477 0 2024-04-25T15:49:44.289 INFO:tasks.workunit.client.1.smithi193.stdout:6/583: chown dd/f2b 10444 1 2024-04-25T15:49:44.291 INFO:tasks.workunit.client.0.smithi110.stdout:6/767: dwrite d1/dd/d3c/f7c [0,4194304] 0 2024-04-25T15:49:44.292 INFO:tasks.workunit.client.1.smithi193.stdout:3/629: creat d8/d30/d45/fc3 x:0 0 0 2024-04-25T15:49:44.296 INFO:tasks.workunit.client.0.smithi110.stdout:9/746: sync 2024-04-25T15:49:44.296 INFO:tasks.workunit.client.0.smithi110.stdout:7/856: write d1b/d29/d38/d8b/d89/dc6/fd3 [3927128,131063] 0 2024-04-25T15:49:44.302 INFO:tasks.workunit.client.0.smithi110.stdout:5/833: write d4/d1d/d85/faf [238786,82703] 0 2024-04-25T15:49:44.303 INFO:tasks.workunit.client.0.smithi110.stdout:0/770: creat d1/d23/d30/d33/dfe/f10d x:0 0 0 2024-04-25T15:49:44.305 INFO:tasks.workunit.client.1.smithi193.stdout:2/623: read d0/d37/d4d/dbe/fd8 [500614,63457] 0 2024-04-25T15:49:44.307 INFO:tasks.workunit.client.0.smithi110.stdout:0/771: dread d1/d23/d30/d33/d5b/d78/da4/feb [0,4194304] 0 2024-04-25T15:49:44.311 INFO:tasks.workunit.client.0.smithi110.stdout:4/801: getdents d13/d24/d2d/db5 0 2024-04-25T15:49:44.313 INFO:tasks.workunit.client.1.smithi193.stdout:6/584: mknod dd/d12/d19/d1c/cb3 0 2024-04-25T15:49:44.316 INFO:tasks.workunit.client.0.smithi110.stdout:8/882: dwrite d14/d32/d36/d48/f104 [0,4194304] 0 2024-04-25T15:49:44.318 INFO:tasks.workunit.client.1.smithi193.stdout:8/591: mkdir d2/d18/dbc 0 2024-04-25T15:49:44.318 INFO:tasks.workunit.client.1.smithi193.stdout:1/545: write dc/f47 [534943,118436] 0 2024-04-25T15:49:44.319 INFO:tasks.workunit.client.1.smithi193.stdout:2/624: creat d0/d42/d51/d4f/ff9 x:0 0 0 2024-04-25T15:49:44.322 INFO:tasks.workunit.client.1.smithi193.stdout:0/625: getdents d0/d49/d5/d7/dd/d78 0 2024-04-25T15:49:44.324 INFO:tasks.workunit.client.0.smithi110.stdout:6/768: dwrite d1/dd/d3c/f5d [4194304,4194304] 0 2024-04-25T15:49:44.324 INFO:tasks.workunit.client.1.smithi193.stdout:7/582: getdents d12 0 2024-04-25T15:49:44.326 INFO:tasks.workunit.client.1.smithi193.stdout:8/592: creat d2/d18/db1/fbd x:0 0 0 2024-04-25T15:49:44.326 INFO:tasks.workunit.client.0.smithi110.stdout:3/814: sync 2024-04-25T15:49:44.326 INFO:tasks.workunit.client.0.smithi110.stdout:2/712: sync 2024-04-25T15:49:44.331 INFO:tasks.workunit.client.0.smithi110.stdout:6/769: dread d1/db/d22/d33/d63/f97 [0,4194304] 0 2024-04-25T15:49:44.332 INFO:tasks.workunit.client.1.smithi193.stdout:0/626: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/f69 [4194304,4194304] 0 2024-04-25T15:49:44.332 INFO:tasks.workunit.client.1.smithi193.stdout:2/625: unlink d0/d37/d25/d5f/d3a/c5a 0 2024-04-25T15:49:44.333 INFO:tasks.workunit.client.1.smithi193.stdout:2/626: stat d0/d42/d51 0 2024-04-25T15:49:44.333 INFO:tasks.workunit.client.1.smithi193.stdout:0/627: readlink d0/d49/d5/d7/dd/d46/l72 0 2024-04-25T15:49:44.335 INFO:tasks.workunit.client.1.smithi193.stdout:8/593: rename d2/d3/d27/l9d to d2/d3/d3a/lbe 0 2024-04-25T15:49:44.336 INFO:tasks.workunit.client.1.smithi193.stdout:1/546: dread dc/d4a/d4d/f57 [0,4194304] 0 2024-04-25T15:49:44.336 INFO:tasks.workunit.client.1.smithi193.stdout:8/594: stat d2/d3/d3a/c4f 0 2024-04-25T15:49:44.337 INFO:tasks.workunit.client.1.smithi193.stdout:7/583: link d12/d31/dc3/fc9 d12/d31/d39/d60/fd4 0 2024-04-25T15:49:44.338 INFO:tasks.workunit.client.1.smithi193.stdout:3/630: dread d8/d13/d4b/f57 [0,4194304] 0 2024-04-25T15:49:44.338 INFO:tasks.workunit.client.1.smithi193.stdout:2/627: dread d0/d42/d51/d4f/fc4 [0,4194304] 0 2024-04-25T15:49:44.340 INFO:tasks.workunit.client.1.smithi193.stdout:2/628: chown d0/d37/d25/d5f/d3a/d6d/d6e/f96 11870532 1 2024-04-25T15:49:44.347 INFO:tasks.workunit.client.1.smithi193.stdout:7/584: rename d12/da1/fc1 to d12/d31/d37/d92/dd1/fd5 0 2024-04-25T15:49:44.348 INFO:tasks.workunit.client.1.smithi193.stdout:7/585: chown d12/d1c/d63/d70/d68 2073601 1 2024-04-25T15:49:44.349 INFO:tasks.workunit.client.1.smithi193.stdout:7/586: chown d12/d31/d37/d62/d6e/d79/l7c 988 1 2024-04-25T15:49:44.349 INFO:tasks.workunit.client.1.smithi193.stdout:0/628: mkdir d0/d49/d5/d61/dd4 0 2024-04-25T15:49:44.349 INFO:tasks.workunit.client.1.smithi193.stdout:7/587: chown d12/fb8 11438 1 2024-04-25T15:49:44.358 INFO:tasks.workunit.client.1.smithi193.stdout:7/588: mkdir d12/d1c/dd6 0 2024-04-25T15:49:44.364 INFO:tasks.workunit.client.0.smithi110.stdout:9/747: creat d9/d13/d5b/d61/db3/dd3/ddb/fec x:0 0 0 2024-04-25T15:49:44.365 INFO:tasks.workunit.client.0.smithi110.stdout:9/748: stat d9/c47 0 2024-04-25T15:49:44.365 INFO:tasks.workunit.client.1.smithi193.stdout:7/589: mkdir d12/d31/d37/d62/d6e/da5/dd7 0 2024-04-25T15:49:44.367 INFO:tasks.workunit.client.0.smithi110.stdout:1/961: link d0/d14/d81/c110 d0/d51/dc8/d12d/c138 0 2024-04-25T15:49:44.367 INFO:tasks.workunit.client.1.smithi193.stdout:9/586: sync 2024-04-25T15:49:44.367 INFO:tasks.workunit.client.1.smithi193.stdout:4/670: sync 2024-04-25T15:49:44.369 INFO:tasks.workunit.client.1.smithi193.stdout:8/595: getdents d2 0 2024-04-25T15:49:44.371 INFO:tasks.workunit.client.1.smithi193.stdout:3/631: write d8/d13/d1f/f16 [2814684,116435] 0 2024-04-25T15:49:44.372 INFO:tasks.workunit.client.1.smithi193.stdout:9/587: chown df/la8 3067830 1 2024-04-25T15:49:44.372 INFO:tasks.workunit.client.1.smithi193.stdout:1/547: dwrite dc/f2b [0,4194304] 0 2024-04-25T15:49:44.373 INFO:tasks.workunit.client.1.smithi193.stdout:7/590: dwrite d12/d13/f24 [0,4194304] 0 2024-04-25T15:49:44.374 INFO:tasks.workunit.client.0.smithi110.stdout:0/772: rename d1/d23/lab to d1/d23/d30/d33/d5b/d78/da4/d10c/l10e 0 2024-04-25T15:49:44.376 INFO:tasks.workunit.client.1.smithi193.stdout:4/671: mknod db/d13/cd3 0 2024-04-25T15:49:44.380 INFO:tasks.workunit.client.1.smithi193.stdout:8/596: rename d2/d3/d27/d69/d86 to d2/d3/d27/d69/dbf 0 2024-04-25T15:49:44.385 INFO:tasks.workunit.client.1.smithi193.stdout:5/616: sync 2024-04-25T15:49:44.385 INFO:tasks.workunit.client.1.smithi193.stdout:6/585: sync 2024-04-25T15:49:44.386 INFO:tasks.workunit.client.1.smithi193.stdout:0/629: write d0/d49/d5/d7/dd/f40 [1936028,98087] 0 2024-04-25T15:49:44.387 INFO:tasks.workunit.client.1.smithi193.stdout:6/586: chown dd/d12/d19/c5a 0 1 2024-04-25T15:49:44.389 INFO:tasks.workunit.client.1.smithi193.stdout:3/632: rename d8/d13/d14/d27/c5c to d8/d13/d1f/d69/d77/cc4 0 2024-04-25T15:49:44.394 INFO:tasks.workunit.client.1.smithi193.stdout:4/672: chown db/d32/fb4 46818637 1 2024-04-25T15:49:44.396 INFO:tasks.workunit.client.1.smithi193.stdout:8/597: symlink d2/d3/d27/d69/dae/lc0 0 2024-04-25T15:49:44.396 INFO:tasks.workunit.client.0.smithi110.stdout:2/713: dread de/d40/d6e/d50/f7b [0,4194304] 0 2024-04-25T15:49:44.397 INFO:tasks.workunit.client.1.smithi193.stdout:2/629: dwrite d0/d37/f67 [0,4194304] 0 2024-04-25T15:49:44.397 INFO:tasks.workunit.client.0.smithi110.stdout:2/714: chown de/d40/d6e/d52/cc2 4087 1 2024-04-25T15:49:44.398 INFO:tasks.workunit.client.0.smithi110.stdout:7/857: link d1b/f88 d1b/d29/db5/dca/d10b/f11d 0 2024-04-25T15:49:44.398 INFO:tasks.workunit.client.1.smithi193.stdout:2/630: truncate d0/d42/d51/d4f/ff9 62773 0 2024-04-25T15:49:44.400 INFO:tasks.workunit.client.1.smithi193.stdout:9/588: creat df/d2c/d95/fb6 x:0 0 0 2024-04-25T15:49:44.401 INFO:tasks.workunit.client.1.smithi193.stdout:3/633: dread d8/d30/d45/fbe [0,4194304] 0 2024-04-25T15:49:44.403 INFO:tasks.workunit.client.1.smithi193.stdout:1/548: creat dc/d4a/d4d/d7a/da6/fb5 x:0 0 0 2024-04-25T15:49:44.406 INFO:tasks.workunit.client.0.smithi110.stdout:1/962: dread d0/d51/dc8/f127 [0,4194304] 0 2024-04-25T15:49:44.407 INFO:tasks.workunit.client.1.smithi193.stdout:6/587: dread dd/d12/d19/d1c/f22 [0,4194304] 0 2024-04-25T15:49:44.409 INFO:tasks.workunit.client.0.smithi110.stdout:8/883: mknod d14/d21/d26/dc0/c131 0 2024-04-25T15:49:44.410 INFO:tasks.workunit.client.1.smithi193.stdout:7/591: mknod d12/d31/d37/d62/d6e/d79/d9d/cd8 0 2024-04-25T15:49:44.411 INFO:tasks.workunit.client.1.smithi193.stdout:7/592: fsync d12/d13/fac 0 2024-04-25T15:49:44.413 INFO:tasks.workunit.client.1.smithi193.stdout:0/630: dwrite d0/d49/d5/d7/dd/d16/d18/fc5 [0,4194304] 0 2024-04-25T15:49:44.414 INFO:tasks.workunit.client.1.smithi193.stdout:4/673: truncate db/d13/d1e/f48 4931838 0 2024-04-25T15:49:44.415 INFO:tasks.workunit.client.0.smithi110.stdout:3/815: creat d0/d7/dd0/d10b/f118 x:0 0 0 2024-04-25T15:49:44.419 INFO:tasks.workunit.client.0.smithi110.stdout:7/858: rename d1b/l21 to d1b/d29/d38/d55/dab/db2/db8/de3/l11e 0 2024-04-25T15:49:44.421 INFO:tasks.workunit.client.0.smithi110.stdout:0/773: write d1/d20/f96 [3205068,98643] 0 2024-04-25T15:49:44.421 INFO:tasks.workunit.client.0.smithi110.stdout:5/834: link d4/d1d/d1f/d35/d3d/d52/d97/fb1 d4/d1d/d1f/d35/d3d/dc8/f115 0 2024-04-25T15:49:44.424 INFO:tasks.workunit.client.1.smithi193.stdout:1/549: rmdir dc/d4a/d4d/d7a 39 2024-04-25T15:49:44.430 INFO:tasks.workunit.client.0.smithi110.stdout:2/715: creat de/d40/d86/fe2 x:0 0 0 2024-04-25T15:49:44.430 INFO:tasks.workunit.client.1.smithi193.stdout:1/550: dwrite dc/d59/f8e [0,4194304] 0 2024-04-25T15:49:44.435 INFO:tasks.workunit.client.0.smithi110.stdout:7/859: creat d1b/d29/d38/d55/d7f/f11f x:0 0 0 2024-04-25T15:49:44.436 INFO:tasks.workunit.client.1.smithi193.stdout:5/617: truncate d17/d18/f2c 3168668 0 2024-04-25T15:49:44.436 INFO:tasks.workunit.client.0.smithi110.stdout:7/860: chown d1b/d29/d38/d55/dab 417948 1 2024-04-25T15:49:44.441 INFO:tasks.workunit.client.0.smithi110.stdout:0/774: rename d1/d23/d30/d33/l4a to d1/d23/d30/d33/d5b/dbb/da8/l10f 0 2024-04-25T15:49:44.443 INFO:tasks.workunit.client.1.smithi193.stdout:5/618: dwrite d17/d18/d9b/fa9 [0,4194304] 0 2024-04-25T15:49:44.445 INFO:tasks.workunit.client.1.smithi193.stdout:5/619: fdatasync d17/d18/d30/d38/fbc 0 2024-04-25T15:49:44.447 INFO:tasks.workunit.client.1.smithi193.stdout:5/620: read d17/d18/d75/d85/d5d/dad/fb5 [727447,113913] 0 2024-04-25T15:49:44.447 INFO:tasks.workunit.client.1.smithi193.stdout:3/634: symlink d8/d13/d4b/lc5 0 2024-04-25T15:49:44.448 INFO:tasks.workunit.client.1.smithi193.stdout:5/621: chown d17/d18/d30/d38/d71/d96/dc4/f56 1 1 2024-04-25T15:49:44.450 INFO:tasks.workunit.client.0.smithi110.stdout:2/716: creat de/d40/d6e/d50/db7/fe3 x:0 0 0 2024-04-25T15:49:44.452 INFO:tasks.workunit.client.0.smithi110.stdout:7/861: rmdir d1b/d29/d38/d55/dab 39 2024-04-25T15:49:44.456 INFO:tasks.workunit.client.1.smithi193.stdout:2/631: truncate d0/d9/d3d/d7d/fd2 1564733 0 2024-04-25T15:49:44.457 INFO:tasks.workunit.client.0.smithi110.stdout:5/835: dwrite d4/d1d/d1f/d35/d3d/d52/d75/fb5 [0,4194304] 0 2024-04-25T15:49:44.459 INFO:tasks.workunit.client.0.smithi110.stdout:1/963: creat d0/d14/d5f/d7d/d72/f139 x:0 0 0 2024-04-25T15:49:44.459 INFO:tasks.workunit.client.0.smithi110.stdout:3/816: write d0/d2e/f39 [1457894,26670] 0 2024-04-25T15:49:44.460 INFO:tasks.workunit.client.1.smithi193.stdout:8/598: dwrite d2/d3/d3a/f61 [0,4194304] 0 2024-04-25T15:49:44.467 INFO:tasks.workunit.client.1.smithi193.stdout:9/589: truncate df/d2c/d34/f75 994270 0 2024-04-25T15:49:44.468 INFO:tasks.workunit.client.0.smithi110.stdout:2/717: fdatasync de/f2d 0 2024-04-25T15:49:44.470 INFO:tasks.workunit.client.1.smithi193.stdout:0/631: sync 2024-04-25T15:49:44.472 INFO:tasks.workunit.client.1.smithi193.stdout:9/590: dwrite df/d2d/f39 [0,4194304] 0 2024-04-25T15:49:44.474 INFO:tasks.workunit.client.0.smithi110.stdout:1/964: mknod d0/d14/d5f/d7d/d72/d88/db6/c13a 0 2024-04-25T15:49:44.475 INFO:tasks.workunit.client.0.smithi110.stdout:8/884: getdents d14/d32/d36/d48/d8f 0 2024-04-25T15:49:44.481 INFO:tasks.workunit.client.1.smithi193.stdout:3/635: rmdir d8 39 2024-04-25T15:49:44.483 INFO:tasks.workunit.client.0.smithi110.stdout:7/862: link d1b/d29/d38/d55/d7f/f7a d1b/d29/d46/d106/f120 0 2024-04-25T15:49:44.484 INFO:tasks.workunit.client.0.smithi110.stdout:7/863: fsync d1b/d29/d38/f61 0 2024-04-25T15:49:44.485 INFO:tasks.workunit.client.0.smithi110.stdout:7/864: chown d1b/d29/d38/d8b/d89/fcc 214635151 1 2024-04-25T15:49:44.486 INFO:tasks.workunit.client.1.smithi193.stdout:5/622: unlink d17/d18/d30/l3d 0 2024-04-25T15:49:44.491 INFO:tasks.workunit.client.0.smithi110.stdout:2/718: rename de/d40/dd7/cd8 to de/d40/d70/dc6/ce4 0 2024-04-25T15:49:44.494 INFO:tasks.workunit.client.1.smithi193.stdout:6/588: link dd/l14 dd/d12/d19/db2/lb4 0 2024-04-25T15:49:44.495 INFO:tasks.workunit.client.0.smithi110.stdout:7/865: mknod d1b/d29/db5/dca/d10b/c121 0 2024-04-25T15:49:44.496 INFO:tasks.workunit.client.0.smithi110.stdout:8/885: dread d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/fa1 [0,4194304] 0 2024-04-25T15:49:44.497 INFO:tasks.workunit.client.0.smithi110.stdout:7/866: write d1b/d29/d38/d55/d95/fa9 [293679,78137] 0 2024-04-25T15:49:44.499 INFO:tasks.workunit.client.1.smithi193.stdout:1/551: creat dc/d4a/d4d/d7a/d85/fb6 x:0 0 0 2024-04-25T15:49:44.500 INFO:tasks.workunit.client.1.smithi193.stdout:1/552: chown dc/d4a/d50/d5c/c62 29 1 2024-04-25T15:49:44.503 INFO:tasks.workunit.client.1.smithi193.stdout:7/593: creat d12/d31/fd9 x:0 0 0 2024-04-25T15:49:44.506 INFO:tasks.workunit.client.0.smithi110.stdout:3/817: getdents d0/d2e/d88 0 2024-04-25T15:49:44.507 INFO:tasks.workunit.client.1.smithi193.stdout:8/599: write d2/d3/d27/d69/dae/f5a [786523,59641] 0 2024-04-25T15:49:44.508 INFO:tasks.workunit.client.1.smithi193.stdout:2/632: dread d0/d42/d51/d4f/f97 [0,4194304] 0 2024-04-25T15:49:44.512 INFO:tasks.workunit.client.0.smithi110.stdout:2/719: dread de/d40/d70/f7f [0,4194304] 0 2024-04-25T15:49:44.512 INFO:tasks.workunit.client.1.smithi193.stdout:9/591: creat df/d2c/d34/d3e/fb7 x:0 0 0 2024-04-25T15:49:44.513 INFO:tasks.workunit.client.0.smithi110.stdout:3/818: rmdir d0/d2e 39 2024-04-25T15:49:44.514 INFO:tasks.workunit.client.0.smithi110.stdout:8/886: dread d14/d32/d36/d48/fb3 [0,4194304] 0 2024-04-25T15:49:44.517 INFO:tasks.workunit.client.1.smithi193.stdout:5/623: dread d17/d18/d30/d34/f46 [0,4194304] 0 2024-04-25T15:49:44.518 INFO:tasks.workunit.client.0.smithi110.stdout:7/867: creat d1b/f122 x:0 0 0 2024-04-25T15:49:44.520 INFO:tasks.workunit.client.0.smithi110.stdout:8/887: fsync d14/d21/d23/fd3 0 2024-04-25T15:49:44.527 INFO:tasks.workunit.client.0.smithi110.stdout:3/819: dread d0/d7/f17 [4194304,4194304] 0 2024-04-25T15:49:44.527 INFO:tasks.workunit.client.1.smithi193.stdout:9/592: dread df/d11/d8b/f52 [0,4194304] 0 2024-04-25T15:49:44.527 INFO:tasks.workunit.client.1.smithi193.stdout:1/553: mkdir dc/d4a/d50/d5c/d5d/d3d/d55/d8a/db7 0 2024-04-25T15:49:44.531 INFO:tasks.workunit.client.0.smithi110.stdout:7/868: write d1b/d29/d46/fb3 [4983690,30682] 0 2024-04-25T15:49:44.539 INFO:tasks.workunit.client.0.smithi110.stdout:3/820: dwrite d0/d2e/d88/dbf/d92/f89 [4194304,4194304] 0 2024-04-25T15:49:44.539 INFO:tasks.workunit.client.1.smithi193.stdout:2/633: creat d0/d37/d4d/dbe/df7/ffa x:0 0 0 2024-04-25T15:49:44.541 INFO:tasks.workunit.client.1.smithi193.stdout:4/674: link db/d13/cd3 db/d4b/d51/d54/d6f/cd4 0 2024-04-25T15:49:44.542 INFO:tasks.workunit.client.0.smithi110.stdout:8/888: link d14/d21/d26/d6a/d7e/da6/de8/c93 d14/d21/d26/d6a/d7e/da6/de8/d71/c132 0 2024-04-25T15:49:44.542 INFO:tasks.workunit.client.1.smithi193.stdout:9/593: dread df/d2c/d2f/d4e/d1c/d30/d3b/f47 [0,4194304] 0 2024-04-25T15:49:44.557 INFO:tasks.workunit.client.0.smithi110.stdout:7/869: rename d1b/d29/d38/d55/dab/db2/c10d to d1b/d29/db5/c123 0 2024-04-25T15:49:44.559 INFO:tasks.workunit.client.0.smithi110.stdout:8/889: creat d14/d32/d36/f133 x:0 0 0 2024-04-25T15:49:44.560 INFO:tasks.workunit.client.1.smithi193.stdout:6/589: write dd/d12/f1e [545183,121608] 0 2024-04-25T15:49:44.561 INFO:tasks.workunit.client.0.smithi110.stdout:7/870: mkdir d1b/d29/d38/d55/dab/db2/db8/de3/d124 0 2024-04-25T15:49:44.564 INFO:tasks.workunit.client.1.smithi193.stdout:3/636: dwrite d8/d13/d1f/dbf/f8c [0,4194304] 0 2024-04-25T15:49:44.564 INFO:tasks.workunit.client.0.smithi110.stdout:3/821: dwrite d0/d16/dcb/f94 [4194304,4194304] 0 2024-04-25T15:49:44.587 INFO:tasks.workunit.client.0.smithi110.stdout:8/890: creat d14/d32/d36/db7/dc9/df9/f134 x:0 0 0 2024-04-25T15:49:44.588 INFO:tasks.workunit.client.0.smithi110.stdout:8/891: chown d14/d32/d36/db7 25859575 1 2024-04-25T15:49:44.588 INFO:tasks.workunit.client.1.smithi193.stdout:7/594: fsync d12/d1c/d63/d70/d68/d8d/f9c 0 2024-04-25T15:49:44.591 INFO:tasks.workunit.client.1.smithi193.stdout:0/632: creat d0/d49/d5/d7/fd5 x:0 0 0 2024-04-25T15:49:44.592 INFO:tasks.workunit.client.1.smithi193.stdout:0/633: readlink d0/d49/l4a 0 2024-04-25T15:49:44.593 INFO:tasks.workunit.client.1.smithi193.stdout:8/600: rmdir d2/d18/da0 39 2024-04-25T15:49:44.594 INFO:tasks.workunit.client.0.smithi110.stdout:3/822: chown d0/d2e/d88/dbf/cdd 2 1 2024-04-25T15:49:44.598 INFO:tasks.workunit.client.0.smithi110.stdout:3/823: rmdir d0/d7/d40 39 2024-04-25T15:49:44.599 INFO:tasks.workunit.client.0.smithi110.stdout:7/871: dwrite d1b/d29/f2b [0,4194304] 0 2024-04-25T15:49:44.600 INFO:tasks.workunit.client.0.smithi110.stdout:8/892: mknod d14/c135 0 2024-04-25T15:49:44.601 INFO:tasks.workunit.client.1.smithi193.stdout:1/554: mkdir dc/d4a/d50/dab/db8 0 2024-04-25T15:49:44.601 INFO:tasks.workunit.client.0.smithi110.stdout:8/893: write d14/d32/d36/db7/dc9/df9/f134 [940023,100590] 0 2024-04-25T15:49:44.602 INFO:tasks.workunit.client.1.smithi193.stdout:6/590: rmdir dd/d12/d19/d1c/d40/d3c/d3f 39 2024-04-25T15:49:44.604 INFO:tasks.workunit.client.0.smithi110.stdout:7/872: readlink d1b/d29/d38/d55/dab/db2/db8/de3/l11e 0 2024-04-25T15:49:44.605 INFO:tasks.workunit.client.1.smithi193.stdout:3/637: mkdir d8/d13/d14/d51/dc6 0 2024-04-25T15:49:44.609 INFO:tasks.workunit.client.0.smithi110.stdout:8/894: fdatasync d14/d21/d26/f29 0 2024-04-25T15:49:44.611 INFO:tasks.workunit.client.1.smithi193.stdout:0/634: unlink d0/d49/d5/d7/dd/f5c 0 2024-04-25T15:49:44.611 INFO:tasks.workunit.client.1.smithi193.stdout:8/601: mkdir d2/d3/d27/d69/dae/d4a/dc1 0 2024-04-25T15:49:44.613 INFO:tasks.workunit.client.0.smithi110.stdout:8/895: creat d14/d21/d26/dab/dac/dbb/f136 x:0 0 0 2024-04-25T15:49:44.615 INFO:tasks.workunit.client.1.smithi193.stdout:4/675: link db/d13/d1e/d1f/d56/d69/d70/d8a/f8c db/d13/dd2/fd5 0 2024-04-25T15:49:44.617 INFO:tasks.workunit.client.0.smithi110.stdout:8/896: creat d14/d21/d26/d6a/d7e/da6/de8/d71/dae/f137 x:0 0 0 2024-04-25T15:49:44.617 INFO:tasks.workunit.client.1.smithi193.stdout:5/624: creat d17/d18/d30/d38/d71/d96/fd2 x:0 0 0 2024-04-25T15:49:44.618 INFO:tasks.workunit.client.1.smithi193.stdout:1/555: mknod dc/d4a/d50/dab/cb9 0 2024-04-25T15:49:44.618 INFO:tasks.workunit.client.1.smithi193.stdout:5/625: fsync d17/d18/d30/d38/f69 0 2024-04-25T15:49:44.619 INFO:tasks.workunit.client.0.smithi110.stdout:7/873: write d1b/d29/d38/d8b/d89/fe9 [230167,86032] 0 2024-04-25T15:49:44.620 INFO:tasks.workunit.client.0.smithi110.stdout:8/897: stat d14/d21/d26/dab/dac/cec 0 2024-04-25T15:49:44.622 INFO:tasks.workunit.client.1.smithi193.stdout:3/638: creat d8/d13/d1f/d69/fc7 x:0 0 0 2024-04-25T15:49:44.624 INFO:tasks.workunit.client.0.smithi110.stdout:7/874: dread - d1b/d29/d38/d55/dab/db2/dcd/de2/fe8 zero size 2024-04-25T15:49:44.625 INFO:tasks.workunit.client.1.smithi193.stdout:2/634: write d0/d42/d51/f46 [2188023,107741] 0 2024-04-25T15:49:44.626 INFO:tasks.workunit.client.0.smithi110.stdout:8/898: creat d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/f138 x:0 0 0 2024-04-25T15:49:44.629 INFO:tasks.workunit.client.0.smithi110.stdout:7/875: truncate d1b/d29/f68 4791647 0 2024-04-25T15:49:44.631 INFO:tasks.workunit.client.1.smithi193.stdout:9/594: truncate df/d2c/d2f/d4e/d1c/f9b 678507 0 2024-04-25T15:49:44.633 INFO:tasks.workunit.client.0.smithi110.stdout:7/876: fsync d1b/f97 0 2024-04-25T15:49:44.637 INFO:tasks.workunit.client.1.smithi193.stdout:1/556: mknod dc/d3c/cba 0 2024-04-25T15:49:44.641 INFO:tasks.workunit.client.1.smithi193.stdout:0/635: dwrite d0/d49/d5/d7/dd/d46/d48/f85 [0,4194304] 0 2024-04-25T15:49:44.643 INFO:tasks.workunit.client.1.smithi193.stdout:5/626: mknod d17/d18/d30/d38/d48/d4f/cd3 0 2024-04-25T15:49:44.643 INFO:tasks.workunit.client.1.smithi193.stdout:3/639: truncate d8/d13/d1f/f22 887202 0 2024-04-25T15:49:44.644 INFO:tasks.workunit.client.1.smithi193.stdout:4/676: write db/d4b/fa3 [5062663,122536] 0 2024-04-25T15:49:44.644 INFO:tasks.workunit.client.0.smithi110.stdout:8/899: write d14/d21/d26/d6a/d7e/da6/de8/d71/fd5 [614097,67141] 0 2024-04-25T15:49:44.644 INFO:tasks.workunit.client.1.smithi193.stdout:5/627: stat d17/d18/f24 0 2024-04-25T15:49:44.647 INFO:tasks.workunit.client.0.smithi110.stdout:8/900: mknod d14/d21/d26/d6a/d7e/da6/de8/d71/c139 0 2024-04-25T15:49:44.647 INFO:tasks.workunit.client.1.smithi193.stdout:6/591: write dd/d12/d19/d1c/d40/d3c/d3f/f73 [657334,117739] 0 2024-04-25T15:49:44.650 INFO:tasks.workunit.client.1.smithi193.stdout:8/602: symlink d2/d3/d27/d6f/dbb/lc2 0 2024-04-25T15:49:44.650 INFO:tasks.workunit.client.0.smithi110.stdout:8/901: rmdir d14/d32/d36/dde/d11e 39 2024-04-25T15:49:44.651 INFO:tasks.workunit.client.1.smithi193.stdout:8/603: chown d2/c1e 725333 1 2024-04-25T15:49:44.652 INFO:tasks.workunit.client.1.smithi193.stdout:9/595: unlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/f4b 0 2024-04-25T15:49:44.653 INFO:tasks.workunit.client.0.smithi110.stdout:7/877: getdents d1b/d29/d38/d55/dab/db2 0 2024-04-25T15:49:44.654 INFO:tasks.workunit.client.1.smithi193.stdout:0/636: dwrite d0/d49/d5/d61/fae [0,4194304] 0 2024-04-25T15:49:44.654 INFO:tasks.workunit.client.1.smithi193.stdout:8/604: write d2/d3/d3a/f7c [4861988,29399] 0 2024-04-25T15:49:44.655 INFO:tasks.workunit.client.0.smithi110.stdout:8/902: creat d14/d21/d26/d6a/f13a x:0 0 0 2024-04-25T15:49:44.655 INFO:tasks.workunit.client.1.smithi193.stdout:7/595: sync 2024-04-25T15:49:44.656 INFO:tasks.workunit.client.1.smithi193.stdout:4/677: dwrite db/d13/d1e/d1f/f98 [0,4194304] 0 2024-04-25T15:49:44.656 INFO:tasks.workunit.client.1.smithi193.stdout:8/605: chown d2/d3/d27/d69/dae/d29 14 1 2024-04-25T15:49:44.657 INFO:tasks.workunit.client.1.smithi193.stdout:0/637: readlink d0/d49/d5/d7/dd/d16/d26/d5e/d99/lc2 0 2024-04-25T15:49:44.658 INFO:tasks.workunit.client.1.smithi193.stdout:8/606: write d2/d3/d20/f8b [4156513,89308] 0 2024-04-25T15:49:44.660 INFO:tasks.workunit.client.1.smithi193.stdout:2/635: chown d0/d9/d15/d31/fd9 0 1 2024-04-25T15:49:44.662 INFO:tasks.workunit.client.1.smithi193.stdout:2/636: dread - d0/d37/d25/d5f/d3a/d6d/d6e/db9/fd6 zero size 2024-04-25T15:49:44.664 INFO:tasks.workunit.client.0.smithi110.stdout:7/878: rename d1b/d29/d87/f118 to d1b/d29/f125 0 2024-04-25T15:49:44.668 INFO:tasks.workunit.client.0.smithi110.stdout:8/903: rename d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/ddd/f100 to d14/d32/d105/d110/f13b 0 2024-04-25T15:49:44.670 INFO:tasks.workunit.client.0.smithi110.stdout:8/904: write d14/d32/d36/db7/dc9/f119 [1021945,58895] 0 2024-04-25T15:49:44.670 INFO:tasks.workunit.client.0.smithi110.stdout:7/879: chown d1b/d29/d46/fe5 2060875103 1 2024-04-25T15:49:44.672 INFO:tasks.workunit.client.1.smithi193.stdout:4/678: creat db/d4b/d51/fd6 x:0 0 0 2024-04-25T15:49:44.673 INFO:tasks.workunit.client.1.smithi193.stdout:4/679: chown db/d13/d1e/d1f/d56/d69/l89 2 1 2024-04-25T15:49:44.688 INFO:tasks.workunit.client.0.smithi110.stdout:8/905: dread d14/d21/fba [0,4194304] 0 2024-04-25T15:49:44.688 INFO:tasks.workunit.client.0.smithi110.stdout:8/906: readlink d14/d21/d26/d6a/db9/lbe 0 2024-04-25T15:49:44.733 INFO:tasks.workunit.client.0.smithi110.stdout:7/880: mknod d1b/d29/d38/d8b/d89/c126 0 2024-04-25T15:49:44.735 INFO:tasks.workunit.client.1.smithi193.stdout:0/638: read d0/d49/d5/f2d [54568,4483] 0 2024-04-25T15:49:44.736 INFO:tasks.workunit.client.0.smithi110.stdout:7/881: mknod d1b/d29/db5/c127 0 2024-04-25T15:49:44.739 INFO:tasks.workunit.client.1.smithi193.stdout:9/596: write df/d2c/d34/d3e/fa5 [2816547,1852] 0 2024-04-25T15:49:44.739 INFO:tasks.workunit.client.1.smithi193.stdout:1/557: write dc/d4a/d50/d5c/fa2 [355606,38611] 0 2024-04-25T15:49:44.741 INFO:tasks.workunit.client.1.smithi193.stdout:6/592: mkdir dd/db5 0 2024-04-25T15:49:44.741 INFO:tasks.workunit.client.0.smithi110.stdout:7/882: mknod d1b/d29/d38/d55/d95/dcb/d105/c128 0 2024-04-25T15:49:44.741 INFO:tasks.workunit.client.1.smithi193.stdout:9/597: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/l96 26 1 2024-04-25T15:49:44.741 INFO:tasks.workunit.client.1.smithi193.stdout:9/598: readlink l7 0 2024-04-25T15:49:44.742 INFO:tasks.workunit.client.0.smithi110.stdout:8/907: write d14/d21/d23/f24 [1600224,124187] 0 2024-04-25T15:49:44.743 INFO:tasks.workunit.client.1.smithi193.stdout:7/596: mkdir d12/d1c/dd6/dda 0 2024-04-25T15:49:44.744 INFO:tasks.workunit.client.1.smithi193.stdout:4/680: symlink db/d13/d1e/d1f/d56/d69/dad/dc0/ld7 0 2024-04-25T15:49:44.744 INFO:tasks.workunit.client.0.smithi110.stdout:7/883: fsync d1b/d29/d38/d55/d7f/f78 0 2024-04-25T15:49:44.745 INFO:tasks.workunit.client.1.smithi193.stdout:2/637: sync 2024-04-25T15:49:44.746 INFO:tasks.workunit.client.1.smithi193.stdout:0/639: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf/ld6 0 2024-04-25T15:49:44.747 INFO:tasks.workunit.client.1.smithi193.stdout:3/640: creat d8/d13/fc8 x:0 0 0 2024-04-25T15:49:44.748 INFO:tasks.workunit.client.1.smithi193.stdout:0/640: chown d0/d49/d5/d7/l44 6 1 2024-04-25T15:49:44.750 INFO:tasks.workunit.client.0.smithi110.stdout:7/884: creat d1b/d29/db5/dca/f129 x:0 0 0 2024-04-25T15:49:44.750 INFO:tasks.workunit.client.1.smithi193.stdout:7/597: dwrite d12/d1c/d63/d70/f34 [0,4194304] 0 2024-04-25T15:49:44.751 INFO:tasks.workunit.client.0.smithi110.stdout:7/885: dread - d1b/d29/d46/d106/f11a zero size 2024-04-25T15:49:44.751 INFO:tasks.workunit.client.1.smithi193.stdout:7/598: chown d12/d1c/d63/fce 49849 1 2024-04-25T15:49:44.751 INFO:tasks.workunit.client.1.smithi193.stdout:5/628: creat d17/d18/d30/fd4 x:0 0 0 2024-04-25T15:49:44.753 INFO:tasks.workunit.client.1.smithi193.stdout:1/558: creat dc/d4a/d4d/d7a/da6/fbb x:0 0 0 2024-04-25T15:49:44.754 INFO:tasks.workunit.client.1.smithi193.stdout:6/593: symlink dd/d12/d19/d45/lb6 0 2024-04-25T15:49:44.755 INFO:tasks.workunit.client.0.smithi110.stdout:7/886: creat d1b/d29/d46/f12a x:0 0 0 2024-04-25T15:49:44.756 INFO:tasks.workunit.client.0.smithi110.stdout:7/887: stat d1b/d29/d38/d8b/d89/dc6/df3 0 2024-04-25T15:49:44.758 INFO:tasks.workunit.client.0.smithi110.stdout:7/888: write d1b/d29/d87/fa2 [1967208,40317] 0 2024-04-25T15:49:44.760 INFO:tasks.workunit.client.1.smithi193.stdout:6/594: dwrite dd/d12/f5d [0,4194304] 0 2024-04-25T15:49:44.762 INFO:tasks.workunit.client.0.smithi110.stdout:8/908: dread d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/fb0 [0,4194304] 0 2024-04-25T15:49:44.763 INFO:tasks.workunit.client.1.smithi193.stdout:4/681: rename db/d32/cc7 to db/d13/d1e/d1f/d22/cd8 0 2024-04-25T15:49:44.765 INFO:tasks.workunit.client.1.smithi193.stdout:8/607: dwrite d2/d3/d27/d69/dae/f1a [0,4194304] 0 2024-04-25T15:49:44.766 INFO:tasks.workunit.client.1.smithi193.stdout:8/608: readlink d2/d3/d27/d6f/d94/l30 0 2024-04-25T15:49:44.770 INFO:tasks.workunit.client.1.smithi193.stdout:3/641: creat d8/d13/d1f/d69/d77/fc9 x:0 0 0 2024-04-25T15:49:44.772 INFO:tasks.workunit.client.1.smithi193.stdout:3/642: read d8/d13/d14/d27/f95 [267914,49906] 0 2024-04-25T15:49:44.773 INFO:tasks.workunit.client.1.smithi193.stdout:0/641: dread - d0/d49/d5/d7/dd/d46/d48/f98 zero size 2024-04-25T15:49:44.774 INFO:tasks.workunit.client.1.smithi193.stdout:3/643: chown d8/d13/d14/d27/f73 1 1 2024-04-25T15:49:44.775 INFO:tasks.workunit.client.1.smithi193.stdout:9/599: dwrite df/d2c/d34/faa [0,4194304] 0 2024-04-25T15:49:44.776 INFO:tasks.workunit.client.0.smithi110.stdout:7/889: rename d1b/d29/d38/d55/dab/db2/db8/de3/cfd to d1b/d29/d46/d106/c12b 0 2024-04-25T15:49:44.788 INFO:tasks.workunit.client.1.smithi193.stdout:2/638: dwrite d0/d37/d25/d5f/d3a/d6d/d85/f8d [0,4194304] 0 2024-04-25T15:49:44.792 INFO:tasks.workunit.client.1.smithi193.stdout:6/595: read dd/d12/d19/f51 [702513,2482] 0 2024-04-25T15:49:44.793 INFO:tasks.workunit.client.0.smithi110.stdout:7/890: dwrite d1b/d29/f68 [0,4194304] 0 2024-04-25T15:49:44.794 INFO:tasks.workunit.client.0.smithi110.stdout:8/909: dread d14/f1c [0,4194304] 0 2024-04-25T15:49:44.794 INFO:tasks.workunit.client.1.smithi193.stdout:3/644: write d8/d13/d4b/f88 [2346362,78064] 0 2024-04-25T15:49:44.795 INFO:tasks.workunit.client.1.smithi193.stdout:5/629: dwrite d17/d18/d30/d34/f46 [0,4194304] 0 2024-04-25T15:49:44.797 INFO:tasks.workunit.client.1.smithi193.stdout:3/645: chown d8/d13/d1f/d69/fb0 989 1 2024-04-25T15:49:44.797 INFO:tasks.workunit.client.0.smithi110.stdout:7/891: chown c12 50947466 1 2024-04-25T15:49:44.800 INFO:tasks.workunit.client.1.smithi193.stdout:4/682: dwrite db/d13/dd2/fd5 [0,4194304] 0 2024-04-25T15:49:44.803 INFO:tasks.workunit.client.0.smithi110.stdout:7/892: creat d1b/d29/d38/d55/d95/f12c x:0 0 0 2024-04-25T15:49:44.811 INFO:tasks.workunit.client.0.smithi110.stdout:7/893: rename d1b/cc3 to d1b/d29/db5/dca/df9/c12d 0 2024-04-25T15:49:44.812 INFO:tasks.workunit.client.1.smithi193.stdout:5/630: read d17/d18/d30/d3b/f60 [2233241,125139] 0 2024-04-25T15:49:44.814 INFO:tasks.workunit.client.1.smithi193.stdout:5/631: dread d17/d18/d30/f8a [0,4194304] 0 2024-04-25T15:49:44.818 INFO:tasks.workunit.client.0.smithi110.stdout:7/894: dread d1b/d29/d46/f51 [4194304,4194304] 0 2024-04-25T15:49:44.819 INFO:tasks.workunit.client.1.smithi193.stdout:7/599: creat d12/d13/dc4/fdb x:0 0 0 2024-04-25T15:49:44.821 INFO:tasks.workunit.client.1.smithi193.stdout:8/609: mknod d2/d3/d27/d69/dae/d4a/dc1/cc3 0 2024-04-25T15:49:44.821 INFO:tasks.workunit.client.0.smithi110.stdout:8/910: dwrite d14/d32/d36/f84 [8388608,4194304] 0 2024-04-25T15:49:44.823 INFO:tasks.workunit.client.0.smithi110.stdout:7/895: unlink d1b/d29/d38/d55/d95/ff4 0 2024-04-25T15:49:44.824 INFO:tasks.workunit.client.0.smithi110.stdout:7/896: chown d1b/d29/db5/dca/d10b/f11d 53165861 1 2024-04-25T15:49:44.827 INFO:tasks.workunit.client.1.smithi193.stdout:0/642: mknod d0/d49/d5/d7/dd/d16/d18/d28/dcf/cd7 0 2024-04-25T15:49:44.830 INFO:tasks.workunit.client.1.smithi193.stdout:9/600: mkdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8 0 2024-04-25T15:49:44.831 INFO:tasks.workunit.client.1.smithi193.stdout:2/639: truncate d0/d42/d51/d4f/f97 1902415 0 2024-04-25T15:49:44.831 INFO:tasks.workunit.client.1.smithi193.stdout:9/601: write df/d2c/d95/fb6 [97258,85950] 0 2024-04-25T15:49:44.833 INFO:tasks.workunit.client.1.smithi193.stdout:6/596: creat dd/d12/d2d/fb7 x:0 0 0 2024-04-25T15:49:44.834 INFO:tasks.workunit.client.1.smithi193.stdout:6/597: chown dd/d12/d19/d45/c98 117325229 1 2024-04-25T15:49:44.838 INFO:tasks.workunit.client.1.smithi193.stdout:4/683: mknod db/d32/d40/cd9 0 2024-04-25T15:49:44.839 INFO:tasks.workunit.client.0.smithi110.stdout:7/897: dread f14 [0,4194304] 0 2024-04-25T15:49:44.840 INFO:tasks.workunit.client.0.smithi110.stdout:7/898: chown d1b/d29/d38/d8b/d89/fcc 89 1 2024-04-25T15:49:44.840 INFO:tasks.workunit.client.0.smithi110.stdout:7/899: readlink d1b/d29/d38/d8b/d89/lc0 0 2024-04-25T15:49:44.841 INFO:tasks.workunit.client.1.smithi193.stdout:5/632: write d17/f97 [3145214,27605] 0 2024-04-25T15:49:44.842 INFO:tasks.workunit.client.1.smithi193.stdout:5/633: fdatasync d17/d18/d75/fbf 0 2024-04-25T15:49:44.842 INFO:tasks.workunit.client.1.smithi193.stdout:7/600: creat d12/da1/fdc x:0 0 0 2024-04-25T15:49:44.844 INFO:tasks.workunit.client.0.smithi110.stdout:7/900: mkdir d1b/d29/db5/dca/df9/d12e 0 2024-04-25T15:49:44.847 INFO:tasks.workunit.client.1.smithi193.stdout:6/598: mknod dd/d12/d2d/cb8 0 2024-04-25T15:49:44.849 INFO:tasks.workunit.client.0.smithi110.stdout:7/901: dread d1b/d29/f60 [0,4194304] 0 2024-04-25T15:49:44.854 INFO:tasks.workunit.client.1.smithi193.stdout:0/643: dwrite d0/d49/d5/d7/dd/f58 [4194304,4194304] 0 2024-04-25T15:49:44.854 INFO:tasks.workunit.client.0.smithi110.stdout:8/911: write fb [1543031,53319] 0 2024-04-25T15:49:44.854 INFO:tasks.workunit.client.0.smithi110.stdout:7/902: creat d1b/d29/d38/d55/d95/dcb/d105/f12f x:0 0 0 2024-04-25T15:49:44.855 INFO:tasks.workunit.client.1.smithi193.stdout:8/610: unlink d2/d3/d20/f22 0 2024-04-25T15:49:44.855 INFO:tasks.workunit.client.1.smithi193.stdout:8/611: fsync d2/d3/d3a/f4c 0 2024-04-25T15:49:44.856 INFO:tasks.workunit.client.1.smithi193.stdout:7/601: creat d12/d13/fdd x:0 0 0 2024-04-25T15:49:44.857 INFO:tasks.workunit.client.0.smithi110.stdout:8/912: unlink d14/d21/f27 0 2024-04-25T15:49:44.858 INFO:tasks.workunit.client.0.smithi110.stdout:4/802: sync 2024-04-25T15:49:44.859 INFO:tasks.workunit.client.1.smithi193.stdout:6/599: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/fb9 x:0 0 0 2024-04-25T15:49:44.861 INFO:tasks.workunit.client.1.smithi193.stdout:8/612: dwrite d2/d3/d27/f7a [0,4194304] 0 2024-04-25T15:49:44.862 INFO:tasks.workunit.client.1.smithi193.stdout:1/559: link dc/d4a/d4d/ca1 dc/d4a/d50/d5c/d63/cbc 0 2024-04-25T15:49:44.866 INFO:tasks.workunit.client.1.smithi193.stdout:4/684: dread db/fc [0,4194304] 0 2024-04-25T15:49:44.867 INFO:tasks.workunit.client.0.smithi110.stdout:7/903: dread d1b/d29/d38/d8b/f92 [0,4194304] 0 2024-04-25T15:49:44.867 INFO:tasks.workunit.client.0.smithi110.stdout:4/803: symlink d13/d24/d26/l116 0 2024-04-25T15:49:44.869 INFO:tasks.workunit.client.1.smithi193.stdout:5/634: mkdir d17/d18/d3e/db6/dd5 0 2024-04-25T15:49:44.871 INFO:tasks.workunit.client.1.smithi193.stdout:0/644: truncate d0/d49/d5/d7/dd/d16/d26/d2e/f34 336043 0 2024-04-25T15:49:44.872 INFO:tasks.workunit.client.1.smithi193.stdout:3/646: dwrite d8/d13/d14/f59 [0,4194304] 0 2024-04-25T15:49:44.873 INFO:tasks.workunit.client.1.smithi193.stdout:0/645: stat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/ccb 0 2024-04-25T15:49:44.875 INFO:tasks.workunit.client.1.smithi193.stdout:0/646: stat d0/d49/d5/d7/dd/d46/d48/d62/f8f 0 2024-04-25T15:49:44.876 INFO:tasks.workunit.client.0.smithi110.stdout:8/913: dread d14/d32/d36/db7/ff7 [0,4194304] 0 2024-04-25T15:49:44.881 INFO:tasks.workunit.client.0.smithi110.stdout:7/904: mkdir d1b/d29/d38/d8b/d89/dc6/df3/d130 0 2024-04-25T15:49:44.886 INFO:tasks.workunit.client.0.smithi110.stdout:7/905: chown d1b/d29/db5/dca/d10b/c114 1597 1 2024-04-25T15:49:44.887 INFO:tasks.workunit.client.0.smithi110.stdout:7/906: truncate d1b/d29/d38/f61 859821 0 2024-04-25T15:49:44.887 INFO:tasks.workunit.client.1.smithi193.stdout:1/560: unlink dc/d3c/l45 0 2024-04-25T15:49:44.890 INFO:tasks.workunit.client.0.smithi110.stdout:7/907: rmdir d1b/d29/d38/d55/dab/db2/db8/de3 39 2024-04-25T15:49:44.890 INFO:tasks.workunit.client.0.smithi110.stdout:7/908: stat d1b 0 2024-04-25T15:49:44.896 INFO:tasks.workunit.client.0.smithi110.stdout:7/909: creat d1b/d29/db5/dca/df9/d12e/f131 x:0 0 0 2024-04-25T15:49:44.897 INFO:tasks.workunit.client.0.smithi110.stdout:7/910: write d1b/d29/db5/f117 [990967,36487] 0 2024-04-25T15:49:44.899 INFO:tasks.workunit.client.0.smithi110.stdout:4/804: dwrite d13/d24/d2d/d5e/f94 [0,4194304] 0 2024-04-25T15:49:44.902 INFO:tasks.workunit.client.1.smithi193.stdout:7/602: dwrite d12/d13/f41 [0,4194304] 0 2024-04-25T15:49:44.905 INFO:tasks.workunit.client.1.smithi193.stdout:5/635: fsync d17/f26 0 2024-04-25T15:49:44.905 INFO:tasks.workunit.client.1.smithi193.stdout:2/640: sync 2024-04-25T15:49:44.905 INFO:tasks.workunit.client.1.smithi193.stdout:6/600: dwrite dd/d2f/d57/f7d [0,4194304] 0 2024-04-25T15:49:44.905 INFO:tasks.workunit.client.0.smithi110.stdout:8/914: dwrite d14/d21/d26/d6a/d7e/da6/fa8 [0,4194304] 0 2024-04-25T15:49:44.906 INFO:tasks.workunit.client.1.smithi193.stdout:3/647: mkdir d8/d13/d4b/d8b/dca 0 2024-04-25T15:49:44.914 INFO:tasks.workunit.client.1.smithi193.stdout:2/641: dwrite d0/d37/d25/d5f/d3a/f58 [0,4194304] 0 2024-04-25T15:49:44.915 INFO:tasks.workunit.client.1.smithi193.stdout:8/613: creat d2/d18/fc4 x:0 0 0 2024-04-25T15:49:44.917 INFO:tasks.workunit.client.1.smithi193.stdout:8/614: write d2/d18/db1/fb3 [4929189,73309] 0 2024-04-25T15:49:44.917 INFO:tasks.workunit.client.0.smithi110.stdout:8/915: rename d14/d21/d26/d6a/cd9 to d14/d21/d26/d6a/c13c 0 2024-04-25T15:49:44.920 INFO:tasks.workunit.client.1.smithi193.stdout:1/561: rename dc/d4a/d50/d5c/d5d/d3d/d55/d8a/lae to dc/d4a/d50/dab/lbd 0 2024-04-25T15:49:44.921 INFO:tasks.workunit.client.1.smithi193.stdout:1/562: write dc/f87 [636100,59903] 0 2024-04-25T15:49:44.922 INFO:tasks.workunit.client.1.smithi193.stdout:4/685: rmdir db/d13/d1e/d1f/d56/d69/d70/d9a 0 2024-04-25T15:49:44.925 INFO:tasks.workunit.client.0.smithi110.stdout:8/916: truncate d14/d21/d26/dab/dac/dbb/f10d 466071 0 2024-04-25T15:49:44.929 INFO:tasks.workunit.client.0.smithi110.stdout:4/805: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/d88/f8f [0,4194304] 0 2024-04-25T15:49:44.929 INFO:tasks.workunit.client.1.smithi193.stdout:0/647: creat d0/d49/d5/d7/fd8 x:0 0 0 2024-04-25T15:49:44.930 INFO:tasks.workunit.client.0.smithi110.stdout:8/917: mknod d14/d32/d36/db7/c13d 0 2024-04-25T15:49:44.931 INFO:tasks.workunit.client.1.smithi193.stdout:3/648: readlink d8/d13/l1e 0 2024-04-25T15:49:44.932 INFO:tasks.workunit.client.1.smithi193.stdout:5/636: readlink d17/d18/d3e/l77 0 2024-04-25T15:49:44.932 INFO:tasks.workunit.client.1.smithi193.stdout:0/648: truncate d0/d49/d5/d7/dd/d16/d18/d28/d56/f70 4736097 0 2024-04-25T15:49:44.932 INFO:tasks.workunit.client.1.smithi193.stdout:5/637: stat d17/d18/d30 0 2024-04-25T15:49:44.934 INFO:tasks.workunit.client.0.smithi110.stdout:4/806: truncate d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/f112 1381436 0 2024-04-25T15:49:44.934 INFO:tasks.workunit.client.0.smithi110.stdout:8/918: mkdir d14/d21/d26/dab/d13e 0 2024-04-25T15:49:44.939 INFO:tasks.workunit.client.1.smithi193.stdout:8/615: mknod d2/cc5 0 2024-04-25T15:49:44.940 INFO:tasks.workunit.client.0.smithi110.stdout:8/919: symlink d14/l13f 0 2024-04-25T15:49:44.941 INFO:tasks.workunit.client.1.smithi193.stdout:4/686: creat db/d32/d45/fda x:0 0 0 2024-04-25T15:49:44.944 INFO:tasks.workunit.client.0.smithi110.stdout:4/807: getdents d13/d24/d2d/d44 0 2024-04-25T15:49:44.944 INFO:tasks.workunit.client.1.smithi193.stdout:3/649: fsync d8/d13/d14/f3e 0 2024-04-25T15:49:44.945 INFO:tasks.workunit.client.1.smithi193.stdout:5/638: chown d17/l58 269 1 2024-04-25T15:49:44.946 INFO:tasks.workunit.client.1.smithi193.stdout:0/649: chown d0/d49/d5/d7/l9d 5 1 2024-04-25T15:49:44.949 INFO:tasks.workunit.client.0.smithi110.stdout:8/920: getdents d14/d32/d105 0 2024-04-25T15:49:44.951 INFO:tasks.workunit.client.1.smithi193.stdout:1/563: mkdir dc/d4a/d50/d5c/d63/dbe 0 2024-04-25T15:49:44.951 INFO:tasks.workunit.client.0.smithi110.stdout:4/808: dwrite d13/d24/d2d/f37 [8388608,4194304] 0 2024-04-25T15:49:44.955 INFO:tasks.workunit.client.1.smithi193.stdout:7/603: link d12/d1c/d63/f65 d12/da1/fde 0 2024-04-25T15:49:44.957 INFO:tasks.workunit.client.0.smithi110.stdout:4/809: rmdir d13 39 2024-04-25T15:49:44.957 INFO:tasks.workunit.client.1.smithi193.stdout:1/564: dwrite dc/d4a/d4d/d7a/d85/fb6 [0,4194304] 0 2024-04-25T15:49:44.960 INFO:tasks.workunit.client.0.smithi110.stdout:8/921: creat d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/f140 x:0 0 0 2024-04-25T15:49:44.961 INFO:tasks.workunit.client.1.smithi193.stdout:9/602: sync 2024-04-25T15:49:44.963 INFO:tasks.workunit.client.1.smithi193.stdout:5/639: creat d17/d18/d30/d38/d48/d4f/fd6 x:0 0 0 2024-04-25T15:49:44.965 INFO:tasks.workunit.client.1.smithi193.stdout:0/650: mkdir d0/d49/d5/d7/dd/d46/d48/d62/dd9 0 2024-04-25T15:49:44.966 INFO:tasks.workunit.client.1.smithi193.stdout:0/651: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf 6550244 1 2024-04-25T15:49:44.968 INFO:tasks.workunit.client.1.smithi193.stdout:6/601: rename dd/d2f/d57/c9d to dd/d12/d19/d1c/d40/d3c/d3f/cba 0 2024-04-25T15:49:44.974 INFO:tasks.workunit.client.1.smithi193.stdout:7/604: mknod d12/d1c/dd6/cdf 0 2024-04-25T15:49:44.975 INFO:tasks.workunit.client.1.smithi193.stdout:7/605: chown d12/d13/f43 979045 1 2024-04-25T15:49:44.975 INFO:tasks.workunit.client.1.smithi193.stdout:2/642: dwrite d0/d37/dbb/df8/f86 [0,4194304] 0 2024-04-25T15:49:44.977 INFO:tasks.workunit.client.1.smithi193.stdout:9/603: mknod df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/cb9 0 2024-04-25T15:49:44.978 INFO:tasks.workunit.client.1.smithi193.stdout:4/687: write db/d13/d1e/f41 [3702462,82899] 0 2024-04-25T15:49:44.979 INFO:tasks.workunit.client.1.smithi193.stdout:5/640: rename d17/d18/dcb to d17/d18/d30/d38/d71/dd7 0 2024-04-25T15:49:44.981 INFO:tasks.workunit.client.1.smithi193.stdout:6/602: fsync dd/d12/d19/d1c/f26 0 2024-04-25T15:49:44.982 INFO:tasks.workunit.client.0.smithi110.stdout:8/922: write d14/d21/d26/dab/fb1 [4812994,101817] 0 2024-04-25T15:49:44.984 INFO:tasks.workunit.client.1.smithi193.stdout:1/565: mknod dc/d4a/d50/d5c/d5d/d3d/db0/cbf 0 2024-04-25T15:49:44.985 INFO:tasks.workunit.client.1.smithi193.stdout:2/643: rmdir d0/d37/d25/d5f 39 2024-04-25T15:49:44.987 INFO:tasks.workunit.client.1.smithi193.stdout:7/606: dwrite d12/d1c/d63/d70/d68/d8f/fab [0,4194304] 0 2024-04-25T15:49:44.987 INFO:tasks.workunit.client.1.smithi193.stdout:6/603: stat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5 0 2024-04-25T15:49:44.989 INFO:tasks.workunit.client.1.smithi193.stdout:3/650: write d8/d13/d4b/f6e [624741,39151] 0 2024-04-25T15:49:44.990 INFO:tasks.workunit.client.0.smithi110.stdout:4/810: dwrite d13/d24/f2a [0,4194304] 0 2024-04-25T15:49:44.996 INFO:tasks.workunit.client.1.smithi193.stdout:0/652: dwrite d0/d49/d5/d7/dd/f73 [0,4194304] 0 2024-04-25T15:49:44.998 INFO:tasks.workunit.client.1.smithi193.stdout:9/604: fsync df/d2c/d2f/d4e/d1c/d30/f7a 0 2024-04-25T15:49:44.999 INFO:tasks.workunit.client.1.smithi193.stdout:3/651: dwrite d8/d30/f40 [4194304,4194304] 0 2024-04-25T15:49:45.003 INFO:tasks.workunit.client.1.smithi193.stdout:1/566: unlink dc/d4a/d77/la4 0 2024-04-25T15:49:45.003 INFO:tasks.workunit.client.0.smithi110.stdout:8/923: dwrite d14/d21/fc8 [0,4194304] 0 2024-04-25T15:49:45.004 INFO:tasks.workunit.client.1.smithi193.stdout:8/616: mknod d2/d3/d27/d69/dbf/d72/cc6 0 2024-04-25T15:49:45.008 INFO:tasks.workunit.client.1.smithi193.stdout:6/604: rmdir dd/d12/d19/d1c/d40 39 2024-04-25T15:49:45.009 INFO:tasks.workunit.client.1.smithi193.stdout:5/641: dread d17/d18/d30/f76 [0,4194304] 0 2024-04-25T15:49:45.012 INFO:tasks.workunit.client.0.smithi110.stdout:8/924: unlink d14/d32/d36/d48/f5e 0 2024-04-25T15:49:45.013 INFO:tasks.workunit.client.0.smithi110.stdout:6/770: sync 2024-04-25T15:49:45.014 INFO:tasks.workunit.client.1.smithi193.stdout:4/688: creat db/d13/dcb/fdb x:0 0 0 2024-04-25T15:49:45.015 INFO:tasks.workunit.client.0.smithi110.stdout:6/771: chown d1/db/d22/d90/l9b 3 1 2024-04-25T15:49:45.015 INFO:tasks.workunit.client.0.smithi110.stdout:8/925: creat d14/d21/d26/d6a/de9/f141 x:0 0 0 2024-04-25T15:49:45.015 INFO:tasks.workunit.client.0.smithi110.stdout:6/772: chown d1/dd/db5/l10c 22 1 2024-04-25T15:49:45.021 INFO:tasks.workunit.client.0.smithi110.stdout:6/773: truncate d1/db/d22/d90/f2b 3185305 0 2024-04-25T15:49:45.021 INFO:tasks.workunit.client.1.smithi193.stdout:7/607: dread d12/d1c/d63/d70/d68/d8f/fc0 [0,4194304] 0 2024-04-25T15:49:45.021 INFO:tasks.workunit.client.1.smithi193.stdout:6/605: read dd/d12/d19/d1c/d40/d3c/d3f/f73 [382295,23163] 0 2024-04-25T15:49:45.022 INFO:tasks.workunit.client.1.smithi193.stdout:5/642: mkdir d17/d18/d75/d85/d5d/dad/dd8 0 2024-04-25T15:49:45.024 INFO:tasks.workunit.client.1.smithi193.stdout:4/689: symlink db/d13/d9d/ldc 0 2024-04-25T15:49:45.025 INFO:tasks.workunit.client.0.smithi110.stdout:8/926: creat d14/d32/d36/dde/f142 x:0 0 0 2024-04-25T15:49:45.027 INFO:tasks.workunit.client.1.smithi193.stdout:6/606: symlink dd/d2f/d57/lbb 0 2024-04-25T15:49:45.029 INFO:tasks.workunit.client.1.smithi193.stdout:0/653: rename d0/d49/d5/d7/dd/d38/fc1 to d0/fda 0 2024-04-25T15:49:45.030 INFO:tasks.workunit.client.1.smithi193.stdout:9/605: dread df/d2c/d34/d3e/f4a [0,4194304] 0 2024-04-25T15:49:45.031 INFO:tasks.workunit.client.1.smithi193.stdout:5/643: mkdir d17/d53/dd9 0 2024-04-25T15:49:45.031 INFO:tasks.workunit.client.1.smithi193.stdout:5/644: read f11 [3198596,8790] 0 2024-04-25T15:49:45.032 INFO:tasks.workunit.client.1.smithi193.stdout:8/617: rename d2/d3/d27/d69/dae/d4a/lb0 to d2/d3/d27/d69/dae/d4a/d63/da2/lc7 0 2024-04-25T15:49:45.033 INFO:tasks.workunit.client.1.smithi193.stdout:5/645: chown d17/d18/d75/d85/l61 16219898 1 2024-04-25T15:49:45.033 INFO:tasks.workunit.client.1.smithi193.stdout:6/607: mknod dd/d12/cbc 0 2024-04-25T15:49:45.034 INFO:tasks.workunit.client.1.smithi193.stdout:0/654: write d0/d49/d5/d7/dd/d16/d26/d5e/f80 [3971461,67127] 0 2024-04-25T15:49:45.034 INFO:tasks.workunit.client.1.smithi193.stdout:7/608: link d12/d13/c4c d12/d31/ce0 0 2024-04-25T15:49:45.036 INFO:tasks.workunit.client.1.smithi193.stdout:6/608: read - dd/d12/d19/d1c/d40/d3c/d3f/d4f/fb9 zero size 2024-04-25T15:49:45.036 INFO:tasks.workunit.client.1.smithi193.stdout:0/655: fdatasync d0/d49/d5/d7/dd/f3c 0 2024-04-25T15:49:45.036 INFO:tasks.workunit.client.1.smithi193.stdout:5/646: mknod d17/d18/d3e/dd1/cda 0 2024-04-25T15:49:45.040 INFO:tasks.workunit.client.1.smithi193.stdout:6/609: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f92 [310371,77933] 0 2024-04-25T15:49:45.041 INFO:tasks.workunit.client.1.smithi193.stdout:5/647: read - d17/d18/d30/d38/d48/d4f/fd6 zero size 2024-04-25T15:49:45.045 INFO:tasks.workunit.client.1.smithi193.stdout:5/648: truncate d17/d53/fce 316400 0 2024-04-25T15:49:45.046 INFO:tasks.workunit.client.1.smithi193.stdout:0/656: mkdir d0/d49/d5/d7/dd/d46/d48/d62/dd9/ddb 0 2024-04-25T15:49:45.047 INFO:tasks.workunit.client.1.smithi193.stdout:7/609: dwrite d12/d13/f41 [0,4194304] 0 2024-04-25T15:49:45.048 INFO:tasks.workunit.client.1.smithi193.stdout:6/610: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dbd 0 2024-04-25T15:49:45.048 INFO:tasks.workunit.client.1.smithi193.stdout:7/610: stat f7 0 2024-04-25T15:49:45.049 INFO:tasks.workunit.client.1.smithi193.stdout:5/649: creat d17/d18/d30/d38/dab/fdb x:0 0 0 2024-04-25T15:49:45.053 INFO:tasks.workunit.client.1.smithi193.stdout:8/618: dread d2/d3/f37 [0,4194304] 0 2024-04-25T15:49:45.053 INFO:tasks.workunit.client.1.smithi193.stdout:3/652: dread d8/fc [4194304,4194304] 0 2024-04-25T15:49:45.055 INFO:tasks.workunit.client.1.smithi193.stdout:5/650: mkdir d17/d18/d3e/dd1/ddc 0 2024-04-25T15:49:45.055 INFO:tasks.workunit.client.1.smithi193.stdout:2/644: write d0/d42/d51/d4f/f97 [2266877,13559] 0 2024-04-25T15:49:45.056 INFO:tasks.workunit.client.1.smithi193.stdout:3/653: chown d8/d13/d1f/d69/d77/f53 14 1 2024-04-25T15:49:45.060 INFO:tasks.workunit.client.1.smithi193.stdout:5/651: mknod d17/d18/d3e/dd1/cdd 0 2024-04-25T15:49:45.061 INFO:tasks.workunit.client.1.smithi193.stdout:9/606: dread df/d11/f2a [0,4194304] 0 2024-04-25T15:49:45.064 INFO:tasks.workunit.client.1.smithi193.stdout:5/652: chown d17/d18/d75/d85/c94 19444 1 2024-04-25T15:49:45.065 INFO:tasks.workunit.client.1.smithi193.stdout:9/607: creat df/d2d/fba x:0 0 0 2024-04-25T15:49:45.066 INFO:tasks.workunit.client.1.smithi193.stdout:0/657: link d0/d49/d5/d7/dd/d16/d26/d5e/d99/lc2 d0/d49/d5/d7/dd/d78/ldc 0 2024-04-25T15:49:45.066 INFO:tasks.workunit.client.1.smithi193.stdout:5/653: stat d17/d18/d30/d38/lb3 0 2024-04-25T15:49:45.068 INFO:tasks.workunit.client.1.smithi193.stdout:3/654: dwrite d8/fb4 [0,4194304] 0 2024-04-25T15:49:45.069 INFO:tasks.workunit.client.1.smithi193.stdout:3/655: dread - d8/d13/fc8 zero size 2024-04-25T15:49:45.072 INFO:tasks.workunit.client.1.smithi193.stdout:3/656: chown d8/d30/c7c 0 1 2024-04-25T15:49:45.072 INFO:tasks.workunit.client.1.smithi193.stdout:5/654: truncate d17/d18/d30/d38/d71/d96/dc4/f70 658714 0 2024-04-25T15:49:45.073 INFO:tasks.workunit.client.1.smithi193.stdout:5/655: chown d17/d18/d30/d34 59 1 2024-04-25T15:49:45.074 INFO:tasks.workunit.client.1.smithi193.stdout:3/657: read d8/d13/d1f/dbf/f8c [2622976,8501] 0 2024-04-25T15:49:45.077 INFO:tasks.workunit.client.1.smithi193.stdout:6/611: dread dd/f13 [0,4194304] 0 2024-04-25T15:49:45.077 INFO:tasks.workunit.client.1.smithi193.stdout:7/611: truncate d12/d13/f41 1740820 0 2024-04-25T15:49:45.078 INFO:tasks.workunit.client.1.smithi193.stdout:8/619: dread d2/d3/d27/d69/dae/d29/f60 [0,4194304] 0 2024-04-25T15:49:45.078 INFO:tasks.workunit.client.1.smithi193.stdout:7/612: fdatasync d12/d13/f43 0 2024-04-25T15:49:45.079 INFO:tasks.workunit.client.1.smithi193.stdout:8/620: chown d2/d3/d3a/f5e 5690650 1 2024-04-25T15:49:45.081 INFO:tasks.workunit.client.1.smithi193.stdout:9/608: rmdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/db3 0 2024-04-25T15:49:45.084 INFO:tasks.workunit.client.1.smithi193.stdout:5/656: mknod d17/d18/d30/d38/dab/cde 0 2024-04-25T15:49:45.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/609: dwrite df/d2c/d2f/d4e/d1c/f76 [0,4194304] 0 2024-04-25T15:49:45.087 INFO:tasks.workunit.client.1.smithi193.stdout:7/613: creat d12/d1c/d63/d70/d68/fe1 x:0 0 0 2024-04-25T15:49:45.087 INFO:tasks.workunit.client.1.smithi193.stdout:8/621: mkdir d2/d3/d3a/dc8 0 2024-04-25T15:49:45.089 INFO:tasks.workunit.client.1.smithi193.stdout:0/658: link d0/c27 d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/cdd 0 2024-04-25T15:49:45.089 INFO:tasks.workunit.client.1.smithi193.stdout:1/567: sync 2024-04-25T15:49:45.090 INFO:tasks.workunit.client.1.smithi193.stdout:3/658: creat d8/d13/d4b/d8b/dca/fcb x:0 0 0 2024-04-25T15:49:45.093 INFO:tasks.workunit.client.1.smithi193.stdout:7/614: mknod d12/d31/d37/d92/ce2 0 2024-04-25T15:49:45.096 INFO:tasks.workunit.client.1.smithi193.stdout:5/657: rename d17/d18/d30/d38/d48/d4f/f98 to d17/d18/d3e/dd1/ddc/fdf 0 2024-04-25T15:49:45.097 INFO:tasks.workunit.client.1.smithi193.stdout:1/568: mkdir dc/d59/dc0 0 2024-04-25T15:49:45.099 INFO:tasks.workunit.client.1.smithi193.stdout:6/612: creat dd/d12/d19/d1c/d40/fbe x:0 0 0 2024-04-25T15:49:45.099 INFO:tasks.workunit.client.1.smithi193.stdout:3/659: dwrite d8/d30/f40 [4194304,4194304] 0 2024-04-25T15:49:45.103 INFO:tasks.workunit.client.1.smithi193.stdout:8/622: dwrite d2/da5/fa7 [0,4194304] 0 2024-04-25T15:49:45.103 INFO:tasks.workunit.client.1.smithi193.stdout:9/610: link df/d2c/d2f/f6e df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/fbb 0 2024-04-25T15:49:45.106 INFO:tasks.workunit.client.1.smithi193.stdout:0/659: dread d0/d49/d5/d7/dd/d16/f30 [0,4194304] 0 2024-04-25T15:49:45.108 INFO:tasks.workunit.client.1.smithi193.stdout:5/658: dread d17/d18/d30/d34/f46 [0,4194304] 0 2024-04-25T15:49:45.108 INFO:tasks.workunit.client.1.smithi193.stdout:0/660: fsync d0/d49/fe 0 2024-04-25T15:49:45.109 INFO:tasks.workunit.client.1.smithi193.stdout:1/569: chown dc/d3c/l46 31 1 2024-04-25T15:49:45.109 INFO:tasks.workunit.client.1.smithi193.stdout:8/623: read d2/d3/d27/d69/dae/d29/fac [163754,12796] 0 2024-04-25T15:49:45.114 INFO:tasks.workunit.client.1.smithi193.stdout:9/611: creat df/d2c/d2f/d4e/d1c/d30/d3b/fbc x:0 0 0 2024-04-25T15:49:45.116 INFO:tasks.workunit.client.1.smithi193.stdout:6/613: creat dd/d12/d19/d1c/fbf x:0 0 0 2024-04-25T15:49:45.116 INFO:tasks.workunit.client.1.smithi193.stdout:5/659: dwrite d17/d18/d75/f99 [0,4194304] 0 2024-04-25T15:49:45.118 INFO:tasks.workunit.client.1.smithi193.stdout:6/614: fdatasync dd/d12/d19/d1c/d40/d3c/d3f/d4f/f8a 0 2024-04-25T15:49:45.118 INFO:tasks.workunit.client.1.smithi193.stdout:0/661: symlink d0/d49/d5/d7/dd/d16/d18/d28/d56/lde 0 2024-04-25T15:49:45.127 INFO:tasks.workunit.client.1.smithi193.stdout:0/662: creat d0/d49/d5/d7/dd/d16/d18/d28/dcf/fdf x:0 0 0 2024-04-25T15:49:45.128 INFO:tasks.workunit.client.1.smithi193.stdout:8/624: symlink d2/d18/da0/lc9 0 2024-04-25T15:49:45.130 INFO:tasks.workunit.client.1.smithi193.stdout:3/660: truncate d8/d30/d72/f9c 2100103 0 2024-04-25T15:49:45.131 INFO:tasks.workunit.client.1.smithi193.stdout:0/663: dread d0/d49/d5/d7/dd/d46/f6b [0,4194304] 0 2024-04-25T15:49:45.131 INFO:tasks.workunit.client.1.smithi193.stdout:6/615: rmdir dd/d12/d19/d1c 39 2024-04-25T15:49:45.133 INFO:tasks.workunit.client.1.smithi193.stdout:3/661: fdatasync d8/d13/d4b/d8b/dca/fcb 0 2024-04-25T15:49:45.134 INFO:tasks.workunit.client.1.smithi193.stdout:7/615: dwrite d12/d1c/d63/f80 [0,4194304] 0 2024-04-25T15:49:45.140 INFO:tasks.workunit.client.1.smithi193.stdout:7/616: symlink d12/d1c/le3 0 2024-04-25T15:49:45.140 INFO:tasks.workunit.client.1.smithi193.stdout:3/662: chown d8/d13/d14/d27/l7a 135809 1 2024-04-25T15:49:45.140 INFO:tasks.workunit.client.1.smithi193.stdout:8/625: dwrite d2/d3/f5f [0,4194304] 0 2024-04-25T15:49:45.144 INFO:tasks.workunit.client.1.smithi193.stdout:3/663: fdatasync d8/d30/d72/fa3 0 2024-04-25T15:49:45.145 INFO:tasks.workunit.client.1.smithi193.stdout:9/612: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f8d [0,4194304] 0 2024-04-25T15:49:45.147 INFO:tasks.workunit.client.1.smithi193.stdout:8/626: symlink d2/da5/lca 0 2024-04-25T15:49:45.148 INFO:tasks.workunit.client.1.smithi193.stdout:6/616: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/lc0 0 2024-04-25T15:49:45.149 INFO:tasks.workunit.client.1.smithi193.stdout:5/660: dwrite d17/d18/d30/d34/f6c [0,4194304] 0 2024-04-25T15:49:45.152 INFO:tasks.workunit.client.1.smithi193.stdout:7/617: creat d12/d13/dcc/fe4 x:0 0 0 2024-04-25T15:49:45.152 INFO:tasks.workunit.client.1.smithi193.stdout:8/627: readlink d2/l41 0 2024-04-25T15:49:45.152 INFO:tasks.workunit.client.1.smithi193.stdout:9/613: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/f59 [2816084,40248] 0 2024-04-25T15:49:45.156 INFO:tasks.workunit.client.1.smithi193.stdout:3/664: mknod d8/d13/d4b/d8b/dca/ccc 0 2024-04-25T15:49:45.157 INFO:tasks.workunit.client.1.smithi193.stdout:9/614: chown df/d2c/d2f/d4e/d1c/f23 8 1 2024-04-25T15:49:45.160 INFO:tasks.workunit.client.1.smithi193.stdout:5/661: rmdir d17/d53/dd9 0 2024-04-25T15:49:45.161 INFO:tasks.workunit.client.1.smithi193.stdout:0/664: dread d0/d49/d5/f24 [0,4194304] 0 2024-04-25T15:49:45.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/665: mkdir d8/d13/d1f/d69/dcd 0 2024-04-25T15:49:45.162 INFO:tasks.workunit.client.1.smithi193.stdout:7/618: creat d12/d31/d37/d62/d6e/da5/dd7/fe5 x:0 0 0 2024-04-25T15:49:45.163 INFO:tasks.workunit.client.1.smithi193.stdout:6/617: dwrite dd/d2f/d57/f8b [4194304,4194304] 0 2024-04-25T15:49:45.163 INFO:tasks.workunit.client.1.smithi193.stdout:9/615: readlink df/l10 0 2024-04-25T15:49:45.167 INFO:tasks.workunit.client.1.smithi193.stdout:3/666: unlink d8/d13/d1f/d69/d77/l6f 0 2024-04-25T15:49:45.168 INFO:tasks.workunit.client.1.smithi193.stdout:0/665: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/de0 0 2024-04-25T15:49:45.169 INFO:tasks.workunit.client.1.smithi193.stdout:7/619: creat d12/d31/d37/d62/d6e/d79/d9d/fe6 x:0 0 0 2024-04-25T15:49:45.169 INFO:tasks.workunit.client.1.smithi193.stdout:9/616: mknod df/d2c/d2f/d4e/d69/cbd 0 2024-04-25T15:49:45.170 INFO:tasks.workunit.client.1.smithi193.stdout:5/662: rmdir d17/d18/d30/d38/dca 0 2024-04-25T15:49:45.171 INFO:tasks.workunit.client.1.smithi193.stdout:3/667: mknod d8/d13/d1f/d69/d77/cce 0 2024-04-25T15:49:45.173 INFO:tasks.workunit.client.1.smithi193.stdout:6/618: getdents dd/d12/d19/db2 0 2024-04-25T15:49:45.174 INFO:tasks.workunit.client.1.smithi193.stdout:5/663: fsync d17/d18/d30/d38/d71/d96/dc4/f56 0 2024-04-25T15:49:45.175 INFO:tasks.workunit.client.0.smithi110.stdout:3/824: sync 2024-04-25T15:49:45.176 INFO:tasks.workunit.client.0.smithi110.stdout:2/720: sync 2024-04-25T15:49:45.176 INFO:tasks.workunit.client.0.smithi110.stdout:0/775: sync 2024-04-25T15:49:45.176 INFO:tasks.workunit.client.0.smithi110.stdout:9/749: sync 2024-04-25T15:49:45.176 INFO:tasks.workunit.client.0.smithi110.stdout:5/836: sync 2024-04-25T15:49:45.176 INFO:tasks.workunit.client.0.smithi110.stdout:1/965: sync 2024-04-25T15:49:45.176 INFO:tasks.workunit.client.1.smithi193.stdout:3/668: write d8/d13/d1f/d69/d77/f50 [272816,1612] 0 2024-04-25T15:49:45.178 INFO:tasks.workunit.client.0.smithi110.stdout:9/750: chown d9/db/df/f33 208 1 2024-04-25T15:49:45.181 INFO:tasks.workunit.client.0.smithi110.stdout:0/776: dread d1/d23/d30/d33/d5b/d72/d77/fd4 [0,4194304] 0 2024-04-25T15:49:45.181 INFO:tasks.workunit.client.1.smithi193.stdout:9/617: dwrite df/d2d/fba [0,4194304] 0 2024-04-25T15:49:45.181 INFO:tasks.workunit.client.0.smithi110.stdout:1/966: fdatasync d0/d14/d81/df8/d33/d8e/d134/fce 0 2024-04-25T15:49:45.182 INFO:tasks.workunit.client.1.smithi193.stdout:7/620: dwrite d12/d31/d39/d60/f64 [0,4194304] 0 2024-04-25T15:49:45.185 INFO:tasks.workunit.client.0.smithi110.stdout:5/837: dread d4/d1d/f65 [0,4194304] 0 2024-04-25T15:49:45.187 INFO:tasks.workunit.client.0.smithi110.stdout:2/721: symlink de/d40/d6e/d52/d6d/d7d/le5 0 2024-04-25T15:49:45.188 INFO:tasks.workunit.client.1.smithi193.stdout:4/690: sync 2024-04-25T15:49:45.196 INFO:tasks.workunit.client.0.smithi110.stdout:9/751: dwrite d9/d13/d5b/d61/db3/dd3/ddb/fdd [0,4194304] 0 2024-04-25T15:49:45.196 INFO:tasks.workunit.client.1.smithi193.stdout:9/618: rename df/d11 to df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe 0 2024-04-25T15:49:45.200 INFO:tasks.workunit.client.0.smithi110.stdout:1/967: rename d0/d14/d5f/d7d/d72 to d0/d51/d6b/df1/d13b 0 2024-04-25T15:49:45.203 INFO:tasks.workunit.client.0.smithi110.stdout:5/838: fdatasync d4/d15/d51/fa4 0 2024-04-25T15:49:45.203 INFO:tasks.workunit.client.0.smithi110.stdout:5/839: chown d4/d1d/c63 0 1 2024-04-25T15:49:45.204 INFO:tasks.workunit.client.1.smithi193.stdout:8/628: truncate d2/d3/ff 907972 0 2024-04-25T15:49:45.207 INFO:tasks.workunit.client.1.smithi193.stdout:1/570: sync 2024-04-25T15:49:45.207 INFO:tasks.workunit.client.1.smithi193.stdout:2/645: sync 2024-04-25T15:49:45.208 INFO:tasks.workunit.client.0.smithi110.stdout:2/722: symlink de/d40/d5b/le6 0 2024-04-25T15:49:45.209 INFO:tasks.workunit.client.1.smithi193.stdout:2/646: chown d0/d9/d3d/d7d/fe1 13057 1 2024-04-25T15:49:45.209 INFO:tasks.workunit.client.0.smithi110.stdout:2/723: chown de/d40/d6e/d52/d6d/d99 11 1 2024-04-25T15:49:45.211 INFO:tasks.workunit.client.1.smithi193.stdout:5/664: dread d17/f97 [0,4194304] 0 2024-04-25T15:49:45.212 INFO:tasks.workunit.client.0.smithi110.stdout:0/777: unlink d1/d47/f86 0 2024-04-25T15:49:45.213 INFO:tasks.workunit.client.0.smithi110.stdout:3/825: dread d0/d2e/d88/dbf/fbe [0,4194304] 0 2024-04-25T15:49:45.215 INFO:tasks.workunit.client.1.smithi193.stdout:7/621: mkdir d12/d1c/d63/db7/de7 0 2024-04-25T15:49:45.217 INFO:tasks.workunit.client.1.smithi193.stdout:7/622: read - d12/d31/d37/d62/d6e/da5/fb6 zero size 2024-04-25T15:49:45.217 INFO:tasks.workunit.client.1.smithi193.stdout:5/665: dwrite d17/d18/d30/d38/d71/d96/fc7 [0,4194304] 0 2024-04-25T15:49:45.217 INFO:tasks.workunit.client.0.smithi110.stdout:1/968: creat d0/d51/d6b/df1/d13b/d61/f13c x:0 0 0 2024-04-25T15:49:45.220 INFO:tasks.workunit.client.0.smithi110.stdout:0/778: dwrite d1/d23/d30/d33/d5b/d78/d107/fb7 [0,4194304] 0 2024-04-25T15:49:45.222 INFO:tasks.workunit.client.1.smithi193.stdout:9/619: unlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f8d 0 2024-04-25T15:49:45.224 INFO:tasks.workunit.client.1.smithi193.stdout:0/666: dwrite d0/d49/d5/d7/dd/d46/f6b [0,4194304] 0 2024-04-25T15:49:45.224 INFO:tasks.workunit.client.1.smithi193.stdout:4/691: mkdir db/d13/ddd 0 2024-04-25T15:49:45.226 INFO:tasks.workunit.client.1.smithi193.stdout:0/667: dread - d0/d49/d5/d7/dd/d16/d18/d28/dcf/fdf zero size 2024-04-25T15:49:45.227 INFO:tasks.workunit.client.0.smithi110.stdout:2/724: mkdir de/d40/d5b/d5c/d5f/db2/dad/dbd/dce/de7 0 2024-04-25T15:49:45.228 INFO:tasks.workunit.client.1.smithi193.stdout:0/668: fsync d0/d49/d5/d7/dd/d16/d1a/fb6 0 2024-04-25T15:49:45.230 INFO:tasks.workunit.client.1.smithi193.stdout:8/629: creat d2/d3/d27/d6f/d94/fcb x:0 0 0 2024-04-25T15:49:45.233 INFO:tasks.workunit.client.1.smithi193.stdout:6/619: rmdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dbd 0 2024-04-25T15:49:45.233 INFO:tasks.workunit.client.0.smithi110.stdout:7/911: sync 2024-04-25T15:49:45.233 INFO:tasks.workunit.client.0.smithi110.stdout:4/811: sync 2024-04-25T15:49:45.235 INFO:tasks.workunit.client.0.smithi110.stdout:7/912: write d1b/d29/d87/d8e/f94 [1998889,93440] 0 2024-04-25T15:49:45.237 INFO:tasks.workunit.client.0.smithi110.stdout:3/826: read - d0/d16/dcb/db0/fef zero size 2024-04-25T15:49:45.237 INFO:tasks.workunit.client.1.smithi193.stdout:3/669: getdents d8/d13/d1f/dbf 0 2024-04-25T15:49:45.238 INFO:tasks.workunit.client.1.smithi193.stdout:3/670: read d8/d30/d72/fa3 [4281026,15212] 0 2024-04-25T15:49:45.242 INFO:tasks.workunit.client.0.smithi110.stdout:0/779: chown d1/d47/l59 1755768511 1 2024-04-25T15:49:45.243 INFO:tasks.workunit.client.1.smithi193.stdout:7/623: mkdir d12/d31/d37/d92/de8 0 2024-04-25T15:49:45.246 INFO:tasks.workunit.client.1.smithi193.stdout:5/666: symlink d17/d18/d30/d38/d71/d96/le0 0 2024-04-25T15:49:45.247 INFO:tasks.workunit.client.0.smithi110.stdout:2/725: creat de/d40/d4f/d9f/fe8 x:0 0 0 2024-04-25T15:49:45.250 INFO:tasks.workunit.client.1.smithi193.stdout:4/692: dread db/d4b/d51/d54/d77/f79 [0,4194304] 0 2024-04-25T15:49:45.251 INFO:tasks.workunit.client.0.smithi110.stdout:7/913: creat d1b/d29/d38/d55/d95/dcb/f132 x:0 0 0 2024-04-25T15:49:45.252 INFO:tasks.workunit.client.1.smithi193.stdout:9/620: creat df/d2c/d2f/d4e/d1c/fbf x:0 0 0 2024-04-25T15:49:45.253 INFO:tasks.workunit.client.1.smithi193.stdout:9/621: dread - df/d2c/d2f/d4e/d1c/d30/d60/f73 zero size 2024-04-25T15:49:45.254 INFO:tasks.workunit.client.0.smithi110.stdout:3/827: chown d0/d7/d40/c109 268579 1 2024-04-25T15:49:45.256 INFO:tasks.workunit.client.1.smithi193.stdout:0/669: mknod d0/d49/d5/d7/dd/d16/d18/d28/ce1 0 2024-04-25T15:49:45.258 INFO:tasks.workunit.client.0.smithi110.stdout:0/780: creat d1/d23/d30/d33/d5b/d72/d77/f110 x:0 0 0 2024-04-25T15:49:45.262 INFO:tasks.workunit.client.0.smithi110.stdout:3/828: dread d0/d7/fa6 [0,4194304] 0 2024-04-25T15:49:45.262 INFO:tasks.workunit.client.0.smithi110.stdout:5/840: link d4/d1d/d1f/d35/cf2 d4/d1d/d1f/d56/c116 0 2024-04-25T15:49:45.262 INFO:tasks.workunit.client.0.smithi110.stdout:9/752: truncate f6 3468310 0 2024-04-25T15:49:45.263 INFO:tasks.workunit.client.1.smithi193.stdout:6/620: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dc1 0 2024-04-25T15:49:45.264 INFO:tasks.workunit.client.0.smithi110.stdout:7/914: mknod d1b/d29/d38/d8b/d89/c133 0 2024-04-25T15:49:45.264 INFO:tasks.workunit.client.1.smithi193.stdout:6/621: chown dd/d2f/f72 205 1 2024-04-25T15:49:45.266 INFO:tasks.workunit.client.1.smithi193.stdout:3/671: chown d8/d13/d1f/laa 2 1 2024-04-25T15:49:45.268 INFO:tasks.workunit.client.0.smithi110.stdout:0/781: mknod d1/d23/d30/d33/d5b/d78/da4/c111 0 2024-04-25T15:49:45.269 INFO:tasks.workunit.client.1.smithi193.stdout:3/672: write d8/d13/d1f/f9e [1886,107739] 0 2024-04-25T15:49:45.270 INFO:tasks.workunit.client.0.smithi110.stdout:9/753: unlink d9/d13/fbf 0 2024-04-25T15:49:45.272 INFO:tasks.workunit.client.0.smithi110.stdout:7/915: creat d1b/d29/d38/d55/dab/db2/dcd/de2/f134 x:0 0 0 2024-04-25T15:49:45.275 INFO:tasks.workunit.client.1.smithi193.stdout:7/624: rename d12/da1/fde to d12/d31/d37/d92/dbf/fe9 0 2024-04-25T15:49:45.276 INFO:tasks.workunit.client.0.smithi110.stdout:0/782: rename d1/d23/d9e/fda to d1/d23/d30/d33/d5b/d83/de1/f112 0 2024-04-25T15:49:45.281 INFO:tasks.workunit.client.1.smithi193.stdout:0/670: truncate d0/d49/d5/d7/dd/d16/f41 260463 0 2024-04-25T15:49:45.284 INFO:tasks.workunit.client.1.smithi193.stdout:1/571: dwrite dc/f12 [0,4194304] 0 2024-04-25T15:49:45.285 INFO:tasks.workunit.client.0.smithi110.stdout:0/783: truncate d1/d23/fc2 1619793 0 2024-04-25T15:49:45.290 INFO:tasks.workunit.client.1.smithi193.stdout:9/622: rename df/d2d/f61 to df/d2c/d34/d3e/d4d/fc0 0 2024-04-25T15:49:45.290 INFO:tasks.workunit.client.0.smithi110.stdout:1/969: dwrite d0/d51/dc8/f127 [0,4194304] 0 2024-04-25T15:49:45.291 INFO:tasks.workunit.client.1.smithi193.stdout:9/623: write df/d2c/d2f/d4e/d1c/d30/d60/f73 [598854,67035] 0 2024-04-25T15:49:45.291 INFO:tasks.workunit.client.1.smithi193.stdout:4/693: link db/d32/d88/da8/lb5 db/d13/d1e/d1f/d56/d69/d70/lde 0 2024-04-25T15:49:45.301 INFO:tasks.workunit.client.1.smithi193.stdout:0/671: fdatasync d0/d49/d5/d7/dd/d46/d48/f57 0 2024-04-25T15:49:45.305 INFO:tasks.workunit.client.0.smithi110.stdout:1/970: fdatasync d0/d14/d5f/d7d/f97 0 2024-04-25T15:49:45.305 INFO:tasks.workunit.client.0.smithi110.stdout:3/829: link d0/l3c d0/d2e/d88/dbf/d99/dad/l119 0 2024-04-25T15:49:45.306 INFO:tasks.workunit.client.1.smithi193.stdout:9/624: rmdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41 39 2024-04-25T15:49:45.307 INFO:tasks.workunit.client.0.smithi110.stdout:0/784: mkdir d1/d47/d113 0 2024-04-25T15:49:45.309 INFO:tasks.workunit.client.1.smithi193.stdout:4/694: rename db/d4b/d51/fd6 to db/d32/d40/fdf 0 2024-04-25T15:49:45.309 INFO:tasks.workunit.client.1.smithi193.stdout:9/625: dread df/d2c/d2f/d4e/d1c/d30/d60/f73 [0,4194304] 0 2024-04-25T15:49:45.310 INFO:tasks.workunit.client.0.smithi110.stdout:2/726: write de/f28 [5763969,126816] 0 2024-04-25T15:49:45.311 INFO:tasks.workunit.client.1.smithi193.stdout:3/673: link d8/d13/d14/f3e d8/d13/d1f/d69/fcf 0 2024-04-25T15:49:45.313 INFO:tasks.workunit.client.1.smithi193.stdout:2/647: write d0/df/f26 [312644,19824] 0 2024-04-25T15:49:45.315 INFO:tasks.workunit.client.1.smithi193.stdout:4/695: read db/d13/d1e/d1f/d22/f2a [823647,129723] 0 2024-04-25T15:49:45.319 INFO:tasks.workunit.client.1.smithi193.stdout:1/572: creat dc/d59/dc0/fc1 x:0 0 0 2024-04-25T15:49:45.323 INFO:tasks.workunit.client.0.smithi110.stdout:7/916: write d1b/f97 [852440,83371] 0 2024-04-25T15:49:45.326 INFO:tasks.workunit.client.1.smithi193.stdout:6/622: truncate dd/f1b 2658415 0 2024-04-25T15:49:45.328 INFO:tasks.workunit.client.1.smithi193.stdout:9/626: creat df/d2c/d2f/d5c/d7e/fc1 x:0 0 0 2024-04-25T15:49:45.329 INFO:tasks.workunit.client.1.smithi193.stdout:9/627: fdatasync df/d2c/d2f/d5c/fb4 0 2024-04-25T15:49:45.331 INFO:tasks.workunit.client.1.smithi193.stdout:5/667: sync 2024-04-25T15:49:45.331 INFO:tasks.workunit.client.1.smithi193.stdout:3/674: mkdir d8/d30/d45/dd0 0 2024-04-25T15:49:45.332 INFO:tasks.workunit.client.0.smithi110.stdout:9/754: dwrite d9/db/df/fdf [0,4194304] 0 2024-04-25T15:49:45.332 INFO:tasks.workunit.client.1.smithi193.stdout:5/668: truncate d17/d18/f21 12794381 0 2024-04-25T15:49:45.332 INFO:tasks.workunit.client.0.smithi110.stdout:9/755: write d9/db/df/d2c/f36 [1603233,94026] 0 2024-04-25T15:49:45.339 INFO:tasks.workunit.client.1.smithi193.stdout:5/669: dwrite d17/d18/d30/d38/dab/fdb [0,4194304] 0 2024-04-25T15:49:45.340 INFO:tasks.workunit.client.1.smithi193.stdout:5/670: read d17/d18/d75/fbf [502515,111346] 0 2024-04-25T15:49:45.345 INFO:tasks.workunit.client.1.smithi193.stdout:1/573: creat dc/d4a/d4d/d7a/d85/fc2 x:0 0 0 2024-04-25T15:49:45.350 INFO:tasks.workunit.client.1.smithi193.stdout:6/623: creat dd/d12/d19/db2/fc2 x:0 0 0 2024-04-25T15:49:45.353 INFO:tasks.workunit.client.1.smithi193.stdout:3/675: mkdir d8/d13/d1f/d69/d77/dd1 0 2024-04-25T15:49:45.356 INFO:tasks.workunit.client.1.smithi193.stdout:7/625: sync 2024-04-25T15:49:45.357 INFO:tasks.workunit.client.1.smithi193.stdout:7/626: fsync d12/d1c/d63/d70/d68/fb4 0 2024-04-25T15:49:45.357 INFO:tasks.workunit.client.1.smithi193.stdout:2/648: fsync d0/d37/d25/d5f/d3a/d6d/d6e/f96 0 2024-04-25T15:49:45.358 INFO:tasks.workunit.client.1.smithi193.stdout:7/627: write d12/d31/d37/d62/d6e/da5/dd7/fe5 [888797,85523] 0 2024-04-25T15:49:45.359 INFO:tasks.workunit.client.1.smithi193.stdout:2/649: dread d0/d37/d25/d5f/d3a/d6d/fcd [0,4194304] 0 2024-04-25T15:49:45.363 INFO:tasks.workunit.client.1.smithi193.stdout:0/672: creat d0/fe2 x:0 0 0 2024-04-25T15:49:45.364 INFO:tasks.workunit.client.1.smithi193.stdout:5/671: rename d17/d18/d30/d38/d48/d4f/lb2 to d17/d18/d30/d38/d71/d96/le1 0 2024-04-25T15:49:45.368 INFO:tasks.workunit.client.1.smithi193.stdout:8/630: write d2/d3/d3a/f54 [913516,47215] 0 2024-04-25T15:49:45.369 INFO:tasks.workunit.client.1.smithi193.stdout:8/631: chown d2/d3/d27/d69/dae/d4a/lb6 5630 1 2024-04-25T15:49:45.369 INFO:tasks.workunit.client.1.smithi193.stdout:8/632: write d2/d3/f16 [2937507,65102] 0 2024-04-25T15:49:45.371 INFO:tasks.workunit.client.1.smithi193.stdout:3/676: mkdir d8/d13/d1f/d69/d77/d71/dd2 0 2024-04-25T15:49:45.378 INFO:tasks.workunit.client.0.smithi110.stdout:1/971: fsync d0/d14/d5f/d6f/f6e 0 2024-04-25T15:49:45.380 INFO:tasks.workunit.client.1.smithi193.stdout:4/696: link db/f61 db/d13/d1e/d1f/d56/d69/fe0 0 2024-04-25T15:49:45.382 INFO:tasks.workunit.client.1.smithi193.stdout:0/673: write d0/fda [709972,34816] 0 2024-04-25T15:49:45.383 INFO:tasks.workunit.client.0.smithi110.stdout:0/785: symlink d1/d23/d30/d33/d5b/d78/da4/d10c/l114 0 2024-04-25T15:49:45.386 INFO:tasks.workunit.client.1.smithi193.stdout:1/574: rename dc/d4a/d50/d5c/c69 to dc/d4a/cc3 0 2024-04-25T15:49:45.387 INFO:tasks.workunit.client.0.smithi110.stdout:2/727: symlink de/d40/d6e/d50/le9 0 2024-04-25T15:49:45.389 INFO:tasks.workunit.client.0.smithi110.stdout:7/917: read - d1b/d29/d38/d55/d7f/faf zero size 2024-04-25T15:49:45.392 INFO:tasks.workunit.client.1.smithi193.stdout:9/628: creat df/d2c/d2f/d4e/fc2 x:0 0 0 2024-04-25T15:49:45.392 INFO:tasks.workunit.client.1.smithi193.stdout:9/629: write df/d2c/d34/d3e/fa5 [2872764,114239] 0 2024-04-25T15:49:45.395 INFO:tasks.workunit.client.1.smithi193.stdout:8/633: mknod d2/d18/db1/ccc 0 2024-04-25T15:49:45.401 INFO:tasks.workunit.client.0.smithi110.stdout:8/927: sync 2024-04-25T15:49:45.401 INFO:tasks.workunit.client.0.smithi110.stdout:6/774: sync 2024-04-25T15:49:45.401 INFO:tasks.workunit.client.1.smithi193.stdout:2/650: chown d0/d42/d51/d4f/d65/la1 51 1 2024-04-25T15:49:45.403 INFO:tasks.workunit.client.1.smithi193.stdout:4/697: mkdir db/d13/d1e/d1f/d56/d69/de1 0 2024-04-25T15:49:45.407 INFO:tasks.workunit.client.1.smithi193.stdout:0/674: dread d0/d49/d5/d7/dd/d46/d48/fa2 [0,4194304] 0 2024-04-25T15:49:45.407 INFO:tasks.workunit.client.1.smithi193.stdout:7/628: rename d12/d31/dc3/fc9 to d12/da1/fea 0 2024-04-25T15:49:45.409 INFO:tasks.workunit.client.1.smithi193.stdout:1/575: rmdir dc/d4a/d50/d5c/d5d/d3d/db0 39 2024-04-25T15:49:45.411 INFO:tasks.workunit.client.1.smithi193.stdout:4/698: dwrite db/d32/d88/fbf [0,4194304] 0 2024-04-25T15:49:45.411 INFO:tasks.workunit.client.1.smithi193.stdout:7/629: chown d12/d31/d37/d62/f9b 24831 1 2024-04-25T15:49:45.411 INFO:tasks.workunit.client.0.smithi110.stdout:9/756: write d9/d13/d5b/d61/d67/dd7/f60 [4441217,60457] 0 2024-04-25T15:49:45.413 INFO:tasks.workunit.client.1.smithi193.stdout:0/675: dread - d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/fbc zero size 2024-04-25T15:49:45.413 INFO:tasks.workunit.client.1.smithi193.stdout:6/624: dwrite dd/d12/d19/d1c/f20 [0,4194304] 0 2024-04-25T15:49:45.415 INFO:tasks.workunit.client.0.smithi110.stdout:8/928: mknod d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/c143 0 2024-04-25T15:49:45.424 INFO:tasks.workunit.client.0.smithi110.stdout:2/728: write de/fa2 [143961,83861] 0 2024-04-25T15:49:45.427 INFO:tasks.workunit.client.0.smithi110.stdout:9/757: symlink d9/d13/d5b/d61/d67/dd7/led 0 2024-04-25T15:49:45.430 INFO:tasks.workunit.client.0.smithi110.stdout:0/786: dwrite d1/d47/d54/f74 [0,4194304] 0 2024-04-25T15:49:45.431 INFO:tasks.workunit.client.1.smithi193.stdout:7/630: creat d12/d31/d37/feb x:0 0 0 2024-04-25T15:49:45.432 INFO:tasks.workunit.client.0.smithi110.stdout:0/787: write d1/d47/f61 [807743,16266] 0 2024-04-25T15:49:45.434 INFO:tasks.workunit.client.1.smithi193.stdout:9/630: unlink df/d2c/d2f/d4e/d1c/f9b 0 2024-04-25T15:49:45.438 INFO:tasks.workunit.client.0.smithi110.stdout:8/929: dread d14/d21/f92 [0,4194304] 0 2024-04-25T15:49:45.439 INFO:tasks.workunit.client.1.smithi193.stdout:5/672: sync 2024-04-25T15:49:45.439 INFO:tasks.workunit.client.0.smithi110.stdout:1/972: dwrite d0/d14/d5f/d7d/f67 [4194304,4194304] 0 2024-04-25T15:49:45.440 INFO:tasks.workunit.client.1.smithi193.stdout:4/699: truncate db/f87 1238833 0 2024-04-25T15:49:45.440 INFO:tasks.workunit.client.1.smithi193.stdout:3/677: dwrite d8/d13/d4b/f5d [0,4194304] 0 2024-04-25T15:49:45.440 INFO:tasks.workunit.client.1.smithi193.stdout:4/700: readlink db/d32/d45/l7d 0 2024-04-25T15:49:45.441 INFO:tasks.workunit.client.0.smithi110.stdout:6/775: dread d1/db/d24/d2d/f3a [0,4194304] 0 2024-04-25T15:49:45.441 INFO:tasks.workunit.client.1.smithi193.stdout:3/678: fdatasync d8/d13/d14/f59 0 2024-04-25T15:49:45.442 INFO:tasks.workunit.client.1.smithi193.stdout:4/701: read db/d13/d1e/d1f/d56/d69/d70/d8a/f8c [3896729,99764] 0 2024-04-25T15:49:45.443 INFO:tasks.workunit.client.1.smithi193.stdout:6/625: unlink dd/d12/d19/d1c/f9f 0 2024-04-25T15:49:45.446 INFO:tasks.workunit.client.0.smithi110.stdout:2/729: rename de/d40/d5b/d5c/d5f/db2/fdd to de/d40/d4f/fea 0 2024-04-25T15:49:45.448 INFO:tasks.workunit.client.1.smithi193.stdout:0/676: creat d0/d49/d5/d7/dd/d16/d1a/fe3 x:0 0 0 2024-04-25T15:49:45.449 INFO:tasks.workunit.client.0.smithi110.stdout:0/788: mkdir d1/d23/d30/d33/d5b/d78/ddb/d115 0 2024-04-25T15:49:45.449 INFO:tasks.workunit.client.0.smithi110.stdout:0/789: dread - d1/d23/d9e/ff9 zero size 2024-04-25T15:49:45.450 INFO:tasks.workunit.client.1.smithi193.stdout:2/651: symlink d0/d9/d15/d31/lfb 0 2024-04-25T15:49:45.450 INFO:tasks.workunit.client.1.smithi193.stdout:4/702: dwrite db/f38 [4194304,4194304] 0 2024-04-25T15:49:45.451 INFO:tasks.workunit.client.0.smithi110.stdout:3/830: link d0/d16/dcb/d2f/d4c/d63/lf6 d0/d7/d40/l11a 0 2024-04-25T15:49:45.455 INFO:tasks.workunit.client.0.smithi110.stdout:1/973: readlink d0/d14/d81/df8/l46 0 2024-04-25T15:49:45.456 INFO:tasks.workunit.client.1.smithi193.stdout:7/631: creat d12/d1c/d45/fec x:0 0 0 2024-04-25T15:49:45.457 INFO:tasks.workunit.client.0.smithi110.stdout:8/930: mkdir d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/ddd/d144 0 2024-04-25T15:49:45.458 INFO:tasks.workunit.client.1.smithi193.stdout:9/631: creat df/d2c/d2f/d4e/d1c/fc3 x:0 0 0 2024-04-25T15:49:45.459 INFO:tasks.workunit.client.0.smithi110.stdout:6/776: truncate d1/db/d24/d2d/f59 1652743 0 2024-04-25T15:49:45.461 INFO:tasks.workunit.client.0.smithi110.stdout:6/777: write d1/db/d22/d33/d43/f60 [4108167,69894] 0 2024-04-25T15:49:45.462 INFO:tasks.workunit.client.0.smithi110.stdout:6/778: dread - d1/d79/fe9 zero size 2024-04-25T15:49:45.466 INFO:tasks.workunit.client.0.smithi110.stdout:3/831: symlink d0/d16/dcb/d50/l11b 0 2024-04-25T15:49:45.469 INFO:tasks.workunit.client.0.smithi110.stdout:8/931: truncate d14/d32/d36/f64 154359 0 2024-04-25T15:49:45.472 INFO:tasks.workunit.client.1.smithi193.stdout:2/652: chown d0/d42/d51/df1/d9f/dc5/ff5 158364409 1 2024-04-25T15:49:45.475 INFO:tasks.workunit.client.0.smithi110.stdout:2/730: dread de/d40/d6e/f21 [0,4194304] 0 2024-04-25T15:49:45.476 INFO:tasks.workunit.client.0.smithi110.stdout:6/779: dread d1/dd/d3c/d91/d8d/dce/f78 [0,4194304] 0 2024-04-25T15:49:45.477 INFO:tasks.workunit.client.0.smithi110.stdout:6/780: dread - d1/d79/fe9 zero size 2024-04-25T15:49:45.478 INFO:tasks.workunit.client.0.smithi110.stdout:0/790: rename d1/d23/d30/d33/d5b/d72 to d1/d6c/d116 0 2024-04-25T15:49:45.480 INFO:tasks.workunit.client.0.smithi110.stdout:0/791: write d1/d23/d30/d33/d5b/d78/ddb/ff8 [436231,128610] 0 2024-04-25T15:49:45.480 INFO:tasks.workunit.client.0.smithi110.stdout:3/832: mknod d0/d16/d8c/c11c 0 2024-04-25T15:49:45.482 INFO:tasks.workunit.client.1.smithi193.stdout:0/677: dread d0/d49/d5/d7/dd/d46/d48/d62/f92 [0,4194304] 0 2024-04-25T15:49:45.483 INFO:tasks.workunit.client.1.smithi193.stdout:0/678: dread - d0/d49/d5/d7/dd/d16/d26/d2e/d93/fb9 zero size 2024-04-25T15:49:45.487 INFO:tasks.workunit.client.1.smithi193.stdout:3/679: unlink d8/d30/d72/f9c 0 2024-04-25T15:49:45.488 INFO:tasks.workunit.client.1.smithi193.stdout:1/576: dwrite dc/d4a/d50/d7c/d7e/f88 [0,4194304] 0 2024-04-25T15:49:45.489 INFO:tasks.workunit.client.1.smithi193.stdout:3/680: write d8/d13/d4b/f6e [314844,105496] 0 2024-04-25T15:49:45.498 INFO:tasks.workunit.client.1.smithi193.stdout:2/653: mknod d0/d37/dbb/df8/cfc 0 2024-04-25T15:49:45.500 INFO:tasks.workunit.client.0.smithi110.stdout:3/833: mknod d0/d2e/d88/dbf/d92/c11d 0 2024-04-25T15:49:45.501 INFO:tasks.workunit.client.1.smithi193.stdout:8/634: truncate d2/f11 3977567 0 2024-04-25T15:49:45.503 INFO:tasks.workunit.client.1.smithi193.stdout:7/632: unlink d12/d31/d39/c55 0 2024-04-25T15:49:45.504 INFO:tasks.workunit.client.1.smithi193.stdout:7/633: fsync d12/d31/d37/d62/d6e/da5/dd7/fe5 0 2024-04-25T15:49:45.505 INFO:tasks.workunit.client.1.smithi193.stdout:0/679: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/de4 0 2024-04-25T15:49:45.506 INFO:tasks.workunit.client.0.smithi110.stdout:1/974: link d0/d51/d6b/df1/d13b/f7e d0/d14/d5f/d7d/d9c/dcc/f13d 0 2024-04-25T15:49:45.508 INFO:tasks.workunit.client.1.smithi193.stdout:5/673: sync 2024-04-25T15:49:45.511 INFO:tasks.workunit.client.0.smithi110.stdout:9/758: getdents d9/d13/d84 0 2024-04-25T15:49:45.511 INFO:tasks.workunit.client.1.smithi193.stdout:0/680: dwrite d0/d49/d5/d7/dd/d46/d48/f85 [4194304,4194304] 0 2024-04-25T15:49:45.512 INFO:tasks.workunit.client.0.smithi110.stdout:9/759: stat d9/d13/d5b/ld5 0 2024-04-25T15:49:45.519 INFO:tasks.workunit.client.1.smithi193.stdout:2/654: dread d0/d9/d15/f1b [0,4194304] 0 2024-04-25T15:49:45.520 INFO:tasks.workunit.client.0.smithi110.stdout:2/731: dwrite de/d40/d5b/d5c/d5f/db2/f9c [0,4194304] 0 2024-04-25T15:49:45.520 INFO:tasks.workunit.client.1.smithi193.stdout:6/626: dwrite dd/d12/d19/d1c/d40/d3c/fa7 [0,4194304] 0 2024-04-25T15:49:45.522 INFO:tasks.workunit.client.0.smithi110.stdout:6/781: dwrite d1/d12/f1f [4194304,4194304] 0 2024-04-25T15:49:45.523 INFO:tasks.workunit.client.1.smithi193.stdout:2/655: dread d0/d42/d51/df1/d9f/dc5/fe7 [0,4194304] 0 2024-04-25T15:49:45.525 INFO:tasks.workunit.client.1.smithi193.stdout:7/634: write d12/d31/d37/d92/dd1/fd5 [1043486,59246] 0 2024-04-25T15:49:45.528 INFO:tasks.workunit.client.0.smithi110.stdout:8/932: write d14/f80 [1926351,118111] 0 2024-04-25T15:49:45.530 INFO:tasks.workunit.client.0.smithi110.stdout:9/760: creat d9/d13/d5b/d61/db3/dd3/fee x:0 0 0 2024-04-25T15:49:45.531 INFO:tasks.workunit.client.0.smithi110.stdout:0/792: dwrite d1/d23/d30/d33/d5b/d83/f53 [0,4194304] 0 2024-04-25T15:49:45.535 INFO:tasks.workunit.client.1.smithi193.stdout:7/635: dwrite d12/d1c/f28 [4194304,4194304] 0 2024-04-25T15:49:45.536 INFO:tasks.workunit.client.0.smithi110.stdout:3/834: symlink d0/d16/dcb/d2f/d4c/d108/l11e 0 2024-04-25T15:49:45.537 INFO:tasks.workunit.client.1.smithi193.stdout:5/674: mknod d17/d18/d75/ce2 0 2024-04-25T15:49:45.540 INFO:tasks.workunit.client.0.smithi110.stdout:9/761: dwrite d9/db/df/f21 [4194304,4194304] 0 2024-04-25T15:49:45.541 INFO:tasks.workunit.client.1.smithi193.stdout:0/681: unlink d0/d49/d5/d7/fd8 0 2024-04-25T15:49:45.542 INFO:tasks.workunit.client.0.smithi110.stdout:1/975: creat d0/d14/d81/df8/d33/d8e/d109/d131/f13e x:0 0 0 2024-04-25T15:49:45.542 INFO:tasks.workunit.client.1.smithi193.stdout:1/577: mknod dc/d4a/d50/d5c/d5d/cc4 0 2024-04-25T15:49:45.543 INFO:tasks.workunit.client.1.smithi193.stdout:4/703: link db/d32/d45/l7d db/d4b/le2 0 2024-04-25T15:49:45.545 INFO:tasks.workunit.client.1.smithi193.stdout:4/704: write db/f3b [45545,107098] 0 2024-04-25T15:49:45.546 INFO:tasks.workunit.client.0.smithi110.stdout:6/782: creat d1/d79/f113 x:0 0 0 2024-04-25T15:49:45.548 INFO:tasks.workunit.client.1.smithi193.stdout:6/627: readlink dd/d12/l27 0 2024-04-25T15:49:45.552 INFO:tasks.workunit.client.0.smithi110.stdout:0/793: rename d1/d23/d30/d33/d5b/d78/da4/d10c/l114 to d1/d6c/d7f/l117 0 2024-04-25T15:49:45.554 INFO:tasks.workunit.client.0.smithi110.stdout:3/835: creat d0/d2e/dfc/f11f x:0 0 0 2024-04-25T15:49:45.554 INFO:tasks.workunit.client.1.smithi193.stdout:9/632: link df/d2c/d2f/d4e/d1c/d30/d60/f73 df/d2c/d2f/fc4 0 2024-04-25T15:49:45.556 INFO:tasks.workunit.client.0.smithi110.stdout:1/976: mknod d0/d14/d81/df8/d33/da9/c13f 0 2024-04-25T15:49:45.557 INFO:tasks.workunit.client.0.smithi110.stdout:2/732: truncate de/d40/d6e/d52/fc8 2649787 0 2024-04-25T15:49:45.557 INFO:tasks.workunit.client.0.smithi110.stdout:1/977: rename d0/d14/d81/df8/d33 to d0/d14/d81/df8/d33/d94/d140 22 2024-04-25T15:49:45.558 INFO:tasks.workunit.client.1.smithi193.stdout:3/681: creat d8/d13/d14/fd3 x:0 0 0 2024-04-25T15:49:45.563 INFO:tasks.workunit.client.0.smithi110.stdout:0/794: fsync d1/d20/f73 0 2024-04-25T15:49:45.563 INFO:tasks.workunit.client.1.smithi193.stdout:0/682: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/ce5 0 2024-04-25T15:49:45.565 INFO:tasks.workunit.client.0.smithi110.stdout:3/836: chown d0/d7/d40/c49 12303560 1 2024-04-25T15:49:45.565 INFO:tasks.workunit.client.1.smithi193.stdout:1/578: mknod dc/d4a/d50/cc5 0 2024-04-25T15:49:45.566 INFO:tasks.workunit.client.0.smithi110.stdout:2/733: dwrite de/d40/d4f/d9f/fe0 [0,4194304] 0 2024-04-25T15:49:45.566 INFO:tasks.workunit.client.0.smithi110.stdout:1/978: dwrite d0/d23/f5a [4194304,4194304] 0 2024-04-25T15:49:45.567 INFO:tasks.workunit.client.0.smithi110.stdout:9/762: mknod d9/d44/d5e/d80/dbb/cef 0 2024-04-25T15:49:45.570 INFO:tasks.workunit.client.1.smithi193.stdout:1/579: dwrite dc/d4a/d4d/d7a/d85/fb6 [0,4194304] 0 2024-04-25T15:49:45.570 INFO:tasks.workunit.client.1.smithi193.stdout:4/705: rename db/d13/d1e/d1f/fcf to db/d13/d1e/d1f/d56/fe3 0 2024-04-25T15:49:45.571 INFO:tasks.workunit.client.1.smithi193.stdout:6/628: rmdir dd/d2f 39 2024-04-25T15:49:45.572 INFO:tasks.workunit.client.1.smithi193.stdout:6/629: stat dd/d12/d2d/la2 0 2024-04-25T15:49:45.572 INFO:tasks.workunit.client.1.smithi193.stdout:1/580: write dc/d4a/d50/d7c/d7e/f88 [3642367,122349] 0 2024-04-25T15:49:45.574 INFO:tasks.workunit.client.0.smithi110.stdout:0/795: symlink d1/d6c/d7f/l118 0 2024-04-25T15:49:45.578 INFO:tasks.workunit.client.0.smithi110.stdout:3/837: rename d0/d16/l82 to d0/d7/dd0/d10b/l120 0 2024-04-25T15:49:45.582 INFO:tasks.workunit.client.1.smithi193.stdout:7/636: link d12/d1c/d45/f7a d12/d31/d37/d62/d6e/d79/d9d/fed 0 2024-04-25T15:49:45.587 INFO:tasks.workunit.client.1.smithi193.stdout:0/683: creat d0/d49/d5/d7/dd/d16/d18/d28/fe6 x:0 0 0 2024-04-25T15:49:45.591 INFO:tasks.workunit.client.1.smithi193.stdout:8/635: getdents d2/d3/d20 0 2024-04-25T15:49:45.594 INFO:tasks.workunit.client.0.smithi110.stdout:9/763: unlink d9/d13/f54 0 2024-04-25T15:49:45.596 INFO:tasks.workunit.client.1.smithi193.stdout:4/706: chown db/d13/d1e/d1f/d56/d69/fe0 120 1 2024-04-25T15:49:45.596 INFO:tasks.workunit.client.1.smithi193.stdout:6/630: dread dd/d12/d19/d1c/f99 [0,4194304] 0 2024-04-25T15:49:45.597 INFO:tasks.workunit.client.0.smithi110.stdout:2/734: creat de/d40/d6e/feb x:0 0 0 2024-04-25T15:49:45.597 INFO:tasks.workunit.client.1.smithi193.stdout:6/631: stat dd/d12/d19/d1c/d40/d3c 0 2024-04-25T15:49:45.598 INFO:tasks.workunit.client.1.smithi193.stdout:6/632: fdatasync dd/d12/d19/d1c/f71 0 2024-04-25T15:49:45.598 INFO:tasks.workunit.client.0.smithi110.stdout:0/796: getdents d1/d23/d30/d33/d5b/dac 0 2024-04-25T15:49:45.599 INFO:tasks.workunit.client.0.smithi110.stdout:0/797: write d1/d20/f96 [2738047,7511] 0 2024-04-25T15:49:45.600 INFO:tasks.workunit.client.0.smithi110.stdout:0/798: write d1/d47/f61 [1014671,114224] 0 2024-04-25T15:49:45.603 INFO:tasks.workunit.client.1.smithi193.stdout:2/656: creat d0/d42/d51/d4f/d65/ffd x:0 0 0 2024-04-25T15:49:45.605 INFO:tasks.workunit.client.0.smithi110.stdout:2/735: mknod de/d40/d86/cec 0 2024-04-25T15:49:45.606 INFO:tasks.workunit.client.1.smithi193.stdout:1/581: dread dc/d3c/f79 [0,4194304] 0 2024-04-25T15:49:45.609 INFO:tasks.workunit.client.0.smithi110.stdout:8/933: write d14/d21/d26/d6a/d7e/da6/de8/d55/f7d [188932,38926] 0 2024-04-25T15:49:45.610 INFO:tasks.workunit.client.1.smithi193.stdout:2/657: dwrite d0/df/dbf/fd5 [0,4194304] 0 2024-04-25T15:49:45.614 INFO:tasks.workunit.client.0.smithi110.stdout:3/838: rename d0/d2e/d88/c10d to d0/c121 0 2024-04-25T15:49:45.614 INFO:tasks.workunit.client.0.smithi110.stdout:6/783: dwrite d1/db/d22/fd2 [0,4194304] 0 2024-04-25T15:49:45.615 INFO:tasks.workunit.client.0.smithi110.stdout:6/784: dread - d1/db/d24/fe8 zero size 2024-04-25T15:49:45.616 INFO:tasks.workunit.client.0.smithi110.stdout:6/785: fsync d1/d12/db8/fe1 0 2024-04-25T15:49:45.618 INFO:tasks.workunit.client.0.smithi110.stdout:6/786: stat d1/db/d22/d90/d76/dad 0 2024-04-25T15:49:45.622 INFO:tasks.workunit.client.0.smithi110.stdout:2/736: fdatasync de/d40/d6e/d52/d6d/fb3 0 2024-04-25T15:49:45.623 INFO:tasks.workunit.client.0.smithi110.stdout:2/737: readlink de/d40/d6e/d50/le9 0 2024-04-25T15:49:45.624 INFO:tasks.workunit.client.0.smithi110.stdout:2/738: chown de/d40/d6e/d50/l9e 107655 1 2024-04-25T15:49:45.628 INFO:tasks.workunit.client.1.smithi193.stdout:9/633: dwrite df/d2c/d2f/d4e/d1c/d30/f7a [0,4194304] 0 2024-04-25T15:49:45.629 INFO:tasks.workunit.client.0.smithi110.stdout:8/934: dread d14/d21/f2a [0,4194304] 0 2024-04-25T15:49:45.630 INFO:tasks.workunit.client.0.smithi110.stdout:1/979: write d0/d14/d81/df8/d33/d94/ddb/ff6 [862244,16914] 0 2024-04-25T15:49:45.633 INFO:tasks.workunit.client.1.smithi193.stdout:7/637: rename d12/f30 to d12/d31/d37/d92/fee 0 2024-04-25T15:49:45.634 INFO:tasks.workunit.client.1.smithi193.stdout:7/638: chown d12/d1c/da3/cad 100 1 2024-04-25T15:49:45.635 INFO:tasks.workunit.client.0.smithi110.stdout:0/799: symlink d1/d47/d113/l119 0 2024-04-25T15:49:45.636 INFO:tasks.workunit.client.0.smithi110.stdout:0/800: dread - d1/d23/d30/d33/d5b/dbb/da8/ffa zero size 2024-04-25T15:49:45.636 INFO:tasks.workunit.client.0.smithi110.stdout:6/787: truncate d1/db/f8c 1679493 0 2024-04-25T15:49:45.637 INFO:tasks.workunit.client.1.smithi193.stdout:2/658: fsync d0/d9/d15/d31/fd9 0 2024-04-25T15:49:45.637 INFO:tasks.workunit.client.1.smithi193.stdout:2/659: chown d0/d9/d15 83482308 1 2024-04-25T15:49:45.639 INFO:tasks.workunit.client.0.smithi110.stdout:9/764: write d9/d44/d5e/d80/f8a [1655829,10375] 0 2024-04-25T15:49:45.644 INFO:tasks.workunit.client.1.smithi193.stdout:8/636: write d2/d3/d27/d69/dae/f88 [5069251,130359] 0 2024-04-25T15:49:45.644 INFO:tasks.workunit.client.0.smithi110.stdout:8/935: truncate d14/d32/fe4 57975 0 2024-04-25T15:49:45.644 INFO:tasks.workunit.client.1.smithi193.stdout:4/707: write db/d13/d1e/f6d [8092875,90652] 0 2024-04-25T15:49:45.645 INFO:tasks.workunit.client.1.smithi193.stdout:8/637: fsync d2/d3/d27/d6f/d94/f90 0 2024-04-25T15:49:45.646 INFO:tasks.workunit.client.1.smithi193.stdout:8/638: stat d2/d3/d27/d69/ca1 0 2024-04-25T15:49:45.650 INFO:tasks.workunit.client.0.smithi110.stdout:4/812: sync 2024-04-25T15:49:45.652 INFO:tasks.workunit.client.0.smithi110.stdout:1/980: rename d0/d51/dd8/fef to d0/d14/d5f/d6f/f141 0 2024-04-25T15:49:45.652 INFO:tasks.workunit.client.0.smithi110.stdout:4/813: chown d13/d24/d2d/d44/d65/d99/daf/d56/d88/f8f 0 1 2024-04-25T15:49:45.654 INFO:tasks.workunit.client.1.smithi193.stdout:1/582: rename dc/d59/cb1 to dc/d4a/d50/d5c/d5d/d3d/d55/d8a/db7/cc6 0 2024-04-25T15:49:45.655 INFO:tasks.workunit.client.0.smithi110.stdout:2/739: dread de/f56 [4194304,4194304] 0 2024-04-25T15:49:45.656 INFO:tasks.workunit.client.1.smithi193.stdout:7/639: creat d12/d1c/d63/fef x:0 0 0 2024-04-25T15:49:45.656 INFO:tasks.workunit.client.1.smithi193.stdout:7/640: stat d12/l4b 0 2024-04-25T15:49:45.658 INFO:tasks.workunit.client.1.smithi193.stdout:3/682: getdents d8/d13/d4b 0 2024-04-25T15:49:45.660 INFO:tasks.workunit.client.0.smithi110.stdout:9/765: truncate d9/d13/fd2 1012920 0 2024-04-25T15:49:45.665 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:45 smithi110 ceph-mon[26344]: pgmap v39: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 103 MiB/s rd, 306 MiB/s wr, 582 op/s 2024-04-25T15:49:45.665 INFO:tasks.workunit.client.1.smithi193.stdout:5/675: mkdir d17/d18/d75/d85/d5d/dad/dd8/de3 0 2024-04-25T15:49:45.665 INFO:tasks.workunit.client.1.smithi193.stdout:2/660: read d0/d37/d25/d5f/d3a/d6d/fcd [636307,34154] 0 2024-04-25T15:49:45.666 INFO:tasks.workunit.client.1.smithi193.stdout:6/633: creat dd/d12/d19/d1c/d40/d3c/d3f/fc3 x:0 0 0 2024-04-25T15:49:45.670 INFO:tasks.workunit.client.0.smithi110.stdout:1/981: truncate d0/d14/d81/df8/d33/f50 741303 0 2024-04-25T15:49:45.671 INFO:tasks.workunit.client.0.smithi110.stdout:3/839: dwrite d0/d16/dcb/d2f/d4c/f112 [0,4194304] 0 2024-04-25T15:49:45.674 INFO:tasks.workunit.client.1.smithi193.stdout:0/684: sync 2024-04-25T15:49:45.675 INFO:tasks.workunit.client.0.smithi110.stdout:2/740: mkdir de/d40/d4f/ded 0 2024-04-25T15:49:45.677 INFO:tasks.workunit.client.1.smithi193.stdout:4/708: creat db/d32/d40/fe4 x:0 0 0 2024-04-25T15:49:45.678 INFO:tasks.workunit.client.1.smithi193.stdout:4/709: write db/d13/d1e/d1f/d56/d57/fa4 [2250292,47556] 0 2024-04-25T15:49:45.681 INFO:tasks.workunit.client.0.smithi110.stdout:2/741: read de/d40/f96 [116849,48930] 0 2024-04-25T15:49:45.682 INFO:tasks.workunit.client.0.smithi110.stdout:1/982: dwrite d0/d51/dd8/df0/f11f [0,4194304] 0 2024-04-25T15:49:45.683 INFO:tasks.workunit.client.1.smithi193.stdout:6/634: dread dd/d12/d19/d1c/d40/d3c/d3f/d4f/f60 [0,4194304] 0 2024-04-25T15:49:45.683 INFO:tasks.workunit.client.1.smithi193.stdout:4/710: dwrite db/d4b/f71 [0,4194304] 0 2024-04-25T15:49:45.685 INFO:tasks.workunit.client.1.smithi193.stdout:1/583: symlink dc/d4a/d50/d7c/d82/lc7 0 2024-04-25T15:49:45.686 INFO:tasks.workunit.client.0.smithi110.stdout:1/983: readlink d0/d14/d5f/d7d/d9c/lcf 0 2024-04-25T15:49:45.686 INFO:tasks.workunit.client.1.smithi193.stdout:9/634: truncate df/d2c/d2f/d4e/f19 3467219 0 2024-04-25T15:49:45.686 INFO:tasks.workunit.client.0.smithi110.stdout:3/840: dwrite d0/d16/dcb/db0/f105 [0,4194304] 0 2024-04-25T15:49:45.689 INFO:tasks.workunit.client.0.smithi110.stdout:3/841: stat d0/c86 0 2024-04-25T15:49:45.692 INFO:tasks.workunit.client.1.smithi193.stdout:6/635: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/fa8 [0,4194304] 0 2024-04-25T15:49:45.695 INFO:tasks.workunit.client.1.smithi193.stdout:6/636: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/f60 [4865091,4338] 0 2024-04-25T15:49:45.696 INFO:tasks.workunit.client.0.smithi110.stdout:0/801: link d1/d23/d30/d33/d5b/d83/de1/lea d1/d23/d81/df4/l11a 0 2024-04-25T15:49:45.696 INFO:tasks.workunit.client.0.smithi110.stdout:4/814: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/d88/f41 [4194304,4194304] 0 2024-04-25T15:49:45.697 INFO:tasks.workunit.client.1.smithi193.stdout:5/676: dread - d17/f89 zero size 2024-04-25T15:49:45.699 INFO:tasks.workunit.client.1.smithi193.stdout:5/677: stat d17/d18/d30/d3b/c7d 0 2024-04-25T15:49:45.701 INFO:tasks.workunit.client.1.smithi193.stdout:2/661: rmdir d0/d37/d25 39 2024-04-25T15:49:45.704 INFO:tasks.workunit.client.0.smithi110.stdout:9/766: rmdir d9/db/d7d 39 2024-04-25T15:49:45.706 INFO:tasks.workunit.client.1.smithi193.stdout:0/685: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/fe7 x:0 0 0 2024-04-25T15:49:45.706 INFO:tasks.workunit.client.1.smithi193.stdout:6/637: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/f8a [0,4194304] 0 2024-04-25T15:49:45.710 INFO:tasks.workunit.client.1.smithi193.stdout:7/641: dread d12/d31/d37/d62/d6e/f8c [0,4194304] 0 2024-04-25T15:49:45.710 INFO:tasks.workunit.client.1.smithi193.stdout:8/639: symlink d2/d18/dbc/lcd 0 2024-04-25T15:49:45.711 INFO:tasks.workunit.client.1.smithi193.stdout:7/642: dread - d12/d1c/d45/fec zero size 2024-04-25T15:49:45.712 INFO:tasks.workunit.client.0.smithi110.stdout:2/742: creat de/d40/d6e/d52/fee x:0 0 0 2024-04-25T15:49:45.712 INFO:tasks.workunit.client.1.smithi193.stdout:7/643: write d12/d13/dc4/fdb [58730,11157] 0 2024-04-25T15:49:45.716 INFO:tasks.workunit.client.1.smithi193.stdout:0/686: read d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f3f [5759,118326] 0 2024-04-25T15:49:45.719 INFO:tasks.workunit.client.1.smithi193.stdout:1/584: creat dc/d4a/d50/d5c/d5d/d3d/d55/d8a/fc8 x:0 0 0 2024-04-25T15:49:45.720 INFO:tasks.workunit.client.0.smithi110.stdout:0/802: read d1/d6c/f94 [172171,1622] 0 2024-04-25T15:49:45.722 INFO:tasks.workunit.client.1.smithi193.stdout:1/585: read dc/d4a/d50/d5c/d5d/d3d/f95 [349625,53694] 0 2024-04-25T15:49:45.724 INFO:tasks.workunit.client.1.smithi193.stdout:7/644: dread d12/d31/d39/db0/fb5 [0,4194304] 0 2024-04-25T15:49:45.724 INFO:tasks.workunit.client.0.smithi110.stdout:4/815: truncate d13/d24/d2d/d44/d90/fc1 4365202 0 2024-04-25T15:49:45.726 INFO:tasks.workunit.client.1.smithi193.stdout:5/678: creat d17/d18/d30/d38/dab/fe4 x:0 0 0 2024-04-25T15:49:45.727 INFO:tasks.workunit.client.1.smithi193.stdout:5/679: write d17/d18/d30/d34/f6c [2545731,48427] 0 2024-04-25T15:49:45.732 INFO:tasks.workunit.client.0.smithi110.stdout:0/803: write d1/f38 [1501187,33441] 0 2024-04-25T15:49:45.732 INFO:tasks.workunit.client.1.smithi193.stdout:8/640: dread d2/d3/d27/d6f/d94/f90 [0,4194304] 0 2024-04-25T15:49:45.733 INFO:tasks.workunit.client.1.smithi193.stdout:3/683: link d8/d13/d14/d27/l7a d8/d30/d45/dd0/ld4 0 2024-04-25T15:49:45.735 INFO:tasks.workunit.client.1.smithi193.stdout:1/586: rmdir dc/d4a/d4d/d7a/d85 39 2024-04-25T15:49:45.737 INFO:tasks.workunit.client.1.smithi193.stdout:7/645: symlink d12/d31/d37/d62/d6e/lf0 0 2024-04-25T15:49:45.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:45 smithi193 ceph-mon[33747]: pgmap v39: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 103 MiB/s rd, 306 MiB/s wr, 582 op/s 2024-04-25T15:49:45.739 INFO:tasks.workunit.client.0.smithi110.stdout:0/804: mknod d1/d23/d30/d33/d35/da1/c11b 0 2024-04-25T15:49:45.739 INFO:tasks.workunit.client.0.smithi110.stdout:8/936: write d14/d32/d36/d48/fb3 [1497391,48502] 0 2024-04-25T15:49:45.742 INFO:tasks.workunit.client.0.smithi110.stdout:1/984: write d0/d23/f83 [559425,79325] 0 2024-04-25T15:49:45.745 INFO:tasks.workunit.client.1.smithi193.stdout:8/641: creat d2/d18/dbc/fce x:0 0 0 2024-04-25T15:49:45.746 INFO:tasks.workunit.client.1.smithi193.stdout:4/711: dwrite f2 [0,4194304] 0 2024-04-25T15:49:45.746 INFO:tasks.workunit.client.0.smithi110.stdout:8/937: symlink d14/d102/l145 0 2024-04-25T15:49:45.747 INFO:tasks.workunit.client.1.smithi193.stdout:4/712: stat db/d13/d1e/f6d 0 2024-04-25T15:49:45.748 INFO:tasks.workunit.client.0.smithi110.stdout:3/842: dwrite d0/d16/dcb/fc8 [0,4194304] 0 2024-04-25T15:49:45.751 INFO:tasks.workunit.client.1.smithi193.stdout:6/638: dread dd/d2f/f56 [4194304,4194304] 0 2024-04-25T15:49:45.753 INFO:tasks.workunit.client.1.smithi193.stdout:9/635: dwrite df/d2c/d2f/f77 [0,4194304] 0 2024-04-25T15:49:45.755 INFO:tasks.workunit.client.0.smithi110.stdout:8/938: dwrite d14/fb2 [0,4194304] 0 2024-04-25T15:49:45.756 INFO:tasks.workunit.client.1.smithi193.stdout:8/642: dread d2/d3/d27/d69/dae/f5a [0,4194304] 0 2024-04-25T15:49:45.760 INFO:tasks.workunit.client.0.smithi110.stdout:1/985: creat d0/d51/dc8/d12d/f142 x:0 0 0 2024-04-25T15:49:45.762 INFO:tasks.workunit.client.1.smithi193.stdout:5/680: symlink d17/le5 0 2024-04-25T15:49:45.766 INFO:tasks.workunit.client.0.smithi110.stdout:9/767: dwrite d9/d2d/f30 [0,4194304] 0 2024-04-25T15:49:45.766 INFO:tasks.workunit.client.0.smithi110.stdout:3/843: unlink d0/d16/f1f 0 2024-04-25T15:49:45.767 INFO:tasks.workunit.client.1.smithi193.stdout:0/687: creat d0/d49/d5/d7/fe8 x:0 0 0 2024-04-25T15:49:45.768 INFO:tasks.workunit.client.0.smithi110.stdout:2/743: write de/d40/d5b/d5c/d5f/f81 [40600,117828] 0 2024-04-25T15:49:45.769 INFO:tasks.workunit.client.1.smithi193.stdout:3/684: creat d8/d13/d1f/d69/d77/dd1/fd5 x:0 0 0 2024-04-25T15:49:45.769 INFO:tasks.workunit.client.0.smithi110.stdout:0/805: link d1/d6c/fd5 d1/d47/d113/f11c 0 2024-04-25T15:49:45.773 INFO:tasks.workunit.client.1.smithi193.stdout:4/713: creat db/d13/d1e/d1f/d56/d69/fe5 x:0 0 0 2024-04-25T15:49:45.773 INFO:tasks.workunit.client.0.smithi110.stdout:1/986: rmdir d0/d14/d81/df8 39 2024-04-25T15:49:45.775 INFO:tasks.workunit.client.0.smithi110.stdout:9/768: fdatasync d9/f24 0 2024-04-25T15:49:45.778 INFO:tasks.workunit.client.1.smithi193.stdout:1/587: mkdir dc/dc9 0 2024-04-25T15:49:45.778 INFO:tasks.workunit.client.1.smithi193.stdout:7/646: creat d12/d1c/dd6/dda/ff1 x:0 0 0 2024-04-25T15:49:45.779 INFO:tasks.workunit.client.0.smithi110.stdout:2/744: dread - de/d40/d5b/d5c/d5f/f73 zero size 2024-04-25T15:49:45.779 INFO:tasks.workunit.client.1.smithi193.stdout:1/588: write dc/f4b [1120276,70903] 0 2024-04-25T15:49:45.780 INFO:tasks.workunit.client.1.smithi193.stdout:1/589: stat dc/d4a/d50/d5c/d5d/d3d/d55/d9c 0 2024-04-25T15:49:45.783 INFO:tasks.workunit.client.0.smithi110.stdout:0/806: rmdir d1/d23/d30/d33/d5b/dbb/da8 39 2024-04-25T15:49:45.786 INFO:tasks.workunit.client.1.smithi193.stdout:8/643: symlink d2/d3/d27/d69/dae/d4a/d53/lcf 0 2024-04-25T15:49:45.786 INFO:tasks.workunit.client.1.smithi193.stdout:2/662: link d0/d9/d15/d31/l2f d0/lfe 0 2024-04-25T15:49:45.787 INFO:tasks.workunit.client.1.smithi193.stdout:5/681: creat d17/d18/d30/d38/d71/dd7/fe6 x:0 0 0 2024-04-25T15:49:45.788 INFO:tasks.workunit.client.0.smithi110.stdout:9/769: symlink d9/db/df/d2c/d3c/d56/da0/lf0 0 2024-04-25T15:49:45.789 INFO:tasks.workunit.client.1.smithi193.stdout:0/688: creat d0/d49/d5/d7/dd/d46/d48/d62/dd9/fe9 x:0 0 0 2024-04-25T15:49:45.792 INFO:tasks.workunit.client.0.smithi110.stdout:2/745: creat de/d40/d6e/d52/d6d/d99/fef x:0 0 0 2024-04-25T15:49:45.793 INFO:tasks.workunit.client.0.smithi110.stdout:0/807: symlink d1/d23/d30/d33/d5b/d83/l11d 0 2024-04-25T15:49:45.794 INFO:tasks.workunit.client.1.smithi193.stdout:6/639: dread dd/d12/d19/f6a [0,4194304] 0 2024-04-25T15:49:45.795 INFO:tasks.workunit.client.0.smithi110.stdout:1/987: dread d0/d51/d6b/df1/d13b/d88/db6/fd4 [0,4194304] 0 2024-04-25T15:49:45.797 INFO:tasks.workunit.client.1.smithi193.stdout:7/647: mkdir d12/d31/d37/d62/d6e/df2 0 2024-04-25T15:49:45.798 INFO:tasks.workunit.client.0.smithi110.stdout:0/808: fsync d1/d47/f8e 0 2024-04-25T15:49:45.801 INFO:tasks.workunit.client.0.smithi110.stdout:9/770: rename d9/db/df/d2c/d3c/f89 to d9/db/df/d2c/d3c/d56/ff1 0 2024-04-25T15:49:45.802 INFO:tasks.workunit.client.1.smithi193.stdout:9/636: truncate df/d2c/d2f/d4e/d1c/d30/d60/f73 630497 0 2024-04-25T15:49:45.803 INFO:tasks.workunit.client.1.smithi193.stdout:9/637: dread - df/d2c/d2f/d5c/fb4 zero size 2024-04-25T15:49:45.811 INFO:tasks.workunit.client.0.smithi110.stdout:3/844: write d0/d16/dcb/db0/fef [160217,118504] 0 2024-04-25T15:49:45.811 INFO:tasks.workunit.client.0.smithi110.stdout:8/939: dwrite d14/f83 [4194304,4194304] 0 2024-04-25T15:49:45.812 INFO:tasks.workunit.client.0.smithi110.stdout:8/940: readlink d14/d32/d36/l68 0 2024-04-25T15:49:45.813 INFO:tasks.workunit.client.0.smithi110.stdout:3/845: write d0/d16/d8c/fea [4496686,89172] 0 2024-04-25T15:49:45.814 INFO:tasks.workunit.client.0.smithi110.stdout:3/846: chown d0/d7/d40/f43 293 1 2024-04-25T15:49:45.815 INFO:tasks.workunit.client.0.smithi110.stdout:2/746: getdents de/d40/d4f 0 2024-04-25T15:49:45.828 INFO:tasks.workunit.client.0.smithi110.stdout:8/941: mkdir d14/d21/d26/d6a/d7e/da6/de8/d146 0 2024-04-25T15:49:45.834 INFO:tasks.workunit.client.0.smithi110.stdout:1/988: dwrite d0/d14/f1d [0,4194304] 0 2024-04-25T15:49:45.837 INFO:tasks.workunit.client.0.smithi110.stdout:9/771: dwrite d9/db/df/f4f [0,4194304] 0 2024-04-25T15:49:45.842 INFO:tasks.workunit.client.0.smithi110.stdout:8/942: link d14/d21/d26/dab/dac/ff4 d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/f147 0 2024-04-25T15:49:45.843 INFO:tasks.workunit.client.0.smithi110.stdout:9/772: rmdir d9/d13/d5b/d61/db3 39 2024-04-25T15:49:45.845 INFO:tasks.workunit.client.0.smithi110.stdout:1/989: truncate d0/d14/d81/df8/d33/d8e/ffa 1611347 0 2024-04-25T15:49:45.848 INFO:tasks.workunit.client.0.smithi110.stdout:8/943: rmdir d14/d21/d26/dc0 39 2024-04-25T15:49:45.850 INFO:tasks.workunit.client.0.smithi110.stdout:3/847: dwrite d0/d16/dcb/d2f/d4c/db2/fe4 [0,4194304] 0 2024-04-25T15:49:45.852 INFO:tasks.workunit.client.0.smithi110.stdout:9/773: creat d9/d13/d5b/ff2 x:0 0 0 2024-04-25T15:49:45.852 INFO:tasks.workunit.client.0.smithi110.stdout:2/747: dwrite de/d40/d6e/d52/f62 [0,4194304] 0 2024-04-25T15:49:45.864 INFO:tasks.workunit.client.0.smithi110.stdout:9/774: mkdir d9/db/df/df3 0 2024-04-25T15:49:45.865 INFO:tasks.workunit.client.0.smithi110.stdout:2/748: mkdir de/d40/d5b/d5c/dbe/df0 0 2024-04-25T15:49:45.867 INFO:tasks.workunit.client.1.smithi193.stdout:2/663: write d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4/feb [655968,18924] 0 2024-04-25T15:49:45.868 INFO:tasks.workunit.client.0.smithi110.stdout:3/848: link d0/d16/dcb/d2f/d33/f44 d0/d16/f122 0 2024-04-25T15:49:45.869 INFO:tasks.workunit.client.1.smithi193.stdout:8/644: chown d2/d3/d27/d69/dae/d4a/d63/l9f 200937 1 2024-04-25T15:49:45.871 INFO:tasks.workunit.client.0.smithi110.stdout:8/944: link d14/d32/d105/d110/f13b d14/d32/d36/db7/dc9/df9/f148 0 2024-04-25T15:49:45.874 INFO:tasks.workunit.client.1.smithi193.stdout:3/685: creat d8/d30/d72/db2/fd6 x:0 0 0 2024-04-25T15:49:45.875 INFO:tasks.workunit.client.1.smithi193.stdout:3/686: dread - d8/d13/d4b/d8b/dca/fcb zero size 2024-04-25T15:49:45.877 INFO:tasks.workunit.client.0.smithi110.stdout:2/749: rename de/d40/d6e/f42 to de/d40/d5b/d5c/d5f/db2/dad/ff1 0 2024-04-25T15:49:45.878 INFO:tasks.workunit.client.0.smithi110.stdout:2/750: write de/d40/d6e/d52/f62 [2783344,121889] 0 2024-04-25T15:49:45.879 INFO:tasks.workunit.client.1.smithi193.stdout:0/689: dread d0/d49/d5/d7/dd/d46/d48/f5b [0,4194304] 0 2024-04-25T15:49:45.881 INFO:tasks.workunit.client.1.smithi193.stdout:4/714: creat db/d13/dce/fe6 x:0 0 0 2024-04-25T15:49:45.881 INFO:tasks.workunit.client.1.smithi193.stdout:4/715: readlink db/d32/d45/l53 0 2024-04-25T15:49:45.882 INFO:tasks.workunit.client.0.smithi110.stdout:8/945: read d14/f40 [2603246,86849] 0 2024-04-25T15:49:45.882 INFO:tasks.workunit.client.0.smithi110.stdout:8/946: chown d14/d32/d36/db7 85161 1 2024-04-25T15:49:45.884 INFO:tasks.workunit.client.1.smithi193.stdout:7/648: symlink d12/d1c/d63/db7/lf3 0 2024-04-25T15:49:45.885 INFO:tasks.workunit.client.1.smithi193.stdout:1/590: truncate dc/f15 1067099 0 2024-04-25T15:49:45.886 INFO:tasks.workunit.client.1.smithi193.stdout:4/716: dwrite f2 [0,4194304] 0 2024-04-25T15:49:45.887 INFO:tasks.workunit.client.0.smithi110.stdout:2/751: truncate de/d40/d6e/d50/fa4 254708 0 2024-04-25T15:49:45.891 INFO:tasks.workunit.client.0.smithi110.stdout:8/947: mkdir d14/d21/d26/d6a/d7e/da6/de8/d55/d149 0 2024-04-25T15:49:45.892 INFO:tasks.workunit.client.1.smithi193.stdout:8/645: fsync d2/d3/f80 0 2024-04-25T15:49:45.896 INFO:tasks.workunit.client.1.smithi193.stdout:3/687: mkdir d8/d13/d14/d27/d54/dd7 0 2024-04-25T15:49:45.896 INFO:tasks.workunit.client.1.smithi193.stdout:3/688: chown d8/d13/d1f/d69/dcd 199971546 1 2024-04-25T15:49:45.897 INFO:tasks.workunit.client.0.smithi110.stdout:8/948: creat d14/d21/d26/d6a/de9/f14a x:0 0 0 2024-04-25T15:49:45.898 INFO:tasks.workunit.client.0.smithi110.stdout:8/949: fdatasync d14/d32/d36/db7/dc9/f119 0 2024-04-25T15:49:45.901 INFO:tasks.workunit.client.0.smithi110.stdout:9/775: link d9/d13/d5b/d61/db3/db4/dc5/de4/fe8 d9/db/df/df3/ff4 0 2024-04-25T15:49:45.901 INFO:tasks.workunit.client.0.smithi110.stdout:9/776: read - d9/d13/d5b/ff2 zero size 2024-04-25T15:49:45.903 INFO:tasks.workunit.client.0.smithi110.stdout:9/777: truncate d9/d13/d5b/d61/db3/db4/dc5/de4/fe8 135244 0 2024-04-25T15:49:45.904 INFO:tasks.workunit.client.0.smithi110.stdout:2/752: creat de/d40/dcd/ddf/ff2 x:0 0 0 2024-04-25T15:49:45.904 INFO:tasks.workunit.client.1.smithi193.stdout:5/682: write d17/d18/d75/d85/d5d/dad/fb5 [292141,28071] 0 2024-04-25T15:49:45.906 INFO:tasks.workunit.client.0.smithi110.stdout:2/753: chown de/d40/d5b/d5c/d5f/db2/dad 47497644 1 2024-04-25T15:49:45.907 INFO:tasks.workunit.client.0.smithi110.stdout:3/849: write d0/d2e/f71 [921803,53865] 0 2024-04-25T15:49:45.912 INFO:tasks.workunit.client.0.smithi110.stdout:5/841: sync 2024-04-25T15:49:45.912 INFO:tasks.workunit.client.0.smithi110.stdout:7/918: sync 2024-04-25T15:49:45.912 INFO:tasks.workunit.client.0.smithi110.stdout:9/778: dwrite d9/db/df/d2c/d3c/fe0 [0,4194304] 0 2024-04-25T15:49:45.917 INFO:tasks.workunit.client.1.smithi193.stdout:1/591: dread dc/f2d [0,4194304] 0 2024-04-25T15:49:45.988 INFO:tasks.workunit.client.1.smithi193.stdout:8/646: rename d2/d3/d27/d69/dbf/d72 to d2/d3/d20/db8/dd0 0 2024-04-25T15:49:45.988 INFO:tasks.workunit.client.1.smithi193.stdout:8/647: chown d2/d3/d27/d69/dae/d4a/d63/l92 2 1 2024-04-25T15:49:45.990 INFO:tasks.workunit.client.0.smithi110.stdout:8/950: link d14/d32/d36/c103 d14/d21/d26/d6a/d7e/da6/de8/d71/dae/c14b 0 2024-04-25T15:49:45.993 INFO:tasks.workunit.client.0.smithi110.stdout:7/919: rmdir d1b/d29/d38/d55/d7f 39 2024-04-25T15:49:45.993 INFO:tasks.workunit.client.1.smithi193.stdout:8/648: dread d2/d3/d27/d69/dae/f49 [0,4194304] 0 2024-04-25T15:49:45.993 INFO:tasks.workunit.client.1.smithi193.stdout:9/638: write df/d2c/d34/f75 [953491,36124] 0 2024-04-25T15:49:45.994 INFO:tasks.workunit.client.1.smithi193.stdout:6/640: creat dd/d12/d19/fc4 x:0 0 0 2024-04-25T15:49:46.001 INFO:tasks.workunit.client.0.smithi110.stdout:5/842: fdatasync d4/f10 0 2024-04-25T15:49:46.003 INFO:tasks.workunit.client.1.smithi193.stdout:6/641: dwrite dd/d2f/d57/f7d [4194304,4194304] 0 2024-04-25T15:49:46.004 INFO:tasks.workunit.client.0.smithi110.stdout:7/920: creat d1b/d29/db5/dca/d10b/f135 x:0 0 0 2024-04-25T15:49:46.005 INFO:tasks.workunit.client.1.smithi193.stdout:7/649: truncate d12/d31/d37/d62/d6e/da5/dd7/fe5 213416 0 2024-04-25T15:49:46.007 INFO:tasks.workunit.client.1.smithi193.stdout:2/664: creat d0/d37/fff x:0 0 0 2024-04-25T15:49:46.007 INFO:tasks.workunit.client.0.smithi110.stdout:9/779: mkdir d9/d13/d5b/d61/d67/dd7/df5 0 2024-04-25T15:49:46.008 INFO:tasks.workunit.client.1.smithi193.stdout:3/689: dwrite d8/d13/d14/d51/f6c [0,4194304] 0 2024-04-25T15:49:46.009 INFO:tasks.workunit.client.1.smithi193.stdout:0/690: dwrite d0/d49/d5/d7/dd/d16/d26/f8d [0,4194304] 0 2024-04-25T15:49:46.009 INFO:tasks.workunit.client.1.smithi193.stdout:7/650: chown d12/d1c/d63/d70/d68/fb4 8014620 1 2024-04-25T15:49:46.009 INFO:tasks.workunit.client.0.smithi110.stdout:2/754: getdents de/d40/d6e 0 2024-04-25T15:49:46.009 INFO:tasks.workunit.client.1.smithi193.stdout:1/592: creat dc/d4a/d50/d7c/fca x:0 0 0 2024-04-25T15:49:46.010 INFO:tasks.workunit.client.0.smithi110.stdout:2/755: fsync de/d40/d4f/d9f/fa3 0 2024-04-25T15:49:46.011 INFO:tasks.workunit.client.0.smithi110.stdout:3/850: creat d0/d16/dcb/d2f/d4c/db2/f123 x:0 0 0 2024-04-25T15:49:46.011 INFO:tasks.workunit.client.1.smithi193.stdout:1/593: fsync dc/f12 0 2024-04-25T15:49:46.011 INFO:tasks.workunit.client.1.smithi193.stdout:7/651: truncate d12/d13/f41 2383374 0 2024-04-25T15:49:46.014 INFO:tasks.workunit.client.0.smithi110.stdout:5/843: mkdir d4/d15/d51/d117 0 2024-04-25T15:49:46.015 INFO:tasks.workunit.client.1.smithi193.stdout:2/665: dwrite d0/d37/d25/d5f/d3a/d6d/d6e/f3e [0,4194304] 0 2024-04-25T15:49:46.016 INFO:tasks.workunit.client.1.smithi193.stdout:2/666: fsync d0/d37/dbb/df8/fca 0 2024-04-25T15:49:46.016 INFO:tasks.workunit.client.0.smithi110.stdout:8/951: dwrite d14/f1c [0,4194304] 0 2024-04-25T15:49:46.017 INFO:tasks.workunit.client.1.smithi193.stdout:2/667: chown d0/d37/d25/d5f/d3a/d6d/d6e/fe2 1842365 1 2024-04-25T15:49:46.017 INFO:tasks.workunit.client.0.smithi110.stdout:9/780: rmdir d9/d13/d5b/d61 39 2024-04-25T15:49:46.022 INFO:tasks.workunit.client.0.smithi110.stdout:2/756: creat de/d6f/ff3 x:0 0 0 2024-04-25T15:49:46.024 INFO:tasks.workunit.client.1.smithi193.stdout:4/717: creat db/d32/fe7 x:0 0 0 2024-04-25T15:49:46.029 INFO:tasks.workunit.client.1.smithi193.stdout:5/683: dread d17/f1f [0,4194304] 0 2024-04-25T15:49:46.031 INFO:tasks.workunit.client.1.smithi193.stdout:2/668: dwrite d0/d37/d4d/dbe/df7/ffa [0,4194304] 0 2024-04-25T15:49:46.031 INFO:tasks.workunit.client.1.smithi193.stdout:9/639: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/f70 [0,4194304] 0 2024-04-25T15:49:46.034 INFO:tasks.workunit.client.1.smithi193.stdout:6/642: rmdir dd/d12/d19/d1c/d40/d3c/d3f 39 2024-04-25T15:49:46.040 INFO:tasks.workunit.client.1.smithi193.stdout:1/594: creat dc/d59/fcb x:0 0 0 2024-04-25T15:49:46.042 INFO:tasks.workunit.client.0.smithi110.stdout:2/757: mkdir de/d40/dd7/df4 0 2024-04-25T15:49:46.042 INFO:tasks.workunit.client.1.smithi193.stdout:0/691: creat d0/d49/d5/d7/dd/d16/d26/d2e/d93/fea x:0 0 0 2024-04-25T15:49:46.043 INFO:tasks.workunit.client.0.smithi110.stdout:7/921: dwrite d1b/d29/fd0 [0,4194304] 0 2024-04-25T15:49:46.044 INFO:tasks.workunit.client.1.smithi193.stdout:2/669: dwrite d0/d37/dbb/df8/f86 [4194304,4194304] 0 2024-04-25T15:49:46.045 INFO:tasks.workunit.client.1.smithi193.stdout:7/652: mkdir d12/d13/dcc/df4 0 2024-04-25T15:49:46.047 INFO:tasks.workunit.client.0.smithi110.stdout:2/758: write de/d40/d6e/f9d [2063908,44455] 0 2024-04-25T15:49:46.048 INFO:tasks.workunit.client.0.smithi110.stdout:7/922: dread - d1b/d29/db5/dca/df9/d12e/f131 zero size 2024-04-25T15:49:46.048 INFO:tasks.workunit.client.1.smithi193.stdout:8/649: creat d2/d3/d27/d69/dae/d4a/d63/da2/dad/fd1 x:0 0 0 2024-04-25T15:49:46.053 INFO:tasks.workunit.client.0.smithi110.stdout:9/781: mknod d9/d13/d5b/cf6 0 2024-04-25T15:49:46.058 INFO:tasks.workunit.client.0.smithi110.stdout:3/851: dwrite d0/f15 [0,4194304] 0 2024-04-25T15:49:46.058 INFO:tasks.workunit.client.0.smithi110.stdout:2/759: creat de/d40/dcd/ff5 x:0 0 0 2024-04-25T15:49:46.060 INFO:tasks.workunit.client.0.smithi110.stdout:2/760: chown de/d40/d4f/c58 7 1 2024-04-25T15:49:46.060 INFO:tasks.workunit.client.0.smithi110.stdout:7/923: rmdir d1b/d29/d46 39 2024-04-25T15:49:46.061 INFO:tasks.workunit.client.0.smithi110.stdout:2/761: fsync de/d40/d6e/d50/db7/fe3 0 2024-04-25T15:49:46.063 INFO:tasks.workunit.client.0.smithi110.stdout:8/952: creat d14/d32/d36/d48/f14c x:0 0 0 2024-04-25T15:49:46.067 INFO:tasks.workunit.client.1.smithi193.stdout:9/640: fsync df/d2c/d2f/d4e/d1c/d30/d3b/f47 0 2024-04-25T15:49:46.070 INFO:tasks.workunit.client.1.smithi193.stdout:0/692: mkdir d0/d49/d5/d7/dd/d16/d26/d5e/d99/deb 0 2024-04-25T15:49:46.070 INFO:tasks.workunit.client.0.smithi110.stdout:8/953: mknod d14/d32/d36/db7/c14d 0 2024-04-25T15:49:46.071 INFO:tasks.workunit.client.1.smithi193.stdout:2/670: fsync d0/d42/d51/df1/d9c/fba 0 2024-04-25T15:49:46.073 INFO:tasks.workunit.client.0.smithi110.stdout:9/782: mkdir d9/d13/d5b/df7 0 2024-04-25T15:49:46.074 INFO:tasks.workunit.client.0.smithi110.stdout:3/852: symlink d0/d2e/dfc/d10f/l124 0 2024-04-25T15:49:46.076 INFO:tasks.workunit.client.1.smithi193.stdout:3/690: dread d8/d13/d14/f93 [0,4194304] 0 2024-04-25T15:49:46.078 INFO:tasks.workunit.client.0.smithi110.stdout:8/954: rename d14/d21/d26/d6a/d7e/da6/de8/f91 to d14/d32/f14e 0 2024-04-25T15:49:46.079 INFO:tasks.workunit.client.0.smithi110.stdout:9/783: creat d9/d44/d5e/d80/dbb/dbe/ff8 x:0 0 0 2024-04-25T15:49:46.091 INFO:tasks.workunit.client.1.smithi193.stdout:5/684: dread d17/d18/f2c [0,4194304] 0 2024-04-25T15:49:46.092 INFO:tasks.workunit.client.1.smithi193.stdout:5/685: write d17/d18/d30/d38/d71/d96/fd2 [25458,48361] 0 2024-04-25T15:49:46.095 INFO:tasks.workunit.client.0.smithi110.stdout:7/924: creat d1b/d29/d38/d55/dab/db2/db8/f136 x:0 0 0 2024-04-25T15:49:46.098 INFO:tasks.workunit.client.1.smithi193.stdout:2/671: dread - d0/d37/d4d/fad zero size 2024-04-25T15:49:46.099 INFO:tasks.workunit.client.0.smithi110.stdout:9/784: unlink c5 0 2024-04-25T15:49:46.101 INFO:tasks.workunit.client.0.smithi110.stdout:8/955: creat d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/d101/f14f x:0 0 0 2024-04-25T15:49:46.101 INFO:tasks.workunit.client.1.smithi193.stdout:9/641: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/f43 [0,4194304] 0 2024-04-25T15:49:46.102 INFO:tasks.workunit.client.1.smithi193.stdout:4/718: rename db/d13/d1e/d1f/d56/d69/de1 to db/d32/de8 0 2024-04-25T15:49:46.103 INFO:tasks.workunit.client.0.smithi110.stdout:5/844: truncate d4/d2c/d2a/f46 2712717 0 2024-04-25T15:49:46.103 INFO:tasks.workunit.client.1.smithi193.stdout:4/719: readlink db/d4b/d51/d54/d77/lb9 0 2024-04-25T15:49:46.103 INFO:tasks.workunit.client.1.smithi193.stdout:4/720: readlink db/d4b/d51/d54/d77/l81 0 2024-04-25T15:49:46.104 INFO:tasks.workunit.client.0.smithi110.stdout:7/925: creat d1b/d29/d87/d8e/f137 x:0 0 0 2024-04-25T15:49:46.107 INFO:tasks.workunit.client.1.smithi193.stdout:1/595: creat dc/d4a/d50/d5c/d5d/d3d/d55/fcc x:0 0 0 2024-04-25T15:49:46.108 INFO:tasks.workunit.client.1.smithi193.stdout:1/596: read - dc/d4a/d50/d5c/d5d/d3d/f83 zero size 2024-04-25T15:49:46.108 INFO:tasks.workunit.client.1.smithi193.stdout:0/693: mknod d0/d49/d5/d7/dd/d16/d26/d9e/cec 0 2024-04-25T15:49:46.109 INFO:tasks.workunit.client.0.smithi110.stdout:2/762: getdents de/d40/d5b/d5c/d5f/db2 0 2024-04-25T15:49:46.109 INFO:tasks.workunit.client.1.smithi193.stdout:1/597: fsync dc/d4a/d50/d5c/d5d/d3d/d55/f93 0 2024-04-25T15:49:46.110 INFO:tasks.workunit.client.1.smithi193.stdout:2/672: creat d0/df/dbf/f100 x:0 0 0 2024-04-25T15:49:46.112 INFO:tasks.workunit.client.0.smithi110.stdout:9/785: readlink d9/d13/d5b/d61/d67/dd7/led 0 2024-04-25T15:49:46.112 INFO:tasks.workunit.client.1.smithi193.stdout:6/643: write dd/fe [1898472,1197] 0 2024-04-25T15:49:46.112 INFO:tasks.workunit.client.1.smithi193.stdout:6/644: readlink l9 0 2024-04-25T15:49:46.114 INFO:tasks.workunit.client.1.smithi193.stdout:7/653: dwrite d12/d13/f14 [4194304,4194304] 0 2024-04-25T15:49:46.115 INFO:tasks.workunit.client.1.smithi193.stdout:7/654: readlink d12/d13/l3d 0 2024-04-25T15:49:46.115 INFO:tasks.workunit.client.1.smithi193.stdout:5/686: rename d17/d18/d75/d85/d5d/dad/dd8/de3 to d17/d18/d30/d38/d71/dd7/de7 0 2024-04-25T15:49:46.117 INFO:tasks.workunit.client.0.smithi110.stdout:2/763: dwrite de/d40/d4f/d9f/fe8 [0,4194304] 0 2024-04-25T15:49:46.118 INFO:tasks.workunit.client.0.smithi110.stdout:7/926: rename d1b/l32 to d1b/d29/d38/d55/dab/db2/l138 0 2024-04-25T15:49:46.119 INFO:tasks.workunit.client.0.smithi110.stdout:3/853: rmdir d0/d16/dcb/d2f/d10a 0 2024-04-25T15:49:46.123 INFO:tasks.workunit.client.1.smithi193.stdout:7/655: dwrite d12/d1c/d63/d70/f34 [4194304,4194304] 0 2024-04-25T15:49:46.125 INFO:tasks.workunit.client.1.smithi193.stdout:0/694: rmdir d0/d49/d5/d7/dd/d16/d26 39 2024-04-25T15:49:46.127 INFO:tasks.workunit.client.0.smithi110.stdout:9/786: creat d9/d44/d5e/d80/dbb/dbe/ff9 x:0 0 0 2024-04-25T15:49:46.129 INFO:tasks.workunit.client.1.smithi193.stdout:2/673: readlink d0/d37/d25/d5f/d3a/ldc 0 2024-04-25T15:49:46.132 INFO:tasks.workunit.client.0.smithi110.stdout:3/854: creat d0/d16/dcb/d50/f125 x:0 0 0 2024-04-25T15:49:46.135 INFO:tasks.workunit.client.1.smithi193.stdout:5/687: chown d17/d18/d75/f99 3088641 1 2024-04-25T15:49:46.136 INFO:tasks.workunit.client.0.smithi110.stdout:2/764: dwrite de/d40/d4f/fea [0,4194304] 0 2024-04-25T15:49:46.137 INFO:tasks.workunit.client.0.smithi110.stdout:7/927: mkdir d1b/d29/d38/d55/d139 0 2024-04-25T15:49:46.138 INFO:tasks.workunit.client.0.smithi110.stdout:7/928: dread - d1b/d29/f125 zero size 2024-04-25T15:49:46.139 INFO:tasks.workunit.client.1.smithi193.stdout:7/656: symlink d12/d1c/dd6/lf5 0 2024-04-25T15:49:46.142 INFO:tasks.workunit.client.0.smithi110.stdout:9/787: fdatasync d9/d13/d84/f92 0 2024-04-25T15:49:46.145 INFO:tasks.workunit.client.0.smithi110.stdout:2/765: truncate de/d40/d5b/d5c/d5f/db2/fcb 907798 0 2024-04-25T15:49:46.147 INFO:tasks.workunit.client.1.smithi193.stdout:9/642: creat df/fc5 x:0 0 0 2024-04-25T15:49:46.150 INFO:tasks.workunit.client.0.smithi110.stdout:3/855: unlink d0/d16/dcb/d2f/d4c/d63/lf6 0 2024-04-25T15:49:46.152 INFO:tasks.workunit.client.1.smithi193.stdout:6/645: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/fc5 x:0 0 0 2024-04-25T15:49:46.152 INFO:tasks.workunit.client.1.smithi193.stdout:2/674: dread d0/d9/d3d/fa3 [0,4194304] 0 2024-04-25T15:49:46.153 INFO:tasks.workunit.client.1.smithi193.stdout:6/646: read dd/d12/d19/f6a [3827611,73643] 0 2024-04-25T15:49:46.156 INFO:tasks.workunit.client.0.smithi110.stdout:2/766: creat de/d40/d5b/d5c/dbe/ff6 x:0 0 0 2024-04-25T15:49:46.159 INFO:tasks.workunit.client.1.smithi193.stdout:7/657: rename d12/d1c/f1d to d12/d31/d37/d92/dd1/ff6 0 2024-04-25T15:49:46.159 INFO:tasks.workunit.client.1.smithi193.stdout:7/658: dread - d12/d31/d37/d92/dbf/fc2 zero size 2024-04-25T15:49:46.160 INFO:tasks.workunit.client.1.smithi193.stdout:7/659: dread - d12/d13/fac zero size 2024-04-25T15:49:46.162 INFO:tasks.workunit.client.1.smithi193.stdout:1/598: creat dc/d4a/d50/d5c/d5d/d3d/fcd x:0 0 0 2024-04-25T15:49:46.162 INFO:tasks.workunit.client.1.smithi193.stdout:1/599: chown dc/f36 7 1 2024-04-25T15:49:46.162 INFO:tasks.workunit.client.1.smithi193.stdout:3/691: truncate d8/d30/f40 7263439 0 2024-04-25T15:49:46.163 INFO:tasks.workunit.client.1.smithi193.stdout:3/692: dread - d8/d13/d14/d27/f96 zero size 2024-04-25T15:49:46.163 INFO:tasks.workunit.client.0.smithi110.stdout:9/788: dread d9/db/df/f68 [0,4194304] 0 2024-04-25T15:49:46.164 INFO:tasks.workunit.client.0.smithi110.stdout:8/956: write d14/d21/d26/d6a/de9/f107 [723042,75547] 0 2024-04-25T15:49:46.165 INFO:tasks.workunit.client.0.smithi110.stdout:9/789: chown d9/db/df/d2c/d3c/d56/l57 25428706 1 2024-04-25T15:49:46.166 INFO:tasks.workunit.client.0.smithi110.stdout:5/845: write d4/d1d/d1f/d35/f6a [2291601,10596] 0 2024-04-25T15:49:46.166 INFO:tasks.workunit.client.1.smithi193.stdout:8/650: sync 2024-04-25T15:49:46.167 INFO:tasks.workunit.client.1.smithi193.stdout:9/643: dread - df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/f84 zero size 2024-04-25T15:49:46.168 INFO:tasks.workunit.client.0.smithi110.stdout:5/846: chown d4/d1d/d1f/d35/d7f/d92/fca 2401 1 2024-04-25T15:49:46.169 INFO:tasks.workunit.client.0.smithi110.stdout:2/767: mkdir de/d40/d6e/d52/df7 0 2024-04-25T15:49:46.169 INFO:tasks.workunit.client.0.smithi110.stdout:5/847: dread - d4/d1d/d85/ffc zero size 2024-04-25T15:49:46.169 INFO:tasks.workunit.client.1.smithi193.stdout:9/644: dread - df/d2c/d2f/d5c/d7e/fc1 zero size 2024-04-25T15:49:46.169 INFO:tasks.workunit.client.1.smithi193.stdout:1/600: dwrite dc/d4a/d4d/d80/f98 [0,4194304] 0 2024-04-25T15:49:46.171 INFO:tasks.workunit.client.1.smithi193.stdout:2/675: rmdir d0/d42/d51/d4f 39 2024-04-25T15:49:46.173 INFO:tasks.workunit.client.0.smithi110.stdout:8/957: creat d14/d32/d105/d110/f150 x:0 0 0 2024-04-25T15:49:46.175 INFO:tasks.workunit.client.1.smithi193.stdout:5/688: link d17/d18/d30/f76 d17/d18/d3e/db6/fe8 0 2024-04-25T15:49:46.176 INFO:tasks.workunit.client.0.smithi110.stdout:9/790: mkdir d9/d44/d5e/db5/dfa 0 2024-04-25T15:49:46.178 INFO:tasks.workunit.client.0.smithi110.stdout:9/791: write d9/d13/d5b/d61/db3/dd3/ddb/fec [876209,1341] 0 2024-04-25T15:49:46.179 INFO:tasks.workunit.client.1.smithi193.stdout:4/721: getdents db/d13/d1e/d1f/d56/d69/d70 0 2024-04-25T15:49:46.179 INFO:tasks.workunit.client.0.smithi110.stdout:7/929: write d1b/d29/db5/dca/fd5 [1592991,767] 0 2024-04-25T15:49:46.180 INFO:tasks.workunit.client.0.smithi110.stdout:9/792: write d9/d13/d5b/d61/db3/dd3/ddb/fec [1836324,117196] 0 2024-04-25T15:49:46.185 INFO:tasks.workunit.client.1.smithi193.stdout:3/693: rename d8/d30/d72/lbd to d8/d30/d45/dd0/ld8 0 2024-04-25T15:49:46.190 INFO:tasks.workunit.client.0.smithi110.stdout:5/848: fdatasync d4/d2c/fdb 0 2024-04-25T15:49:46.196 INFO:tasks.workunit.client.0.smithi110.stdout:3/856: dwrite d0/d16/dcb/d2f/d33/f44 [0,4194304] 0 2024-04-25T15:49:46.201 INFO:tasks.workunit.client.0.smithi110.stdout:9/793: symlink d9/d13/d5b/d61/db3/dd3/lfb 0 2024-04-25T15:49:46.202 INFO:tasks.workunit.client.0.smithi110.stdout:6/788: sync 2024-04-25T15:49:46.203 INFO:tasks.workunit.client.0.smithi110.stdout:6/789: stat d1/dd/d3c/d91/f7f 0 2024-04-25T15:49:46.203 INFO:tasks.workunit.client.1.smithi193.stdout:0/695: truncate d0/d49/d5/d7/dd/d16/d26/d5e/f69 1081276 0 2024-04-25T15:49:46.205 INFO:tasks.workunit.client.1.smithi193.stdout:2/676: creat d0/d9/d3d/d7d/f101 x:0 0 0 2024-04-25T15:49:46.206 INFO:tasks.workunit.client.1.smithi193.stdout:5/689: mknod d17/d18/d3e/dd1/ddc/ce9 0 2024-04-25T15:49:46.208 INFO:tasks.workunit.client.1.smithi193.stdout:6/647: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dc1/fc6 x:0 0 0 2024-04-25T15:49:46.209 INFO:tasks.workunit.client.1.smithi193.stdout:4/722: mknod db/d13/d1e/d1f/d56/d69/dad/dc0/ce9 0 2024-04-25T15:49:46.210 INFO:tasks.workunit.client.0.smithi110.stdout:7/930: creat d1b/d29/db5/dca/d10b/d112/f13a x:0 0 0 2024-04-25T15:49:46.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/660: mknod d12/d31/d37/d62/db3/cf7 0 2024-04-25T15:49:46.211 INFO:tasks.workunit.client.0.smithi110.stdout:3/857: truncate d0/d7/f69 4747420 0 2024-04-25T15:49:46.217 INFO:tasks.workunit.client.1.smithi193.stdout:9/645: mkdir df/d2c/d2f/d5c/d7e/da2/dc6 0 2024-04-25T15:49:46.217 INFO:tasks.workunit.client.0.smithi110.stdout:6/790: read d1/db/d22/f57 [199650,57146] 0 2024-04-25T15:49:46.219 INFO:tasks.workunit.client.0.smithi110.stdout:2/768: dwrite de/d40/d6e/d50/fa4 [0,4194304] 0 2024-04-25T15:49:46.222 INFO:tasks.workunit.client.0.smithi110.stdout:3/858: dwrite d0/d16/d8c/fea [4194304,4194304] 0 2024-04-25T15:49:46.223 INFO:tasks.workunit.client.0.smithi110.stdout:5/849: dread d4/f10 [0,4194304] 0 2024-04-25T15:49:46.224 INFO:tasks.workunit.client.0.smithi110.stdout:5/850: dread - d4/d1d/d85/ffc zero size 2024-04-25T15:49:46.228 INFO:tasks.workunit.client.0.smithi110.stdout:8/958: getdents d14/d32/d105/d110 0 2024-04-25T15:49:46.233 INFO:tasks.workunit.client.0.smithi110.stdout:7/931: mkdir d1b/d29/d38/d55/d139/d13b 0 2024-04-25T15:49:46.234 INFO:tasks.workunit.client.1.smithi193.stdout:8/651: truncate d2/d3/f16 391022 0 2024-04-25T15:49:46.235 INFO:tasks.workunit.client.1.smithi193.stdout:6/648: creat dd/d12/d19/d45/fc7 x:0 0 0 2024-04-25T15:49:46.236 INFO:tasks.workunit.client.1.smithi193.stdout:6/649: chown dd/d12/d19/d1c/d40/d3c/f68 15 1 2024-04-25T15:49:46.239 INFO:tasks.workunit.client.0.smithi110.stdout:2/769: unlink de/d40/d6e/d50/db7/cd2 0 2024-04-25T15:49:46.240 INFO:tasks.workunit.client.0.smithi110.stdout:2/770: chown de/d40/d6e/d52/f62 4413201 1 2024-04-25T15:49:46.240 INFO:tasks.workunit.client.1.smithi193.stdout:3/694: mkdir d8/d13/d1f/d69/dcd/dd9 0 2024-04-25T15:49:46.240 INFO:tasks.workunit.client.1.smithi193.stdout:0/696: write d0/d49/d5/d7/dd/d16/d18/f22 [4440994,78392] 0 2024-04-25T15:49:46.241 INFO:tasks.workunit.client.0.smithi110.stdout:9/794: write d9/db/df/d25/f51 [1562073,127898] 0 2024-04-25T15:49:46.242 INFO:tasks.workunit.client.1.smithi193.stdout:9/646: creat df/d2c/d2f/d4e/d1c/fc7 x:0 0 0 2024-04-25T15:49:46.242 INFO:tasks.workunit.client.0.smithi110.stdout:9/795: chown d9/db/df/f33 2703633 1 2024-04-25T15:49:46.243 INFO:tasks.workunit.client.0.smithi110.stdout:9/796: fsync d9/d44/d5e/db5/fda 0 2024-04-25T15:49:46.244 INFO:tasks.workunit.client.0.smithi110.stdout:5/851: mknod d4/d1d/d1f/d35/d3d/d52/d75/dfa/c118 0 2024-04-25T15:49:46.244 INFO:tasks.workunit.client.1.smithi193.stdout:1/601: link dc/d4a/d50/l64 dc/dc9/lce 0 2024-04-25T15:49:46.245 INFO:tasks.workunit.client.0.smithi110.stdout:5/852: fdatasync d4/d1d/d1f/d35/d3d/d93/ff1 0 2024-04-25T15:49:46.248 INFO:tasks.workunit.client.1.smithi193.stdout:5/690: symlink d17/lea 0 2024-04-25T15:49:46.249 INFO:tasks.workunit.client.0.smithi110.stdout:7/932: fdatasync d1b/f48 0 2024-04-25T15:49:46.250 INFO:tasks.workunit.client.0.smithi110.stdout:7/933: stat d1b/d29/db5/dca/df9 0 2024-04-25T15:49:46.252 INFO:tasks.workunit.client.1.smithi193.stdout:8/652: fdatasync d2/fab 0 2024-04-25T15:49:46.252 INFO:tasks.workunit.client.1.smithi193.stdout:8/653: fdatasync d2/da5/fa7 0 2024-04-25T15:49:46.253 INFO:tasks.workunit.client.0.smithi110.stdout:6/791: creat d1/dd/d3c/df8/f114 x:0 0 0 2024-04-25T15:49:46.253 INFO:tasks.workunit.client.1.smithi193.stdout:4/723: mkdir db/d4b/dea 0 2024-04-25T15:49:46.254 INFO:tasks.workunit.client.1.smithi193.stdout:7/661: write d12/d31/d37/d62/d6e/d79/f94 [322585,10720] 0 2024-04-25T15:49:46.254 INFO:tasks.workunit.client.0.smithi110.stdout:3/859: mknod d0/d2e/d88/dbf/c126 0 2024-04-25T15:49:46.256 INFO:tasks.workunit.client.1.smithi193.stdout:6/650: rmdir dd/d12/d19/d1c/d40/d3c/d3f/d4f 39 2024-04-25T15:49:46.257 INFO:tasks.workunit.client.1.smithi193.stdout:6/651: read - dd/f59 zero size 2024-04-25T15:49:46.257 INFO:tasks.workunit.client.0.smithi110.stdout:2/771: rename de/d40/d6e/d52/d6d/l7c to de/d6f/dd9/lf8 0 2024-04-25T15:49:46.259 INFO:tasks.workunit.client.0.smithi110.stdout:9/797: rmdir d9/db/df/d2c 39 2024-04-25T15:49:46.259 INFO:tasks.workunit.client.0.smithi110.stdout:8/959: write d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/fa7 [498986,51505] 0 2024-04-25T15:49:46.260 INFO:tasks.workunit.client.1.smithi193.stdout:7/662: dwrite d12/d31/d37/d92/dbf/fc2 [0,4194304] 0 2024-04-25T15:49:46.261 INFO:tasks.workunit.client.1.smithi193.stdout:7/663: write d12/d13/dcc/fe4 [33639,74660] 0 2024-04-25T15:49:46.261 INFO:tasks.workunit.client.0.smithi110.stdout:8/960: write d14/d21/d26/f29 [5104633,82679] 0 2024-04-25T15:49:46.263 INFO:tasks.workunit.client.1.smithi193.stdout:3/695: symlink d8/d13/d1f/d9f/lda 0 2024-04-25T15:49:46.264 INFO:tasks.workunit.client.0.smithi110.stdout:5/853: fsync d4/d1d/d1f/fcf 0 2024-04-25T15:49:46.268 INFO:tasks.workunit.client.1.smithi193.stdout:9/647: creat df/d2c/d2f/d4e/d1c/d30/d3b/fc8 x:0 0 0 2024-04-25T15:49:46.268 INFO:tasks.workunit.client.0.smithi110.stdout:7/934: mkdir d1b/d29/db5/dca/d10b/d112/d13c 0 2024-04-25T15:49:46.271 INFO:tasks.workunit.client.0.smithi110.stdout:3/860: mkdir d0/d16/dcb/d50/d127 0 2024-04-25T15:49:46.272 INFO:tasks.workunit.client.1.smithi193.stdout:2/677: link d0/d42/d51/cd0 d0/d37/d4d/dbe/c102 0 2024-04-25T15:49:46.275 INFO:tasks.workunit.client.1.smithi193.stdout:5/691: mkdir d17/d18/d30/d38/d48/d4f/deb 0 2024-04-25T15:49:46.278 INFO:tasks.workunit.client.1.smithi193.stdout:2/678: dwrite d0/d37/d25/d5f/d98/fec [0,4194304] 0 2024-04-25T15:49:46.288 INFO:tasks.workunit.client.1.smithi193.stdout:6/652: readlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/l6e 0 2024-04-25T15:49:46.289 INFO:tasks.workunit.client.0.smithi110.stdout:9/798: write d9/d44/fab [4151591,129920] 0 2024-04-25T15:49:46.289 INFO:tasks.workunit.client.0.smithi110.stdout:5/854: fdatasync d4/d1d/d1f/d35/d3d/f60 0 2024-04-25T15:49:46.290 INFO:tasks.workunit.client.0.smithi110.stdout:5/855: readlink d4/lc 0 2024-04-25T15:49:46.292 INFO:tasks.workunit.client.1.smithi193.stdout:7/664: mknod d12/d13/dcc/cf8 0 2024-04-25T15:49:46.292 INFO:tasks.workunit.client.1.smithi193.stdout:4/724: write db/d32/f94 [2012868,100971] 0 2024-04-25T15:49:46.293 INFO:tasks.workunit.client.0.smithi110.stdout:8/961: write d14/d21/d26/d6a/d7e/da6/de8/d55/fb4 [2672522,36961] 0 2024-04-25T15:49:46.293 INFO:tasks.workunit.client.1.smithi193.stdout:7/665: chown d12/d1c/dd6/dda/ff1 24992 1 2024-04-25T15:49:46.294 INFO:tasks.workunit.client.1.smithi193.stdout:3/696: mknod d8/d13/d4b/d8b/d9a/cdb 0 2024-04-25T15:49:46.295 INFO:tasks.workunit.client.0.smithi110.stdout:7/935: readlink d1b/d29/l4d 0 2024-04-25T15:49:46.297 INFO:tasks.workunit.client.0.smithi110.stdout:2/772: symlink de/d40/d5b/d5c/dbe/df0/lf9 0 2024-04-25T15:49:46.299 INFO:tasks.workunit.client.1.smithi193.stdout:7/666: dwrite d12/d13/dc4/fdb [0,4194304] 0 2024-04-25T15:49:46.301 INFO:tasks.workunit.client.1.smithi193.stdout:1/602: rename dc/d4a/d50/d5c/d63/cbc to dc/d59/dc0/ccf 0 2024-04-25T15:49:46.302 INFO:tasks.workunit.client.1.smithi193.stdout:5/692: unlink d17/d53/lcf 0 2024-04-25T15:49:46.304 INFO:tasks.workunit.client.1.smithi193.stdout:2/679: unlink d0/d9/f73 0 2024-04-25T15:49:46.305 INFO:tasks.workunit.client.1.smithi193.stdout:8/654: link d2/d3/d27/d69/dae/f1a d2/d3/d27/d6f/fd2 0 2024-04-25T15:49:46.305 INFO:tasks.workunit.client.1.smithi193.stdout:9/648: dwrite df/d2d/f99 [0,4194304] 0 2024-04-25T15:49:46.306 INFO:tasks.workunit.client.1.smithi193.stdout:6/653: creat dd/d12/d2d/fc8 x:0 0 0 2024-04-25T15:49:46.307 INFO:tasks.workunit.client.1.smithi193.stdout:8/655: chown d2/d3/d27/d69/f9c 818556316 1 2024-04-25T15:49:46.308 INFO:tasks.workunit.client.1.smithi193.stdout:8/656: chown d2/d3/d27/d69/dae/d4a/d63/l92 2257 1 2024-04-25T15:49:46.308 INFO:tasks.workunit.client.1.smithi193.stdout:4/725: fdatasync db/d13/d1e/d1f/d56/d69/d70/d8a/f8c 0 2024-04-25T15:49:46.311 INFO:tasks.workunit.client.0.smithi110.stdout:8/962: symlink d14/d32/d36/db7/l151 0 2024-04-25T15:49:46.313 INFO:tasks.workunit.client.1.smithi193.stdout:8/657: read d2/d3/ff [607801,78438] 0 2024-04-25T15:49:46.315 INFO:tasks.workunit.client.0.smithi110.stdout:7/936: read d1b/f26 [2748831,109934] 0 2024-04-25T15:49:46.316 INFO:tasks.workunit.client.1.smithi193.stdout:0/697: creat d0/d49/d5/d7/dd/d16/d26/fed x:0 0 0 2024-04-25T15:49:46.317 INFO:tasks.workunit.client.1.smithi193.stdout:7/667: creat d12/d13/ff9 x:0 0 0 2024-04-25T15:49:46.319 INFO:tasks.workunit.client.1.smithi193.stdout:5/693: creat d17/d18/d30/d34/d52/fec x:0 0 0 2024-04-25T15:49:46.320 INFO:tasks.workunit.client.1.smithi193.stdout:1/603: dread dc/f15 [0,4194304] 0 2024-04-25T15:49:46.320 INFO:tasks.workunit.client.1.smithi193.stdout:2/680: fdatasync d0/fc 0 2024-04-25T15:49:46.321 INFO:tasks.workunit.client.1.smithi193.stdout:9/649: mknod df/d2c/d2f/d4e/d69/cc9 0 2024-04-25T15:49:46.329 INFO:tasks.workunit.client.1.smithi193.stdout:4/726: fdatasync db/d32/d45/dab/fc8 0 2024-04-25T15:49:46.332 INFO:tasks.workunit.client.1.smithi193.stdout:3/697: link d8/d13/d14/d27/f96 d8/d30/d72/db2/fdc 0 2024-04-25T15:49:46.334 INFO:tasks.workunit.client.0.smithi110.stdout:9/799: dwrite d9/db/df/d2c/d3c/fd6 [0,4194304] 0 2024-04-25T15:49:46.338 INFO:tasks.workunit.client.1.smithi193.stdout:5/694: rmdir d17/d18/d30/d38/d71/dd7 39 2024-04-25T15:49:46.341 INFO:tasks.workunit.client.1.smithi193.stdout:5/695: dread d17/d18/d30/d38/dab/fc9 [0,4194304] 0 2024-04-25T15:49:46.342 INFO:tasks.workunit.client.1.smithi193.stdout:7/668: dread d12/d1c/f8a [0,4194304] 0 2024-04-25T15:49:46.344 INFO:tasks.workunit.client.1.smithi193.stdout:6/654: mkdir dd/db5/dc9 0 2024-04-25T15:49:46.349 INFO:tasks.workunit.client.1.smithi193.stdout:0/698: write d0/f35 [1365379,101009] 0 2024-04-25T15:49:46.350 INFO:tasks.workunit.client.1.smithi193.stdout:3/698: mknod d8/d13/d1f/cdd 0 2024-04-25T15:49:46.352 INFO:tasks.workunit.client.1.smithi193.stdout:3/699: read d8/d13/d1f/dbf/f8c [2026345,124419] 0 2024-04-25T15:49:46.357 INFO:tasks.workunit.client.1.smithi193.stdout:3/700: dwrite d8/d30/d72/f7e [0,4194304] 0 2024-04-25T15:49:46.358 INFO:tasks.workunit.client.1.smithi193.stdout:1/604: mkdir dc/d4a/d4d/d7a/dd0 0 2024-04-25T15:49:46.359 INFO:tasks.workunit.client.1.smithi193.stdout:1/605: truncate dc/d59/dc0/fc1 771516 0 2024-04-25T15:49:46.362 INFO:tasks.workunit.client.1.smithi193.stdout:1/606: write dc/d4a/d50/d5c/d5d/d3d/f97 [2816338,26279] 0 2024-04-25T15:49:46.362 INFO:tasks.workunit.client.1.smithi193.stdout:4/727: dread db/d13/d1e/d1f/d22/f2c [0,4194304] 0 2024-04-25T15:49:46.363 INFO:tasks.workunit.client.1.smithi193.stdout:4/728: chown db/d13/d9d/fb7 6756 1 2024-04-25T15:49:46.367 INFO:tasks.workunit.client.1.smithi193.stdout:8/658: truncate d2/d18/db1/fb3 2301892 0 2024-04-25T15:49:46.367 INFO:tasks.workunit.client.0.smithi110.stdout:4/816: sync 2024-04-25T15:49:46.367 INFO:tasks.workunit.client.0.smithi110.stdout:0/809: sync 2024-04-25T15:49:46.368 INFO:tasks.workunit.client.0.smithi110.stdout:1/990: sync 2024-04-25T15:49:46.369 INFO:tasks.workunit.client.0.smithi110.stdout:4/817: dread - d13/d24/d2d/dd9/de1/f109 zero size 2024-04-25T15:49:46.370 INFO:tasks.workunit.client.1.smithi193.stdout:6/655: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/fca x:0 0 0 2024-04-25T15:49:46.373 INFO:tasks.workunit.client.1.smithi193.stdout:0/699: mknod d0/d49/d6a/cee 0 2024-04-25T15:49:46.382 INFO:tasks.workunit.client.1.smithi193.stdout:5/696: dwrite d17/d18/f2c [0,4194304] 0 2024-04-25T15:49:46.383 INFO:tasks.workunit.client.1.smithi193.stdout:3/701: dwrite d8/d30/d45/f70 [0,4194304] 0 2024-04-25T15:49:46.383 INFO:tasks.workunit.client.1.smithi193.stdout:1/607: unlink dc/d4a/d50/cb3 0 2024-04-25T15:49:46.384 INFO:tasks.workunit.client.1.smithi193.stdout:5/697: truncate d17/d53/fce 802813 0 2024-04-25T15:49:46.386 INFO:tasks.workunit.client.1.smithi193.stdout:9/650: creat df/d2c/fca x:0 0 0 2024-04-25T15:49:46.387 INFO:tasks.workunit.client.1.smithi193.stdout:5/698: fdatasync d17/d18/d30/d34/d52/fec 0 2024-04-25T15:49:46.392 INFO:tasks.workunit.client.1.smithi193.stdout:7/669: truncate d12/d13/f43 2041885 0 2024-04-25T15:49:46.392 INFO:tasks.workunit.client.1.smithi193.stdout:9/651: truncate df/d2c/d2f/d4e/d1c/fbf 1012841 0 2024-04-25T15:49:46.392 INFO:tasks.workunit.client.1.smithi193.stdout:8/659: rename d2/d18/da0 to d2/d18/dbc/dd3 0 2024-04-25T15:49:46.399 INFO:tasks.workunit.client.1.smithi193.stdout:2/681: getdents d0/d9/d3d 0 2024-04-25T15:49:46.400 INFO:tasks.workunit.client.0.smithi110.stdout:3/861: rename d0/d7/d40/f43 to d0/d16/dcb/d2f/f128 0 2024-04-25T15:49:46.400 INFO:tasks.workunit.client.1.smithi193.stdout:3/702: fdatasync d8/d13/d1f/dbf/f8c 0 2024-04-25T15:49:46.402 INFO:tasks.workunit.client.1.smithi193.stdout:4/729: creat db/d13/dd2/feb x:0 0 0 2024-04-25T15:49:46.403 INFO:tasks.workunit.client.1.smithi193.stdout:4/730: chown db/d4b/le2 1284972 1 2024-04-25T15:49:46.403 INFO:tasks.workunit.client.1.smithi193.stdout:2/682: dread d0/d37/dbb/df8/f86 [4194304,4194304] 0 2024-04-25T15:49:46.404 INFO:tasks.workunit.client.1.smithi193.stdout:5/699: creat d17/d18/d30/d3b/fed x:0 0 0 2024-04-25T15:49:46.404 INFO:tasks.workunit.client.1.smithi193.stdout:5/700: dread - d17/d18/d9b/fc3 zero size 2024-04-25T15:49:46.405 INFO:tasks.workunit.client.1.smithi193.stdout:7/670: mkdir d12/d13/dc4/dfa 0 2024-04-25T15:49:46.407 INFO:tasks.workunit.client.1.smithi193.stdout:9/652: symlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/lcb 0 2024-04-25T15:49:46.412 INFO:tasks.workunit.client.1.smithi193.stdout:8/660: dread d2/d18/dbc/dd3/fb2 [0,4194304] 0 2024-04-25T15:49:46.413 INFO:tasks.workunit.client.0.smithi110.stdout:2/773: rename de/d40/d5b/d5c/dbe/df0 to de/d40/d70/dfa 0 2024-04-25T15:49:46.415 INFO:tasks.workunit.client.0.smithi110.stdout:8/963: mkdir d14/d21/d26/d6a/d7e/da6/de8/d55/d149/d152 0 2024-04-25T15:49:46.416 INFO:tasks.workunit.client.1.smithi193.stdout:0/700: truncate d0/d49/d5/d7/dd/d46/d48/f57 1085440 0 2024-04-25T15:49:46.419 INFO:tasks.workunit.client.0.smithi110.stdout:7/937: mkdir d1b/d29/d38/d55/d7f/dc8/d13d 0 2024-04-25T15:49:46.421 INFO:tasks.workunit.client.0.smithi110.stdout:5/856: write d4/d2c/fd0 [649987,74863] 0 2024-04-25T15:49:46.422 INFO:tasks.workunit.client.1.smithi193.stdout:3/703: dread d8/d13/d1f/f16 [0,4194304] 0 2024-04-25T15:49:46.422 INFO:tasks.workunit.client.1.smithi193.stdout:6/656: dwrite dd/d12/d19/f7f [0,4194304] 0 2024-04-25T15:49:46.423 INFO:tasks.workunit.client.1.smithi193.stdout:4/731: mkdir db/d32/d40/dec 0 2024-04-25T15:49:46.425 INFO:tasks.workunit.client.1.smithi193.stdout:4/732: truncate db/d13/d1e/d1f/d56/d69/fe5 20208 0 2024-04-25T15:49:46.426 INFO:tasks.workunit.client.1.smithi193.stdout:2/683: rename d0/d9/d3d/d7d/fe1 to d0/d42/d51/df1/d9f/dc5/f103 0 2024-04-25T15:49:46.426 INFO:tasks.workunit.client.0.smithi110.stdout:9/800: write d9/f34 [524176,49757] 0 2024-04-25T15:49:46.426 INFO:tasks.workunit.client.1.smithi193.stdout:4/733: write db/fd0 [239826,23491] 0 2024-04-25T15:49:46.429 INFO:tasks.workunit.client.1.smithi193.stdout:5/701: mkdir d17/d18/d30/d34/d52/dee 0 2024-04-25T15:49:46.430 INFO:tasks.workunit.client.1.smithi193.stdout:7/671: dread d12/d31/d37/d92/dd1/ff6 [0,4194304] 0 2024-04-25T15:49:46.430 INFO:tasks.workunit.client.0.smithi110.stdout:1/991: write d0/d23/fd9 [588167,36811] 0 2024-04-25T15:49:46.430 INFO:tasks.workunit.client.1.smithi193.stdout:5/702: chown d17/d18/d3e/db6 118 1 2024-04-25T15:49:46.431 INFO:tasks.workunit.client.0.smithi110.stdout:8/964: symlink d14/d21/d26/d6a/d7e/l153 0 2024-04-25T15:49:46.433 INFO:tasks.workunit.client.0.smithi110.stdout:3/862: truncate d0/d16/dcb/f35 5011226 0 2024-04-25T15:49:46.436 INFO:tasks.workunit.client.0.smithi110.stdout:7/938: chown d1b/d29/d46/c56 981712304 1 2024-04-25T15:49:46.437 INFO:tasks.workunit.client.0.smithi110.stdout:5/857: rmdir d4/d1d/d1f/d35/d3d/d52 39 2024-04-25T15:49:46.437 INFO:tasks.workunit.client.0.smithi110.stdout:7/939: readlink d1b/d29/la1 0 2024-04-25T15:49:46.437 INFO:tasks.workunit.client.0.smithi110.stdout:4/818: dwrite d13/d24/d2d/dd9/fc2 [0,4194304] 0 2024-04-25T15:49:46.442 INFO:tasks.workunit.client.0.smithi110.stdout:2/774: creat de/d40/d6e/d52/dda/ffb x:0 0 0 2024-04-25T15:49:46.450 INFO:tasks.workunit.client.0.smithi110.stdout:5/858: read d4/d15/f1a [2809606,100523] 0 2024-04-25T15:49:46.452 INFO:tasks.workunit.client.0.smithi110.stdout:7/940: mkdir d1b/d29/db5/dca/d13e 0 2024-04-25T15:49:46.454 INFO:tasks.workunit.client.0.smithi110.stdout:7/941: chown d1b/d29/d38/d55/d95/f9c 100 1 2024-04-25T15:49:46.454 INFO:tasks.workunit.client.0.smithi110.stdout:0/810: getdents d1/d23/d30/d33/d35/da1 0 2024-04-25T15:49:46.456 INFO:tasks.workunit.client.0.smithi110.stdout:8/965: write d14/d32/d36/db7/ff7 [2901216,27599] 0 2024-04-25T15:49:46.458 INFO:tasks.workunit.client.0.smithi110.stdout:9/801: dwrite d9/d13/f8e [0,4194304] 0 2024-04-25T15:49:46.458 INFO:tasks.workunit.client.0.smithi110.stdout:8/966: chown d14/d21/d26/d6a/d7e/da6/de8/l51 2 1 2024-04-25T15:49:46.460 INFO:tasks.workunit.client.0.smithi110.stdout:2/775: mkdir de/d40/d6e/d52/dfc 0 2024-04-25T15:49:46.462 INFO:tasks.workunit.client.1.smithi193.stdout:2/684: mknod d0/d42/d51/df1/c104 0 2024-04-25T15:49:46.462 INFO:tasks.workunit.client.1.smithi193.stdout:4/734: chown db/d13/d9d/ca6 358252 1 2024-04-25T15:49:46.462 INFO:tasks.workunit.client.1.smithi193.stdout:7/672: mknod d12/d31/d39/db0/cfb 0 2024-04-25T15:49:46.463 INFO:tasks.workunit.client.1.smithi193.stdout:5/703: dread d17/d18/d75/d85/d5d/dad/fb5 [0,4194304] 0 2024-04-25T15:49:46.464 INFO:tasks.workunit.client.1.smithi193.stdout:1/608: link dc/c39 dc/d4a/d50/d5c/d63/cd1 0 2024-04-25T15:49:46.464 INFO:tasks.workunit.client.1.smithi193.stdout:4/735: dread - db/d32/fe7 zero size 2024-04-25T15:49:46.465 INFO:tasks.workunit.client.1.smithi193.stdout:5/704: fsync d17/d18/d30/fd4 0 2024-04-25T15:49:46.470 INFO:tasks.workunit.client.1.smithi193.stdout:8/661: dread d2/d3/ff [0,4194304] 0 2024-04-25T15:49:46.471 INFO:tasks.workunit.client.1.smithi193.stdout:1/609: dwrite dc/f47 [0,4194304] 0 2024-04-25T15:49:46.471 INFO:tasks.workunit.client.1.smithi193.stdout:9/653: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/fcc x:0 0 0 2024-04-25T15:49:46.473 INFO:tasks.workunit.client.0.smithi110.stdout:1/992: creat d0/d14/d81/df8/f143 x:0 0 0 2024-04-25T15:49:46.474 INFO:tasks.workunit.client.0.smithi110.stdout:3/863: truncate d0/d16/dcb/fc8 1997089 0 2024-04-25T15:49:46.475 INFO:tasks.workunit.client.0.smithi110.stdout:9/802: creat d9/db/df/d2c/d3c/ffc x:0 0 0 2024-04-25T15:49:46.476 INFO:tasks.workunit.client.0.smithi110.stdout:4/819: write d13/d24/d2d/d44/d90/fc1 [4599378,36995] 0 2024-04-25T15:49:46.477 INFO:tasks.workunit.client.1.smithi193.stdout:3/704: write d8/d13/f64 [558237,70601] 0 2024-04-25T15:49:46.479 INFO:tasks.workunit.client.0.smithi110.stdout:7/942: write d1b/fb4 [559977,7100] 0 2024-04-25T15:49:46.480 INFO:tasks.workunit.client.1.smithi193.stdout:7/673: symlink d12/d1c/lfc 0 2024-04-25T15:49:46.481 INFO:tasks.workunit.client.0.smithi110.stdout:7/943: fsync d1b/d29/d87/d8e/f94 0 2024-04-25T15:49:46.485 INFO:tasks.workunit.client.0.smithi110.stdout:5/859: mkdir d4/d1d/d1f/d35/d3d/d52/d97/d119 0 2024-04-25T15:49:46.486 INFO:tasks.workunit.client.1.smithi193.stdout:0/701: rename d0/d49/d5/d7/dd/d16/d1a/fe3 to d0/d49/d5/d7/dd/d16/fef 0 2024-04-25T15:49:46.486 INFO:tasks.workunit.client.1.smithi193.stdout:8/662: write d2/fb4 [986115,105476] 0 2024-04-25T15:49:46.488 INFO:tasks.workunit.client.1.smithi193.stdout:1/610: write dc/d4a/d77/f7d [5359,67718] 0 2024-04-25T15:49:46.504 INFO:tasks.workunit.client.0.smithi110.stdout:5/860: dread d4/d1d/d1f/f47 [0,4194304] 0 2024-04-25T15:49:46.507 INFO:tasks.workunit.client.1.smithi193.stdout:0/702: dread d0/d49/d5/d7/dd/d16/d26/d2e/f4b [4194304,4194304] 0 2024-04-25T15:49:46.508 INFO:tasks.workunit.client.1.smithi193.stdout:0/703: chown d0/d49/d5/d7/dd/d16/f30 211918 1 2024-04-25T15:49:46.509 INFO:tasks.workunit.client.1.smithi193.stdout:0/704: truncate d0/d49/d5/d7/fe8 695059 0 2024-04-25T15:49:46.542 INFO:tasks.workunit.client.1.smithi193.stdout:6/657: sync 2024-04-25T15:49:46.568 INFO:tasks.workunit.client.0.smithi110.stdout:6/792: sync 2024-04-25T15:49:46.570 INFO:tasks.workunit.client.0.smithi110.stdout:1/993: rename d0/d14/d5f/d6f/f141 to d0/d14/d5f/d7d/f144 0 2024-04-25T15:49:46.570 INFO:tasks.workunit.client.0.smithi110.stdout:1/994: chown d0/d14/d5f/d7d/f67 1097481232 1 2024-04-25T15:49:46.571 INFO:tasks.workunit.client.0.smithi110.stdout:4/820: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d117 0 2024-04-25T15:49:46.579 INFO:tasks.workunit.client.0.smithi110.stdout:2/776: symlink de/d40/d6e/d52/df7/lfd 0 2024-04-25T15:49:46.581 INFO:tasks.workunit.client.0.smithi110.stdout:2/777: read de/d40/d6e/f9d [243610,110761] 0 2024-04-25T15:49:46.581 INFO:tasks.workunit.client.0.smithi110.stdout:8/967: dwrite f11 [0,4194304] 0 2024-04-25T15:49:46.582 INFO:tasks.workunit.client.0.smithi110.stdout:3/864: write d0/d2e/f39 [2046714,90193] 0 2024-04-25T15:49:46.584 INFO:tasks.workunit.client.0.smithi110.stdout:2/778: write de/d40/d5b/d5c/d5f/db2/dad/fdb [218975,99566] 0 2024-04-25T15:49:46.588 INFO:tasks.workunit.client.1.smithi193.stdout:8/663: symlink d2/d3/d27/d69/dae/d4a/ld4 0 2024-04-25T15:49:46.589 INFO:tasks.workunit.client.1.smithi193.stdout:8/664: readlink d2/d3/d27/l64 0 2024-04-25T15:49:46.590 INFO:tasks.workunit.client.0.smithi110.stdout:5/861: symlink d4/d2c/d83/l11a 0 2024-04-25T15:49:46.590 INFO:tasks.workunit.client.1.smithi193.stdout:1/611: rmdir dc/d4a/d50/d5c/d5d/d3d/d55/d8a/db7 39 2024-04-25T15:49:46.594 INFO:tasks.workunit.client.1.smithi193.stdout:9/654: fdatasync df/d2c/d2f/d4e/d1c/d30/d60/f85 0 2024-04-25T15:49:46.596 INFO:tasks.workunit.client.1.smithi193.stdout:7/674: link d12/d13/fac d12/d13/ffd 0 2024-04-25T15:49:46.599 INFO:tasks.workunit.client.1.smithi193.stdout:5/705: link d17/d53/fc1 d17/fef 0 2024-04-25T15:49:46.599 INFO:tasks.workunit.client.1.smithi193.stdout:9/655: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/f70 [0,4194304] 0 2024-04-25T15:49:46.599 INFO:tasks.workunit.client.0.smithi110.stdout:8/968: mknod d14/d21/d26/dab/c154 0 2024-04-25T15:49:46.602 INFO:tasks.workunit.client.0.smithi110.stdout:2/779: rmdir de/d40/d5b/d5c/d5f/db2 39 2024-04-25T15:49:46.606 INFO:tasks.workunit.client.1.smithi193.stdout:6/658: dread dd/f74 [0,4194304] 0 2024-04-25T15:49:46.607 INFO:tasks.workunit.client.1.smithi193.stdout:0/705: dwrite d0/d49/d6a/f97 [0,4194304] 0 2024-04-25T15:49:46.609 INFO:tasks.workunit.client.1.smithi193.stdout:7/675: symlink d12/d13/dcc/lfe 0 2024-04-25T15:49:46.609 INFO:tasks.workunit.client.1.smithi193.stdout:0/706: stat d0/d49/d5/d7/dd/d16/fef 0 2024-04-25T15:49:46.610 INFO:tasks.workunit.client.1.smithi193.stdout:1/612: dread dc/f4b [0,4194304] 0 2024-04-25T15:49:46.611 INFO:tasks.workunit.client.1.smithi193.stdout:7/676: dread - d12/d31/d37/d62/d6e/da5/fb6 zero size 2024-04-25T15:49:46.612 INFO:tasks.workunit.client.1.smithi193.stdout:4/736: rename db/d4b/d51/d54/c66 to db/d13/ced 0 2024-04-25T15:49:46.612 INFO:tasks.workunit.client.0.smithi110.stdout:0/811: link d1/d23/d30/d33/d5b/dbb/da8/fb6 d1/d23/f11e 0 2024-04-25T15:49:46.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:46 smithi110 ceph-mon[26344]: pgmap v40: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 76 MiB/s rd, 202 MiB/s wr, 401 op/s 2024-04-25T15:49:46.613 INFO:tasks.workunit.client.1.smithi193.stdout:1/613: write dc/d4a/d50/d5c/d5d/d3d/f83 [366436,29571] 0 2024-04-25T15:49:46.614 INFO:tasks.workunit.client.1.smithi193.stdout:5/706: mknod d17/d18/d75/d85/d5d/dad/cf0 0 2024-04-25T15:49:46.615 INFO:tasks.workunit.client.0.smithi110.stdout:8/969: rmdir d14/d102 39 2024-04-25T15:49:46.617 INFO:tasks.workunit.client.1.smithi193.stdout:9/656: unlink df/d2c/d2f/d4e/d69/cc9 0 2024-04-25T15:49:46.617 INFO:tasks.workunit.client.1.smithi193.stdout:5/707: truncate d17/d18/d75/d85/d5d/fb0 692757 0 2024-04-25T15:49:46.619 INFO:tasks.workunit.client.0.smithi110.stdout:6/793: write d1/d12/daf/fbb [3076663,15693] 0 2024-04-25T15:49:46.620 INFO:tasks.workunit.client.1.smithi193.stdout:5/708: truncate d17/d18/d30/d3b/fed 618284 0 2024-04-25T15:49:46.620 INFO:tasks.workunit.client.0.smithi110.stdout:6/794: chown d1/dd/d3c/d91/lab 65079 1 2024-04-25T15:49:46.623 INFO:tasks.workunit.client.0.smithi110.stdout:9/803: rename d9/d44/d5e/d80/dbb/dbe/c76 to d9/d44/d5e/cfd 0 2024-04-25T15:49:46.623 INFO:tasks.workunit.client.1.smithi193.stdout:8/665: dread d2/d18/db1/fb3 [0,4194304] 0 2024-04-25T15:49:46.624 INFO:tasks.workunit.client.1.smithi193.stdout:8/666: write d2/d3/d3a/f4c [637836,87840] 0 2024-04-25T15:49:46.625 INFO:tasks.workunit.client.0.smithi110.stdout:4/821: creat d13/d24/d2d/d44/d65/d99/daf/d56/f118 x:0 0 0 2024-04-25T15:49:46.626 INFO:tasks.workunit.client.1.smithi193.stdout:4/737: unlink db/fbb 0 2024-04-25T15:49:46.626 INFO:tasks.workunit.client.0.smithi110.stdout:4/822: write d13/d24/d2d/d36/d51/f8c [1474618,10401] 0 2024-04-25T15:49:46.627 INFO:tasks.workunit.client.0.smithi110.stdout:4/823: chown d13/d24/d2d/d55/lc6 49093107 1 2024-04-25T15:49:46.627 INFO:tasks.workunit.client.0.smithi110.stdout:0/812: symlink d1/d23/d30/d33/d35/l11f 0 2024-04-25T15:49:46.629 INFO:tasks.workunit.client.0.smithi110.stdout:3/865: creat d0/d16/dcb/d2f/f129 x:0 0 0 2024-04-25T15:49:46.629 INFO:tasks.workunit.client.0.smithi110.stdout:3/866: readlink d0/d16/l4b 0 2024-04-25T15:49:46.630 INFO:tasks.workunit.client.1.smithi193.stdout:2/685: rename d0/df/ccf to d0/d42/d51/d4f/c105 0 2024-04-25T15:49:46.631 INFO:tasks.workunit.client.1.smithi193.stdout:9/657: symlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/lcd 0 2024-04-25T15:49:46.634 INFO:tasks.workunit.client.0.smithi110.stdout:2/780: rename de/d40/d5b/l91 to de/d40/dcd/ddf/lfe 0 2024-04-25T15:49:46.634 INFO:tasks.workunit.client.1.smithi193.stdout:0/707: symlink d0/d49/d5/d7/dd/lf0 0 2024-04-25T15:49:46.635 INFO:tasks.workunit.client.0.smithi110.stdout:4/824: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/f118 [0,4194304] 0 2024-04-25T15:49:46.635 INFO:tasks.workunit.client.1.smithi193.stdout:1/614: dread dc/f18 [0,4194304] 0 2024-04-25T15:49:46.639 INFO:tasks.workunit.client.1.smithi193.stdout:9/658: dwrite df/d2c/d2f/d5c/d7e/fc1 [0,4194304] 0 2024-04-25T15:49:46.640 INFO:tasks.workunit.client.1.smithi193.stdout:4/738: mkdir db/d32/d40/dee 0 2024-04-25T15:49:46.641 INFO:tasks.workunit.client.1.smithi193.stdout:7/677: mkdir d12/d13/dc4/dfa/dff 0 2024-04-25T15:49:46.641 INFO:tasks.workunit.client.1.smithi193.stdout:4/739: chown db/d4b 128646644 1 2024-04-25T15:49:46.643 INFO:tasks.workunit.client.0.smithi110.stdout:2/781: dwrite de/d40/d6e/d50/fa4 [4194304,4194304] 0 2024-04-25T15:49:46.645 INFO:tasks.workunit.client.1.smithi193.stdout:3/705: rename d8/d13/d14/d27/d6d to d8/d13/d1f/d69/dcd/dd9/dde 0 2024-04-25T15:49:46.645 INFO:tasks.workunit.client.1.smithi193.stdout:0/708: dwrite d0/d49/d5/d7/dd/d78/fb3 [0,4194304] 0 2024-04-25T15:49:46.646 INFO:tasks.workunit.client.1.smithi193.stdout:4/740: dread - db/d13/d1e/d1f/d56/d69/d70/d8a/fb8 zero size 2024-04-25T15:49:46.647 INFO:tasks.workunit.client.1.smithi193.stdout:3/706: chown d8/d13/d4b/lc5 46 1 2024-04-25T15:49:46.647 INFO:tasks.workunit.client.1.smithi193.stdout:2/686: creat d0/d37/d25/d5f/f106 x:0 0 0 2024-04-25T15:49:46.649 INFO:tasks.workunit.client.1.smithi193.stdout:3/707: write d8/d30/f42 [4579270,24343] 0 2024-04-25T15:49:46.658 INFO:tasks.workunit.client.0.smithi110.stdout:9/804: rename d9/f8c to d9/db/d7d/ffe 0 2024-04-25T15:49:46.660 INFO:tasks.workunit.client.0.smithi110.stdout:4/825: rmdir d13/d24/d2d/d36 39 2024-04-25T15:49:46.662 INFO:tasks.workunit.client.1.smithi193.stdout:1/615: truncate dc/f40 1165477 0 2024-04-25T15:49:46.662 INFO:tasks.workunit.client.1.smithi193.stdout:1/616: dread - dc/d4a/d50/fa7 zero size 2024-04-25T15:49:46.663 INFO:tasks.workunit.client.0.smithi110.stdout:0/813: creat d1/d23/d30/d33/d5b/d78/ddb/d115/f120 x:0 0 0 2024-04-25T15:49:46.666 INFO:tasks.workunit.client.0.smithi110.stdout:5/862: link d4/d1d/d1f/d56/c71 d4/d15/d51/d117/c11b 0 2024-04-25T15:49:46.667 INFO:tasks.workunit.client.0.smithi110.stdout:5/863: readlink d4/d15/d51/d90/de8/lea 0 2024-04-25T15:49:46.667 INFO:tasks.workunit.client.1.smithi193.stdout:7/678: mkdir d12/d13/dcc/d100 0 2024-04-25T15:49:46.668 INFO:tasks.workunit.client.1.smithi193.stdout:7/679: chown d12/d1c/f7e 202551614 1 2024-04-25T15:49:46.671 INFO:tasks.workunit.client.1.smithi193.stdout:8/667: write d2/d3/d27/d69/dae/f5a [158213,95234] 0 2024-04-25T15:49:46.673 INFO:tasks.workunit.client.0.smithi110.stdout:9/805: symlink d9/d44/d5e/d80/lff 0 2024-04-25T15:49:46.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/668: fdatasync d2/d3/d27/d69/dae/f5a 0 2024-04-25T15:49:46.676 INFO:tasks.workunit.client.1.smithi193.stdout:0/709: dread d0/d49/d5/d7/dd/d16/f41 [0,4194304] 0 2024-04-25T15:49:46.677 INFO:tasks.workunit.client.1.smithi193.stdout:0/710: write d0/d49/faa [776195,30773] 0 2024-04-25T15:49:46.679 INFO:tasks.workunit.client.1.smithi193.stdout:4/741: mkdir db/d13/d1e/d1f/d56/d69/dad/def 0 2024-04-25T15:49:46.680 INFO:tasks.workunit.client.0.smithi110.stdout:6/795: write d1/dd/f64 [1085330,23787] 0 2024-04-25T15:49:46.689 INFO:tasks.workunit.client.0.smithi110.stdout:3/867: write d0/f1e [4271580,110944] 0 2024-04-25T15:49:46.691 INFO:tasks.workunit.client.1.smithi193.stdout:7/680: fsync d12/d31/d39/d60/fa8 0 2024-04-25T15:49:46.695 INFO:tasks.workunit.client.0.smithi110.stdout:2/782: dwrite de/d40/d5b/d5c/d5f/f7a [0,4194304] 0 2024-04-25T15:49:46.695 INFO:tasks.workunit.client.1.smithi193.stdout:3/708: write d8/d13/d1f/f78 [657271,117318] 0 2024-04-25T15:49:46.701 INFO:tasks.workunit.client.1.smithi193.stdout:9/659: truncate df/d2c/d2f/d4e/d1c/f76 3796896 0 2024-04-25T15:49:46.701 INFO:tasks.workunit.client.1.smithi193.stdout:3/709: dwrite d8/d13/d1f/d69/d77/fc9 [0,4194304] 0 2024-04-25T15:49:46.702 INFO:tasks.workunit.client.0.smithi110.stdout:5/864: dwrite d4/d1d/d1f/d35/d3d/d52/d97/fb1 [0,4194304] 0 2024-04-25T15:49:46.703 INFO:tasks.workunit.client.0.smithi110.stdout:5/865: chown d4/d15/c79 4348 1 2024-04-25T15:49:46.704 INFO:tasks.workunit.client.0.smithi110.stdout:4/826: rmdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d10d/d114 0 2024-04-25T15:49:46.705 INFO:tasks.workunit.client.0.smithi110.stdout:3/868: read d0/d2e/f87 [3729630,30180] 0 2024-04-25T15:49:46.708 INFO:tasks.workunit.client.0.smithi110.stdout:5/866: dwrite d4/fb2 [0,4194304] 0 2024-04-25T15:49:46.711 INFO:tasks.workunit.client.0.smithi110.stdout:5/867: truncate d4/d1d/d1f/d35/d3d/dc8/f115 4464225 0 2024-04-25T15:49:46.778 INFO:tasks.workunit.client.0.smithi110.stdout:0/814: creat d1/d23/d30/d33/f121 x:0 0 0 2024-04-25T15:49:46.779 INFO:tasks.workunit.client.1.smithi193.stdout:2/687: readlink d0/d37/d25/d5f/d3a/ldc 0 2024-04-25T15:49:46.780 INFO:tasks.workunit.client.0.smithi110.stdout:2/783: fsync de/d40/d5b/d5c/fa5 0 2024-04-25T15:49:46.780 INFO:tasks.workunit.client.1.smithi193.stdout:2/688: fsync d0/d37/f67 0 2024-04-25T15:49:46.780 INFO:tasks.workunit.client.0.smithi110.stdout:2/784: readlink de/d40/d70/lb9 0 2024-04-25T15:49:46.780 INFO:tasks.workunit.client.1.smithi193.stdout:5/709: link d17/f1f d17/d18/ff1 0 2024-04-25T15:49:46.782 INFO:tasks.workunit.client.0.smithi110.stdout:9/806: link d9/d13/d5b/ff2 d9/d13/f100 0 2024-04-25T15:49:46.783 INFO:tasks.workunit.client.0.smithi110.stdout:9/807: write f6 [2643336,41216] 0 2024-04-25T15:49:46.786 INFO:tasks.workunit.client.1.smithi193.stdout:2/689: dwrite d0/d42/d51/df1/fea [0,4194304] 0 2024-04-25T15:49:46.786 INFO:tasks.workunit.client.1.smithi193.stdout:1/617: mkdir dc/d4a/d4d/d7a/dd0/dd2 0 2024-04-25T15:49:46.790 INFO:tasks.workunit.client.1.smithi193.stdout:7/681: mkdir d12/d1c/dd6/d101 0 2024-04-25T15:49:46.791 INFO:tasks.workunit.client.1.smithi193.stdout:2/690: dread - d0/d37/d25/d5f/d3a/d6d/d6e/db9/fe8 zero size 2024-04-25T15:49:46.792 INFO:tasks.workunit.client.1.smithi193.stdout:7/682: truncate d12/d13/dcc/fe4 922067 0 2024-04-25T15:49:46.834 INFO:tasks.workunit.client.1.smithi193.stdout:6/659: rename dd/d2f/d57/f90 to dd/d12/d19/d1c/d40/fcb 0 2024-04-25T15:49:46.835 INFO:tasks.workunit.client.0.smithi110.stdout:6/796: rename d1/dd/d3c/lfa to d1/db/d24/l115 0 2024-04-25T15:49:46.838 INFO:tasks.workunit.client.0.smithi110.stdout:0/815: truncate d1/f8 56130 0 2024-04-25T15:49:46.839 INFO:tasks.workunit.client.1.smithi193.stdout:9/660: unlink df/d2d/c50 0 2024-04-25T15:49:46.840 INFO:tasks.workunit.client.1.smithi193.stdout:9/661: write df/d2c/d2f/d4e/fa0 [4545090,52452] 0 2024-04-25T15:49:46.841 INFO:tasks.workunit.client.1.smithi193.stdout:8/669: mknod d2/cd5 0 2024-04-25T15:49:46.852 INFO:tasks.workunit.client.1.smithi193.stdout:0/711: write d0/d49/d5/d7/dd/d16/d26/d5e/fc8 [3167426,83114] 0 2024-04-25T15:49:46.855 INFO:tasks.workunit.client.1.smithi193.stdout:4/742: truncate db/d13/d1e/d1f/d56/d57/fa4 4071958 0 2024-04-25T15:49:46.856 INFO:tasks.workunit.client.1.smithi193.stdout:4/743: dread - db/d13/d1e/d1f/d22/fcc zero size 2024-04-25T15:49:46.857 INFO:tasks.workunit.client.1.smithi193.stdout:4/744: dread - db/d32/d45/fcd zero size 2024-04-25T15:49:46.858 INFO:tasks.workunit.client.0.smithi110.stdout:4/827: write d13/d24/d2d/d36/d51/f68 [3112293,69743] 0 2024-04-25T15:49:46.859 INFO:tasks.workunit.client.0.smithi110.stdout:6/797: creat d1/db/d24/daa/f116 x:0 0 0 2024-04-25T15:49:46.861 INFO:tasks.workunit.client.0.smithi110.stdout:0/816: fdatasync d1/d6c/f94 0 2024-04-25T15:49:46.863 INFO:tasks.workunit.client.1.smithi193.stdout:4/745: dwrite db/d4b/fa3 [4194304,4194304] 0 2024-04-25T15:49:46.863 INFO:tasks.workunit.client.1.smithi193.stdout:4/746: chown db/f8d 131 1 2024-04-25T15:49:46.866 INFO:tasks.workunit.client.0.smithi110.stdout:3/869: dwrite d0/d16/dcb/f22 [4194304,4194304] 0 2024-04-25T15:49:46.869 INFO:tasks.workunit.client.0.smithi110.stdout:9/808: truncate d9/d13/fd2 1253762 0 2024-04-25T15:49:46.877 INFO:tasks.workunit.client.0.smithi110.stdout:6/798: readlink d1/dd/db5/df4/lff 0 2024-04-25T15:49:46.879 INFO:tasks.workunit.client.0.smithi110.stdout:6/799: truncate d1/db/d22/d90/d9e/ffd 42226 0 2024-04-25T15:49:46.880 INFO:tasks.workunit.client.0.smithi110.stdout:6/800: chown d1/db/d22/d90/cb7 46676360 1 2024-04-25T15:49:46.887 INFO:tasks.workunit.client.0.smithi110.stdout:3/870: rename d0/d16/dcb/d50/c64 to d0/d16/c12a 0 2024-04-25T15:49:46.889 INFO:tasks.workunit.client.0.smithi110.stdout:9/809: truncate d9/d44/f7f 316127 0 2024-04-25T15:49:46.891 INFO:tasks.workunit.client.0.smithi110.stdout:6/801: dread d1/dd/d3c/d91/f5a [0,4194304] 0 2024-04-25T15:49:46.893 INFO:tasks.workunit.client.0.smithi110.stdout:5/868: getdents d4/d15 0 2024-04-25T15:49:46.894 INFO:tasks.workunit.client.0.smithi110.stdout:4/828: symlink d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d113/l119 0 2024-04-25T15:49:46.898 INFO:tasks.workunit.client.0.smithi110.stdout:2/785: rmdir de/d40/d5b/d5c/d5f/db2/dad/dbd/dce/de7 0 2024-04-25T15:49:46.901 INFO:tasks.workunit.client.0.smithi110.stdout:3/871: mkdir d0/d2e/d88/dbf/da5/d12b 0 2024-04-25T15:49:46.902 INFO:tasks.workunit.client.0.smithi110.stdout:3/872: write d0/d2e/f39 [441602,118475] 0 2024-04-25T15:49:46.903 INFO:tasks.workunit.client.1.smithi193.stdout:1/618: mkdir dc/d4a/d77/dd3 0 2024-04-25T15:49:46.903 INFO:tasks.workunit.client.0.smithi110.stdout:0/817: write d1/d23/d30/d33/d5b/dbb/da8/fb6 [6436305,124560] 0 2024-04-25T15:49:46.904 INFO:tasks.workunit.client.0.smithi110.stdout:9/810: mknod d9/db/d7d/c101 0 2024-04-25T15:49:46.905 INFO:tasks.workunit.client.0.smithi110.stdout:6/802: creat d1/dd/d3c/d91/d8d/dce/f117 x:0 0 0 2024-04-25T15:49:46.907 INFO:tasks.workunit.client.1.smithi193.stdout:2/691: symlink d0/d37/dbb/df8/l107 0 2024-04-25T15:49:46.908 INFO:tasks.workunit.client.1.smithi193.stdout:2/692: write d0/d42/d51/f40 [393372,63891] 0 2024-04-25T15:49:46.908 INFO:tasks.workunit.client.0.smithi110.stdout:5/869: mkdir d4/d1d/d1f/d35/d3d/d52/d75/d11c 0 2024-04-25T15:49:46.909 INFO:tasks.workunit.client.1.smithi193.stdout:7/683: creat d12/d1c/d96/f102 x:0 0 0 2024-04-25T15:49:46.911 INFO:tasks.workunit.client.1.smithi193.stdout:3/710: rename d8/d13/d1f/d69/d77/dd1/fd5 to d8/d13/d1f/d9f/fdf 0 2024-04-25T15:49:46.912 INFO:tasks.workunit.client.0.smithi110.stdout:2/786: mkdir de/d40/d6e/d52/d6d/d7d/dff 0 2024-04-25T15:49:46.913 INFO:tasks.workunit.client.0.smithi110.stdout:2/787: chown de/d40/d70/dfa 3903 1 2024-04-25T15:49:46.917 INFO:tasks.workunit.client.0.smithi110.stdout:0/818: rmdir d1/d23/dc5 39 2024-04-25T15:49:46.919 INFO:tasks.workunit.client.1.smithi193.stdout:8/670: dread - d2/d3/d27/d69/dae/d4a/d63/da2/fb9 zero size 2024-04-25T15:49:46.924 INFO:tasks.workunit.client.0.smithi110.stdout:5/870: symlink d4/d1d/d1f/d35/d3d/d93/l11d 0 2024-04-25T15:49:46.924 INFO:tasks.workunit.client.0.smithi110.stdout:5/871: chown d4/d1d/d1f/l33 9985055 1 2024-04-25T15:49:46.926 INFO:tasks.workunit.client.0.smithi110.stdout:2/788: creat de/d40/d4f/d9f/f100 x:0 0 0 2024-04-25T15:49:46.927 INFO:tasks.workunit.client.1.smithi193.stdout:4/747: symlink db/d13/d1e/d1f/d56/d69/dad/lf0 0 2024-04-25T15:49:46.928 INFO:tasks.workunit.client.1.smithi193.stdout:4/748: write db/d13/dcb/fdb [64506,47021] 0 2024-04-25T15:49:46.929 INFO:tasks.workunit.client.0.smithi110.stdout:0/819: rename d1/d6c/d116/d77/fcc to d1/d23/d9e/f122 0 2024-04-25T15:49:46.932 INFO:tasks.workunit.client.1.smithi193.stdout:1/619: write dc/d3c/f79 [240601,103277] 0 2024-04-25T15:49:46.938 INFO:tasks.workunit.client.1.smithi193.stdout:7/684: truncate d12/f2c 468067 0 2024-04-25T15:49:46.938 INFO:tasks.workunit.client.1.smithi193.stdout:7/685: chown la 0 1 2024-04-25T15:49:46.939 INFO:tasks.workunit.client.1.smithi193.stdout:7/686: read - d12/d1c/d45/fec zero size 2024-04-25T15:49:46.940 INFO:tasks.workunit.client.0.smithi110.stdout:0/820: truncate d1/d23/d30/d33/d5b/d83/f84 4012192 0 2024-04-25T15:49:46.942 INFO:tasks.workunit.client.1.smithi193.stdout:6/660: rename dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f92 to dd/d2f/d57/daf/fcc 0 2024-04-25T15:49:46.944 INFO:tasks.workunit.client.0.smithi110.stdout:5/872: mkdir d4/d1d/d1f/d35/d3d/d11e 0 2024-04-25T15:49:46.944 INFO:tasks.workunit.client.0.smithi110.stdout:9/811: write d9/d44/d5e/fc1 [1083027,21700] 0 2024-04-25T15:49:46.945 INFO:tasks.workunit.client.0.smithi110.stdout:6/803: write d1/fdf [993052,119824] 0 2024-04-25T15:49:46.945 INFO:tasks.workunit.client.1.smithi193.stdout:7/687: dread d12/d31/d37/d92/dd1/fd5 [0,4194304] 0 2024-04-25T15:49:46.948 INFO:tasks.workunit.client.0.smithi110.stdout:3/873: dwrite d0/d2e/fc6 [0,4194304] 0 2024-04-25T15:49:46.949 INFO:tasks.workunit.client.1.smithi193.stdout:9/662: dwrite df/d2c/d2f/d4e/f3a [4194304,4194304] 0 2024-04-25T15:49:46.949 INFO:tasks.workunit.client.0.smithi110.stdout:3/874: readlink d0/d7/l4a 0 2024-04-25T15:49:46.953 INFO:tasks.workunit.client.1.smithi193.stdout:3/711: dread d8/d30/f40 [4194304,4194304] 0 2024-04-25T15:49:46.953 INFO:tasks.workunit.client.1.smithi193.stdout:4/749: chown db/d4b/d51/cae 26 1 2024-04-25T15:49:46.956 INFO:tasks.workunit.client.0.smithi110.stdout:0/821: unlink d1/d23/d30/d33/d35/f36 0 2024-04-25T15:49:46.956 INFO:tasks.workunit.client.0.smithi110.stdout:9/812: dread d9/d13/d5b/d61/db3/dd3/ddb/fec [0,4194304] 0 2024-04-25T15:49:46.959 INFO:tasks.workunit.client.0.smithi110.stdout:9/813: chown d9/d13/d5b/ld5 546478 1 2024-04-25T15:49:46.959 INFO:tasks.workunit.client.0.smithi110.stdout:0/822: chown d1/d23/d30/d33/d5b/d78/d107/dee 166586 1 2024-04-25T15:49:46.959 INFO:tasks.workunit.client.0.smithi110.stdout:3/875: dwrite d0/d16/dcb/d2f/d4c/fa0 [0,4194304] 0 2024-04-25T15:49:46.960 INFO:tasks.workunit.client.1.smithi193.stdout:1/620: symlink dc/d4a/d50/d7c/d7e/ld4 0 2024-04-25T15:49:46.971 INFO:tasks.workunit.client.0.smithi110.stdout:6/804: rename d1/db/d24/fe8 to d1/db/d22/d90/d76/f118 0 2024-04-25T15:49:46.977 INFO:tasks.workunit.client.1.smithi193.stdout:8/671: rename d2/d3/d27/d69/dae/d4a/d63/da2/dad to d2/d3/d27/d69/dae/dd6 0 2024-04-25T15:49:46.977 INFO:tasks.workunit.client.0.smithi110.stdout:2/789: dwrite de/d40/d5b/d5c/d5f/db2/fcb [0,4194304] 0 2024-04-25T15:49:46.979 INFO:tasks.workunit.client.0.smithi110.stdout:2/790: chown de/d40/d6e/d50/fa4 1955116 1 2024-04-25T15:49:46.979 INFO:tasks.workunit.client.0.smithi110.stdout:3/876: chown d0/lab 1954244 1 2024-04-25T15:49:46.981 INFO:tasks.workunit.client.1.smithi193.stdout:0/712: creat d0/d49/d5/d7/dd/d16/ff1 x:0 0 0 2024-04-25T15:49:46.986 INFO:tasks.workunit.client.0.smithi110.stdout:2/791: dread de/d40/d5b/d5c/d5f/f81 [0,4194304] 0 2024-04-25T15:49:46.986 INFO:tasks.workunit.client.1.smithi193.stdout:5/710: getdents d17/d18/d75/d85/d5d/dad 0 2024-04-25T15:49:46.988 INFO:tasks.workunit.client.1.smithi193.stdout:7/688: dread d12/d31/d37/d62/d6e/d79/f4a [0,4194304] 0 2024-04-25T15:49:46.988 INFO:tasks.workunit.client.1.smithi193.stdout:5/711: write d17/d18/d30/d38/d71/d96/dc4/f70 [826000,92774] 0 2024-04-25T15:49:46.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:46 smithi193 ceph-mon[33747]: pgmap v40: 65 pgs: 65 active+clean; 3.1 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 76 MiB/s rd, 202 MiB/s wr, 401 op/s 2024-04-25T15:49:46.989 INFO:tasks.workunit.client.0.smithi110.stdout:3/877: dwrite d0/d16/d8c/fea [4194304,4194304] 0 2024-04-25T15:49:46.989 INFO:tasks.workunit.client.1.smithi193.stdout:9/663: mknod df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/cce 0 2024-04-25T15:49:46.990 INFO:tasks.workunit.client.1.smithi193.stdout:5/712: chown d17/d18/d30/d38/d71/d96/fc7 22055267 1 2024-04-25T15:49:46.990 INFO:tasks.workunit.client.0.smithi110.stdout:2/792: fsync de/d40/d5b/d5c/d5f/db2/f9c 0 2024-04-25T15:49:46.992 INFO:tasks.workunit.client.1.smithi193.stdout:3/712: creat d8/d30/d45/fe0 x:0 0 0 2024-04-25T15:49:46.992 INFO:tasks.workunit.client.1.smithi193.stdout:5/713: truncate d17/d18/d75/d85/d5d/dad/fb5 1641681 0 2024-04-25T15:49:46.993 INFO:tasks.workunit.client.0.smithi110.stdout:5/873: dwrite d4/d1d/d1f/d35/d3d/f60 [0,4194304] 0 2024-04-25T15:49:46.995 INFO:tasks.workunit.client.1.smithi193.stdout:4/750: fdatasync db/d13/d1e/d1f/d22/f5a 0 2024-04-25T15:49:46.997 INFO:tasks.workunit.client.0.smithi110.stdout:6/805: unlink d1/db/d22/d33/de4/fed 0 2024-04-25T15:49:46.997 INFO:tasks.workunit.client.0.smithi110.stdout:6/806: readlink d1/d12/l5f 0 2024-04-25T15:49:46.998 INFO:tasks.workunit.client.1.smithi193.stdout:2/693: creat d0/d37/d25/d5f/f108 x:0 0 0 2024-04-25T15:49:47.004 INFO:tasks.workunit.client.0.smithi110.stdout:0/823: mknod d1/d23/d30/d33/d5b/d78/ddb/de7/c123 0 2024-04-25T15:49:47.005 INFO:tasks.workunit.client.1.smithi193.stdout:8/672: truncate d2/d3/d27/d69/dae/d29/fac 888700 0 2024-04-25T15:49:47.009 INFO:tasks.workunit.client.1.smithi193.stdout:0/713: read d0/d49/d5/d7/dd/d46/d48/f57 [180030,116582] 0 2024-04-25T15:49:47.010 INFO:tasks.workunit.client.1.smithi193.stdout:7/689: creat d12/d13/f103 x:0 0 0 2024-04-25T15:49:47.014 INFO:tasks.workunit.client.1.smithi193.stdout:9/664: unlink df/d2d/f31 0 2024-04-25T15:49:47.015 INFO:tasks.workunit.client.0.smithi110.stdout:5/874: symlink d4/d1d/d1f/d35/d7f/d92/d9b/dcb/l11f 0 2024-04-25T15:49:47.016 INFO:tasks.workunit.client.1.smithi193.stdout:5/714: write d17/f97 [913912,77809] 0 2024-04-25T15:49:47.017 INFO:tasks.workunit.client.1.smithi193.stdout:5/715: truncate d17/d18/d30/d38/dab/fe4 852336 0 2024-04-25T15:49:47.018 INFO:tasks.workunit.client.1.smithi193.stdout:3/713: mkdir d8/d30/d72/de1 0 2024-04-25T15:49:47.022 INFO:tasks.workunit.client.1.smithi193.stdout:1/621: creat dc/d4a/d50/d5c/d5d/d3d/d55/d9c/fd5 x:0 0 0 2024-04-25T15:49:47.023 INFO:tasks.workunit.client.0.smithi110.stdout:0/824: symlink d1/d23/d30/d33/d5b/d78/ddb/de7/l124 0 2024-04-25T15:49:47.025 INFO:tasks.workunit.client.0.smithi110.stdout:2/793: fsync de/d40/f8f 0 2024-04-25T15:49:47.027 INFO:tasks.workunit.client.1.smithi193.stdout:8/673: fsync d2/d3/f80 0 2024-04-25T15:49:47.028 INFO:tasks.workunit.client.1.smithi193.stdout:8/674: stat d2/d3/d20/f8b 0 2024-04-25T15:49:47.029 INFO:tasks.workunit.client.0.smithi110.stdout:5/875: symlink d4/d15/d51/l120 0 2024-04-25T15:49:47.033 INFO:tasks.workunit.client.0.smithi110.stdout:9/814: rename d9/lb2 to d9/db/df/d2c/l102 0 2024-04-25T15:49:47.034 INFO:tasks.workunit.client.1.smithi193.stdout:7/690: truncate d12/d31/d37/d62/d6e/f8c 1188243 0 2024-04-25T15:49:47.036 INFO:tasks.workunit.client.0.smithi110.stdout:3/878: write d0/d7/dd0/ddf/fe3 [722680,43971] 0 2024-04-25T15:49:47.036 INFO:tasks.workunit.client.0.smithi110.stdout:0/825: read - d1/d47/ff2 zero size 2024-04-25T15:49:47.039 INFO:tasks.workunit.client.1.smithi193.stdout:5/716: symlink d17/d18/d75/d85/d5d/lf2 0 2024-04-25T15:49:47.039 INFO:tasks.workunit.client.1.smithi193.stdout:5/717: fsync d17/f97 0 2024-04-25T15:49:47.041 INFO:tasks.workunit.client.0.smithi110.stdout:6/807: write d1/d79/f7a [400336,110228] 0 2024-04-25T15:49:47.042 INFO:tasks.workunit.client.0.smithi110.stdout:2/794: read de/d40/d70/f84 [143694,104479] 0 2024-04-25T15:49:47.045 INFO:tasks.workunit.client.0.smithi110.stdout:5/876: creat d4/d1d/d1f/d35/d3d/d52/d75/dfa/f121 x:0 0 0 2024-04-25T15:49:47.050 INFO:tasks.workunit.client.0.smithi110.stdout:0/826: dread d1/d47/f95 [0,4194304] 0 2024-04-25T15:49:47.052 INFO:tasks.workunit.client.1.smithi193.stdout:6/661: rename dd/d12/d2d/c9e to dd/d12/d19/d1c/d40/ccd 0 2024-04-25T15:49:47.058 INFO:tasks.workunit.client.1.smithi193.stdout:0/714: dwrite d0/d49/d5/f2d [0,4194304] 0 2024-04-25T15:49:47.059 INFO:tasks.workunit.client.1.smithi193.stdout:8/675: creat d2/d3/d27/fd7 x:0 0 0 2024-04-25T15:49:47.062 INFO:tasks.workunit.client.1.smithi193.stdout:7/691: chown d12/d13/f43 96 1 2024-04-25T15:49:47.064 INFO:tasks.workunit.client.0.smithi110.stdout:0/827: mkdir d1/d23/d30/d33/d35/da1/db4/db9/d125 0 2024-04-25T15:49:47.065 INFO:tasks.workunit.client.1.smithi193.stdout:2/694: dwrite d0/d37/d25/d5f/d3a/d6d/d6e/db9/fe8 [0,4194304] 0 2024-04-25T15:49:47.067 INFO:tasks.workunit.client.0.smithi110.stdout:9/815: dwrite d9/f5f [0,4194304] 0 2024-04-25T15:49:47.070 INFO:tasks.workunit.client.0.smithi110.stdout:3/879: creat d0/d2e/d88/dbf/da5/d12b/f12c x:0 0 0 2024-04-25T15:49:47.071 INFO:tasks.workunit.client.1.smithi193.stdout:1/622: mknod dc/d4a/d4d/d7a/d85/cd6 0 2024-04-25T15:49:47.073 INFO:tasks.workunit.client.0.smithi110.stdout:5/877: mknod d4/d1d/d1f/c122 0 2024-04-25T15:49:47.073 INFO:tasks.workunit.client.1.smithi193.stdout:1/623: write dc/d4a/d4d/d7a/f96 [4335947,26753] 0 2024-04-25T15:49:47.075 INFO:tasks.workunit.client.0.smithi110.stdout:0/828: unlink d1/l4f 0 2024-04-25T15:49:47.080 INFO:tasks.workunit.client.0.smithi110.stdout:6/808: rename d1/d79 to d1/db/d119 0 2024-04-25T15:49:47.084 INFO:tasks.workunit.client.0.smithi110.stdout:0/829: mknod d1/d47/d113/c126 0 2024-04-25T15:49:47.085 INFO:tasks.workunit.client.1.smithi193.stdout:0/715: symlink d0/d49/d5/d7/dd/d46/d48/d62/lf2 0 2024-04-25T15:49:47.086 INFO:tasks.workunit.client.1.smithi193.stdout:8/676: truncate d2/da5/fa7 4305819 0 2024-04-25T15:49:47.088 INFO:tasks.workunit.client.0.smithi110.stdout:2/795: rename de/d40/d6e/d50/lc7 to de/d40/d5b/d5c/d5f/db2/dad/dbd/dce/l101 0 2024-04-25T15:49:47.091 INFO:tasks.workunit.client.0.smithi110.stdout:5/878: mkdir d4/d1d/d1f/d35/d3d/d52/d75/d113/d123 0 2024-04-25T15:49:47.094 INFO:tasks.workunit.client.0.smithi110.stdout:9/816: dread d9/d13/f1a [4194304,4194304] 0 2024-04-25T15:49:47.094 INFO:tasks.workunit.client.0.smithi110.stdout:3/880: dread d0/d16/f18 [0,4194304] 0 2024-04-25T15:49:47.097 INFO:tasks.workunit.client.1.smithi193.stdout:6/662: dread dd/d12/d19/d1c/f22 [0,4194304] 0 2024-04-25T15:49:47.100 INFO:tasks.workunit.client.0.smithi110.stdout:0/830: unlink d1/d6c/ff1 0 2024-04-25T15:49:47.102 INFO:tasks.workunit.client.1.smithi193.stdout:2/695: creat d0/d37/d4d/f109 x:0 0 0 2024-04-25T15:49:47.104 INFO:tasks.workunit.client.0.smithi110.stdout:2/796: unlink de/d40/d6e/f39 0 2024-04-25T15:49:47.105 INFO:tasks.workunit.client.0.smithi110.stdout:9/817: dread d9/d13/d7c/fb7 [0,4194304] 0 2024-04-25T15:49:47.109 INFO:tasks.workunit.client.1.smithi193.stdout:6/663: dread dd/d12/d19/d1c/d40/d3c/d3f/f61 [0,4194304] 0 2024-04-25T15:49:47.109 INFO:tasks.workunit.client.1.smithi193.stdout:6/664: chown dd/d2f 5275791 1 2024-04-25T15:49:47.110 INFO:tasks.workunit.client.1.smithi193.stdout:6/665: chown dd/f59 299170960 1 2024-04-25T15:49:47.110 INFO:tasks.workunit.client.1.smithi193.stdout:6/666: chown dd/d12/f1e 24943 1 2024-04-25T15:49:47.115 INFO:tasks.workunit.client.1.smithi193.stdout:6/667: dwrite dd/d12/d19/d1c/f20 [0,4194304] 0 2024-04-25T15:49:47.116 INFO:tasks.workunit.client.0.smithi110.stdout:0/831: unlink d1/d20/le2 0 2024-04-25T15:49:47.117 INFO:tasks.workunit.client.0.smithi110.stdout:6/809: dwrite d1/dd/d3c/d91/d8d/dce/d102/d75/d82/fa6 [0,4194304] 0 2024-04-25T15:49:47.120 INFO:tasks.workunit.client.0.smithi110.stdout:6/810: chown d1/db/d24/daa/cd7 807 1 2024-04-25T15:49:47.124 INFO:tasks.workunit.client.1.smithi193.stdout:8/677: mkdir d2/d3/d27/d69/dae/dd8 0 2024-04-25T15:49:47.125 INFO:tasks.workunit.client.1.smithi193.stdout:3/714: sync 2024-04-25T15:49:47.129 INFO:tasks.workunit.client.1.smithi193.stdout:5/718: truncate d17/d18/d30/d38/d71/d96/fc7 2729650 0 2024-04-25T15:49:47.130 INFO:tasks.workunit.client.0.smithi110.stdout:3/881: dread d0/d16/dcb/d2f/d4c/d63/fc0 [0,4194304] 0 2024-04-25T15:49:47.131 INFO:tasks.workunit.client.0.smithi110.stdout:5/879: dwrite d4/f8 [0,4194304] 0 2024-04-25T15:49:47.131 INFO:tasks.workunit.client.1.smithi193.stdout:7/692: dread d12/d31/d37/d62/d6e/f8c [0,4194304] 0 2024-04-25T15:49:47.131 INFO:tasks.workunit.client.1.smithi193.stdout:9/665: link df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/l78 df/d2c/d2f/d5c/d7e/da2/lcf 0 2024-04-25T15:49:47.133 INFO:tasks.workunit.client.1.smithi193.stdout:2/696: stat d0/d9/d15/d31/c9d 0 2024-04-25T15:49:47.133 INFO:tasks.workunit.client.1.smithi193.stdout:9/666: dread df/d2c/d2f/d4e/d1c/fbf [0,4194304] 0 2024-04-25T15:49:47.137 INFO:tasks.workunit.client.0.smithi110.stdout:9/818: fsync d9/d44/f7f 0 2024-04-25T15:49:47.139 INFO:tasks.workunit.client.0.smithi110.stdout:6/811: mkdir d1/db/d24/d2d/d11a 0 2024-04-25T15:49:47.142 INFO:tasks.workunit.client.1.smithi193.stdout:4/751: rename db/d32/d40/l91 to db/d4b/d51/d54/lf1 0 2024-04-25T15:49:47.142 INFO:tasks.workunit.client.0.smithi110.stdout:3/882: rmdir d0/d16/d8c 39 2024-04-25T15:49:47.143 INFO:tasks.workunit.client.0.smithi110.stdout:3/883: readlink d0/d2e/l32 0 2024-04-25T15:49:47.145 INFO:tasks.workunit.client.0.smithi110.stdout:5/880: mknod d4/d15/d51/d90/de8/c124 0 2024-04-25T15:49:47.146 INFO:tasks.workunit.client.1.smithi193.stdout:0/716: mknod d0/d49/d5/cf3 0 2024-04-25T15:49:47.148 INFO:tasks.workunit.client.1.smithi193.stdout:8/678: symlink d2/d3/d27/d69/dae/d4a/dc1/ld9 0 2024-04-25T15:49:47.150 INFO:tasks.workunit.client.0.smithi110.stdout:3/884: dwrite d0/f1e [4194304,4194304] 0 2024-04-25T15:49:47.150 INFO:tasks.workunit.client.1.smithi193.stdout:3/715: creat d8/d13/d1f/d69/dcd/dd9/fe2 x:0 0 0 2024-04-25T15:49:47.151 INFO:tasks.workunit.client.0.smithi110.stdout:3/885: fsync d0/d2e/d88/dbf/da5/f106 0 2024-04-25T15:49:47.153 INFO:tasks.workunit.client.0.smithi110.stdout:6/812: mknod d1/db/d22/d33/d43/c11b 0 2024-04-25T15:49:47.157 INFO:tasks.workunit.client.1.smithi193.stdout:2/697: creat d0/d42/d51/df1/f10a x:0 0 0 2024-04-25T15:49:47.158 INFO:tasks.workunit.client.0.smithi110.stdout:5/881: fsync d4/d1d/d1f/d35/d7f/d92/d9b/dcb/fe3 0 2024-04-25T15:49:47.160 INFO:tasks.workunit.client.1.smithi193.stdout:9/667: mknod df/d2c/d2f/d4e/d1c/d30/d3b/cd0 0 2024-04-25T15:49:47.167 INFO:tasks.workunit.client.1.smithi193.stdout:7/693: rename d12/d31/d37/d62/d6e/d79 to d12/d31/d37/d104 0 2024-04-25T15:49:47.169 INFO:tasks.workunit.client.1.smithi193.stdout:4/752: fsync db/d13/d1e/d1f/d56/fe3 0 2024-04-25T15:49:47.170 INFO:tasks.workunit.client.1.smithi193.stdout:4/753: read - db/d13/d1e/d1f/d22/fcc zero size 2024-04-25T15:49:47.171 INFO:tasks.workunit.client.0.smithi110.stdout:2/797: dwrite de/d40/d6e/f9b [4194304,4194304] 0 2024-04-25T15:49:47.173 INFO:tasks.workunit.client.0.smithi110.stdout:6/813: rename d1/db/d24/d2d/f7b to d1/dd/d3c/d91/d8d/dce/da1/f11c 0 2024-04-25T15:49:47.173 INFO:tasks.workunit.client.0.smithi110.stdout:9/819: write d9/db/f1b [436000,123082] 0 2024-04-25T15:49:47.174 INFO:tasks.workunit.client.0.smithi110.stdout:9/820: chown d9/db/df/d25 5286 1 2024-04-25T15:49:47.177 INFO:tasks.workunit.client.1.smithi193.stdout:6/668: mkdir dd/d12/d19/d1c/d40/dce 0 2024-04-25T15:49:47.178 INFO:tasks.workunit.client.1.smithi193.stdout:6/669: fsync dd/d12/d2d/fb7 0 2024-04-25T15:49:47.181 INFO:tasks.workunit.client.0.smithi110.stdout:6/814: dwrite d1/dd/d3c/d91/d8d/dce/d102/d75/d82/fa6 [0,4194304] 0 2024-04-25T15:49:47.181 INFO:tasks.workunit.client.1.smithi193.stdout:8/679: unlink d2/da5/lca 0 2024-04-25T15:49:47.185 INFO:tasks.workunit.client.1.smithi193.stdout:3/716: readlink d8/d30/d45/dd0/ld8 0 2024-04-25T15:49:47.186 INFO:tasks.workunit.client.1.smithi193.stdout:0/717: dwrite d0/d49/d5/d7/dd/d46/d48/fb4 [0,4194304] 0 2024-04-25T15:49:47.187 INFO:tasks.workunit.client.1.smithi193.stdout:6/670: dwrite dd/d2f/d57/f8b [4194304,4194304] 0 2024-04-25T15:49:47.189 INFO:tasks.workunit.client.1.smithi193.stdout:5/719: symlink d17/d18/d30/lf3 0 2024-04-25T15:49:47.194 INFO:tasks.workunit.client.0.smithi110.stdout:3/886: truncate d0/d16/dcb/d2f/d4c/db2/fe4 1163138 0 2024-04-25T15:49:47.198 INFO:tasks.workunit.client.1.smithi193.stdout:1/624: getdents dc/d4a/d4d/d7a/dd0 0 2024-04-25T15:49:47.199 INFO:tasks.workunit.client.0.smithi110.stdout:6/815: rmdir d1/db/d22/d90/d76 39 2024-04-25T15:49:47.201 INFO:tasks.workunit.client.1.smithi193.stdout:7/694: creat d12/d13/dc4/dfa/f105 x:0 0 0 2024-04-25T15:49:47.202 INFO:tasks.workunit.client.1.smithi193.stdout:7/695: truncate d12/d1c/d45/f5d 4628041 0 2024-04-25T15:49:47.202 INFO:tasks.workunit.client.0.smithi110.stdout:2/798: mknod de/d40/d5b/d5c/c102 0 2024-04-25T15:49:47.204 INFO:tasks.workunit.client.1.smithi193.stdout:4/754: chown db/d13/d1e/d1f/d22/cd8 362407 1 2024-04-25T15:49:47.205 INFO:tasks.workunit.client.0.smithi110.stdout:3/887: creat d0/d16/dcb/d2f/d4c/db2/df2/f12d x:0 0 0 2024-04-25T15:49:47.213 INFO:tasks.workunit.client.0.smithi110.stdout:2/799: creat de/d40/d5b/d5c/dbe/f103 x:0 0 0 2024-04-25T15:49:47.214 INFO:tasks.workunit.client.1.smithi193.stdout:3/717: mkdir d8/d13/d4b/d8b/de3 0 2024-04-25T15:49:47.215 INFO:tasks.workunit.client.1.smithi193.stdout:3/718: chown d8/d30/c7c 16757124 1 2024-04-25T15:49:47.216 INFO:tasks.workunit.client.1.smithi193.stdout:3/719: fsync d8/d30/f42 0 2024-04-25T15:49:47.217 INFO:tasks.workunit.client.0.smithi110.stdout:3/888: chown d0/d7/d40/c58 0 1 2024-04-25T15:49:47.218 INFO:tasks.workunit.client.1.smithi193.stdout:6/671: read dd/d12/d19/d1c/f26 [1162821,4264] 0 2024-04-25T15:49:47.222 INFO:tasks.workunit.client.1.smithi193.stdout:5/720: unlink d17/d18/d30/d38/d48/d4f/fb8 0 2024-04-25T15:49:47.222 INFO:tasks.workunit.client.0.smithi110.stdout:5/882: getdents d4/d1d/d1f/d35/d3d/d93 0 2024-04-25T15:49:47.224 INFO:tasks.workunit.client.0.smithi110.stdout:3/889: dwrite d0/d2e/d88/dbf/d92/f89 [0,4194304] 0 2024-04-25T15:49:47.224 INFO:tasks.workunit.client.1.smithi193.stdout:9/668: creat df/d2c/d2f/d5c/da3/dac/fd1 x:0 0 0 2024-04-25T15:49:47.225 INFO:tasks.workunit.client.0.smithi110.stdout:5/883: rename d4/d1d/d1f/d35/d3d/d52 to d4/d1d/d1f/d35/d3d/d52/d75/d125 22 2024-04-25T15:49:47.227 INFO:tasks.workunit.client.0.smithi110.stdout:5/884: stat d4/d1d/d1f/d35/d3d/d52/d97/fb1 0 2024-04-25T15:49:47.229 INFO:tasks.workunit.client.1.smithi193.stdout:5/721: dwrite d17/d18/d3e/f3f [0,4194304] 0 2024-04-25T15:49:47.229 INFO:tasks.workunit.client.0.smithi110.stdout:2/800: mkdir de/d40/d70/d104 0 2024-04-25T15:49:47.252 INFO:tasks.workunit.client.0.smithi110.stdout:9/821: dwrite d9/d13/d5b/d61/f9d [0,4194304] 0 2024-04-25T15:49:47.255 INFO:tasks.workunit.client.0.smithi110.stdout:9/822: chown d9/d13/d7c/la5 953 1 2024-04-25T15:49:47.256 INFO:tasks.workunit.client.1.smithi193.stdout:7/696: mkdir d12/d1c/d63/d70/d68/d8d/d106 0 2024-04-25T15:49:47.260 INFO:tasks.workunit.client.0.smithi110.stdout:6/816: write d1/dd/d3c/d91/d8d/dce/da1/fa2 [166754,96659] 0 2024-04-25T15:49:47.262 INFO:tasks.workunit.client.1.smithi193.stdout:8/680: write d2/d3/d27/d6f/fd2 [3948077,80437] 0 2024-04-25T15:49:47.264 INFO:tasks.workunit.client.0.smithi110.stdout:5/885: rmdir d4/d15/d51/d90/ded 39 2024-04-25T15:49:47.267 INFO:tasks.workunit.client.0.smithi110.stdout:5/886: truncate d4/d1d/d1f/d35/d7f/d92/d9b/f101 433429 0 2024-04-25T15:49:47.268 INFO:tasks.workunit.client.0.smithi110.stdout:5/887: write d4/d1d/d1f/d35/f6a [691634,128637] 0 2024-04-25T15:49:47.269 INFO:tasks.workunit.client.0.smithi110.stdout:2/801: dread - de/d40/d6e/d52/dda/ffb zero size 2024-04-25T15:49:47.272 INFO:tasks.workunit.client.0.smithi110.stdout:9/823: rmdir d9/d13/d5b/d61/d67/dd7 39 2024-04-25T15:49:47.273 INFO:tasks.workunit.client.1.smithi193.stdout:6/672: dread dd/d12/d19/d1c/f46 [0,4194304] 0 2024-04-25T15:49:47.274 INFO:tasks.workunit.client.1.smithi193.stdout:6/673: chown dd/d12/d19/db2/fc2 245 1 2024-04-25T15:49:47.275 INFO:tasks.workunit.client.1.smithi193.stdout:6/674: truncate dd/d12/d19/d1c/d40/d3c/d3f/f85 601673 0 2024-04-25T15:49:47.276 INFO:tasks.workunit.client.1.smithi193.stdout:9/669: symlink df/d2c/d95/ld2 0 2024-04-25T15:49:47.282 INFO:tasks.workunit.client.1.smithi193.stdout:4/755: dread db/d13/d1e/d1f/d22/f2a [0,4194304] 0 2024-04-25T15:49:47.283 INFO:tasks.workunit.client.1.smithi193.stdout:5/722: rmdir d17/d18/d75 39 2024-04-25T15:49:47.284 INFO:tasks.workunit.client.1.smithi193.stdout:1/625: dwrite dc/d4a/d50/d5c/d5d/f3a [4194304,4194304] 0 2024-04-25T15:49:47.284 INFO:tasks.workunit.client.1.smithi193.stdout:6/675: dwrite dd/d12/d19/d1c/fa4 [0,4194304] 0 2024-04-25T15:49:47.285 INFO:tasks.workunit.client.1.smithi193.stdout:6/676: readlink dd/l6d 0 2024-04-25T15:49:47.285 INFO:tasks.workunit.client.1.smithi193.stdout:1/626: readlink dc/d4a/d50/d7c/d82/lc7 0 2024-04-25T15:49:47.286 INFO:tasks.workunit.client.1.smithi193.stdout:7/697: symlink d12/d31/d37/d62/d6e/da5/l107 0 2024-04-25T15:49:47.286 INFO:tasks.workunit.client.1.smithi193.stdout:7/698: chown d12/l35 347287100 1 2024-04-25T15:49:47.288 INFO:tasks.workunit.client.1.smithi193.stdout:7/699: fdatasync d12/d1c/d63/d70/d68/fe1 0 2024-04-25T15:49:47.289 INFO:tasks.workunit.client.0.smithi110.stdout:3/890: dwrite d0/d16/dcb/d2f/d4c/ffd [0,4194304] 0 2024-04-25T15:49:47.290 INFO:tasks.workunit.client.1.smithi193.stdout:8/681: creat d2/d3/d3a/fda x:0 0 0 2024-04-25T15:49:47.291 INFO:tasks.workunit.client.0.smithi110.stdout:6/817: mkdir d1/db/d22/d90/d11d 0 2024-04-25T15:49:47.292 INFO:tasks.workunit.client.0.smithi110.stdout:5/888: dread d4/fd [0,4194304] 0 2024-04-25T15:49:47.293 INFO:tasks.workunit.client.1.smithi193.stdout:3/720: truncate d8/d13/f32 2539692 0 2024-04-25T15:49:47.296 INFO:tasks.workunit.client.1.smithi193.stdout:8/682: dwrite d2/d3/d27/d69/dae/d4a/d53/f58 [0,4194304] 0 2024-04-25T15:49:47.296 INFO:tasks.workunit.client.0.smithi110.stdout:9/824: read d9/d13/d5b/d61/d67/dd7/f6b [1912568,26857] 0 2024-04-25T15:49:47.298 INFO:tasks.workunit.client.1.smithi193.stdout:9/670: readlink df/d2c/d2f/d4e/l9d 0 2024-04-25T15:49:47.302 INFO:tasks.workunit.client.1.smithi193.stdout:2/698: rename d0/d9/d15/d31/dd3 to d0/d9/d15/d10b 0 2024-04-25T15:49:47.302 INFO:tasks.workunit.client.0.smithi110.stdout:2/802: dwrite de/f24 [0,4194304] 0 2024-04-25T15:49:47.305 INFO:tasks.workunit.client.1.smithi193.stdout:4/756: unlink db/d32/fe7 0 2024-04-25T15:49:47.305 INFO:tasks.workunit.client.0.smithi110.stdout:5/889: dread - d4/d1d/d1f/d35/d3d/d93/dc4/fd7 zero size 2024-04-25T15:49:47.308 INFO:tasks.workunit.client.0.smithi110.stdout:9/825: rmdir d9/d44/d5e/db5 39 2024-04-25T15:49:47.310 INFO:tasks.workunit.client.0.smithi110.stdout:6/818: mkdir d1/db/d22/d90/df7/d11e 0 2024-04-25T15:49:47.314 INFO:tasks.workunit.client.1.smithi193.stdout:7/700: unlink d12/d31/d37/d62/f9b 0 2024-04-25T15:49:47.316 INFO:tasks.workunit.client.0.smithi110.stdout:2/803: mkdir de/d40/d4f/d105 0 2024-04-25T15:49:47.317 INFO:tasks.workunit.client.0.smithi110.stdout:3/891: truncate d0/d16/d8c/fcf 1374293 0 2024-04-25T15:49:47.317 INFO:tasks.workunit.client.0.smithi110.stdout:6/819: chown d1/db/d22/d33/d43/l6e 490 1 2024-04-25T15:49:47.322 INFO:tasks.workunit.client.1.smithi193.stdout:0/718: rename d0/d49/d6a/cc7 to d0/d49/d5/d7/dd/d38/cf4 0 2024-04-25T15:49:47.323 INFO:tasks.workunit.client.1.smithi193.stdout:0/719: read - d0/d49/d5/d7/dd/d16/d18/d28/fe6 zero size 2024-04-25T15:49:47.324 INFO:tasks.workunit.client.1.smithi193.stdout:2/699: symlink d0/df/l10c 0 2024-04-25T15:49:47.325 INFO:tasks.workunit.client.0.smithi110.stdout:6/820: dwrite d1/d12/db8/fe1 [4194304,4194304] 0 2024-04-25T15:49:47.329 INFO:tasks.workunit.client.1.smithi193.stdout:6/677: fsync dd/d12/d19/d1c/fa4 0 2024-04-25T15:49:47.341 INFO:tasks.workunit.client.1.smithi193.stdout:5/723: dwrite d17/f26 [0,4194304] 0 2024-04-25T15:49:47.345 INFO:tasks.workunit.client.1.smithi193.stdout:3/721: write d8/d13/d14/d23/fa1 [403903,53657] 0 2024-04-25T15:49:47.350 INFO:tasks.workunit.client.1.smithi193.stdout:2/700: mkdir d0/d37/d25/d5f/d3a/d6d/d85/d10d 0 2024-04-25T15:49:47.353 INFO:tasks.workunit.client.1.smithi193.stdout:6/678: fdatasync dd/d12/d19/d45/f7e 0 2024-04-25T15:49:47.355 INFO:tasks.workunit.client.1.smithi193.stdout:4/757: dread db/f8d [0,4194304] 0 2024-04-25T15:49:47.355 INFO:tasks.workunit.client.1.smithi193.stdout:4/758: chown db/c17 101 1 2024-04-25T15:49:47.357 INFO:tasks.workunit.client.1.smithi193.stdout:8/683: mkdir d2/d18/ddb 0 2024-04-25T15:49:47.359 INFO:tasks.workunit.client.1.smithi193.stdout:5/724: rmdir d17/d18/d30/d3b 39 2024-04-25T15:49:47.362 INFO:tasks.workunit.client.1.smithi193.stdout:1/627: truncate dc/d4a/d50/d5c/d5d/f3a 56324 0 2024-04-25T15:49:47.366 INFO:tasks.workunit.client.1.smithi193.stdout:2/701: dread d0/df/da5/fc2 [0,4194304] 0 2024-04-25T15:49:47.367 INFO:tasks.workunit.client.1.smithi193.stdout:2/702: fdatasync d0/d42/d51/df1/fea 0 2024-04-25T15:49:47.368 INFO:tasks.workunit.client.1.smithi193.stdout:0/720: write d0/d49/d5/d61/f74 [1029506,48756] 0 2024-04-25T15:49:47.369 INFO:tasks.workunit.client.1.smithi193.stdout:7/701: link f8 d12/d31/d37/d62/d6e/da5/f108 0 2024-04-25T15:49:47.371 INFO:tasks.workunit.client.1.smithi193.stdout:8/684: mknod d2/d3/d27/d6f/dbb/cdc 0 2024-04-25T15:49:47.376 INFO:tasks.workunit.client.1.smithi193.stdout:9/671: getdents df/d2c/d2f/d5c/da3 0 2024-04-25T15:49:47.378 INFO:tasks.workunit.client.1.smithi193.stdout:3/722: mknod d8/d13/d1f/d69/d77/d71/dd2/ce4 0 2024-04-25T15:49:47.381 INFO:tasks.workunit.client.1.smithi193.stdout:7/702: rename d12/da1 to d12/d1c/d63/d109 0 2024-04-25T15:49:47.383 INFO:tasks.workunit.client.1.smithi193.stdout:1/628: dwrite dc/f3f [0,4194304] 0 2024-04-25T15:49:47.383 INFO:tasks.workunit.client.1.smithi193.stdout:9/672: rmdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d 39 2024-04-25T15:49:47.383 INFO:tasks.workunit.client.1.smithi193.stdout:7/703: chown d12/d13/ff9 1 1 2024-04-25T15:49:47.384 INFO:tasks.workunit.client.1.smithi193.stdout:7/704: stat d12/d13/f71 0 2024-04-25T15:49:47.385 INFO:tasks.workunit.client.1.smithi193.stdout:3/723: fdatasync d8/d30/f6b 0 2024-04-25T15:49:47.385 INFO:tasks.workunit.client.0.smithi110.stdout:2/804: chown de/d40/d6e/d50/fa4 122174014 1 2024-04-25T15:49:47.386 INFO:tasks.workunit.client.0.smithi110.stdout:5/890: creat d4/d1d/d1f/d35/d3d/d52/d75/dd3/f126 x:0 0 0 2024-04-25T15:49:47.387 INFO:tasks.workunit.client.1.smithi193.stdout:8/685: dread d2/d3/f37 [0,4194304] 0 2024-04-25T15:49:47.387 INFO:tasks.workunit.client.0.smithi110.stdout:3/892: fdatasync d0/d16/dcb/d50/f59 0 2024-04-25T15:49:47.387 INFO:tasks.workunit.client.1.smithi193.stdout:5/725: dread d17/d18/d30/d38/d71/d96/fc7 [0,4194304] 0 2024-04-25T15:49:47.388 INFO:tasks.workunit.client.0.smithi110.stdout:5/891: chown d4/d1d/d1f/d35/d3d/d52/d75/dd3 215386 1 2024-04-25T15:49:47.390 INFO:tasks.workunit.client.1.smithi193.stdout:9/673: creat df/d2c/d34/d3e/d4d/fd3 x:0 0 0 2024-04-25T15:49:47.392 INFO:tasks.workunit.client.1.smithi193.stdout:7/705: unlink d12/d31/d39/l44 0 2024-04-25T15:49:47.392 INFO:tasks.workunit.client.1.smithi193.stdout:7/706: chown d12/d1c/dd6/dda 4 1 2024-04-25T15:49:47.396 INFO:tasks.workunit.client.1.smithi193.stdout:5/726: unlink d17/d18/d30/d38/d48/f8f 0 2024-04-25T15:49:47.397 INFO:tasks.workunit.client.0.smithi110.stdout:3/893: symlink d0/d2e/d88/dbf/d99/dad/daf/l12e 0 2024-04-25T15:49:47.398 INFO:tasks.workunit.client.1.smithi193.stdout:8/686: mknod d2/d3/d27/d6f/cdd 0 2024-04-25T15:49:47.399 INFO:tasks.workunit.client.0.smithi110.stdout:5/892: truncate d4/d15/d51/f9c 762449 0 2024-04-25T15:49:47.400 INFO:tasks.workunit.client.0.smithi110.stdout:5/893: chown d4/d15/d51/l120 75918173 1 2024-04-25T15:49:47.401 INFO:tasks.workunit.client.0.smithi110.stdout:5/894: readlink d4/d1d/d85/le5 0 2024-04-25T15:49:47.402 INFO:tasks.workunit.client.0.smithi110.stdout:2/805: unlink de/c27 0 2024-04-25T15:49:47.402 INFO:tasks.workunit.client.1.smithi193.stdout:7/707: creat d12/d31/dc3/f10a x:0 0 0 2024-04-25T15:49:47.403 INFO:tasks.workunit.client.0.smithi110.stdout:3/894: dread - d0/d7/f104 zero size 2024-04-25T15:49:47.407 INFO:tasks.workunit.client.1.smithi193.stdout:0/721: rename d0/d49/d5/d7/dd/d38 to d0/d49/d5/d7/dd/d16/d26/df5 0 2024-04-25T15:49:47.408 INFO:tasks.workunit.client.1.smithi193.stdout:0/722: chown d0/d49/d5/d7/dd/d46/d48 55890 1 2024-04-25T15:49:47.412 INFO:tasks.workunit.client.0.smithi110.stdout:3/895: mkdir d0/d2e/d88/dbf/d92/d12f 0 2024-04-25T15:49:47.413 INFO:tasks.workunit.client.1.smithi193.stdout:5/727: dread d17/d18/d30/d38/d71/f74 [0,4194304] 0 2024-04-25T15:49:47.415 INFO:tasks.workunit.client.1.smithi193.stdout:0/723: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/fc8 [4194304,4194304] 0 2024-04-25T15:49:47.415 INFO:tasks.workunit.client.1.smithi193.stdout:3/724: rename d8/d13/d14/d27/f95 to d8/d13/d14/d27/d54/dd7/fe5 0 2024-04-25T15:49:47.416 INFO:tasks.workunit.client.1.smithi193.stdout:0/724: chown d0/d49/d5/d7/dd/d46/ccc 72571 1 2024-04-25T15:49:47.417 INFO:tasks.workunit.client.1.smithi193.stdout:9/674: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/fd4 x:0 0 0 2024-04-25T15:49:47.418 INFO:tasks.workunit.client.0.smithi110.stdout:2/806: creat de/d40/d4f/d105/f106 x:0 0 0 2024-04-25T15:49:47.419 INFO:tasks.workunit.client.1.smithi193.stdout:9/675: fdatasync df/d2c/d2f/d4e/f9c 0 2024-04-25T15:49:47.419 INFO:tasks.workunit.client.1.smithi193.stdout:0/725: write d0/d49/d5/d7/dd/d16/d1a/f8e [3476473,32180] 0 2024-04-25T15:49:47.419 INFO:tasks.workunit.client.1.smithi193.stdout:9/676: chown df/d2c/d2f/c89 8 1 2024-04-25T15:49:47.425 INFO:tasks.workunit.client.0.smithi110.stdout:2/807: fsync de/f2d 0 2024-04-25T15:49:47.427 INFO:tasks.workunit.client.0.smithi110.stdout:3/896: symlink d0/d2e/d88/dbf/d92/d12f/l130 0 2024-04-25T15:49:47.433 INFO:tasks.workunit.client.0.smithi110.stdout:2/808: creat de/d40/d6e/d50/db7/f107 x:0 0 0 2024-04-25T15:49:47.435 INFO:tasks.workunit.client.1.smithi193.stdout:3/725: creat d8/d13/d1f/dbf/fe6 x:0 0 0 2024-04-25T15:49:47.441 INFO:tasks.workunit.client.1.smithi193.stdout:9/677: creat df/d2c/d34/d3e/d4d/d5e/fd5 x:0 0 0 2024-04-25T15:49:47.441 INFO:tasks.workunit.client.0.smithi110.stdout:6/821: write d1/db/d22/f57 [139880,30548] 0 2024-04-25T15:49:47.442 INFO:tasks.workunit.client.1.smithi193.stdout:3/726: dwrite d8/d13/d14/f80 [0,4194304] 0 2024-04-25T15:49:47.442 INFO:tasks.workunit.client.0.smithi110.stdout:6/822: readlink d1/db/d22/d90/l9b 0 2024-04-25T15:49:47.443 INFO:tasks.workunit.client.0.smithi110.stdout:6/823: stat d1/dd/d3c/f5d 0 2024-04-25T15:49:47.444 INFO:tasks.workunit.client.1.smithi193.stdout:5/728: symlink d17/d18/d30/d34/d52/dee/lf4 0 2024-04-25T15:49:47.446 INFO:tasks.workunit.client.1.smithi193.stdout:1/629: dwrite dc/f15 [0,4194304] 0 2024-04-25T15:49:47.447 INFO:tasks.workunit.client.0.smithi110.stdout:6/824: mknod d1/db/d22/d33/c11f 0 2024-04-25T15:49:47.448 INFO:tasks.workunit.client.0.smithi110.stdout:6/825: fsync d1/dd/f64 0 2024-04-25T15:49:47.449 INFO:tasks.workunit.client.1.smithi193.stdout:1/630: write dc/d4a/d50/d7c/fca [805722,128568] 0 2024-04-25T15:49:47.450 INFO:tasks.workunit.client.1.smithi193.stdout:9/678: dread df/d2c/d2f/d5c/d7e/fc1 [0,4194304] 0 2024-04-25T15:49:47.452 INFO:tasks.workunit.client.1.smithi193.stdout:8/687: rename d2/d3/d27/d6f to d2/d3/d20/d8c/dde 0 2024-04-25T15:49:47.454 INFO:tasks.workunit.client.0.smithi110.stdout:7/944: sync 2024-04-25T15:49:47.454 INFO:tasks.workunit.client.0.smithi110.stdout:8/970: sync 2024-04-25T15:49:47.454 INFO:tasks.workunit.client.0.smithi110.stdout:1/995: sync 2024-04-25T15:49:47.457 INFO:tasks.workunit.client.0.smithi110.stdout:6/826: stat d1/db/d22/d90/d76/dad 0 2024-04-25T15:49:47.459 INFO:tasks.workunit.client.0.smithi110.stdout:6/827: fsync d1/dd/d3c/d91/d8d/dce/d102/d75/d82/fa6 0 2024-04-25T15:49:47.460 INFO:tasks.workunit.client.1.smithi193.stdout:0/726: creat d0/d49/d5/d7/dd/d16/d26/ff6 x:0 0 0 2024-04-25T15:49:47.461 INFO:tasks.workunit.client.0.smithi110.stdout:5/895: write d4/d1d/d1f/d35/d3d/f5d [2398472,128442] 0 2024-04-25T15:49:47.463 INFO:tasks.workunit.client.1.smithi193.stdout:1/631: readlink dc/l2a 0 2024-04-25T15:49:47.469 INFO:tasks.workunit.client.0.smithi110.stdout:8/971: rename d14/d21/d23 to d14/d21/d26/d6a/d118/d155 0 2024-04-25T15:49:47.472 INFO:tasks.workunit.client.1.smithi193.stdout:3/727: dread d8/d13/f32 [0,4194304] 0 2024-04-25T15:49:47.473 INFO:tasks.workunit.client.1.smithi193.stdout:8/688: creat d2/d3/d27/fdf x:0 0 0 2024-04-25T15:49:47.474 INFO:tasks.workunit.client.1.smithi193.stdout:3/728: chown d8/d13/d4b/lc5 1450 1 2024-04-25T15:49:47.475 INFO:tasks.workunit.client.1.smithi193.stdout:7/708: dwrite d12/f2c [0,4194304] 0 2024-04-25T15:49:47.479 INFO:tasks.workunit.client.1.smithi193.stdout:5/729: rename c10 to d17/d83/cf5 0 2024-04-25T15:49:47.479 INFO:tasks.workunit.client.1.smithi193.stdout:8/689: dread d2/d3/d27/d69/dae/d4a/d53/f58 [0,4194304] 0 2024-04-25T15:49:47.481 INFO:tasks.workunit.client.1.smithi193.stdout:7/709: dwrite d12/d1c/d63/d70/f34 [0,4194304] 0 2024-04-25T15:49:47.482 INFO:tasks.workunit.client.1.smithi193.stdout:7/710: fdatasync d12/d1c/d45/f5d 0 2024-04-25T15:49:47.483 INFO:tasks.workunit.client.0.smithi110.stdout:2/809: write f8 [5246034,79726] 0 2024-04-25T15:49:47.488 INFO:tasks.workunit.client.0.smithi110.stdout:3/897: dwrite d0/d7/f68 [8388608,4194304] 0 2024-04-25T15:49:47.492 INFO:tasks.workunit.client.1.smithi193.stdout:6/679: sync 2024-04-25T15:49:47.492 INFO:tasks.workunit.client.0.smithi110.stdout:6/828: creat d1/db/d119/d112/f120 x:0 0 0 2024-04-25T15:49:47.492 INFO:tasks.workunit.client.0.smithi110.stdout:3/898: read d0/d7/f69 [1560817,54823] 0 2024-04-25T15:49:47.493 INFO:tasks.workunit.client.1.smithi193.stdout:6/680: chown dd/d2f/d57/f5c 15303989 1 2024-04-25T15:49:47.495 INFO:tasks.workunit.client.0.smithi110.stdout:5/896: mknod d4/d1d/d1f/d56/c127 0 2024-04-25T15:49:47.505 INFO:tasks.workunit.client.0.smithi110.stdout:2/810: dwrite de/d40/d6e/d50/db7/fe3 [0,4194304] 0 2024-04-25T15:49:47.506 INFO:tasks.workunit.client.0.smithi110.stdout:8/972: creat d14/d32/d36/dde/d111/f156 x:0 0 0 2024-04-25T15:49:47.509 INFO:tasks.workunit.client.1.smithi193.stdout:5/730: symlink d17/d18/d30/d38/d71/d96/lf6 0 2024-04-25T15:49:47.509 INFO:tasks.workunit.client.0.smithi110.stdout:7/945: write f19 [480666,44204] 0 2024-04-25T15:49:47.513 INFO:tasks.workunit.client.1.smithi193.stdout:1/632: unlink dc/d59/f65 0 2024-04-25T15:49:47.515 INFO:tasks.workunit.client.0.smithi110.stdout:6/829: creat d1/db/da0/da5/f121 x:0 0 0 2024-04-25T15:49:47.520 INFO:tasks.workunit.client.0.smithi110.stdout:1/996: dwrite d0/d14/d81/df8/d33/d94/f115 [0,4194304] 0 2024-04-25T15:49:47.520 INFO:tasks.workunit.client.1.smithi193.stdout:7/711: creat d12/f10b x:0 0 0 2024-04-25T15:49:47.524 INFO:tasks.workunit.client.1.smithi193.stdout:8/690: write d2/d18/dbc/dd3/fb2 [1121666,57933] 0 2024-04-25T15:49:47.526 INFO:tasks.workunit.client.0.smithi110.stdout:2/811: rename de/d40/d6e/d52/c79 to de/d40/d4f/ded/c108 0 2024-04-25T15:49:47.527 INFO:tasks.workunit.client.0.smithi110.stdout:2/812: stat de/d40/d4f/d9f/fe0 0 2024-04-25T15:49:47.533 INFO:tasks.workunit.client.0.smithi110.stdout:3/899: dwrite d0/d7/d7d/fd3 [0,4194304] 0 2024-04-25T15:49:47.533 INFO:tasks.workunit.client.1.smithi193.stdout:5/731: mkdir d17/d18/d30/d34/d52/dee/df7 0 2024-04-25T15:49:47.536 INFO:tasks.workunit.client.0.smithi110.stdout:3/900: chown d0/c86 462 1 2024-04-25T15:49:47.538 INFO:tasks.workunit.client.1.smithi193.stdout:9/679: getdents df/d2c/d2f/d4e 0 2024-04-25T15:49:47.539 INFO:tasks.workunit.client.0.smithi110.stdout:5/897: dwrite d4/d1d/d1f/d35/d3d/f53 [8388608,4194304] 0 2024-04-25T15:49:47.539 INFO:tasks.workunit.client.1.smithi193.stdout:9/680: write df/d2d/fba [3859984,33682] 0 2024-04-25T15:49:47.542 INFO:tasks.workunit.client.1.smithi193.stdout:8/691: symlink d2/da5/le0 0 2024-04-25T15:49:47.543 INFO:tasks.workunit.client.1.smithi193.stdout:8/692: readlink d2/d3/d20/d8c/dde/d94/l51 0 2024-04-25T15:49:47.544 INFO:tasks.workunit.client.0.smithi110.stdout:5/898: chown d4/d15/d51/d90/fa3 0 1 2024-04-25T15:49:47.548 INFO:tasks.workunit.client.1.smithi193.stdout:0/727: rename d0/d49/d5/d61/la9 to d0/d49/d5/d7/dd/d16/lf7 0 2024-04-25T15:49:47.552 INFO:tasks.workunit.client.0.smithi110.stdout:3/901: rmdir d0/d16/dcb/d2f 39 2024-04-25T15:49:47.553 INFO:tasks.workunit.client.0.smithi110.stdout:5/899: dwrite d4/d1d/d1f/d9e/f10d [0,4194304] 0 2024-04-25T15:49:47.554 INFO:tasks.workunit.client.0.smithi110.stdout:7/946: link d1b/d29/d46/l66 d1b/d29/d38/d55/dab/db2/dcd/l13f 0 2024-04-25T15:49:47.556 INFO:tasks.workunit.client.1.smithi193.stdout:0/728: dwrite d0/d49/d5/d7/dd/f6e [0,4194304] 0 2024-04-25T15:49:47.556 INFO:tasks.workunit.client.0.smithi110.stdout:2/813: mknod de/d40/c109 0 2024-04-25T15:49:47.560 INFO:tasks.workunit.client.1.smithi193.stdout:7/712: dwrite d12/d31/d39/d60/fd4 [0,4194304] 0 2024-04-25T15:49:47.566 INFO:tasks.workunit.client.0.smithi110.stdout:2/814: dread de/d40/d5b/d5c/d5f/db2/dad/fdb [0,4194304] 0 2024-04-25T15:49:47.571 INFO:tasks.workunit.client.1.smithi193.stdout:9/681: rmdir df/d2c 39 2024-04-25T15:49:47.574 INFO:tasks.workunit.client.0.smithi110.stdout:6/830: dread d1/db/d24/f29 [0,4194304] 0 2024-04-25T15:49:47.575 INFO:tasks.workunit.client.1.smithi193.stdout:8/693: unlink d2/d3/c17 0 2024-04-25T15:49:47.576 INFO:tasks.workunit.client.1.smithi193.stdout:8/694: stat d2/d3/d27/d69/dae/d4a/d53/c74 0 2024-04-25T15:49:47.576 INFO:tasks.workunit.client.1.smithi193.stdout:3/729: link d8/d13/d14/d23/lb9 d8/d13/d1f/d69/d77/d71/le7 0 2024-04-25T15:49:47.581 INFO:tasks.workunit.client.0.smithi110.stdout:7/947: rmdir d1b/d29/d46/d106 39 2024-04-25T15:49:47.581 INFO:tasks.workunit.client.0.smithi110.stdout:3/902: fsync d0/d16/dcb/d2f/d33/f9e 0 2024-04-25T15:49:47.582 INFO:tasks.workunit.client.0.smithi110.stdout:2/815: rmdir de/d6f/dbb 39 2024-04-25T15:49:47.583 INFO:tasks.workunit.client.0.smithi110.stdout:5/900: dwrite d4/d15/d51/d90/ded/ff6 [0,4194304] 0 2024-04-25T15:49:47.583 INFO:tasks.workunit.client.1.smithi193.stdout:1/633: creat dc/d4a/d4d/d7a/fd7 x:0 0 0 2024-04-25T15:49:47.585 INFO:tasks.workunit.client.0.smithi110.stdout:5/901: chown d4/f25 1 1 2024-04-25T15:49:47.586 INFO:tasks.workunit.client.1.smithi193.stdout:4/759: sync 2024-04-25T15:49:47.586 INFO:tasks.workunit.client.1.smithi193.stdout:2/703: sync 2024-04-25T15:49:47.587 INFO:tasks.workunit.client.1.smithi193.stdout:4/760: chown db/d32/fb4 6879620 1 2024-04-25T15:49:47.588 INFO:tasks.workunit.client.1.smithi193.stdout:7/713: mknod d12/d1c/dd6/c10c 0 2024-04-25T15:49:47.589 INFO:tasks.workunit.client.0.smithi110.stdout:6/831: dwrite d1/dd/d3c/d91/d8d/dce/d102/d75/d82/fa6 [0,4194304] 0 2024-04-25T15:49:47.594 INFO:tasks.workunit.client.1.smithi193.stdout:8/695: mknod d2/d3/d27/d69/dae/d4a/dc1/ce1 0 2024-04-25T15:49:47.597 INFO:tasks.workunit.client.1.smithi193.stdout:2/704: dwrite d0/d37/d25/d5f/d3a/d6d/d6e/f3e [0,4194304] 0 2024-04-25T15:49:47.597 INFO:tasks.workunit.client.0.smithi110.stdout:6/832: dwrite d1/d12/f49 [0,4194304] 0 2024-04-25T15:49:47.600 INFO:tasks.workunit.client.1.smithi193.stdout:9/682: dwrite df/d2c/d2f/f77 [0,4194304] 0 2024-04-25T15:49:47.604 INFO:tasks.workunit.client.0.smithi110.stdout:2/816: unlink de/l5e 0 2024-04-25T15:49:47.606 INFO:tasks.workunit.client.0.smithi110.stdout:3/903: rename d0/d2e/d88/dbf/le2 to d0/d16/dcb/d2f/d4c/d108/l131 0 2024-04-25T15:49:47.607 INFO:tasks.workunit.client.0.smithi110.stdout:2/817: dread de/d40/d6e/d50/db7/fe3 [0,4194304] 0 2024-04-25T15:49:47.609 INFO:tasks.workunit.client.0.smithi110.stdout:3/904: dread - d0/d2e/d88/dbf/da5/d12b/f12c zero size 2024-04-25T15:49:47.609 INFO:tasks.workunit.client.1.smithi193.stdout:6/681: sync 2024-04-25T15:49:47.614 INFO:tasks.workunit.client.0.smithi110.stdout:6/833: creat d1/db/d22/d90/d9e/f122 x:0 0 0 2024-04-25T15:49:47.615 INFO:tasks.workunit.client.0.smithi110.stdout:6/834: chown d1/c28 2206 1 2024-04-25T15:49:47.617 INFO:tasks.workunit.client.0.smithi110.stdout:2/818: creat de/d40/dcd/f10a x:0 0 0 2024-04-25T15:49:47.621 INFO:tasks.workunit.client.1.smithi193.stdout:5/732: getdents d17/d18/d3e/dd1/ddc 0 2024-04-25T15:49:47.622 INFO:tasks.workunit.client.1.smithi193.stdout:9/683: rmdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf 39 2024-04-25T15:49:47.625 INFO:tasks.workunit.client.1.smithi193.stdout:1/634: symlink dc/d4a/d50/dab/db8/ld8 0 2024-04-25T15:49:47.626 INFO:tasks.workunit.client.1.smithi193.stdout:7/714: dread d12/d13/f14 [4194304,4194304] 0 2024-04-25T15:49:47.627 INFO:tasks.workunit.client.1.smithi193.stdout:1/635: dread - dc/d4a/d50/d5c/d5d/d3d/d55/f93 zero size 2024-04-25T15:49:47.629 INFO:tasks.workunit.client.0.smithi110.stdout:3/905: rename d0/l19 to d0/d2e/d88/dbf/d92/d12f/l132 0 2024-04-25T15:49:47.631 INFO:tasks.workunit.client.0.smithi110.stdout:3/906: write d0/d16/dcb/f94 [245331,56044] 0 2024-04-25T15:49:47.638 INFO:tasks.workunit.client.0.smithi110.stdout:7/948: dwrite d1b/d29/d38/d8b/d89/f96 [0,4194304] 0 2024-04-25T15:49:47.639 INFO:tasks.workunit.client.1.smithi193.stdout:4/761: dread f9 [0,4194304] 0 2024-04-25T15:49:47.645 INFO:tasks.workunit.client.1.smithi193.stdout:9/684: dread df/d2c/d2f/d4e/f1a [0,4194304] 0 2024-04-25T15:49:47.646 INFO:tasks.workunit.client.1.smithi193.stdout:9/685: stat df/d2c/d2f/d4e/d1c/fbf 0 2024-04-25T15:49:47.647 INFO:tasks.workunit.client.1.smithi193.stdout:0/729: write d0/d49/d5/d7/dd/d16/d26/d2e/f34 [1198192,90752] 0 2024-04-25T15:49:47.647 INFO:tasks.workunit.client.1.smithi193.stdout:9/686: readlink df/d2c/d2f/l66 0 2024-04-25T15:49:47.649 INFO:tasks.workunit.client.0.smithi110.stdout:7/949: dwrite d1b/d29/db5/dca/df9/d12e/f131 [0,4194304] 0 2024-04-25T15:49:47.650 INFO:tasks.workunit.client.1.smithi193.stdout:1/636: creat dc/d4a/d50/d7c/fd9 x:0 0 0 2024-04-25T15:49:47.650 INFO:tasks.workunit.client.0.smithi110.stdout:3/907: rename d0/d7/f104 to d0/d16/d8c/dce/f133 0 2024-04-25T15:49:47.651 INFO:tasks.workunit.client.1.smithi193.stdout:1/637: chown dc/f19 54682 1 2024-04-25T15:49:47.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/682: dread dd/d12/d19/d1c/d40/d3c/d3f/d4f/f8a [0,4194304] 0 2024-04-25T15:49:47.653 INFO:tasks.workunit.client.1.smithi193.stdout:1/638: write dc/f47 [1641679,80458] 0 2024-04-25T15:49:47.654 INFO:tasks.workunit.client.0.smithi110.stdout:5/902: truncate d4/d1d/d1f/d35/f6a 109336 0 2024-04-25T15:49:47.654 INFO:tasks.workunit.client.1.smithi193.stdout:6/683: fdatasync dd/d12/d19/d1c/fa4 0 2024-04-25T15:49:47.655 INFO:tasks.workunit.client.1.smithi193.stdout:6/684: fsync dd/d12/d19/d45/fc7 0 2024-04-25T15:49:47.657 INFO:tasks.workunit.client.1.smithi193.stdout:6/685: truncate dd/d2f/fa1 111063 0 2024-04-25T15:49:47.657 INFO:tasks.workunit.client.1.smithi193.stdout:2/705: write d0/d42/d51/d4f/fa7 [436966,116064] 0 2024-04-25T15:49:47.658 INFO:tasks.workunit.client.1.smithi193.stdout:3/730: getdents d8/d30/d45 0 2024-04-25T15:49:47.660 INFO:tasks.workunit.client.1.smithi193.stdout:3/731: dread - d8/d13/d4b/d8b/fab zero size 2024-04-25T15:49:47.661 INFO:tasks.workunit.client.0.smithi110.stdout:6/835: write d1/db/d22/d90/d9e/f9f [1122670,127861] 0 2024-04-25T15:49:47.662 INFO:tasks.workunit.client.1.smithi193.stdout:8/696: link d2/d3/f80 d2/d3/d27/d69/dae/dd8/fe2 0 2024-04-25T15:49:47.662 INFO:tasks.workunit.client.1.smithi193.stdout:8/697: stat d2/cd5 0 2024-04-25T15:49:47.663 INFO:tasks.workunit.client.0.smithi110.stdout:2/819: write de/d40/d4f/f8b [3490794,100390] 0 2024-04-25T15:49:47.664 INFO:tasks.workunit.client.1.smithi193.stdout:8/698: fsync d2/d3/d27/fdf 0 2024-04-25T15:49:47.666 INFO:tasks.workunit.client.0.smithi110.stdout:3/908: rename d0/d16/d8c/fd6 to d0/d16/dcb/d2f/d4c/db2/df2/f134 0 2024-04-25T15:49:47.666 INFO:tasks.workunit.client.0.smithi110.stdout:3/909: fdatasync d0/d2e/fc6 0 2024-04-25T15:49:47.670 INFO:tasks.workunit.client.0.smithi110.stdout:5/903: creat d4/d1d/d1f/d35/d3d/d52/d75/dfa/d105/d10a/f128 x:0 0 0 2024-04-25T15:49:47.670 INFO:tasks.workunit.client.1.smithi193.stdout:4/762: fdatasync db/d4b/f7b 0 2024-04-25T15:49:47.673 INFO:tasks.workunit.client.1.smithi193.stdout:0/730: chown d0/d49/d5/d7/dd/ld0 0 1 2024-04-25T15:49:47.678 INFO:tasks.workunit.client.0.smithi110.stdout:6/836: dwrite d1/db/d22/d33/d63/dc2/f107 [0,4194304] 0 2024-04-25T15:49:47.678 INFO:tasks.workunit.client.0.smithi110.stdout:6/837: readlink d1/dd/l10 0 2024-04-25T15:49:47.680 INFO:tasks.workunit.client.0.smithi110.stdout:4/829: sync 2024-04-25T15:49:47.680 INFO:tasks.workunit.client.0.smithi110.stdout:0/832: sync 2024-04-25T15:49:47.680 INFO:tasks.workunit.client.0.smithi110.stdout:6/838: chown d1/dd/l98 2 1 2024-04-25T15:49:47.682 INFO:tasks.workunit.client.1.smithi193.stdout:9/687: creat df/d2d/fd6 x:0 0 0 2024-04-25T15:49:47.682 INFO:tasks.workunit.client.0.smithi110.stdout:6/839: chown d1/db/d22/d33/d43/fe2 289493 1 2024-04-25T15:49:47.682 INFO:tasks.workunit.client.0.smithi110.stdout:0/833: stat d1/d23/d30/d33/d5b/d78/ddb/de7/l124 0 2024-04-25T15:49:47.682 INFO:tasks.workunit.client.0.smithi110.stdout:6/840: readlink d1/db/l35 0 2024-04-25T15:49:47.689 INFO:tasks.workunit.client.0.smithi110.stdout:7/950: dwrite d1b/d29/f37 [0,4194304] 0 2024-04-25T15:49:47.689 INFO:tasks.workunit.client.0.smithi110.stdout:3/910: stat d0/l1b 0 2024-04-25T15:49:47.691 INFO:tasks.workunit.client.1.smithi193.stdout:7/715: dwrite d12/d31/d39/d60/fa8 [0,4194304] 0 2024-04-25T15:49:47.691 INFO:tasks.workunit.client.1.smithi193.stdout:1/639: mkdir dc/d4a/d50/d7c/dda 0 2024-04-25T15:49:47.692 INFO:tasks.workunit.client.0.smithi110.stdout:7/951: write d1b/d29/db5/dca/d10b/d112/f13a [616743,113914] 0 2024-04-25T15:49:47.696 INFO:tasks.workunit.client.1.smithi193.stdout:2/706: chown d0/d42/d51/df1/d9f/dc5/f103 577872712 1 2024-04-25T15:49:47.697 INFO:tasks.workunit.client.0.smithi110.stdout:2/820: write de/d40/d5b/d5c/d5f/db2/da9/dac/fe1 [259326,82569] 0 2024-04-25T15:49:47.698 INFO:tasks.workunit.client.1.smithi193.stdout:6/686: fsync dd/d12/d19/d1c/d40/d3c/d3f/f61 0 2024-04-25T15:49:47.700 INFO:tasks.workunit.client.1.smithi193.stdout:3/732: mkdir d8/d13/d14/d23/de8 0 2024-04-25T15:49:47.701 INFO:tasks.workunit.client.1.smithi193.stdout:1/640: dwrite dc/d4a/d50/d7c/f9e [0,4194304] 0 2024-04-25T15:49:47.701 INFO:tasks.workunit.client.1.smithi193.stdout:3/733: dread - d8/d13/d14/fd3 zero size 2024-04-25T15:49:47.704 INFO:tasks.workunit.client.1.smithi193.stdout:8/699: creat d2/d3/d20/db8/dd0/fe3 x:0 0 0 2024-04-25T15:49:47.705 INFO:tasks.workunit.client.1.smithi193.stdout:3/734: chown d8/d30/d45/ca8 3973100 1 2024-04-25T15:49:47.710 INFO:tasks.workunit.client.1.smithi193.stdout:3/735: chown d8/d13/d14/f80 0 1 2024-04-25T15:49:47.710 INFO:tasks.workunit.client.1.smithi193.stdout:4/763: truncate db/d13/d1e/d1f/d22/f2a 833241 0 2024-04-25T15:49:47.711 INFO:tasks.workunit.client.0.smithi110.stdout:6/841: read d1/dd/f8e [3660986,46862] 0 2024-04-25T15:49:47.711 INFO:tasks.workunit.client.0.smithi110.stdout:3/911: symlink d0/d16/d8c/l135 0 2024-04-25T15:49:47.711 INFO:tasks.workunit.client.1.smithi193.stdout:1/641: dwrite dc/d4a/d4d/d80/faa [0,4194304] 0 2024-04-25T15:49:47.713 INFO:tasks.workunit.client.1.smithi193.stdout:0/731: rename d0/d49/d5/d7/dd/d46/d48/f5b to d0/d49/d5/d7/dd/d46/d48/d62/dd9/ff8 0 2024-04-25T15:49:47.716 INFO:tasks.workunit.client.1.smithi193.stdout:9/688: mknod df/d2c/d34/d3e/cd7 0 2024-04-25T15:49:47.717 INFO:tasks.workunit.client.0.smithi110.stdout:2/821: chown de/d40/d5b/d5c/d5f/db2/da9/dac/fe1 79786 1 2024-04-25T15:49:47.718 INFO:tasks.workunit.client.1.smithi193.stdout:9/689: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/fcc [808791,81768] 0 2024-04-25T15:49:47.721 INFO:tasks.workunit.client.1.smithi193.stdout:6/687: unlink dd/d12/d19/d1c/l49 0 2024-04-25T15:49:47.722 INFO:tasks.workunit.client.1.smithi193.stdout:0/732: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/d99/f9a [0,4194304] 0 2024-04-25T15:49:47.723 INFO:tasks.workunit.client.1.smithi193.stdout:6/688: stat dd/d2f/d57/la3 0 2024-04-25T15:49:47.723 INFO:tasks.workunit.client.0.smithi110.stdout:3/912: creat d0/d16/dcb/d2f/d4c/d63/f136 x:0 0 0 2024-04-25T15:49:47.724 INFO:tasks.workunit.client.1.smithi193.stdout:5/733: creat d17/d18/d75/ff8 x:0 0 0 2024-04-25T15:49:47.727 INFO:tasks.workunit.client.1.smithi193.stdout:8/700: mkdir d2/d3/d27/d69/dae/d4a/d63/da2/de4 0 2024-04-25T15:49:47.728 INFO:tasks.workunit.client.1.smithi193.stdout:4/764: mkdir db/d13/d1e/d1f/d56/d57/df2 0 2024-04-25T15:49:47.729 INFO:tasks.workunit.client.0.smithi110.stdout:6/842: symlink d1/l123 0 2024-04-25T15:49:47.730 INFO:tasks.workunit.client.1.smithi193.stdout:1/642: truncate dc/d4a/d50/d5c/d5d/d3d/f95 4086549 0 2024-04-25T15:49:47.730 INFO:tasks.workunit.client.1.smithi193.stdout:7/716: mkdir d12/d31/d37/d92/de8/d10d 0 2024-04-25T15:49:47.731 INFO:tasks.workunit.client.0.smithi110.stdout:7/952: link d1b/d29/d38/d55/ffc d1b/d29/d38/d55/d7f/f140 0 2024-04-25T15:49:47.732 INFO:tasks.workunit.client.1.smithi193.stdout:1/643: dread - dc/d4a/d50/d5c/d5d/d3d/fcd zero size 2024-04-25T15:49:47.734 INFO:tasks.workunit.client.0.smithi110.stdout:0/834: rename d1/d23/f2b to d1/d23/d30/d33/f127 0 2024-04-25T15:49:47.734 INFO:tasks.workunit.client.1.smithi193.stdout:1/644: read - dc/d4a/d4d/d7a/da6/fbb zero size 2024-04-25T15:49:47.735 INFO:tasks.workunit.client.1.smithi193.stdout:8/701: dwrite d2/d18/db1/fbd [0,4194304] 0 2024-04-25T15:49:47.735 INFO:tasks.workunit.client.0.smithi110.stdout:2/822: dread de/d40/d6e/f8e [0,4194304] 0 2024-04-25T15:49:47.736 INFO:tasks.workunit.client.1.smithi193.stdout:8/702: chown d2/d3/d27/l8d 3 1 2024-04-25T15:49:47.738 INFO:tasks.workunit.client.0.smithi110.stdout:7/953: mknod d1b/d29/d38/d55/d95/dcb/c141 0 2024-04-25T15:49:47.739 INFO:tasks.workunit.client.0.smithi110.stdout:6/843: dread d1/db/f44 [0,4194304] 0 2024-04-25T15:49:47.739 INFO:tasks.workunit.client.0.smithi110.stdout:3/913: creat d0/f137 x:0 0 0 2024-04-25T15:49:47.741 INFO:tasks.workunit.client.1.smithi193.stdout:0/733: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf/ff9 x:0 0 0 2024-04-25T15:49:47.746 INFO:tasks.workunit.client.1.smithi193.stdout:4/765: mkdir db/d84/df3 0 2024-04-25T15:49:47.747 INFO:tasks.workunit.client.1.smithi193.stdout:4/766: write db/d4b/fba [538854,37707] 0 2024-04-25T15:49:47.754 INFO:tasks.workunit.client.0.smithi110.stdout:5/904: write d4/d15/d51/f9c [1421386,85909] 0 2024-04-25T15:49:47.755 INFO:tasks.workunit.client.1.smithi193.stdout:7/717: dwrite f2 [0,4194304] 0 2024-04-25T15:49:47.756 INFO:tasks.workunit.client.1.smithi193.stdout:7/718: readlink d12/d31/d39/d60/ld2 0 2024-04-25T15:49:47.761 INFO:tasks.workunit.client.0.smithi110.stdout:4/830: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/f57 [0,4194304] 0 2024-04-25T15:49:47.763 INFO:tasks.workunit.client.1.smithi193.stdout:1/645: dread dc/f2b [0,4194304] 0 2024-04-25T15:49:47.764 INFO:tasks.workunit.client.1.smithi193.stdout:1/646: dread - dc/d4a/d50/d5c/d5d/f51 zero size 2024-04-25T15:49:47.776 INFO:tasks.workunit.client.1.smithi193.stdout:0/734: mknod d0/d49/d5/d7/dd/d16/d18/d28/dcf/cfa 0 2024-04-25T15:49:47.776 INFO:tasks.workunit.client.0.smithi110.stdout:0/835: link d1/d47/d54/f6d d1/d6c/d7f/d79/f128 0 2024-04-25T15:49:47.777 INFO:tasks.workunit.client.1.smithi193.stdout:0/735: fdatasync d0/d49/d5/d7/dd/d16/d26/df5/f4c 0 2024-04-25T15:49:47.778 INFO:tasks.workunit.client.1.smithi193.stdout:0/736: fdatasync d0/d49/d5/d7/dd/d16/d26/ff6 0 2024-04-25T15:49:47.778 INFO:tasks.workunit.client.1.smithi193.stdout:3/736: creat d8/d30/fe9 x:0 0 0 2024-04-25T15:49:47.780 INFO:tasks.workunit.client.1.smithi193.stdout:4/767: symlink db/d4b/d51/d54/d77/lf4 0 2024-04-25T15:49:47.782 INFO:tasks.workunit.client.1.smithi193.stdout:2/707: getdents d0/d42 0 2024-04-25T15:49:47.784 INFO:tasks.workunit.client.1.smithi193.stdout:2/708: write d0/d42/d51/f40 [8664417,81937] 0 2024-04-25T15:49:47.785 INFO:tasks.workunit.client.1.smithi193.stdout:0/737: dwrite d0/fda [0,4194304] 0 2024-04-25T15:49:47.786 INFO:tasks.workunit.client.0.smithi110.stdout:5/905: unlink d4/d1d/d85/l9f 0 2024-04-25T15:49:47.786 INFO:tasks.workunit.client.0.smithi110.stdout:7/954: write d1b/d29/d38/d55/d7f/f72 [4187715,12692] 0 2024-04-25T15:49:47.787 INFO:tasks.workunit.client.1.smithi193.stdout:0/738: write d0/d49/d5/d7/dd/f3c [3008185,81833] 0 2024-04-25T15:49:47.789 INFO:tasks.workunit.client.0.smithi110.stdout:9/826: sync 2024-04-25T15:49:47.789 INFO:tasks.workunit.client.0.smithi110.stdout:8/973: sync 2024-04-25T15:49:47.790 INFO:tasks.workunit.client.0.smithi110.stdout:7/955: dread - d1b/d29/d38/d55/d7f/f11f zero size 2024-04-25T15:49:47.790 INFO:tasks.workunit.client.0.smithi110.stdout:2/823: dwrite de/f2d [0,4194304] 0 2024-04-25T15:49:47.798 INFO:tasks.workunit.client.0.smithi110.stdout:6/844: getdents d1/dd/db5 0 2024-04-25T15:49:47.801 INFO:tasks.workunit.client.0.smithi110.stdout:5/906: rmdir d4/d1d/d1f/d35/d3d/d93 39 2024-04-25T15:49:47.803 INFO:tasks.workunit.client.0.smithi110.stdout:3/914: write d0/f2c [2888101,53706] 0 2024-04-25T15:49:47.804 INFO:tasks.workunit.client.0.smithi110.stdout:3/915: chown d0/d7/d7d/l9c 2025 1 2024-04-25T15:49:47.804 INFO:tasks.workunit.client.1.smithi193.stdout:6/689: truncate dd/d12/d19/f7f 3786324 0 2024-04-25T15:49:47.805 INFO:tasks.workunit.client.0.smithi110.stdout:3/916: dread - d0/d2e/dfc/f11f zero size 2024-04-25T15:49:47.807 INFO:tasks.workunit.client.0.smithi110.stdout:9/827: creat d9/d13/d5b/d61/db3/f103 x:0 0 0 2024-04-25T15:49:47.808 INFO:tasks.workunit.client.1.smithi193.stdout:9/690: dwrite df/d2c/d2f/d4e/f19 [0,4194304] 0 2024-04-25T15:49:47.810 INFO:tasks.workunit.client.1.smithi193.stdout:9/691: readlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/l96 0 2024-04-25T15:49:47.811 INFO:tasks.workunit.client.0.smithi110.stdout:4/831: write d13/dba/ffb [488852,83238] 0 2024-04-25T15:49:47.815 INFO:tasks.workunit.client.1.smithi193.stdout:3/737: rmdir d8/d13 39 2024-04-25T15:49:47.818 INFO:tasks.workunit.client.1.smithi193.stdout:7/719: write d12/d1c/d63/d70/d68/d8d/f9c [844316,39730] 0 2024-04-25T15:49:47.819 INFO:tasks.workunit.client.0.smithi110.stdout:5/907: dread d4/d1d/d1f/d35/d3d/f5d [0,4194304] 0 2024-04-25T15:49:47.819 INFO:tasks.workunit.client.1.smithi193.stdout:7/720: write d12/f2e [2807463,52964] 0 2024-04-25T15:49:47.821 INFO:tasks.workunit.client.0.smithi110.stdout:7/956: mkdir d1b/d29/db5/dca/d10b/d112/d142 0 2024-04-25T15:49:47.821 INFO:tasks.workunit.client.0.smithi110.stdout:2/824: truncate de/d40/d5b/d5c/d5f/db2/f95 1321921 0 2024-04-25T15:49:47.825 INFO:tasks.workunit.client.0.smithi110.stdout:2/825: stat de/d40/d6e/d50/db7/faa 0 2024-04-25T15:49:47.825 INFO:tasks.workunit.client.0.smithi110.stdout:7/957: truncate d1b/d29/db5/dca/fd5 2601620 0 2024-04-25T15:49:47.826 INFO:tasks.workunit.client.0.smithi110.stdout:0/836: rename d1/d23/d30/d33/d35/da1 to d1/d23/d30/d33/d129 0 2024-04-25T15:49:47.831 INFO:tasks.workunit.client.0.smithi110.stdout:9/828: rmdir d9/db/df/d25 39 2024-04-25T15:49:47.834 INFO:tasks.workunit.client.1.smithi193.stdout:4/768: write db/d13/d1e/d1f/f64 [285355,93358] 0 2024-04-25T15:49:47.837 INFO:tasks.workunit.client.0.smithi110.stdout:4/832: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d11a 0 2024-04-25T15:49:47.839 INFO:tasks.workunit.client.0.smithi110.stdout:5/908: rmdir d4/d2c/d83 39 2024-04-25T15:49:47.839 INFO:tasks.workunit.client.0.smithi110.stdout:5/909: dread - d4/d15/d51/fa4 zero size 2024-04-25T15:49:47.842 INFO:tasks.workunit.client.0.smithi110.stdout:8/974: write d14/d32/d36/f64 [844226,111878] 0 2024-04-25T15:49:47.844 INFO:tasks.workunit.client.0.smithi110.stdout:8/975: chown d14/d21/f62 485 1 2024-04-25T15:49:47.845 INFO:tasks.workunit.client.0.smithi110.stdout:7/958: unlink d1b/d29/d46/f47 0 2024-04-25T15:49:47.847 INFO:tasks.workunit.client.0.smithi110.stdout:2/826: mknod de/d40/dd7/c10b 0 2024-04-25T15:49:47.858 INFO:tasks.workunit.client.0.smithi110.stdout:0/837: mknod d1/d23/d30/d33/d5b/dbb/c12a 0 2024-04-25T15:49:47.858 INFO:tasks.workunit.client.0.smithi110.stdout:3/917: link d0/d16/dcb/c23 d0/d16/d8c/c138 0 2024-04-25T15:49:47.865 INFO:tasks.workunit.client.0.smithi110.stdout:8/976: rename d14/d21/d26/dab/dac/ff4 to d14/d32/d36/db7/f157 0 2024-04-25T15:49:47.868 INFO:tasks.workunit.client.0.smithi110.stdout:7/959: mknod d1b/d29/d38/d55/d139/d13b/c143 0 2024-04-25T15:49:47.869 INFO:tasks.workunit.client.0.smithi110.stdout:7/960: read d1b/f97 [681783,98201] 0 2024-04-25T15:49:47.870 INFO:tasks.workunit.client.0.smithi110.stdout:2/827: mkdir de/d40/d6e/d52/d6d/d7d/dff/d10c 0 2024-04-25T15:49:47.872 INFO:tasks.workunit.client.0.smithi110.stdout:2/828: fdatasync de/d40/d6e/d52/d6d/d99/fef 0 2024-04-25T15:49:47.873 INFO:tasks.workunit.client.0.smithi110.stdout:2/829: readlink de/d40/d6e/d50/le9 0 2024-04-25T15:49:47.873 INFO:tasks.workunit.client.0.smithi110.stdout:0/838: dread d1/d6c/f94 [0,4194304] 0 2024-04-25T15:49:47.874 INFO:tasks.workunit.client.0.smithi110.stdout:7/961: dread d1b/d29/db5/dca/d10b/d112/f13a [0,4194304] 0 2024-04-25T15:49:47.875 INFO:tasks.workunit.client.0.smithi110.stdout:3/918: creat d0/d16/dcb/d50/f139 x:0 0 0 2024-04-25T15:49:47.878 INFO:tasks.workunit.client.0.smithi110.stdout:4/833: creat d13/d24/d2d/f11b x:0 0 0 2024-04-25T15:49:47.880 INFO:tasks.workunit.client.0.smithi110.stdout:5/910: creat d4/f129 x:0 0 0 2024-04-25T15:49:47.881 INFO:tasks.workunit.client.0.smithi110.stdout:5/911: write d4/d1d/d1f/d9e/f10d [1145265,2506] 0 2024-04-25T15:49:47.883 INFO:tasks.workunit.client.0.smithi110.stdout:8/977: readlink d14/d32/d36/db7/dc9/lce 0 2024-04-25T15:49:47.884 INFO:tasks.workunit.client.0.smithi110.stdout:8/978: write d14/d21/d26/d6a/de9/f107 [1645005,76408] 0 2024-04-25T15:49:47.887 INFO:tasks.workunit.client.1.smithi193.stdout:5/734: mknod d17/d18/d75/d85/d5d/dad/cf9 0 2024-04-25T15:49:47.892 INFO:tasks.workunit.client.0.smithi110.stdout:9/829: dwrite d9/db/d7d/ffe [0,4194304] 0 2024-04-25T15:49:47.895 INFO:tasks.workunit.client.0.smithi110.stdout:2/830: dread de/d40/f96 [0,4194304] 0 2024-04-25T15:49:47.895 INFO:tasks.workunit.client.0.smithi110.stdout:0/839: unlink d1/f8d 0 2024-04-25T15:49:47.896 INFO:tasks.workunit.client.0.smithi110.stdout:0/840: chown d1/d23/d30/d33/d35/l55 693 1 2024-04-25T15:49:47.897 INFO:tasks.workunit.client.1.smithi193.stdout:3/738: readlink d8/d13/d1f/d69/d77/la4 0 2024-04-25T15:49:47.901 INFO:tasks.workunit.client.1.smithi193.stdout:7/721: fdatasync d12/d13/f17 0 2024-04-25T15:49:47.902 INFO:tasks.workunit.client.1.smithi193.stdout:7/722: fdatasync d12/d1c/d63/d70/d68/d8d/f9c 0 2024-04-25T15:49:47.902 INFO:tasks.workunit.client.0.smithi110.stdout:2/831: dwrite de/fa2 [0,4194304] 0 2024-04-25T15:49:47.905 INFO:tasks.workunit.client.0.smithi110.stdout:4/834: rename d13/d24/d2d/d5e/def/c10e to d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d10d/c11c 0 2024-04-25T15:49:47.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/835: chown d13/l21 4062605 1 2024-04-25T15:49:47.907 INFO:tasks.workunit.client.0.smithi110.stdout:2/832: read de/f2d [4350300,78065] 0 2024-04-25T15:49:47.908 INFO:tasks.workunit.client.1.smithi193.stdout:0/739: fdatasync d0/d49/d5/d7/dd/d16/d26/d5e/f69 0 2024-04-25T15:49:47.911 INFO:tasks.workunit.client.1.smithi193.stdout:4/769: creat db/d4b/d51/d54/d77/ff5 x:0 0 0 2024-04-25T15:49:47.915 INFO:tasks.workunit.client.0.smithi110.stdout:2/833: dwrite de/d40/d4f/f8b [0,4194304] 0 2024-04-25T15:49:47.920 INFO:tasks.workunit.client.0.smithi110.stdout:2/834: dread de/d40/d6e/f9b [4194304,4194304] 0 2024-04-25T15:49:47.921 INFO:tasks.workunit.client.0.smithi110.stdout:2/835: chown de/d40/d5b/d5c/d5f/db2/fc4 11 1 2024-04-25T15:49:47.922 INFO:tasks.workunit.client.0.smithi110.stdout:2/836: chown de/d40/dcd/ddf/ff2 1471026146 1 2024-04-25T15:49:47.923 INFO:tasks.workunit.client.0.smithi110.stdout:2/837: chown de/d40/d6e/d52/dda/ffb 1149585 1 2024-04-25T15:49:47.924 INFO:tasks.workunit.client.0.smithi110.stdout:8/979: creat d14/d21/d26/d6a/d118/f158 x:0 0 0 2024-04-25T15:49:47.925 INFO:tasks.workunit.client.1.smithi193.stdout:1/647: unlink dc/d4a/d50/d5c/d5d/d3d/d55/d8a/db7/cc6 0 2024-04-25T15:49:47.931 INFO:tasks.workunit.client.1.smithi193.stdout:5/735: rmdir d17/d18/d3e/dd1/ddc 39 2024-04-25T15:49:47.935 INFO:tasks.workunit.client.0.smithi110.stdout:9/830: write d9/d44/d5e/db5/fda [624210,38273] 0 2024-04-25T15:49:47.936 INFO:tasks.workunit.client.1.smithi193.stdout:5/736: dwrite d17/d18/d3e/f4c [0,4194304] 0 2024-04-25T15:49:47.937 INFO:tasks.workunit.client.1.smithi193.stdout:9/692: unlink df/d2c/d2f/d5c/fb2 0 2024-04-25T15:49:47.937 INFO:tasks.workunit.client.1.smithi193.stdout:5/737: chown d17/d18/d3e/db6 82 1 2024-04-25T15:49:47.941 INFO:tasks.workunit.client.0.smithi110.stdout:3/919: mknod d0/d16/c13a 0 2024-04-25T15:49:47.942 INFO:tasks.workunit.client.0.smithi110.stdout:3/920: chown d0/d7/fe5 91825 1 2024-04-25T15:49:47.944 INFO:tasks.workunit.client.1.smithi193.stdout:7/723: creat d12/d31/d37/d92/dbf/f10e x:0 0 0 2024-04-25T15:49:47.944 INFO:tasks.workunit.client.1.smithi193.stdout:7/724: stat d12/d31/d37/d104/f94 0 2024-04-25T15:49:47.945 INFO:tasks.workunit.client.1.smithi193.stdout:7/725: dread - d12/d1c/d63/d109/fdc zero size 2024-04-25T15:49:47.945 INFO:tasks.workunit.client.0.smithi110.stdout:7/962: mknod d1b/d29/d38/d55/d7f/dc8/d13d/c144 0 2024-04-25T15:49:47.949 INFO:tasks.workunit.client.0.smithi110.stdout:4/836: mknod d13/d24/d2d/d44/d65/c11d 0 2024-04-25T15:49:47.951 INFO:tasks.workunit.client.1.smithi193.stdout:4/770: read db/d13/d1e/d1f/f63 [2728887,42961] 0 2024-04-25T15:49:47.951 INFO:tasks.workunit.client.0.smithi110.stdout:5/912: getdents d4/d15/d51/d90/ded/d106 0 2024-04-25T15:49:47.954 INFO:tasks.workunit.client.1.smithi193.stdout:8/703: link d2/d3/d3a/lbe d2/d3/d27/d69/dbf/le5 0 2024-04-25T15:49:47.959 INFO:tasks.workunit.client.0.smithi110.stdout:9/831: creat d9/db/d58/f104 x:0 0 0 2024-04-25T15:49:47.959 INFO:tasks.workunit.client.0.smithi110.stdout:9/832: chown d9/d44/d5e 0 1 2024-04-25T15:49:47.966 INFO:tasks.workunit.client.0.smithi110.stdout:4/837: rename d13/d24/d2d/d5e/fa9 to d13/dba/dd8/f11e 0 2024-04-25T15:49:47.969 INFO:tasks.workunit.client.1.smithi193.stdout:2/709: getdents d0/d9 0 2024-04-25T15:49:47.970 INFO:tasks.workunit.client.1.smithi193.stdout:2/710: write d0/d42/d51/d4f/f69 [2626838,62546] 0 2024-04-25T15:49:47.971 INFO:tasks.workunit.client.1.smithi193.stdout:7/726: dread d12/d31/d39/db0/fb5 [0,4194304] 0 2024-04-25T15:49:47.971 INFO:tasks.workunit.client.0.smithi110.stdout:1/997: sync 2024-04-25T15:49:47.973 INFO:tasks.workunit.client.0.smithi110.stdout:7/963: dwrite d1b/d29/d38/d8b/d89/fcc [4194304,4194304] 0 2024-04-25T15:49:47.973 INFO:tasks.workunit.client.1.smithi193.stdout:0/740: symlink d0/d49/d5/d7/dd/d46/d48/lfb 0 2024-04-25T15:49:47.974 INFO:tasks.workunit.client.0.smithi110.stdout:1/998: chown d0/d51/dc8/f127 1 1 2024-04-25T15:49:47.975 INFO:tasks.workunit.client.0.smithi110.stdout:8/980: truncate d14/d32/fe4 1029597 0 2024-04-25T15:49:47.975 INFO:tasks.workunit.client.1.smithi193.stdout:4/771: mkdir db/d4b/d51/d54/d77/df6 0 2024-04-25T15:49:47.976 INFO:tasks.workunit.client.0.smithi110.stdout:0/841: creat d1/d23/d30/d33/d5b/f12b x:0 0 0 2024-04-25T15:49:47.978 INFO:tasks.workunit.client.0.smithi110.stdout:8/981: dread d14/f94 [0,4194304] 0 2024-04-25T15:49:47.978 INFO:tasks.workunit.client.1.smithi193.stdout:1/648: rename dc/d4a/d4d/l4e to dc/d4a/d50/d7c/dda/ldb 0 2024-04-25T15:49:47.981 INFO:tasks.workunit.client.1.smithi193.stdout:8/704: creat d2/d3/d20/d8c/da6/fe6 x:0 0 0 2024-04-25T15:49:47.981 INFO:tasks.workunit.client.0.smithi110.stdout:9/833: dread d9/d13/d5b/d61/db3/dd3/ddb/fec [0,4194304] 0 2024-04-25T15:49:47.981 INFO:tasks.workunit.client.1.smithi193.stdout:6/690: link dd/d12/d19/d1c/d40/d3c/d3f/d4f/c82 dd/d12/d19/d1c/d40/d3c/d3f/ccf 0 2024-04-25T15:49:47.982 INFO:tasks.workunit.client.1.smithi193.stdout:6/691: stat dd/d12/d19/d1c/d40/d3c 0 2024-04-25T15:49:47.985 INFO:tasks.workunit.client.1.smithi193.stdout:3/739: link d8/d13/d1f/d9f/fa5 d8/d13/d14/d23/de8/fea 0 2024-04-25T15:49:47.985 INFO:tasks.workunit.client.0.smithi110.stdout:4/838: creat d13/d24/d2d/d36/d51/f11f x:0 0 0 2024-04-25T15:49:47.986 INFO:tasks.workunit.client.1.smithi193.stdout:3/740: chown d8/d30/d72/fa3 893 1 2024-04-25T15:49:47.986 INFO:tasks.workunit.client.1.smithi193.stdout:3/741: chown d8/d13/d14/d23/de8 20 1 2024-04-25T15:49:47.988 INFO:tasks.workunit.client.0.smithi110.stdout:2/838: link de/d40/d6e/f9b de/d40/f10d 0 2024-04-25T15:49:47.989 INFO:tasks.workunit.client.0.smithi110.stdout:7/964: rmdir d1b/d29/d38/d8b 39 2024-04-25T15:49:47.995 INFO:tasks.workunit.client.0.smithi110.stdout:1/999: creat d0/d51/dd8/df0/f145 x:0 0 0 2024-04-25T15:49:47.996 INFO:tasks.workunit.client.0.smithi110.stdout:5/913: dread d4/d1d/d1f/f64 [0,4194304] 0 2024-04-25T15:49:48.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/982: symlink d14/d32/d36/l159 0 2024-04-25T15:49:48.000 INFO:tasks.workunit.client.0.smithi110.stdout:8/983: chown d14/d32/d36/l5f 974466 1 2024-04-25T15:49:48.001 INFO:tasks.workunit.client.1.smithi193.stdout:1/649: mkdir dc/d4a/d50/d7c/d82/ddc 0 2024-04-25T15:49:48.006 INFO:tasks.workunit.client.1.smithi193.stdout:9/693: creat df/fd8 x:0 0 0 2024-04-25T15:49:48.007 INFO:tasks.workunit.client.1.smithi193.stdout:9/694: readlink df/l10 0 2024-04-25T15:49:48.007 INFO:tasks.workunit.client.1.smithi193.stdout:1/650: dwrite dc/d4a/d4d/d80/faa [0,4194304] 0 2024-04-25T15:49:48.008 INFO:tasks.workunit.client.1.smithi193.stdout:9/695: dread - df/d2c/d34/d3e/d4d/d5e/fd5 zero size 2024-04-25T15:49:48.009 INFO:tasks.workunit.client.1.smithi193.stdout:8/705: rmdir d2/d3/d20/d8c/da6 39 2024-04-25T15:49:48.010 INFO:tasks.workunit.client.0.smithi110.stdout:2/839: mknod de/d40/d6e/d52/d6d/d99/c10e 0 2024-04-25T15:49:48.011 INFO:tasks.workunit.client.1.smithi193.stdout:8/706: write d2/da5/fa7 [3687,106239] 0 2024-04-25T15:49:48.012 INFO:tasks.workunit.client.1.smithi193.stdout:6/692: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0 0 2024-04-25T15:49:48.012 INFO:tasks.workunit.client.1.smithi193.stdout:8/707: chown d2/d3/d27/d69/dbf 153 1 2024-04-25T15:49:48.014 INFO:tasks.workunit.client.1.smithi193.stdout:6/693: write dd/d12/d19/d1c/d40/d3c/d3f/f54 [4200387,10188] 0 2024-04-25T15:49:48.014 INFO:tasks.workunit.client.1.smithi193.stdout:3/742: creat d8/d13/d1f/feb x:0 0 0 2024-04-25T15:49:48.015 INFO:tasks.workunit.client.0.smithi110.stdout:5/914: dread d4/d1d/d1f/d35/d3d/dc8/f115 [0,4194304] 0 2024-04-25T15:49:48.015 INFO:tasks.workunit.client.1.smithi193.stdout:3/743: chown d8/d13/d1f/d9f/ca9 176149985 1 2024-04-25T15:49:48.016 INFO:tasks.workunit.client.0.smithi110.stdout:0/842: mknod d1/d23/dc5/c12c 0 2024-04-25T15:49:48.022 INFO:tasks.workunit.client.1.smithi193.stdout:2/711: dwrite d0/d44/fa8 [0,4194304] 0 2024-04-25T15:49:48.022 INFO:tasks.workunit.client.0.smithi110.stdout:9/834: symlink d9/db/df/d2c/l105 0 2024-04-25T15:49:48.023 INFO:tasks.workunit.client.0.smithi110.stdout:4/839: symlink d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d11a/l120 0 2024-04-25T15:49:48.024 INFO:tasks.workunit.client.0.smithi110.stdout:9/835: dread d9/d13/d5b/d61/db3/db4/dc5/de4/fe8 [0,4194304] 0 2024-04-25T15:49:48.025 INFO:tasks.workunit.client.1.smithi193.stdout:4/772: dwrite db/d32/d45/dab/fc8 [0,4194304] 0 2024-04-25T15:49:48.026 INFO:tasks.workunit.client.1.smithi193.stdout:1/651: rmdir dc/d4a/d50/d5c/d5d/d3d 39 2024-04-25T15:49:48.030 INFO:tasks.workunit.client.1.smithi193.stdout:9/696: readlink df/l12 0 2024-04-25T15:49:48.031 INFO:tasks.workunit.client.1.smithi193.stdout:4/773: dwrite db/d32/d45/dab/fc8 [0,4194304] 0 2024-04-25T15:49:48.032 INFO:tasks.workunit.client.1.smithi193.stdout:1/652: dread dc/d4a/d50/d7c/f9e [0,4194304] 0 2024-04-25T15:49:48.032 INFO:tasks.workunit.client.1.smithi193.stdout:9/697: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/l72 2029687 1 2024-04-25T15:49:48.033 INFO:tasks.workunit.client.1.smithi193.stdout:9/698: readlink df/l12 0 2024-04-25T15:49:48.034 INFO:tasks.workunit.client.1.smithi193.stdout:5/738: link d17/d18/d30/d3b/c7d d17/d18/d30/d34/d52/cfa 0 2024-04-25T15:49:48.035 INFO:tasks.workunit.client.0.smithi110.stdout:0/843: unlink d1/l1e 0 2024-04-25T15:49:48.037 INFO:tasks.workunit.client.1.smithi193.stdout:9/699: dread df/d2c/d2f/d4e/f19 [0,4194304] 0 2024-04-25T15:49:48.039 INFO:tasks.workunit.client.1.smithi193.stdout:9/700: truncate df/d2c/d34/faa 4259696 0 2024-04-25T15:49:48.042 INFO:tasks.workunit.client.0.smithi110.stdout:8/984: rename c7 to d14/d32/d36/dde/d111/c15a 0 2024-04-25T15:49:48.044 INFO:tasks.workunit.client.1.smithi193.stdout:8/708: fdatasync d2/d3/f37 0 2024-04-25T15:49:48.051 INFO:tasks.workunit.client.0.smithi110.stdout:4/840: creat d13/dba/dd8/f121 x:0 0 0 2024-04-25T15:49:48.051 INFO:tasks.workunit.client.0.smithi110.stdout:8/985: dwrite d14/d21/d26/dab/dac/dbb/f136 [0,4194304] 0 2024-04-25T15:49:48.051 INFO:tasks.workunit.client.1.smithi193.stdout:6/694: fsync dd/f74 0 2024-04-25T15:49:48.054 INFO:tasks.workunit.client.0.smithi110.stdout:7/965: link f14 d1b/d29/db5/dca/d10b/f145 0 2024-04-25T15:49:48.058 INFO:tasks.workunit.client.0.smithi110.stdout:0/844: symlink d1/d23/d30/d33/d5b/d78/ddb/d115/l12d 0 2024-04-25T15:49:48.060 INFO:tasks.workunit.client.1.smithi193.stdout:0/741: creat d0/d49/d5/d7/dd/ffc x:0 0 0 2024-04-25T15:49:48.062 INFO:tasks.workunit.client.0.smithi110.stdout:8/986: mknod d14/d21/d26/dab/dac/dbb/c15b 0 2024-04-25T15:49:48.064 INFO:tasks.workunit.client.0.smithi110.stdout:7/966: dwrite d1b/d29/f37 [0,4194304] 0 2024-04-25T15:49:48.067 INFO:tasks.workunit.client.0.smithi110.stdout:4/841: unlink d13/d24/d2d/d36/d51/fe4 0 2024-04-25T15:49:48.074 INFO:tasks.workunit.client.1.smithi193.stdout:4/774: creat db/d4b/d51/d54/d77/ff7 x:0 0 0 2024-04-25T15:49:48.076 INFO:tasks.workunit.client.1.smithi193.stdout:6/695: dread dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f87 [0,4194304] 0 2024-04-25T15:49:48.076 INFO:tasks.workunit.client.0.smithi110.stdout:0/845: creat d1/d47/d54/f12e x:0 0 0 2024-04-25T15:49:48.077 INFO:tasks.workunit.client.1.smithi193.stdout:1/653: chown dc/d4a/d50/d5c/d5d/d3d/c8d 425 1 2024-04-25T15:49:48.084 INFO:tasks.workunit.client.1.smithi193.stdout:9/701: mkdir df/d2c/d2f/d5c/dd9 0 2024-04-25T15:49:48.084 INFO:tasks.workunit.client.1.smithi193.stdout:9/702: write df/d2d/fb1 [249642,69947] 0 2024-04-25T15:49:48.086 INFO:tasks.workunit.client.0.smithi110.stdout:2/840: rename de/d40/d6e/d50/f7b to de/d6f/f10f 0 2024-04-25T15:49:48.087 INFO:tasks.workunit.client.0.smithi110.stdout:0/846: mknod d1/d23/d30/d33/d129/c12f 0 2024-04-25T15:49:48.090 INFO:tasks.workunit.client.0.smithi110.stdout:8/987: unlink d14/d32/d36/d48/d8f/c125 0 2024-04-25T15:49:48.092 INFO:tasks.workunit.client.1.smithi193.stdout:5/739: write d17/fef [1018463,13397] 0 2024-04-25T15:49:48.093 INFO:tasks.workunit.client.1.smithi193.stdout:7/727: getdents d12/d31/d39/db0 0 2024-04-25T15:49:48.094 INFO:tasks.workunit.client.1.smithi193.stdout:5/740: dread d17/d18/d30/d34/f6c [4194304,4194304] 0 2024-04-25T15:49:48.094 INFO:tasks.workunit.client.1.smithi193.stdout:7/728: write d12/d13/f103 [442069,48336] 0 2024-04-25T15:49:48.096 INFO:tasks.workunit.client.1.smithi193.stdout:5/741: fsync d17/d18/d30/d38/d71/d96/dc4/f56 0 2024-04-25T15:49:48.096 INFO:tasks.workunit.client.0.smithi110.stdout:2/841: truncate de/d40/d6e/feb 667654 0 2024-04-25T15:49:48.096 INFO:tasks.workunit.client.1.smithi193.stdout:3/744: rename d8/d13/d14/d27/d54/dd7/fe5 to d8/d13/d1f/d69/dcd/dd9/fec 0 2024-04-25T15:49:48.097 INFO:tasks.workunit.client.1.smithi193.stdout:3/745: chown d8/d30/d72 392 1 2024-04-25T15:49:48.097 INFO:tasks.workunit.client.0.smithi110.stdout:4/842: dwrite d13/d24/d2d/d5e/f7b [0,4194304] 0 2024-04-25T15:49:48.100 INFO:tasks.workunit.client.0.smithi110.stdout:5/915: rename d4/d1d/d1f/d35/d7f/fac to d4/d15/f12a 0 2024-04-25T15:49:48.108 INFO:tasks.workunit.client.1.smithi193.stdout:4/775: creat db/d32/d45/ff8 x:0 0 0 2024-04-25T15:49:48.110 INFO:tasks.workunit.client.0.smithi110.stdout:2/842: read de/d40/f36 [751554,105118] 0 2024-04-25T15:49:48.110 INFO:tasks.workunit.client.0.smithi110.stdout:4/843: mkdir d13/d24/d2d/d5e/d122 0 2024-04-25T15:49:48.110 INFO:tasks.workunit.client.1.smithi193.stdout:1/654: readlink dc/d3c/l86 0 2024-04-25T15:49:48.111 INFO:tasks.workunit.client.1.smithi193.stdout:1/655: stat dc/d4a/d50/d7c/d7e/ld4 0 2024-04-25T15:49:48.111 INFO:tasks.workunit.client.0.smithi110.stdout:5/916: truncate d4/d1d/d1f/f8f 56156 0 2024-04-25T15:49:48.112 INFO:tasks.workunit.client.0.smithi110.stdout:4/844: write d13/d24/d2d/d36/d51/f11f [913504,121192] 0 2024-04-25T15:49:48.114 INFO:tasks.workunit.client.0.smithi110.stdout:0/847: write d1/d23/d30/d33/d5b/d78/fb1 [969201,38886] 0 2024-04-25T15:49:48.117 INFO:tasks.workunit.client.0.smithi110.stdout:7/967: getdents d1b/d29/d38/d55/dab/db2/dcd/de2 0 2024-04-25T15:49:48.118 INFO:tasks.workunit.client.1.smithi193.stdout:9/703: truncate df/d2c/d2f/d4e/d1c/f3d 1755575 0 2024-04-25T15:49:48.118 INFO:tasks.workunit.client.1.smithi193.stdout:9/704: write df/d2d/fb1 [469236,46660] 0 2024-04-25T15:49:48.118 INFO:tasks.workunit.client.1.smithi193.stdout:9/705: fdatasync df/d2c/d2f/d5c/fb4 0 2024-04-25T15:49:48.118 INFO:tasks.workunit.client.0.smithi110.stdout:5/917: stat d4/d2c/d83/l11a 0 2024-04-25T15:49:48.119 INFO:tasks.workunit.client.1.smithi193.stdout:8/709: symlink d2/d18/le7 0 2024-04-25T15:49:48.120 INFO:tasks.workunit.client.0.smithi110.stdout:2/843: mkdir de/d40/d5b/d5c/d5f/db2/d110 0 2024-04-25T15:49:48.122 INFO:tasks.workunit.client.0.smithi110.stdout:2/844: read de/d40/d4f/f49 [2651253,74722] 0 2024-04-25T15:49:48.122 INFO:tasks.workunit.client.0.smithi110.stdout:0/848: mkdir d1/d23/dc5/d130 0 2024-04-25T15:49:48.125 INFO:tasks.workunit.client.1.smithi193.stdout:5/742: truncate d17/d18/ff1 3064136 0 2024-04-25T15:49:48.126 INFO:tasks.workunit.client.0.smithi110.stdout:2/845: dread de/d40/d5b/d5c/d5f/db2/fcb [0,4194304] 0 2024-04-25T15:49:48.127 INFO:tasks.workunit.client.0.smithi110.stdout:7/968: rename d1b/d29/d38/d55/d7f/lea to d1b/d29/db5/dca/d10b/d112/d13c/l146 0 2024-04-25T15:49:48.129 INFO:tasks.workunit.client.0.smithi110.stdout:4/845: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d117/f123 x:0 0 0 2024-04-25T15:49:48.129 INFO:tasks.workunit.client.1.smithi193.stdout:2/712: link d0/c7c d0/d9/d3d/c10e 0 2024-04-25T15:49:48.131 INFO:tasks.workunit.client.0.smithi110.stdout:8/988: getdents d14/d21/d26/dab 0 2024-04-25T15:49:48.136 INFO:tasks.workunit.client.1.smithi193.stdout:6/696: link dd/d12/d19/d1c/fa4 dd/d12/d2d/fd1 0 2024-04-25T15:49:48.140 INFO:tasks.workunit.client.0.smithi110.stdout:5/918: chown d4/d15/d51/d90/ded 0 1 2024-04-25T15:49:48.140 INFO:tasks.workunit.client.0.smithi110.stdout:4/846: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/f118 [0,4194304] 0 2024-04-25T15:49:48.144 INFO:tasks.workunit.client.0.smithi110.stdout:8/989: dwrite d14/d21/d26/d6a/d7e/da6/de8/d55/fb4 [4194304,4194304] 0 2024-04-25T15:49:48.145 INFO:tasks.workunit.client.0.smithi110.stdout:2/846: read - de/d40/d5b/d5c/fba zero size 2024-04-25T15:49:48.146 INFO:tasks.workunit.client.1.smithi193.stdout:9/706: rmdir df/d2c/d34/d3e 39 2024-04-25T15:49:48.148 INFO:tasks.workunit.client.0.smithi110.stdout:0/849: mknod d1/d23/dc5/d108/c131 0 2024-04-25T15:49:48.149 INFO:tasks.workunit.client.1.smithi193.stdout:8/710: truncate d2/d3/f37 1217531 0 2024-04-25T15:49:48.151 INFO:tasks.workunit.client.0.smithi110.stdout:7/969: truncate d1b/d29/d38/d55/dab/fb0 969356 0 2024-04-25T15:49:48.157 INFO:tasks.workunit.client.1.smithi193.stdout:7/729: creat d12/d1c/d63/db7/de7/f10f x:0 0 0 2024-04-25T15:49:48.159 INFO:tasks.workunit.client.1.smithi193.stdout:4/776: write db/d13/d1e/d1f/d56/d69/d70/f76 [1465401,49292] 0 2024-04-25T15:49:48.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/746: link d8/d30/fe9 d8/d13/d14/d27/d54/dd7/fed 0 2024-04-25T15:49:48.162 INFO:tasks.workunit.client.1.smithi193.stdout:0/742: creat d0/d49/ffd x:0 0 0 2024-04-25T15:49:48.163 INFO:tasks.workunit.client.1.smithi193.stdout:0/743: dread - d0/d49/d5/fca zero size 2024-04-25T15:49:48.216 INFO:tasks.workunit.client.0.smithi110.stdout:6/845: sync 2024-04-25T15:49:48.220 INFO:tasks.workunit.client.0.smithi110.stdout:2/847: rmdir de/d40/d6e/d52/d6d/d99 39 2024-04-25T15:49:48.226 INFO:tasks.workunit.client.1.smithi193.stdout:6/697: unlink dd/f2b 0 2024-04-25T15:49:48.226 INFO:tasks.workunit.client.1.smithi193.stdout:6/698: stat dd/d12/d19/d45/c6b 0 2024-04-25T15:49:48.228 INFO:tasks.workunit.client.1.smithi193.stdout:6/699: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/fb9 [221392,125942] 0 2024-04-25T15:49:48.228 INFO:tasks.workunit.client.1.smithi193.stdout:1/656: mkdir dc/d4a/d50/d5c/d5d/ddd 0 2024-04-25T15:49:48.232 INFO:tasks.workunit.client.1.smithi193.stdout:9/707: chown df/d2c/d34/d3e/d4d/d5e 194306 1 2024-04-25T15:49:48.240 INFO:tasks.workunit.client.1.smithi193.stdout:7/730: rename d12/d31/d37/d62/lb9 to d12/d1c/dd6/l110 0 2024-04-25T15:49:48.241 INFO:tasks.workunit.client.1.smithi193.stdout:8/711: dwrite d2/d3/d20/d8c/da6/fe6 [0,4194304] 0 2024-04-25T15:49:48.241 INFO:tasks.workunit.client.1.smithi193.stdout:7/731: dread - d12/d13/ffd zero size 2024-04-25T15:49:48.243 INFO:tasks.workunit.client.1.smithi193.stdout:8/712: dread - d2/d18/dbc/fce zero size 2024-04-25T15:49:48.243 INFO:tasks.workunit.client.0.smithi110.stdout:4/847: write d13/f1a [1921405,92926] 0 2024-04-25T15:49:48.245 INFO:tasks.workunit.client.0.smithi110.stdout:0/850: mkdir d1/d47/d132 0 2024-04-25T15:49:48.246 INFO:tasks.workunit.client.0.smithi110.stdout:0/851: write d1/d23/f11e [83613,28227] 0 2024-04-25T15:49:48.248 INFO:tasks.workunit.client.1.smithi193.stdout:3/747: unlink d8/d13/d14/d23/lac 0 2024-04-25T15:49:48.249 INFO:tasks.workunit.client.1.smithi193.stdout:3/748: write d8/d13/d1f/d69/dcd/dd9/fe2 [919480,31510] 0 2024-04-25T15:49:48.249 INFO:tasks.workunit.client.1.smithi193.stdout:0/744: chown d0/d49/d5/c31 18931927 1 2024-04-25T15:49:48.252 INFO:tasks.workunit.client.1.smithi193.stdout:9/708: dread df/d2c/d34/d3e/f4a [0,4194304] 0 2024-04-25T15:49:48.253 INFO:tasks.workunit.client.1.smithi193.stdout:9/709: fsync df/d2d/fb1 0 2024-04-25T15:49:48.254 INFO:tasks.workunit.client.1.smithi193.stdout:9/710: chown df/d2c/d2f/d4e/f37 3622903 1 2024-04-25T15:49:48.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/846: mkdir d1/db/da0/da5/dea/d124 0 2024-04-25T15:49:48.255 INFO:tasks.workunit.client.0.smithi110.stdout:6/847: fsync d1/dd/d3c/d91/d8d/dce/d102/d75/d82/fa6 0 2024-04-25T15:49:48.256 INFO:tasks.workunit.client.1.smithi193.stdout:3/749: dread d8/d13/d14/d23/f48 [0,4194304] 0 2024-04-25T15:49:48.257 INFO:tasks.workunit.client.0.smithi110.stdout:7/970: write d1b/d29/f60 [720943,105169] 0 2024-04-25T15:49:48.258 INFO:tasks.workunit.client.0.smithi110.stdout:7/971: chown d1b/d29/d46/l9b 661 1 2024-04-25T15:49:48.261 INFO:tasks.workunit.client.1.smithi193.stdout:4/777: rename db/d13/d1e/f41 to db/d32/d45/ff9 0 2024-04-25T15:49:48.262 INFO:tasks.workunit.client.0.smithi110.stdout:0/852: symlink d1/d23/d30/l133 0 2024-04-25T15:49:48.263 INFO:tasks.workunit.client.1.smithi193.stdout:7/732: dread d12/d1c/d63/d70/f25 [0,4194304] 0 2024-04-25T15:49:48.263 INFO:tasks.workunit.client.1.smithi193.stdout:3/750: dwrite d8/d30/d45/fe0 [0,4194304] 0 2024-04-25T15:49:48.265 INFO:tasks.workunit.client.1.smithi193.stdout:7/733: truncate d12/d1c/d63/d109/fdc 408762 0 2024-04-25T15:49:48.270 INFO:tasks.workunit.client.0.smithi110.stdout:5/919: getdents d4/d1d 0 2024-04-25T15:49:48.270 INFO:tasks.workunit.client.1.smithi193.stdout:5/743: dwrite d17/d18/d30/f8a [0,4194304] 0 2024-04-25T15:49:48.271 INFO:tasks.workunit.client.0.smithi110.stdout:5/920: chown d4/d2c/d2a 199158 1 2024-04-25T15:49:48.272 INFO:tasks.workunit.client.0.smithi110.stdout:4/848: dread d13/d24/d2d/f62 [0,4194304] 0 2024-04-25T15:49:48.275 INFO:tasks.workunit.client.0.smithi110.stdout:6/848: mknod d1/db/d22/d90/d76/c125 0 2024-04-25T15:49:48.275 INFO:tasks.workunit.client.0.smithi110.stdout:2/848: write de/d40/d6e/f21 [2405318,126799] 0 2024-04-25T15:49:48.279 INFO:tasks.workunit.client.1.smithi193.stdout:2/713: creat d0/d9/d15/f10f x:0 0 0 2024-04-25T15:49:48.279 INFO:tasks.workunit.client.0.smithi110.stdout:7/972: read d1b/d29/d87/fe6 [635034,117835] 0 2024-04-25T15:49:48.280 INFO:tasks.workunit.client.1.smithi193.stdout:2/714: fdatasync d0/d37/f67 0 2024-04-25T15:49:48.281 INFO:tasks.workunit.client.0.smithi110.stdout:7/973: chown d1b/d29/fd0 29551 1 2024-04-25T15:49:48.281 INFO:tasks.workunit.client.1.smithi193.stdout:0/745: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/ffe x:0 0 0 2024-04-25T15:49:48.282 INFO:tasks.workunit.client.1.smithi193.stdout:0/746: fdatasync d0/d49/d5/f2d 0 2024-04-25T15:49:48.282 INFO:tasks.workunit.client.0.smithi110.stdout:4/849: dread - d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/fd3 zero size 2024-04-25T15:49:48.288 INFO:tasks.workunit.client.0.smithi110.stdout:2/849: write de/d40/d5b/d5c/dbe/f103 [136125,121120] 0 2024-04-25T15:49:48.289 INFO:tasks.workunit.client.0.smithi110.stdout:5/921: dread d4/f6d [0,4194304] 0 2024-04-25T15:49:48.291 INFO:tasks.workunit.client.0.smithi110.stdout:5/922: chown d4/d1d/d1f/d9e/fb8 91 1 2024-04-25T15:49:48.291 INFO:tasks.workunit.client.0.smithi110.stdout:4/850: symlink d13/d24/d2d/db5/l124 0 2024-04-25T15:49:48.293 INFO:tasks.workunit.client.1.smithi193.stdout:7/734: creat d12/d1c/d63/f111 x:0 0 0 2024-04-25T15:49:48.294 INFO:tasks.workunit.client.1.smithi193.stdout:7/735: dread - d12/d31/d37/d62/d6e/da5/fb6 zero size 2024-04-25T15:49:48.294 INFO:tasks.workunit.client.0.smithi110.stdout:2/850: mknod de/d40/d6e/d50/db7/c111 0 2024-04-25T15:49:48.296 INFO:tasks.workunit.client.1.smithi193.stdout:8/713: write d2/d3/d27/d69/dae/f49 [815486,120620] 0 2024-04-25T15:49:48.297 INFO:tasks.workunit.client.0.smithi110.stdout:3/921: sync 2024-04-25T15:49:48.297 INFO:tasks.workunit.client.0.smithi110.stdout:9/836: sync 2024-04-25T15:49:48.297 INFO:tasks.workunit.client.0.smithi110.stdout:7/974: rename d1b/d29/d38/d8b/d89/c133 to d1b/d29/d38/d8b/d89/c147 0 2024-04-25T15:49:48.298 INFO:tasks.workunit.client.0.smithi110.stdout:9/837: chown d9/d44/d5e/l64 3 1 2024-04-25T15:49:48.299 INFO:tasks.workunit.client.1.smithi193.stdout:5/744: rename d17/d18/d3e to d17/d18/d3e/dd1/dfb 22 2024-04-25T15:49:48.299 INFO:tasks.workunit.client.1.smithi193.stdout:5/745: fdatasync d17/d18/d9b/fa9 0 2024-04-25T15:49:48.300 INFO:tasks.workunit.client.1.smithi193.stdout:5/746: chown d17/d18/d30/d38/c92 3 1 2024-04-25T15:49:48.300 INFO:tasks.workunit.client.1.smithi193.stdout:5/747: readlink d17/d18/d30/d38/d48/la3 0 2024-04-25T15:49:48.325 INFO:tasks.workunit.client.0.smithi110.stdout:0/853: write d1/d47/ff2 [626281,124049] 0 2024-04-25T15:49:48.327 INFO:tasks.workunit.client.1.smithi193.stdout:2/715: mkdir d0/d37/dbb/df8/d110 0 2024-04-25T15:49:48.328 INFO:tasks.workunit.client.1.smithi193.stdout:2/716: stat d0/d9/d3d/d7d/fd2 0 2024-04-25T15:49:48.329 INFO:tasks.workunit.client.0.smithi110.stdout:6/849: write d1/dd/d3c/d91/f8b [868922,10033] 0 2024-04-25T15:49:48.331 INFO:tasks.workunit.client.0.smithi110.stdout:6/850: read - d1/dd/d3c/d91/d8d/dce/d102/d75/d82/f85 zero size 2024-04-25T15:49:48.332 INFO:tasks.workunit.client.0.smithi110.stdout:7/975: symlink d1b/d29/d38/d55/d7f/dc8/d13d/l148 0 2024-04-25T15:49:48.334 INFO:tasks.workunit.client.1.smithi193.stdout:2/717: dwrite d0/d37/d25/d5f/d3a/d6d/d6e/f3e [0,4194304] 0 2024-04-25T15:49:48.334 INFO:tasks.workunit.client.1.smithi193.stdout:9/711: dwrite f9 [0,4194304] 0 2024-04-25T15:49:48.335 INFO:tasks.workunit.client.1.smithi193.stdout:9/712: readlink df/l12 0 2024-04-25T15:49:48.337 INFO:tasks.workunit.client.0.smithi110.stdout:5/923: write d4/d1d/d1f/d35/d3d/f58 [4297090,53963] 0 2024-04-25T15:49:48.337 INFO:tasks.workunit.client.1.smithi193.stdout:0/747: unlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/cb5 0 2024-04-25T15:49:48.339 INFO:tasks.workunit.client.1.smithi193.stdout:6/700: creat dd/d12/d19/d1c/d40/d3c/fd2 x:0 0 0 2024-04-25T15:49:48.340 INFO:tasks.workunit.client.0.smithi110.stdout:3/922: write d0/d2e/d88/dbf/da5/ff8 [451881,28746] 0 2024-04-25T15:49:48.344 INFO:tasks.workunit.client.1.smithi193.stdout:7/736: creat d12/d31/d37/d104/d9d/f112 x:0 0 0 2024-04-25T15:49:48.345 INFO:tasks.workunit.client.1.smithi193.stdout:7/737: fsync f8 0 2024-04-25T15:49:48.346 INFO:tasks.workunit.client.1.smithi193.stdout:8/714: write d2/fab [806096,66515] 0 2024-04-25T15:49:48.352 INFO:tasks.workunit.client.1.smithi193.stdout:9/713: symlink df/d2c/d2f/d4e/lda 0 2024-04-25T15:49:48.355 INFO:tasks.workunit.client.1.smithi193.stdout:0/748: truncate d0/d49/d5/d7/dd/d16/d18/f7c 1008966 0 2024-04-25T15:49:48.357 INFO:tasks.workunit.client.1.smithi193.stdout:6/701: rename dd/d12/d19/d1c/d40/lab to dd/d2f/d57/ld3 0 2024-04-25T15:49:48.359 INFO:tasks.workunit.client.1.smithi193.stdout:8/715: rmdir d2/d18/dbc/dd3 39 2024-04-25T15:49:48.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/718: dread d0/d37/d25/f77 [0,4194304] 0 2024-04-25T15:49:48.365 INFO:tasks.workunit.client.1.smithi193.stdout:4/778: link db/d13/l19 db/lfa 0 2024-04-25T15:49:48.366 INFO:tasks.workunit.client.1.smithi193.stdout:4/779: dread - db/d13/d1e/d1f/d56/fe3 zero size 2024-04-25T15:49:48.366 INFO:tasks.workunit.client.1.smithi193.stdout:4/780: stat db/cd 0 2024-04-25T15:49:48.366 INFO:tasks.workunit.client.1.smithi193.stdout:4/781: chown db/d84/df3 0 1 2024-04-25T15:49:48.367 INFO:tasks.workunit.client.1.smithi193.stdout:3/751: getdents d8/d13/d1f/d69/d77 0 2024-04-25T15:49:48.368 INFO:tasks.workunit.client.1.smithi193.stdout:3/752: chown d8/d30/d72/c92 6 1 2024-04-25T15:49:48.368 INFO:tasks.workunit.client.1.smithi193.stdout:4/782: read db/d32/d88/fbf [2186734,122169] 0 2024-04-25T15:49:48.369 INFO:tasks.workunit.client.1.smithi193.stdout:6/702: creat dd/d12/d19/d45/fd4 x:0 0 0 2024-04-25T15:49:48.370 INFO:tasks.workunit.client.1.smithi193.stdout:6/703: readlink dd/l75 0 2024-04-25T15:49:48.371 INFO:tasks.workunit.client.1.smithi193.stdout:7/738: getdents d12/d13/dc4/dfa/dff 0 2024-04-25T15:49:48.371 INFO:tasks.workunit.client.1.smithi193.stdout:3/753: dread d8/d13/d1f/f78 [0,4194304] 0 2024-04-25T15:49:48.375 INFO:tasks.workunit.client.1.smithi193.stdout:1/657: sync 2024-04-25T15:49:48.377 INFO:tasks.workunit.client.1.smithi193.stdout:1/658: write dc/d4a/d4d/d7a/da6/fbb [545100,71645] 0 2024-04-25T15:49:48.383 INFO:tasks.workunit.client.1.smithi193.stdout:7/739: rmdir d12/d1c/d96 39 2024-04-25T15:49:48.385 INFO:tasks.workunit.client.1.smithi193.stdout:5/748: truncate d17/d18/d3e/f3f 4758893 0 2024-04-25T15:49:48.388 INFO:tasks.workunit.client.1.smithi193.stdout:3/754: fsync d8/d13/d1f/d69/fc2 0 2024-04-25T15:49:48.390 INFO:tasks.workunit.client.1.smithi193.stdout:9/714: creat df/d2c/d34/fdb x:0 0 0 2024-04-25T15:49:48.396 INFO:tasks.workunit.client.1.smithi193.stdout:0/749: rename d0/d49/d5/d7/dd/d16/d1a to d0/d49/d5/d7/dff 0 2024-04-25T15:49:48.397 INFO:tasks.workunit.client.1.smithi193.stdout:0/750: chown d0/d49/d5/d7/dd/d16/d18/d28 2771 1 2024-04-25T15:49:48.400 INFO:tasks.workunit.client.1.smithi193.stdout:4/783: creat db/d32/d40/dec/ffb x:0 0 0 2024-04-25T15:49:48.401 INFO:tasks.workunit.client.1.smithi193.stdout:7/740: readlink d12/d31/d37/d62/d6e/lc8 0 2024-04-25T15:49:48.405 INFO:tasks.workunit.client.1.smithi193.stdout:2/719: rmdir d0/d37/d25/d5f/d3a/d6d/d85/d10d 0 2024-04-25T15:49:48.406 INFO:tasks.workunit.client.0.smithi110.stdout:6/851: rmdir d1/db/d22/d90/df7 39 2024-04-25T15:49:48.406 INFO:tasks.workunit.client.0.smithi110.stdout:6/852: read - d1/db/d24/daa/f116 zero size 2024-04-25T15:49:48.409 INFO:tasks.workunit.client.1.smithi193.stdout:9/715: mkdir df/d2c/d2f/d5c/da3/dac/ddc 0 2024-04-25T15:49:48.411 INFO:tasks.workunit.client.1.smithi193.stdout:8/716: rename d2/da5/fa7 to d2/d3/d20/fe8 0 2024-04-25T15:49:48.412 INFO:tasks.workunit.client.1.smithi193.stdout:8/717: chown d2/d3/d20/d8c/dde/d94/c47 12 1 2024-04-25T15:49:48.413 INFO:tasks.workunit.client.0.smithi110.stdout:4/851: rename d13/d24/d26/f9b to d13/d24/d2d/d44/d65/d99/daf/f125 0 2024-04-25T15:49:48.414 INFO:tasks.workunit.client.0.smithi110.stdout:4/852: chown d13/d24/d2d/d44/d65/d99/daf/d56/d88/l30 2140 1 2024-04-25T15:49:48.415 INFO:tasks.workunit.client.0.smithi110.stdout:4/853: chown d13/d24/d2d/d44/da4/fb3 464 1 2024-04-25T15:49:48.416 INFO:tasks.workunit.client.0.smithi110.stdout:5/924: chown d4/d15/d51/d90/ded/def/f10f 216800 1 2024-04-25T15:49:48.417 INFO:tasks.workunit.client.1.smithi193.stdout:4/784: symlink db/d13/d1e/lfc 0 2024-04-25T15:49:48.420 INFO:tasks.workunit.client.1.smithi193.stdout:7/741: creat d12/d31/d37/d62/d6e/da5/dd7/f113 x:0 0 0 2024-04-25T15:49:48.421 INFO:tasks.workunit.client.1.smithi193.stdout:5/749: symlink d17/d18/d75/d85/lfc 0 2024-04-25T15:49:48.425 INFO:tasks.workunit.client.1.smithi193.stdout:2/720: creat d0/d37/dbb/f111 x:0 0 0 2024-04-25T15:49:48.426 INFO:tasks.workunit.client.0.smithi110.stdout:7/976: getdents d1b/d29/db5/dca/d10b/d112/d142 0 2024-04-25T15:49:48.427 INFO:tasks.workunit.client.0.smithi110.stdout:7/977: dread - d1b/d29/db5/dca/d10b/f135 zero size 2024-04-25T15:49:48.427 INFO:tasks.workunit.client.0.smithi110.stdout:7/978: stat d1b/d29/d38/d55/d7f/c63 0 2024-04-25T15:49:48.428 INFO:tasks.workunit.client.1.smithi193.stdout:3/755: dwrite d8/d13/d14/da0/fb5 [0,4194304] 0 2024-04-25T15:49:48.429 INFO:tasks.workunit.client.0.smithi110.stdout:9/838: creat d9/d13/d5b/f106 x:0 0 0 2024-04-25T15:49:48.431 INFO:tasks.workunit.client.1.smithi193.stdout:9/716: creat df/d2c/d2f/d5c/fdd x:0 0 0 2024-04-25T15:49:48.439 INFO:tasks.workunit.client.0.smithi110.stdout:3/923: dwrite d0/d7/f69 [4194304,4194304] 0 2024-04-25T15:49:48.439 INFO:tasks.workunit.client.0.smithi110.stdout:0/854: dread d1/d23/d30/d33/d5b/d78/d107/fb7 [4194304,4194304] 0 2024-04-25T15:49:48.442 INFO:tasks.workunit.client.0.smithi110.stdout:3/924: readlink d0/d16/d8c/l135 0 2024-04-25T15:49:48.443 INFO:tasks.workunit.client.0.smithi110.stdout:8/990: sync 2024-04-25T15:49:48.444 INFO:tasks.workunit.client.1.smithi193.stdout:8/718: read d2/f48 [1611434,17953] 0 2024-04-25T15:49:48.444 INFO:tasks.workunit.client.0.smithi110.stdout:0/855: dread d1/d23/f24 [0,4194304] 0 2024-04-25T15:49:48.445 INFO:tasks.workunit.client.1.smithi193.stdout:8/719: write d2/d3/d20/db8/dd0/fe3 [190324,19097] 0 2024-04-25T15:49:48.447 INFO:tasks.workunit.client.0.smithi110.stdout:3/925: write d0/d16/dcb/d50/f139 [81138,44616] 0 2024-04-25T15:49:48.448 INFO:tasks.workunit.client.0.smithi110.stdout:6/853: dwrite d1/dd/d3c/d91/d8d/dce/da1/fc1 [0,4194304] 0 2024-04-25T15:49:48.451 INFO:tasks.workunit.client.0.smithi110.stdout:7/979: unlink d1b/d29/fd0 0 2024-04-25T15:49:48.451 INFO:tasks.workunit.client.0.smithi110.stdout:3/926: write d0/d2e/dfc/f11f [817277,69594] 0 2024-04-25T15:49:48.451 INFO:tasks.workunit.client.0.smithi110.stdout:0/856: chown d1/d23/d30/d33/d35/ld2 417165 1 2024-04-25T15:49:48.452 INFO:tasks.workunit.client.0.smithi110.stdout:5/925: write d4/d1d/d1f/fcf [86740,10534] 0 2024-04-25T15:49:48.457 INFO:tasks.workunit.client.0.smithi110.stdout:3/927: chown d0/d16/f117 1 1 2024-04-25T15:49:48.459 INFO:tasks.workunit.client.0.smithi110.stdout:8/991: dwrite d14/d21/d26/d6a/d7e/da6/fa8 [4194304,4194304] 0 2024-04-25T15:49:48.459 INFO:tasks.workunit.client.0.smithi110.stdout:3/928: stat d0/d16/d8c/dce 0 2024-04-25T15:49:48.459 INFO:tasks.workunit.client.1.smithi193.stdout:2/721: rmdir d0/d37/d25 39 2024-04-25T15:49:48.460 INFO:tasks.workunit.client.1.smithi193.stdout:3/756: fsync d8/d4d/f67 0 2024-04-25T15:49:48.461 INFO:tasks.workunit.client.0.smithi110.stdout:5/926: dread d4/d15/d51/d90/ded/def/f10f [0,4194304] 0 2024-04-25T15:49:48.464 INFO:tasks.workunit.client.1.smithi193.stdout:6/704: rename dd/d2f/d57/l7a to dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/ld5 0 2024-04-25T15:49:48.464 INFO:tasks.workunit.client.0.smithi110.stdout:8/992: chown d14/d21/d26/d6a/d7e/l8c 2000711 1 2024-04-25T15:49:48.466 INFO:tasks.workunit.client.0.smithi110.stdout:5/927: stat d4/d15/d51/d99/fe7 0 2024-04-25T15:49:48.467 INFO:tasks.workunit.client.0.smithi110.stdout:8/993: readlink d14/d21/d26/d6a/d7e/da6/de8/d71/d9d/lc6 0 2024-04-25T15:49:48.467 INFO:tasks.workunit.client.1.smithi193.stdout:8/720: mknod d2/d3/d27/d69/dae/ce9 0 2024-04-25T15:49:48.468 INFO:tasks.workunit.client.0.smithi110.stdout:9/839: fdatasync d9/db/df/d25/fc4 0 2024-04-25T15:49:48.469 INFO:tasks.workunit.client.0.smithi110.stdout:0/857: chown d1/f8 481 1 2024-04-25T15:49:48.470 INFO:tasks.workunit.client.0.smithi110.stdout:9/840: chown d9/db/df/df3/ff4 2058 1 2024-04-25T15:49:48.472 INFO:tasks.workunit.client.1.smithi193.stdout:9/717: mkdir df/d2c/dde 0 2024-04-25T15:49:48.473 INFO:tasks.workunit.client.1.smithi193.stdout:9/718: dread - df/d2c/d34/d3e/d4d/fd3 zero size 2024-04-25T15:49:48.475 INFO:tasks.workunit.client.0.smithi110.stdout:7/980: creat d1b/d29/d38/d55/dab/db2/f149 x:0 0 0 2024-04-25T15:49:48.476 INFO:tasks.workunit.client.0.smithi110.stdout:3/929: dread d0/d2e/d88/dbf/d92/f93 [0,4194304] 0 2024-04-25T15:49:48.478 INFO:tasks.workunit.client.0.smithi110.stdout:3/930: chown d0/d7/d40/f7e 815706 1 2024-04-25T15:49:48.479 INFO:tasks.workunit.client.0.smithi110.stdout:3/931: readlink d0/d2e/d88/dbf/da5/lbd 0 2024-04-25T15:49:48.481 INFO:tasks.workunit.client.0.smithi110.stdout:4/854: dwrite d13/d24/f45 [4194304,4194304] 0 2024-04-25T15:49:48.481 INFO:tasks.workunit.client.0.smithi110.stdout:5/928: symlink d4/d15/d51/d90/ded/def/l12b 0 2024-04-25T15:49:48.482 INFO:tasks.workunit.client.1.smithi193.stdout:0/751: getdents d0/d49/d5/d7/dd/d16/d26/d5e 0 2024-04-25T15:49:48.485 INFO:tasks.workunit.client.1.smithi193.stdout:6/705: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/ld6 0 2024-04-25T15:49:48.487 INFO:tasks.workunit.client.0.smithi110.stdout:6/854: write d1/db/d24/d2d/f3a [5023339,28216] 0 2024-04-25T15:49:48.488 INFO:tasks.workunit.client.0.smithi110.stdout:0/858: read d1/fbc [3174909,26250] 0 2024-04-25T15:49:48.492 INFO:tasks.workunit.client.1.smithi193.stdout:3/757: creat d8/d13/d4b/d8b/de3/fee x:0 0 0 2024-04-25T15:49:48.493 INFO:tasks.workunit.client.1.smithi193.stdout:1/659: sync 2024-04-25T15:49:48.493 INFO:tasks.workunit.client.1.smithi193.stdout:6/706: dwrite dd/d12/d19/d1c/d40/d3c/d3f/f85 [0,4194304] 0 2024-04-25T15:49:48.495 INFO:tasks.workunit.client.0.smithi110.stdout:8/994: dwrite d14/f1d [0,4194304] 0 2024-04-25T15:49:48.495 INFO:tasks.workunit.client.1.smithi193.stdout:8/721: dread d2/d3/d3a/f54 [0,4194304] 0 2024-04-25T15:49:48.495 INFO:tasks.workunit.client.1.smithi193.stdout:9/719: creat df/d2c/d2f/d4e/d1c/d30/d60/fdf x:0 0 0 2024-04-25T15:49:48.496 INFO:tasks.workunit.client.1.smithi193.stdout:1/660: write dc/d4a/d50/fa7 [485918,8930] 0 2024-04-25T15:49:48.497 INFO:tasks.workunit.client.1.smithi193.stdout:1/661: read - dc/d59/fcb zero size 2024-04-25T15:49:48.497 INFO:tasks.workunit.client.1.smithi193.stdout:2/722: link d0/d37/d4d/dbe/df7/ffa d0/d37/d4d/dbe/df7/f112 0 2024-04-25T15:49:48.498 INFO:tasks.workunit.client.1.smithi193.stdout:9/720: dread - df/d2c/d34/d3e/d4d/d5e/fd5 zero size 2024-04-25T15:49:48.500 INFO:tasks.workunit.client.1.smithi193.stdout:2/723: dread - d0/d42/d51/df1/d9f/dc5/ff5 zero size 2024-04-25T15:49:48.504 INFO:tasks.workunit.client.1.smithi193.stdout:5/750: getdents d17/d18/d30/d38 0 2024-04-25T15:49:48.506 INFO:tasks.workunit.client.0.smithi110.stdout:9/841: creat d9/d13/d5b/d61/d67/dd7/df5/f107 x:0 0 0 2024-04-25T15:49:48.508 INFO:tasks.workunit.client.1.smithi193.stdout:3/758: mknod d8/d13/d14/d23/cef 0 2024-04-25T15:49:48.508 INFO:tasks.workunit.client.0.smithi110.stdout:9/842: write d9/d13/d5b/d61/db3/dd3/ddb/fdd [195694,447] 0 2024-04-25T15:49:48.509 INFO:tasks.workunit.client.0.smithi110.stdout:8/995: chown d14/d21/d26/dc0/c131 236864 1 2024-04-25T15:49:48.510 INFO:tasks.workunit.client.1.smithi193.stdout:6/707: fsync dd/f13 0 2024-04-25T15:49:48.511 INFO:tasks.workunit.client.0.smithi110.stdout:4/855: mknod d13/d24/d2d/d44/d65/d99/dbc/c126 0 2024-04-25T15:49:48.511 INFO:tasks.workunit.client.0.smithi110.stdout:8/996: dread - d14/d32/d105/d110/f150 zero size 2024-04-25T15:49:48.511 INFO:tasks.workunit.client.1.smithi193.stdout:1/662: creat dc/d4a/fde x:0 0 0 2024-04-25T15:49:48.513 INFO:tasks.workunit.client.0.smithi110.stdout:6/855: rename d1/dd/fe6 to d1/db/da0/da5/f126 0 2024-04-25T15:49:48.514 INFO:tasks.workunit.client.0.smithi110.stdout:0/859: truncate d1/d6c/d7f/d79/f128 636472 0 2024-04-25T15:49:48.516 INFO:tasks.workunit.client.0.smithi110.stdout:3/932: creat d0/d2e/d88/dbf/d99/dad/f13b x:0 0 0 2024-04-25T15:49:48.519 INFO:tasks.workunit.client.0.smithi110.stdout:3/933: read d0/d16/dcb/f22 [255194,48147] 0 2024-04-25T15:49:48.519 INFO:tasks.workunit.client.0.smithi110.stdout:5/929: dread d4/f7c [0,4194304] 0 2024-04-25T15:49:48.522 INFO:tasks.workunit.client.0.smithi110.stdout:9/843: unlink d9/db/df/d2c/l105 0 2024-04-25T15:49:48.522 INFO:tasks.workunit.client.0.smithi110.stdout:9/844: readlink d9/db/l9e 0 2024-04-25T15:49:48.525 INFO:tasks.workunit.client.1.smithi193.stdout:0/752: write d0/d49/d5/d7/dd/d46/d48/fa2 [1639970,24500] 0 2024-04-25T15:49:48.525 INFO:tasks.workunit.client.0.smithi110.stdout:9/845: readlink d9/l10 0 2024-04-25T15:49:48.526 INFO:tasks.workunit.client.1.smithi193.stdout:5/751: symlink d17/d18/d30/d38/d48/d4f/lfd 0 2024-04-25T15:49:48.527 INFO:tasks.workunit.client.0.smithi110.stdout:4/856: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/f127 x:0 0 0 2024-04-25T15:49:48.529 INFO:tasks.workunit.client.1.smithi193.stdout:6/708: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dd7 0 2024-04-25T15:49:48.529 INFO:tasks.workunit.client.1.smithi193.stdout:6/709: write dd/fe [3102504,35444] 0 2024-04-25T15:49:48.530 INFO:tasks.workunit.client.0.smithi110.stdout:0/860: dread d1/d23/d30/d33/f88 [0,4194304] 0 2024-04-25T15:49:48.531 INFO:tasks.workunit.client.1.smithi193.stdout:1/663: unlink dc/d4a/d50/d5c/d5d/c68 0 2024-04-25T15:49:48.536 INFO:tasks.workunit.client.1.smithi193.stdout:4/785: rename db/d13/ced to db/cfd 0 2024-04-25T15:49:48.539 INFO:tasks.workunit.client.1.smithi193.stdout:4/786: dread f9 [4194304,4194304] 0 2024-04-25T15:49:48.540 INFO:tasks.workunit.client.1.smithi193.stdout:4/787: stat db/d13/d1e/d1f 0 2024-04-25T15:49:48.542 INFO:tasks.workunit.client.0.smithi110.stdout:3/934: rename d0/d2e/d88/dbf/lc3 to d0/d16/dcb/dd7/l13c 0 2024-04-25T15:49:48.543 INFO:tasks.workunit.client.1.smithi193.stdout:5/752: chown d17/d18/d75/cb7 449177285 1 2024-04-25T15:49:48.545 INFO:tasks.workunit.client.0.smithi110.stdout:9/846: creat d9/d13/d5b/d61/db3/db4/dc5/f108 x:0 0 0 2024-04-25T15:49:48.545 INFO:tasks.workunit.client.0.smithi110.stdout:8/997: dwrite d14/d32/d36/f115 [0,4194304] 0 2024-04-25T15:49:48.546 INFO:tasks.workunit.client.1.smithi193.stdout:3/759: mkdir d8/df0 0 2024-04-25T15:49:48.547 INFO:tasks.workunit.client.1.smithi193.stdout:3/760: truncate d8/d13/d14/fd3 17244 0 2024-04-25T15:49:48.552 INFO:tasks.workunit.client.1.smithi193.stdout:6/710: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/ld8 0 2024-04-25T15:49:48.553 INFO:tasks.workunit.client.1.smithi193.stdout:2/724: write d0/d37/d25/d5f/d3a/d6d/d6e/f96 [1050092,69914] 0 2024-04-25T15:49:48.554 INFO:tasks.workunit.client.0.smithi110.stdout:8/998: dwrite d14/d32/d105/d110/f150 [0,4194304] 0 2024-04-25T15:49:48.554 INFO:tasks.workunit.client.1.smithi193.stdout:2/725: chown d0 36928146 1 2024-04-25T15:49:48.555 INFO:tasks.workunit.client.1.smithi193.stdout:2/726: dread - d0/d42/d51/df1/d9c/fdd zero size 2024-04-25T15:49:48.556 INFO:tasks.workunit.client.1.smithi193.stdout:1/664: unlink dc/d4a/f71 0 2024-04-25T15:49:48.556 INFO:tasks.workunit.client.0.smithi110.stdout:8/999: truncate d14/d32/d36/db7/ff7 5061309 0 2024-04-25T15:49:48.556 INFO:tasks.workunit.client.0.smithi110.stdout:0/861: mknod d1/d23/d30/d33/d5b/dbb/da8/c134 0 2024-04-25T15:49:48.560 INFO:tasks.workunit.client.1.smithi193.stdout:7/742: rename d12/d1c/c21 to d12/d13/dc4/dfa/dff/c114 0 2024-04-25T15:49:48.562 INFO:tasks.workunit.client.0.smithi110.stdout:9/847: dread d9/d44/fab [0,4194304] 0 2024-04-25T15:49:48.562 INFO:tasks.workunit.client.1.smithi193.stdout:7/743: read d12/d1c/d63/d70/d68/d8d/f9c [9405,93948] 0 2024-04-25T15:49:48.562 INFO:tasks.workunit.client.1.smithi193.stdout:7/744: stat f2 0 2024-04-25T15:49:48.563 INFO:tasks.workunit.client.0.smithi110.stdout:0/862: dwrite d1/d47/fa9 [4194304,4194304] 0 2024-04-25T15:49:48.563 INFO:tasks.workunit.client.1.smithi193.stdout:7/745: fsync d12/d1c/f28 0 2024-04-25T15:49:48.563 INFO:tasks.workunit.client.0.smithi110.stdout:5/930: symlink d4/l12c 0 2024-04-25T15:49:48.571 INFO:tasks.workunit.client.0.smithi110.stdout:9/848: dwrite d9/d13/d5b/d61/d67/dd7/f60 [0,4194304] 0 2024-04-25T15:49:48.589 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:48 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:48.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:48 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:48.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:48 smithi193 ceph-mon[33747]: Reconfiguring grafana.smithi110 (dependencies changed)... 2024-04-25T15:49:48.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:48 smithi193 ceph-mon[33747]: Reconfiguring daemon grafana.smithi110 on smithi110 2024-04-25T15:49:48.590 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:48 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:49:48.593 INFO:tasks.workunit.client.1.smithi193.stdout:8/722: sync 2024-04-25T15:49:48.612 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:48 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:48.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:48 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:48.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:48 smithi110 ceph-mon[26344]: Reconfiguring grafana.smithi110 (dependencies changed)... 2024-04-25T15:49:48.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:48 smithi110 ceph-mon[26344]: Reconfiguring daemon grafana.smithi110 on smithi110 2024-04-25T15:49:48.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:48 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:49:48.623 INFO:tasks.workunit.client.0.smithi110.stdout:3/935: read d0/d16/dcb/d2f/d4c/db2/fe4 [686406,20384] 0 2024-04-25T15:49:48.675 INFO:tasks.workunit.client.1.smithi193.stdout:2/727: write d0/d37/dbb/df8/f86 [3662591,115946] 0 2024-04-25T15:49:48.683 INFO:tasks.workunit.client.0.smithi110.stdout:6/856: link d1/d12/db8/ld9 d1/db/d22/d90/d76/dee/l127 0 2024-04-25T15:49:48.684 INFO:tasks.workunit.client.0.smithi110.stdout:2/851: sync 2024-04-25T15:49:48.690 INFO:tasks.workunit.client.1.smithi193.stdout:9/721: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/f2e [4194304,4194304] 0 2024-04-25T15:49:48.692 INFO:tasks.workunit.client.1.smithi193.stdout:3/761: write d8/d13/d14/d23/f48 [1762090,23618] 0 2024-04-25T15:49:48.693 INFO:tasks.workunit.client.1.smithi193.stdout:7/746: rmdir d12/d31/d37/d92 39 2024-04-25T15:49:48.693 INFO:tasks.workunit.client.1.smithi193.stdout:7/747: readlink d12/d31/d37/l7b 0 2024-04-25T15:49:48.695 INFO:tasks.workunit.client.0.smithi110.stdout:5/931: truncate d4/d15/d51/d99/fe7 1447421 0 2024-04-25T15:49:48.698 INFO:tasks.workunit.client.0.smithi110.stdout:9/849: creat d9/db/d7d/f109 x:0 0 0 2024-04-25T15:49:48.701 INFO:tasks.workunit.client.0.smithi110.stdout:3/936: rmdir d0/d7/d40 39 2024-04-25T15:49:48.709 INFO:tasks.workunit.client.1.smithi193.stdout:1/665: creat dc/d4a/d77/dd3/fdf x:0 0 0 2024-04-25T15:49:48.710 INFO:tasks.workunit.client.0.smithi110.stdout:6/857: unlink d1/l7 0 2024-04-25T15:49:48.715 INFO:tasks.workunit.client.0.smithi110.stdout:0/863: symlink d1/d6c/d116/l135 0 2024-04-25T15:49:48.719 INFO:tasks.workunit.client.0.smithi110.stdout:0/864: dread d1/d23/d30/d33/d35/f6b [0,4194304] 0 2024-04-25T15:49:48.720 INFO:tasks.workunit.client.1.smithi193.stdout:3/762: rmdir d8/d13/d1f/d69/dcd 39 2024-04-25T15:49:48.723 INFO:tasks.workunit.client.0.smithi110.stdout:9/850: rename d9/db/df/d25/c94 to d9/d44/d5e/d80/dbb/dbe/c10a 0 2024-04-25T15:49:48.724 INFO:tasks.workunit.client.0.smithi110.stdout:9/851: stat d9/db/df/f33 0 2024-04-25T15:49:48.730 INFO:tasks.workunit.client.0.smithi110.stdout:6/858: mknod d1/d12/db8/c128 0 2024-04-25T15:49:48.730 INFO:tasks.workunit.client.1.smithi193.stdout:8/723: symlink d2/d3/d3a/lea 0 2024-04-25T15:49:48.732 INFO:tasks.workunit.client.0.smithi110.stdout:2/852: link de/d40/dcd/ff5 de/d40/d6e/d52/f112 0 2024-04-25T15:49:48.742 INFO:tasks.workunit.client.1.smithi193.stdout:3/763: mknod d8/d13/d1f/d69/d77/cf1 0 2024-04-25T15:49:48.744 INFO:tasks.workunit.client.0.smithi110.stdout:0/865: unlink d1/d20/d69/le3 0 2024-04-25T15:49:48.745 INFO:tasks.workunit.client.1.smithi193.stdout:7/748: creat d12/d1c/dd6/d101/f115 x:0 0 0 2024-04-25T15:49:48.748 INFO:tasks.workunit.client.1.smithi193.stdout:7/749: truncate d12/d31/d37/d62/d6e/da5/dd7/f113 358798 0 2024-04-25T15:49:48.749 INFO:tasks.workunit.client.1.smithi193.stdout:1/666: dread dc/f5b [0,4194304] 0 2024-04-25T15:49:48.753 INFO:tasks.workunit.client.0.smithi110.stdout:7/981: sync 2024-04-25T15:49:48.754 INFO:tasks.workunit.client.0.smithi110.stdout:7/982: chown d1b/d29/c86 0 1 2024-04-25T15:49:48.757 INFO:tasks.workunit.client.1.smithi193.stdout:3/764: fdatasync d8/fc 0 2024-04-25T15:49:48.758 INFO:tasks.workunit.client.0.smithi110.stdout:5/932: creat d4/d1d/d1f/f12d x:0 0 0 2024-04-25T15:49:48.760 INFO:tasks.workunit.client.1.smithi193.stdout:8/724: creat d2/d18/ddb/feb x:0 0 0 2024-04-25T15:49:48.761 INFO:tasks.workunit.client.1.smithi193.stdout:8/725: dread d2/d3/ff [0,4194304] 0 2024-04-25T15:49:48.763 INFO:tasks.workunit.client.1.smithi193.stdout:7/750: rmdir d12/d13/dcc 39 2024-04-25T15:49:48.767 INFO:tasks.workunit.client.1.smithi193.stdout:2/728: getdents d0/d42/d51/d4f 0 2024-04-25T15:49:48.768 INFO:tasks.workunit.client.1.smithi193.stdout:2/729: write d0/d42/d51/df1/d9c/fba [1093943,110524] 0 2024-04-25T15:49:48.769 INFO:tasks.workunit.client.1.smithi193.stdout:2/730: write d0/d37/d25/d5f/f108 [634708,92868] 0 2024-04-25T15:49:48.770 INFO:tasks.workunit.client.1.smithi193.stdout:2/731: write d0/d42/d51/df1/d9f/dc5/ff5 [300142,102647] 0 2024-04-25T15:49:48.774 INFO:tasks.workunit.client.0.smithi110.stdout:2/853: symlink de/d40/d6e/d52/dfc/l113 0 2024-04-25T15:49:48.784 INFO:tasks.workunit.client.0.smithi110.stdout:7/983: link d1b/d29/d38/d55/d7f/faf d1b/d29/db5/dca/d10b/f14a 0 2024-04-25T15:49:48.788 INFO:tasks.workunit.client.0.smithi110.stdout:7/984: mkdir d1b/d29/db5/dca/d10b/d112/d13c/d14b 0 2024-04-25T15:49:48.790 INFO:tasks.workunit.client.1.smithi193.stdout:7/751: dread d12/d31/d37/d92/dbf/fc2 [0,4194304] 0 2024-04-25T15:49:48.790 INFO:tasks.workunit.client.1.smithi193.stdout:8/726: getdents d2/d3/d20/d8c 0 2024-04-25T15:49:48.791 INFO:tasks.workunit.client.1.smithi193.stdout:8/727: stat d2/d3/d27/d69/dae/d4a/d53/f58 0 2024-04-25T15:49:48.792 INFO:tasks.workunit.client.1.smithi193.stdout:7/752: truncate d12/d31/d37/d62/d6e/da5/dd7/f113 865566 0 2024-04-25T15:49:48.793 INFO:tasks.workunit.client.0.smithi110.stdout:7/985: link d1b/d29/d38/d55/fd9 d1b/d29/d38/d55/d7f/dc8/f14c 0 2024-04-25T15:49:48.798 INFO:tasks.workunit.client.0.smithi110.stdout:6/859: rename d1/db/d22/d33/d63/dc2/dc7/lc9 to d1/dd/d3c/d91/d8d/dce/d102/l129 0 2024-04-25T15:49:48.800 INFO:tasks.workunit.client.0.smithi110.stdout:7/986: dwrite d1b/d29/db5/f117 [0,4194304] 0 2024-04-25T15:49:48.805 INFO:tasks.workunit.client.0.smithi110.stdout:7/987: mkdir d1b/d29/d38/d55/d139/d13b/d14d 0 2024-04-25T15:49:48.806 INFO:tasks.workunit.client.0.smithi110.stdout:7/988: chown d1b/d29/d38/d55/d95/dcb 2 1 2024-04-25T15:49:48.850 INFO:tasks.workunit.client.1.smithi193.stdout:7/753: mkdir d12/d13/d116 0 2024-04-25T15:49:48.851 INFO:tasks.workunit.client.1.smithi193.stdout:7/754: write d12/d31/d37/d104/f74 [8477789,15444] 0 2024-04-25T15:49:48.851 INFO:tasks.workunit.client.1.smithi193.stdout:8/728: rmdir d2/d3/d3a/dc8 0 2024-04-25T15:49:48.852 INFO:tasks.workunit.client.1.smithi193.stdout:7/755: write d12/d1c/dd6/d101/f115 [292657,125035] 0 2024-04-25T15:49:48.853 INFO:tasks.workunit.client.1.smithi193.stdout:8/729: dread - d2/d3/d20/d8c/dde/d94/fcb zero size 2024-04-25T15:49:48.860 INFO:tasks.workunit.client.1.smithi193.stdout:8/730: dwrite d2/d3/d27/fdf [0,4194304] 0 2024-04-25T15:49:48.861 INFO:tasks.workunit.client.1.smithi193.stdout:7/756: dwrite d12/d31/fc7 [0,4194304] 0 2024-04-25T15:49:48.866 INFO:tasks.workunit.client.1.smithi193.stdout:7/757: symlink d12/d31/d37/d104/d9d/l117 0 2024-04-25T15:49:48.866 INFO:tasks.workunit.client.1.smithi193.stdout:7/758: dread - d12/d31/d37/d62/d6e/da5/fb6 zero size 2024-04-25T15:49:48.872 INFO:tasks.workunit.client.1.smithi193.stdout:4/788: truncate db/d13/d1e/d1f/d22/f6b 785135 0 2024-04-25T15:49:48.878 INFO:tasks.workunit.client.1.smithi193.stdout:4/789: truncate db/d13/d1e/d1f/d56/d57/fa4 2693307 0 2024-04-25T15:49:48.881 INFO:tasks.workunit.client.1.smithi193.stdout:3/765: unlink d8/d13/d4b/lc5 0 2024-04-25T15:49:48.883 INFO:tasks.workunit.client.1.smithi193.stdout:4/790: mknod db/d13/d1e/d1f/d56/d69/dad/def/cfe 0 2024-04-25T15:49:48.885 INFO:tasks.workunit.client.1.smithi193.stdout:4/791: write db/d32/f94 [2006769,70618] 0 2024-04-25T15:49:48.886 INFO:tasks.workunit.client.1.smithi193.stdout:5/753: dwrite d17/d18/d75/f88 [0,4194304] 0 2024-04-25T15:49:48.888 INFO:tasks.workunit.client.1.smithi193.stdout:4/792: mknod db/d32/de8/cff 0 2024-04-25T15:49:48.890 INFO:tasks.workunit.client.1.smithi193.stdout:5/754: mknod d17/d18/d30/d38/dab/cfe 0 2024-04-25T15:49:48.891 INFO:tasks.workunit.client.1.smithi193.stdout:8/731: dread d2/d3/d20/f8b [0,4194304] 0 2024-04-25T15:49:48.892 INFO:tasks.workunit.client.1.smithi193.stdout:9/722: mkdir df/d2c/d2f/d4e/d1c/d30/de0 0 2024-04-25T15:49:48.897 INFO:tasks.workunit.client.1.smithi193.stdout:9/723: fsync df/d2c/d2f/d4e/f19 0 2024-04-25T15:49:48.897 INFO:tasks.workunit.client.1.smithi193.stdout:5/755: dwrite d17/d18/d30/f8a [0,4194304] 0 2024-04-25T15:49:48.898 INFO:tasks.workunit.client.1.smithi193.stdout:5/756: chown d17/d83 7561682 1 2024-04-25T15:49:48.898 INFO:tasks.workunit.client.1.smithi193.stdout:9/724: chown df/d2c/d2f/d4e/d1c/fc3 223 1 2024-04-25T15:49:48.898 INFO:tasks.workunit.client.1.smithi193.stdout:5/757: chown d17/d18/d75 62556 1 2024-04-25T15:49:48.899 INFO:tasks.workunit.client.1.smithi193.stdout:9/725: chown df/d2c/d2f/d4e/l20 60680792 1 2024-04-25T15:49:48.900 INFO:tasks.workunit.client.0.smithi110.stdout:4/857: rename d13/d24/f48 to d13/d24/f128 0 2024-04-25T15:49:48.901 INFO:tasks.workunit.client.1.smithi193.stdout:9/726: dread - df/d2c/d2f/d4e/d1c/fc3 zero size 2024-04-25T15:49:48.901 INFO:tasks.workunit.client.1.smithi193.stdout:9/727: readlink df/l12 0 2024-04-25T15:49:48.902 INFO:tasks.workunit.client.0.smithi110.stdout:4/858: chown d13/d24/d2d/d44/da4/c107 436258 1 2024-04-25T15:49:48.903 INFO:tasks.workunit.client.1.smithi193.stdout:8/732: getdents d2/d18/dbc 0 2024-04-25T15:49:48.905 INFO:tasks.workunit.client.1.smithi193.stdout:9/728: mknod df/d2c/d2f/d5c/da3/ce1 0 2024-04-25T15:49:48.907 INFO:tasks.workunit.client.0.smithi110.stdout:4/859: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d129 0 2024-04-25T15:49:48.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/729: chown df/d2c/d2f/l58 510 1 2024-04-25T15:49:48.910 INFO:tasks.workunit.client.0.smithi110.stdout:4/860: mknod d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d117/c12a 0 2024-04-25T15:49:48.911 INFO:tasks.workunit.client.0.smithi110.stdout:4/861: fsync d13/d24/d2d/d5e/f94 0 2024-04-25T15:49:48.915 INFO:tasks.workunit.client.0.smithi110.stdout:4/862: truncate d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d117/f123 823608 0 2024-04-25T15:49:48.917 INFO:tasks.workunit.client.0.smithi110.stdout:4/863: creat d13/d24/d2d/d44/f12b x:0 0 0 2024-04-25T15:49:48.919 INFO:tasks.workunit.client.0.smithi110.stdout:4/864: fdatasync d13/d24/d2d/db5/fe5 0 2024-04-25T15:49:48.927 INFO:tasks.workunit.client.0.smithi110.stdout:4/865: dwrite d13/d24/d2d/d5e/f94 [4194304,4194304] 0 2024-04-25T15:49:48.930 INFO:tasks.workunit.client.0.smithi110.stdout:9/852: dwrite d9/db/df/d2c/d3c/fac [0,4194304] 0 2024-04-25T15:49:48.933 INFO:tasks.workunit.client.0.smithi110.stdout:3/937: dwrite d0/d16/dcb/db0/feb [0,4194304] 0 2024-04-25T15:49:48.933 INFO:tasks.workunit.client.1.smithi193.stdout:0/753: rename d0/d49/d5/d7/dd/d16/d26/d5e/f80 to d0/d49/d5/d7/dd/f100 0 2024-04-25T15:49:48.934 INFO:tasks.workunit.client.1.smithi193.stdout:1/667: dwrite dc/d4a/d50/d5c/d5d/f51 [0,4194304] 0 2024-04-25T15:49:48.938 INFO:tasks.workunit.client.0.smithi110.stdout:9/853: symlink d9/d13/d5b/d61/d67/dd7/l10b 0 2024-04-25T15:49:48.938 INFO:tasks.workunit.client.0.smithi110.stdout:9/854: creat d9/db/d7d/f10c x:0 0 0 2024-04-25T15:49:48.939 INFO:tasks.workunit.client.0.smithi110.stdout:9/855: chown d9/d13/d5b/d61/db3/db4/dc5/f108 582888 1 2024-04-25T15:49:48.946 INFO:tasks.workunit.client.0.smithi110.stdout:9/856: unlink d9/db/df/df3/ff4 0 2024-04-25T15:49:48.947 INFO:tasks.workunit.client.0.smithi110.stdout:5/933: dwrite d4/f6d [0,4194304] 0 2024-04-25T15:49:48.948 INFO:tasks.workunit.client.0.smithi110.stdout:9/857: chown d9/db/df/l23 299843204 1 2024-04-25T15:49:48.949 INFO:tasks.workunit.client.1.smithi193.stdout:8/733: rename d2/d3/d27/d69/dae/d4a/d53/c6b to d2/d3/d27/d69/dae/d4a/d63/da2/de4/cec 0 2024-04-25T15:49:48.950 INFO:tasks.workunit.client.1.smithi193.stdout:8/734: dread - d2/d3/d20/d8c/dde/d94/fcb zero size 2024-04-25T15:49:48.952 INFO:tasks.workunit.client.0.smithi110.stdout:9/858: fdatasync d9/d44/fab 0 2024-04-25T15:49:48.956 INFO:tasks.workunit.client.0.smithi110.stdout:2/854: dwrite de/d40/d6e/d52/d6d/d7d/f85 [0,4194304] 0 2024-04-25T15:49:48.962 INFO:tasks.workunit.client.0.smithi110.stdout:9/859: dwrite d9/d13/f8e [0,4194304] 0 2024-04-25T15:49:48.965 INFO:tasks.workunit.client.0.smithi110.stdout:5/934: mkdir d4/d1d/d1f/d35/d3d/d52/d75/d11c/d12e 0 2024-04-25T15:49:48.967 INFO:tasks.workunit.client.0.smithi110.stdout:5/935: chown d4/d1d/d1f/d35/d7f/d92/d9b/dcb/ce9 7 1 2024-04-25T15:49:48.972 INFO:tasks.workunit.client.0.smithi110.stdout:5/936: mknod d4/d2c/c12f 0 2024-04-25T15:49:48.973 INFO:tasks.workunit.client.0.smithi110.stdout:9/860: getdents d9/d44/d5e/d80 0 2024-04-25T15:49:48.974 INFO:tasks.workunit.client.0.smithi110.stdout:9/861: chown d9/db/df/d2c/d3c/fd6 1521 1 2024-04-25T15:49:48.976 INFO:tasks.workunit.client.0.smithi110.stdout:9/862: creat d9/d13/d5b/d61/d67/dd7/f10d x:0 0 0 2024-04-25T15:49:48.977 INFO:tasks.workunit.client.1.smithi193.stdout:5/758: rmdir d17/d18/d3e 39 2024-04-25T15:49:48.979 INFO:tasks.workunit.client.0.smithi110.stdout:7/989: write d1b/d29/d38/f3d [6094350,59648] 0 2024-04-25T15:49:48.981 INFO:tasks.workunit.client.0.smithi110.stdout:9/863: dread d9/db/df/fcb [0,4194304] 0 2024-04-25T15:49:48.982 INFO:tasks.workunit.client.1.smithi193.stdout:1/668: dread dc/d4a/d50/d5c/d5d/d3d/d55/d8a/fa5 [0,4194304] 0 2024-04-25T15:49:48.982 INFO:tasks.workunit.client.1.smithi193.stdout:5/759: rename d17/d18/d75/fbf to d17/d18/d30/d34/d52/daf/fff 0 2024-04-25T15:49:48.987 INFO:tasks.workunit.client.1.smithi193.stdout:1/669: creat dc/d4a/d50/d5c/d5d/d3d/d55/fe0 x:0 0 0 2024-04-25T15:49:48.991 INFO:tasks.workunit.client.1.smithi193.stdout:2/732: creat d0/d9/d15/f113 x:0 0 0 2024-04-25T15:49:48.995 INFO:tasks.workunit.client.0.smithi110.stdout:7/990: rename d1b/d29/db5/dca/d10b/l10c to d1b/d29/db5/l14e 0 2024-04-25T15:49:48.996 INFO:tasks.workunit.client.0.smithi110.stdout:7/991: stat d1b/c43 0 2024-04-25T15:49:48.998 INFO:tasks.workunit.client.0.smithi110.stdout:7/992: mknod d1b/d29/d38/d55/d95/dcb/c14f 0 2024-04-25T15:49:49.000 INFO:tasks.workunit.client.0.smithi110.stdout:7/993: truncate d1b/d29/d46/fcf 4634726 0 2024-04-25T15:49:49.002 INFO:tasks.workunit.client.0.smithi110.stdout:6/860: unlink d1/dd/d3c/d91/d8d/dce/ffc 0 2024-04-25T15:49:49.003 INFO:tasks.workunit.client.0.smithi110.stdout:6/861: mknod d1/d48/c12a 0 2024-04-25T15:49:49.006 INFO:tasks.workunit.client.0.smithi110.stdout:6/862: creat d1/dd/d3c/d91/d8d/dce/da1/f12b x:0 0 0 2024-04-25T15:49:49.007 INFO:tasks.workunit.client.0.smithi110.stdout:6/863: chown d1/dd/d3c/d91/lab 419378 1 2024-04-25T15:49:49.008 INFO:tasks.workunit.client.0.smithi110.stdout:0/866: creat d1/d47/d54/f136 x:0 0 0 2024-04-25T15:49:49.009 INFO:tasks.workunit.client.1.smithi193.stdout:6/711: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/fd9 x:0 0 0 2024-04-25T15:49:49.010 INFO:tasks.workunit.client.1.smithi193.stdout:6/712: chown dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80 1781224 1 2024-04-25T15:49:49.011 INFO:tasks.workunit.client.0.smithi110.stdout:0/867: creat d1/d23/d81/f137 x:0 0 0 2024-04-25T15:49:49.016 INFO:tasks.workunit.client.0.smithi110.stdout:5/937: symlink d4/l130 0 2024-04-25T15:49:49.017 INFO:tasks.workunit.client.1.smithi193.stdout:7/759: write d12/d13/fac [843799,46479] 0 2024-04-25T15:49:49.020 INFO:tasks.workunit.client.0.smithi110.stdout:5/938: rename d4/d1d/d1f/d35/d3d/d52/d97/d119 to d4/d1d/d1f/d35/d3d/d52/d131 0 2024-04-25T15:49:49.021 INFO:tasks.workunit.client.1.smithi193.stdout:6/713: dread dd/f13 [0,4194304] 0 2024-04-25T15:49:49.022 INFO:tasks.workunit.client.1.smithi193.stdout:6/714: stat dd/d12/d19/d1c/d40/d3c/fa7 0 2024-04-25T15:49:49.023 INFO:tasks.workunit.client.1.smithi193.stdout:6/715: truncate dd/d12/d19/d45/fc7 296572 0 2024-04-25T15:49:49.027 INFO:tasks.workunit.client.1.smithi193.stdout:4/793: getdents db/d13/d1e/d1f/d56/d69/dad/def 0 2024-04-25T15:49:49.027 INFO:tasks.workunit.client.1.smithi193.stdout:6/716: stat dd/c10 0 2024-04-25T15:49:49.029 INFO:tasks.workunit.client.1.smithi193.stdout:6/717: truncate dd/d2f/d57/daf/fcc 676337 0 2024-04-25T15:49:49.030 INFO:tasks.workunit.client.1.smithi193.stdout:3/766: mkdir d8/d13/d4b/d8b/dca/df2 0 2024-04-25T15:49:49.030 INFO:tasks.workunit.client.1.smithi193.stdout:6/718: readlink dd/d2f/l4a 0 2024-04-25T15:49:49.032 INFO:tasks.workunit.client.1.smithi193.stdout:6/719: readlink dd/d2f/d57/l7c 0 2024-04-25T15:49:49.033 INFO:tasks.workunit.client.1.smithi193.stdout:4/794: dwrite db/f61 [4194304,4194304] 0 2024-04-25T15:49:49.033 INFO:tasks.workunit.client.1.smithi193.stdout:4/795: readlink db/d13/l1b 0 2024-04-25T15:49:49.034 INFO:tasks.workunit.client.1.smithi193.stdout:6/720: chown dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0 3720 1 2024-04-25T15:49:49.038 INFO:tasks.workunit.client.1.smithi193.stdout:4/796: dread db/f3b [0,4194304] 0 2024-04-25T15:49:49.039 INFO:tasks.workunit.client.1.smithi193.stdout:9/730: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/f40 [0,4194304] 0 2024-04-25T15:49:49.039 INFO:tasks.workunit.client.1.smithi193.stdout:6/721: mkdir dd/d12/d19/d1c/d40/dda 0 2024-04-25T15:49:49.042 INFO:tasks.workunit.client.1.smithi193.stdout:9/731: truncate df/d2c/d2f/d4e/d1c/fc7 655351 0 2024-04-25T15:49:49.042 INFO:tasks.workunit.client.1.smithi193.stdout:4/797: mkdir db/d84/df3/d100 0 2024-04-25T15:49:49.043 INFO:tasks.workunit.client.1.smithi193.stdout:4/798: read f9 [1397298,36922] 0 2024-04-25T15:49:49.043 INFO:tasks.workunit.client.1.smithi193.stdout:6/722: dread - dd/d12/d19/d1c/d40/d3c/d3f/d4f/f7b zero size 2024-04-25T15:49:49.046 INFO:tasks.workunit.client.1.smithi193.stdout:0/754: rmdir d0/d49/d5/d7/dd 39 2024-04-25T15:49:49.049 INFO:tasks.workunit.client.1.smithi193.stdout:4/799: dwrite db/d13/dcb/fdb [0,4194304] 0 2024-04-25T15:49:49.051 INFO:tasks.workunit.client.1.smithi193.stdout:9/732: rename df/d2c/d2f/d4e/d1c/d30/l65 to df/d2c/d2f/d4e/d1c/d30/d60/le2 0 2024-04-25T15:49:49.052 INFO:tasks.workunit.client.0.smithi110.stdout:4/866: link d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d11a/l120 d13/dba/l12c 0 2024-04-25T15:49:49.059 INFO:tasks.workunit.client.1.smithi193.stdout:9/733: mkdir df/d2c/d2f/d4e/d69/d6b/de3 0 2024-04-25T15:49:49.062 INFO:tasks.workunit.client.1.smithi193.stdout:6/723: creat dd/d12/d19/d1c/d40/d3c/d3f/fdb x:0 0 0 2024-04-25T15:49:49.063 INFO:tasks.workunit.client.1.smithi193.stdout:4/800: link db/d13/c2b db/d13/dcb/c101 0 2024-04-25T15:49:49.064 INFO:tasks.workunit.client.0.smithi110.stdout:2/855: write de/d40/d5b/d5c/d5f/db2/dad/fdb [456899,18936] 0 2024-04-25T15:49:49.064 INFO:tasks.workunit.client.1.smithi193.stdout:8/735: dwrite d2/d3/d27/f9b [0,4194304] 0 2024-04-25T15:49:49.065 INFO:tasks.workunit.client.0.smithi110.stdout:2/856: dread - de/d40/d5b/d5c/d5f/db2/fc4 zero size 2024-04-25T15:49:49.068 INFO:tasks.workunit.client.1.smithi193.stdout:5/760: write d17/d18/d30/d3b/f60 [2374095,24966] 0 2024-04-25T15:49:49.069 INFO:tasks.workunit.client.1.smithi193.stdout:4/801: symlink db/d84/df3/l102 0 2024-04-25T15:49:49.070 INFO:tasks.workunit.client.0.smithi110.stdout:2/857: dread de/d40/d6e/d50/db7/fe3 [0,4194304] 0 2024-04-25T15:49:49.071 INFO:tasks.workunit.client.1.smithi193.stdout:8/736: dread d2/d18/db1/fbd [0,4194304] 0 2024-04-25T15:49:49.072 INFO:tasks.workunit.client.0.smithi110.stdout:2/858: stat de/d40/d5b/d5c/caf 0 2024-04-25T15:49:49.080 INFO:tasks.workunit.client.1.smithi193.stdout:8/737: read d2/d3/f37 [996294,34787] 0 2024-04-25T15:49:49.082 INFO:tasks.workunit.client.1.smithi193.stdout:1/670: dwrite dc/d4a/d50/d5c/f91 [0,4194304] 0 2024-04-25T15:49:49.082 INFO:tasks.workunit.client.0.smithi110.stdout:7/994: dwrite d1b/d29/d38/d55/d7f/f64 [0,4194304] 0 2024-04-25T15:49:49.083 INFO:tasks.workunit.client.0.smithi110.stdout:6/864: write d1/dd/d3c/d91/d8d/dce/f73 [1899499,122517] 0 2024-04-25T15:49:49.084 INFO:tasks.workunit.client.1.smithi193.stdout:5/761: dwrite d17/d18/d30/d38/d71/d96/dc4/f70 [0,4194304] 0 2024-04-25T15:49:49.085 INFO:tasks.workunit.client.1.smithi193.stdout:2/733: dwrite d0/d37/d25/d5f/d3a/d6d/d6e/f60 [0,4194304] 0 2024-04-25T15:49:49.085 INFO:tasks.workunit.client.0.smithi110.stdout:0/868: write d1/d6c/d116/fd1 [1000274,33443] 0 2024-04-25T15:49:49.090 INFO:tasks.workunit.client.1.smithi193.stdout:0/755: dread d0/d49/d5/d7/dd/d16/d26/df5/f4c [0,4194304] 0 2024-04-25T15:49:49.091 INFO:tasks.workunit.client.1.smithi193.stdout:7/760: truncate d12/d31/d37/fc5 4131155 0 2024-04-25T15:49:49.096 INFO:tasks.workunit.client.1.smithi193.stdout:3/767: truncate d8/d13/d4b/f88 2180946 0 2024-04-25T15:49:49.097 INFO:tasks.workunit.client.1.smithi193.stdout:5/762: creat d17/d18/d30/d34/d52/dee/f100 x:0 0 0 2024-04-25T15:49:49.097 INFO:tasks.workunit.client.1.smithi193.stdout:8/738: symlink d2/d3/d27/d69/dae/d29/led 0 2024-04-25T15:49:49.100 INFO:tasks.workunit.client.0.smithi110.stdout:5/939: write d4/d1d/d1f/d35/d7f/d92/fca [4654010,128587] 0 2024-04-25T15:49:49.100 INFO:tasks.workunit.client.0.smithi110.stdout:6/865: fsync d1/db/f88 0 2024-04-25T15:49:49.102 INFO:tasks.workunit.client.1.smithi193.stdout:0/756: dwrite d0/d49/d5/d7/dd/d46/d48/d62/f92 [0,4194304] 0 2024-04-25T15:49:49.105 INFO:tasks.workunit.client.0.smithi110.stdout:0/869: symlink d1/d23/dc5/d108/l138 0 2024-04-25T15:49:49.106 INFO:tasks.workunit.client.0.smithi110.stdout:7/995: dwrite d1b/d29/d38/d55/d95/dcb/f132 [0,4194304] 0 2024-04-25T15:49:49.114 INFO:tasks.workunit.client.0.smithi110.stdout:4/867: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/d88/f47 [0,4194304] 0 2024-04-25T15:49:49.114 INFO:tasks.workunit.client.1.smithi193.stdout:1/671: dread dc/d4a/d4d/f57 [0,4194304] 0 2024-04-25T15:49:49.114 INFO:tasks.workunit.client.1.smithi193.stdout:3/768: unlink d8/d13/d14/d23/f48 0 2024-04-25T15:49:49.115 INFO:tasks.workunit.client.1.smithi193.stdout:1/672: chown dc/d4a/d50/d7c/dda 482220250 1 2024-04-25T15:49:49.119 INFO:tasks.workunit.client.0.smithi110.stdout:6/866: unlink d1/dd/l98 0 2024-04-25T15:49:49.121 INFO:tasks.workunit.client.1.smithi193.stdout:8/739: mknod d2/d3/d20/cee 0 2024-04-25T15:49:49.123 INFO:tasks.workunit.client.0.smithi110.stdout:0/870: truncate d1/d47/d54/f9c 2501022 0 2024-04-25T15:49:49.123 INFO:tasks.workunit.client.0.smithi110.stdout:2/859: write de/d40/d6e/f93 [167446,31614] 0 2024-04-25T15:49:49.123 INFO:tasks.workunit.client.1.smithi193.stdout:9/734: dwrite df/d2c/d34/d3e/f6a [0,4194304] 0 2024-04-25T15:49:49.126 INFO:tasks.workunit.client.0.smithi110.stdout:5/940: creat d4/d15/d51/d117/f132 x:0 0 0 2024-04-25T15:49:49.126 INFO:tasks.workunit.client.1.smithi193.stdout:6/724: dwrite dd/f42 [0,4194304] 0 2024-04-25T15:49:49.128 INFO:tasks.workunit.client.1.smithi193.stdout:6/725: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa [1084386,61178] 0 2024-04-25T15:49:49.128 INFO:tasks.workunit.client.0.smithi110.stdout:6/867: rmdir d1/db/da0/da5/dea 39 2024-04-25T15:49:49.131 INFO:tasks.workunit.client.0.smithi110.stdout:7/996: mkdir d1b/d29/d150 0 2024-04-25T15:49:49.131 INFO:tasks.workunit.client.0.smithi110.stdout:6/868: write d1/dd/d3c/d91/d8d/dce/f73 [1304545,63956] 0 2024-04-25T15:49:49.134 INFO:tasks.workunit.client.1.smithi193.stdout:3/769: fsync d8/d13/d4b/f5d 0 2024-04-25T15:49:49.135 INFO:tasks.workunit.client.1.smithi193.stdout:1/673: read dc/d4a/d50/d5c/d5d/d3d/f95 [679334,102514] 0 2024-04-25T15:49:49.136 INFO:tasks.workunit.client.1.smithi193.stdout:1/674: chown dc/d4a/d4d/d7a/da6 0 1 2024-04-25T15:49:49.140 INFO:tasks.workunit.client.0.smithi110.stdout:5/941: rename d4/d1d/d1f/d35/d3d/d52/d75/dfa/d105/d10a/dd4/cf7 to d4/d1d/d1f/d35/d3d/d52/d75/c133 0 2024-04-25T15:49:49.143 INFO:tasks.workunit.client.1.smithi193.stdout:9/735: mknod df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/ce4 0 2024-04-25T15:49:49.145 INFO:tasks.workunit.client.1.smithi193.stdout:6/726: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/cdc 0 2024-04-25T15:49:49.147 INFO:tasks.workunit.client.1.smithi193.stdout:3/770: rmdir d8/d30/d72 39 2024-04-25T15:49:49.148 INFO:tasks.workunit.client.1.smithi193.stdout:9/736: dwrite f9 [0,4194304] 0 2024-04-25T15:49:49.148 INFO:tasks.workunit.client.1.smithi193.stdout:1/675: symlink dc/d4a/d4d/d7a/da6/le1 0 2024-04-25T15:49:49.149 INFO:tasks.workunit.client.1.smithi193.stdout:8/740: mknod d2/d18/cef 0 2024-04-25T15:49:49.150 INFO:tasks.workunit.client.1.smithi193.stdout:2/734: getdents d0/df 0 2024-04-25T15:49:49.152 INFO:tasks.workunit.client.1.smithi193.stdout:0/757: link d0/d49/d5/d7/f71 d0/d49/d5/d61/f101 0 2024-04-25T15:49:49.156 INFO:tasks.workunit.client.1.smithi193.stdout:3/771: symlink d8/d13/d4b/d8b/de3/lf3 0 2024-04-25T15:49:49.158 INFO:tasks.workunit.client.1.smithi193.stdout:9/737: unlink df/d2c/d2f/d4e/c44 0 2024-04-25T15:49:49.159 INFO:tasks.workunit.client.1.smithi193.stdout:8/741: creat d2/da5/ff0 x:0 0 0 2024-04-25T15:49:49.160 INFO:tasks.workunit.client.1.smithi193.stdout:1/676: mkdir dc/d4a/d4d/d7a/dd0/de2 0 2024-04-25T15:49:49.161 INFO:tasks.workunit.client.1.smithi193.stdout:2/735: mkdir d0/d9/d3d/d114 0 2024-04-25T15:49:49.163 INFO:tasks.workunit.client.1.smithi193.stdout:0/758: creat d0/d49/d5/d7/dd/d16/d18/d28/f102 x:0 0 0 2024-04-25T15:49:49.166 INFO:tasks.workunit.client.1.smithi193.stdout:8/742: readlink d2/l28 0 2024-04-25T15:49:49.167 INFO:tasks.workunit.client.1.smithi193.stdout:2/736: creat d0/d9/d3d/d7d/dc3/f115 x:0 0 0 2024-04-25T15:49:49.168 INFO:tasks.workunit.client.1.smithi193.stdout:7/761: write d12/d1c/f7e [203225,6216] 0 2024-04-25T15:49:49.172 INFO:tasks.workunit.client.1.smithi193.stdout:8/743: symlink d2/d3/d27/d69/dae/d29/lf1 0 2024-04-25T15:49:49.174 INFO:tasks.workunit.client.1.smithi193.stdout:7/762: creat d12/d31/d39/d60/f118 x:0 0 0 2024-04-25T15:49:49.174 INFO:tasks.workunit.client.1.smithi193.stdout:7/763: readlink d12/l35 0 2024-04-25T15:49:49.174 INFO:tasks.workunit.client.1.smithi193.stdout:7/764: chown d12/d13/l3d 3 1 2024-04-25T15:49:49.175 INFO:tasks.workunit.client.1.smithi193.stdout:2/737: symlink d0/d37/d25/d5f/d3a/d6d/d6e/db9/l116 0 2024-04-25T15:49:49.175 INFO:tasks.workunit.client.1.smithi193.stdout:5/763: write d17/d18/d30/d3b/fa1 [1000262,121659] 0 2024-04-25T15:49:49.175 INFO:tasks.workunit.client.1.smithi193.stdout:6/727: link dd/d12/d19/d1c/d40/ccd dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/cdd 0 2024-04-25T15:49:49.176 INFO:tasks.workunit.client.1.smithi193.stdout:2/738: stat d0/d37/d25/dda/fe0 0 2024-04-25T15:49:49.176 INFO:tasks.workunit.client.1.smithi193.stdout:3/772: link d8/d13/d14/d23/caf d8/df0/cf4 0 2024-04-25T15:49:49.178 INFO:tasks.workunit.client.1.smithi193.stdout:8/744: rmdir d2/d3/d27/d69/dae/d4a 39 2024-04-25T15:49:49.178 INFO:tasks.workunit.client.1.smithi193.stdout:6/728: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/fb9 [272903,9542] 0 2024-04-25T15:49:49.180 INFO:tasks.workunit.client.1.smithi193.stdout:7/765: mknod d12/d1c/d63/d70/d68/d8f/c119 0 2024-04-25T15:49:49.185 INFO:tasks.workunit.client.0.smithi110.stdout:2/860: write de/d40/d5b/d5c/d5f/f81 [695428,86383] 0 2024-04-25T15:49:49.185 INFO:tasks.workunit.client.1.smithi193.stdout:8/745: creat d2/d3/d20/db8/dd0/ff2 x:0 0 0 2024-04-25T15:49:49.186 INFO:tasks.workunit.client.1.smithi193.stdout:2/739: dwrite d0/d42/d51/d4f/ff9 [0,4194304] 0 2024-04-25T15:49:49.188 INFO:tasks.workunit.client.1.smithi193.stdout:6/729: dwrite dd/f13 [0,4194304] 0 2024-04-25T15:49:49.188 INFO:tasks.workunit.client.0.smithi110.stdout:0/871: dwrite d1/f16 [4194304,4194304] 0 2024-04-25T15:49:49.195 INFO:tasks.workunit.client.1.smithi193.stdout:5/764: symlink d17/d18/d30/d38/d71/d96/dc4/da7/l101 0 2024-04-25T15:49:49.195 INFO:tasks.workunit.client.0.smithi110.stdout:4/868: dwrite d13/d24/d2d/d44/d65/d99/daf/d56/f8e [0,4194304] 0 2024-04-25T15:49:49.198 INFO:tasks.workunit.client.0.smithi110.stdout:7/997: write d1b/f79 [281465,123185] 0 2024-04-25T15:49:49.199 INFO:tasks.workunit.client.0.smithi110.stdout:7/998: dread - d1b/d29/d38/d55/d95/f9e zero size 2024-04-25T15:49:49.199 INFO:tasks.workunit.client.1.smithi193.stdout:8/746: creat d2/d3/d20/db8/dd0/ff3 x:0 0 0 2024-04-25T15:49:49.206 INFO:tasks.workunit.client.0.smithi110.stdout:6/869: write d1/db/d22/d90/d76/dad/fbc [1351604,87050] 0 2024-04-25T15:49:49.208 INFO:tasks.workunit.client.0.smithi110.stdout:2/861: creat de/d40/d6e/d52/d6d/d99/f114 x:0 0 0 2024-04-25T15:49:49.217 INFO:tasks.workunit.client.1.smithi193.stdout:8/747: chown d2/d3/d27/d69/dae/d4a/d63/da2/de4 74970801 1 2024-04-25T15:49:49.217 INFO:tasks.workunit.client.1.smithi193.stdout:7/766: dread d12/d1c/d63/d109/fea [0,4194304] 0 2024-04-25T15:49:49.221 INFO:tasks.workunit.client.1.smithi193.stdout:6/730: symlink dd/d12/d19/d1c/d40/dce/lde 0 2024-04-25T15:49:49.223 INFO:tasks.workunit.client.1.smithi193.stdout:5/765: chown d17/d18/d75/d85/d5d/dad/fb5 426393910 1 2024-04-25T15:49:49.225 INFO:tasks.workunit.client.0.smithi110.stdout:6/870: rename d1/dd/d3c/d91/d8d/dce/d102/lbf to d1/db/da0/da5/dea/d124/l12c 0 2024-04-25T15:49:49.226 INFO:tasks.workunit.client.1.smithi193.stdout:0/759: write d0/d49/d5/d7/dd/d78/fa6 [82816,873] 0 2024-04-25T15:49:49.230 INFO:tasks.workunit.client.1.smithi193.stdout:5/766: dwrite d17/d18/d30/d38/d48/d4f/fd6 [0,4194304] 0 2024-04-25T15:49:49.232 INFO:tasks.workunit.client.1.smithi193.stdout:6/731: truncate dd/d12/d19/d1c/d40/d3c/f68 3140781 0 2024-04-25T15:49:49.235 INFO:tasks.workunit.client.1.smithi193.stdout:8/748: creat d2/d3/d27/d69/dae/dd8/ff4 x:0 0 0 2024-04-25T15:49:49.239 INFO:tasks.workunit.client.1.smithi193.stdout:6/732: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa [0,4194304] 0 2024-04-25T15:49:49.247 INFO:tasks.workunit.client.1.smithi193.stdout:2/740: write d0/d37/d25/d5f/d3a/d6d/d6e/db9/fee [390607,71004] 0 2024-04-25T15:49:49.248 INFO:tasks.workunit.client.0.smithi110.stdout:7/999: rmdir d1b/d29/d38/d8b/d89/dc6/df3/d130 0 2024-04-25T15:49:49.248 INFO:tasks.workunit.client.1.smithi193.stdout:3/773: dwrite d8/d13/d14/d23/de8/fea [0,4194304] 0 2024-04-25T15:49:49.248 INFO:tasks.workunit.client.1.smithi193.stdout:2/741: chown d0/d37/dbb/df8 17 1 2024-04-25T15:49:49.250 INFO:tasks.workunit.client.1.smithi193.stdout:2/742: stat d0/d9/d15/d31/lfb 0 2024-04-25T15:49:49.251 INFO:tasks.workunit.client.0.smithi110.stdout:0/872: write d1/d23/d30/d33/d5b/d83/f84 [2593385,49791] 0 2024-04-25T15:49:49.251 INFO:tasks.workunit.client.1.smithi193.stdout:3/774: write d8/d30/f42 [1980565,90908] 0 2024-04-25T15:49:49.252 INFO:tasks.workunit.client.1.smithi193.stdout:8/749: mknod d2/d3/d27/d69/cf5 0 2024-04-25T15:49:49.253 INFO:tasks.workunit.client.1.smithi193.stdout:6/733: rmdir dd/d12/d19/d1c 39 2024-04-25T15:49:49.253 INFO:tasks.workunit.client.0.smithi110.stdout:4/869: write d13/d24/d2d/d36/f74 [1388398,31011] 0 2024-04-25T15:49:49.255 INFO:tasks.workunit.client.1.smithi193.stdout:7/767: rmdir d12/d31/d37/dcd 0 2024-04-25T15:49:49.257 INFO:tasks.workunit.client.1.smithi193.stdout:8/750: dread d2/d3/d27/fdf [0,4194304] 0 2024-04-25T15:49:49.260 INFO:tasks.workunit.client.1.smithi193.stdout:7/768: read d12/d1c/d45/f7a [91275,91671] 0 2024-04-25T15:49:49.260 INFO:tasks.workunit.client.1.smithi193.stdout:7/769: stat d12/d13/c16 0 2024-04-25T15:49:49.261 INFO:tasks.workunit.client.0.smithi110.stdout:4/870: dwrite d13/dba/ffb [0,4194304] 0 2024-04-25T15:49:49.263 INFO:tasks.workunit.client.0.smithi110.stdout:0/873: creat d1/d23/d30/d33/d129/db4/f139 x:0 0 0 2024-04-25T15:49:49.266 INFO:tasks.workunit.client.1.smithi193.stdout:3/775: symlink d8/d13/d1f/d9f/lf5 0 2024-04-25T15:49:49.267 INFO:tasks.workunit.client.0.smithi110.stdout:4/871: unlink d13/d24/d2d/d44/d65/d99/daf/d56/d88/lae 0 2024-04-25T15:49:49.270 INFO:tasks.workunit.client.1.smithi193.stdout:6/734: truncate dd/d2f/f50 742377 0 2024-04-25T15:49:49.274 INFO:tasks.workunit.client.1.smithi193.stdout:3/776: dread d8/d13/d1f/dbf/fba [0,4194304] 0 2024-04-25T15:49:49.274 INFO:tasks.workunit.client.1.smithi193.stdout:0/760: creat d0/d49/d5/d7/dd/d16/d18/f103 x:0 0 0 2024-04-25T15:49:49.276 INFO:tasks.workunit.client.1.smithi193.stdout:0/761: dread - d0/d49/d5/d7/dd/ffc zero size 2024-04-25T15:49:49.276 INFO:tasks.workunit.client.1.smithi193.stdout:5/767: creat d17/d18/f102 x:0 0 0 2024-04-25T15:49:49.277 INFO:tasks.workunit.client.1.smithi193.stdout:7/770: mkdir d12/d31/d37/d104/d11a 0 2024-04-25T15:49:49.278 INFO:tasks.workunit.client.1.smithi193.stdout:0/762: dread - d0/d49/d5/d7/dd/d16/d26/d2e/d3d/ffe zero size 2024-04-25T15:49:49.279 INFO:tasks.workunit.client.1.smithi193.stdout:6/735: fdatasync dd/d2f/d57/f69 0 2024-04-25T15:49:49.280 INFO:tasks.workunit.client.1.smithi193.stdout:0/763: write d0/d49/d5/d7/dd/d16/d26/d2e/d93/fb9 [979949,4111] 0 2024-04-25T15:49:49.281 INFO:tasks.workunit.client.0.smithi110.stdout:6/871: truncate d1/dd/d3c/f7c 3668574 0 2024-04-25T15:49:49.281 INFO:tasks.workunit.client.1.smithi193.stdout:6/736: chown dd/d2f/f56 74 1 2024-04-25T15:49:49.285 INFO:tasks.workunit.client.1.smithi193.stdout:7/771: dwrite d12/d1c/d63/f111 [0,4194304] 0 2024-04-25T15:49:49.288 INFO:tasks.workunit.client.1.smithi193.stdout:8/751: link d2/f48 d2/d3/d27/ff6 0 2024-04-25T15:49:49.291 INFO:tasks.workunit.client.0.smithi110.stdout:6/872: truncate d1/db/d22/d33/f56 1011485 0 2024-04-25T15:49:49.293 INFO:tasks.workunit.client.1.smithi193.stdout:2/743: write d0/d37/d25/d5f/d3a/d6d/d85/f99 [562188,57275] 0 2024-04-25T15:49:49.295 INFO:tasks.workunit.client.1.smithi193.stdout:9/738: sync 2024-04-25T15:49:49.295 INFO:tasks.workunit.client.1.smithi193.stdout:4/802: sync 2024-04-25T15:49:49.295 INFO:tasks.workunit.client.1.smithi193.stdout:1/677: sync 2024-04-25T15:49:49.298 INFO:tasks.workunit.client.0.smithi110.stdout:0/874: write d1/d23/d30/d33/f88 [4410319,53837] 0 2024-04-25T15:49:49.298 INFO:tasks.workunit.client.1.smithi193.stdout:7/772: symlink d12/d31/d39/db0/l11b 0 2024-04-25T15:49:49.301 INFO:tasks.workunit.client.1.smithi193.stdout:8/752: symlink d2/d3/d27/d69/dae/dd6/lf7 0 2024-04-25T15:49:49.305 INFO:tasks.workunit.client.0.smithi110.stdout:6/873: link d1/dd/d3c/d91/d8d/dce/fde d1/dd/f12d 0 2024-04-25T15:49:49.305 INFO:tasks.workunit.client.1.smithi193.stdout:1/678: dwrite dc/d4a/d77/dd3/fdf [0,4194304] 0 2024-04-25T15:49:49.306 INFO:tasks.workunit.client.1.smithi193.stdout:9/739: rmdir df/d2c/d2f/d4e/d1c/d30/d3b 39 2024-04-25T15:49:49.306 INFO:tasks.workunit.client.0.smithi110.stdout:6/874: chown d1/dd/d3c/d91/d10d 338719 1 2024-04-25T15:49:49.307 INFO:tasks.workunit.client.0.smithi110.stdout:4/872: dread d13/d24/d2d/d44/f60 [0,4194304] 0 2024-04-25T15:49:49.307 INFO:tasks.workunit.client.1.smithi193.stdout:0/764: dread d0/d49/d5/d61/fae [0,4194304] 0 2024-04-25T15:49:49.309 INFO:tasks.workunit.client.1.smithi193.stdout:4/803: symlink db/d13/d1e/d1f/d56/d69/dad/l103 0 2024-04-25T15:49:49.309 INFO:tasks.workunit.client.1.smithi193.stdout:4/804: chown fa 2 1 2024-04-25T15:49:49.309 INFO:tasks.workunit.client.0.smithi110.stdout:6/875: creat d1/d12/daf/f12e x:0 0 0 2024-04-25T15:49:49.310 INFO:tasks.workunit.client.1.smithi193.stdout:2/744: truncate d0/d9/d3d/d7d/fab 2941620 0 2024-04-25T15:49:49.311 INFO:tasks.workunit.client.1.smithi193.stdout:2/745: chown d0/d44/lf4 20955845 1 2024-04-25T15:49:49.311 INFO:tasks.workunit.client.0.smithi110.stdout:0/875: link d1/d23/d30/d33/d5b/d83/de1/lea d1/d23/d30/d33/d5b/d78/l13a 0 2024-04-25T15:49:49.315 INFO:tasks.workunit.client.1.smithi193.stdout:5/768: write d17/d18/d30/d38/d71/d96/dc4/f47 [4110807,115600] 0 2024-04-25T15:49:49.320 INFO:tasks.workunit.client.0.smithi110.stdout:4/873: dread - d13/d24/d2d/d44/da4/fca zero size 2024-04-25T15:49:49.323 INFO:tasks.workunit.client.1.smithi193.stdout:8/753: truncate d2/d3/d20/d8c/dde/d94/f2b 5012094 0 2024-04-25T15:49:49.323 INFO:tasks.workunit.client.1.smithi193.stdout:3/777: dwrite d8/d13/d1f/d69/fcf [4194304,4194304] 0 2024-04-25T15:49:49.326 INFO:tasks.workunit.client.0.smithi110.stdout:4/874: fsync fa 0 2024-04-25T15:49:49.333 INFO:tasks.workunit.client.1.smithi193.stdout:3/778: dwrite d8/d30/f42 [4194304,4194304] 0 2024-04-25T15:49:49.335 INFO:tasks.workunit.client.1.smithi193.stdout:5/769: fdatasync d17/d18/d30/d38/d48/d4f/f5f 0 2024-04-25T15:49:49.336 INFO:tasks.workunit.client.1.smithi193.stdout:7/773: mknod d12/d31/d37/c11c 0 2024-04-25T15:49:49.343 INFO:tasks.workunit.client.1.smithi193.stdout:8/754: readlink d2/d3/d20/d8c/dde/lb7 0 2024-04-25T15:49:49.344 INFO:tasks.workunit.client.1.smithi193.stdout:8/755: chown d2/d3/d3a/lea 43 1 2024-04-25T15:49:49.345 INFO:tasks.workunit.client.1.smithi193.stdout:8/756: chown d2/d3/d27/d69/dae/dd6/lf7 0 1 2024-04-25T15:49:49.345 INFO:tasks.workunit.client.1.smithi193.stdout:8/757: chown d2/d18/c99 56 1 2024-04-25T15:49:49.346 INFO:tasks.workunit.client.1.smithi193.stdout:8/758: readlink d2/d3/d27/d69/dae/d4a/l82 0 2024-04-25T15:49:49.348 INFO:tasks.workunit.client.0.smithi110.stdout:6/876: dwrite d1/db/d22/d33/d63/fd4 [0,4194304] 0 2024-04-25T15:49:49.350 INFO:tasks.workunit.client.0.smithi110.stdout:6/877: write d1/dd/f64 [1098706,55422] 0 2024-04-25T15:49:49.354 INFO:tasks.workunit.client.1.smithi193.stdout:1/679: dwrite dc/d4a/d50/d7c/d7e/f8c [0,4194304] 0 2024-04-25T15:49:49.355 INFO:tasks.workunit.client.1.smithi193.stdout:2/746: symlink d0/d37/dbb/df8/d110/l117 0 2024-04-25T15:49:49.358 INFO:tasks.workunit.client.1.smithi193.stdout:3/779: mkdir d8/d13/d1f/dbf/df6 0 2024-04-25T15:49:49.358 INFO:tasks.workunit.client.1.smithi193.stdout:2/747: truncate d0/d37/fb3 807533 0 2024-04-25T15:49:49.359 INFO:tasks.workunit.client.1.smithi193.stdout:4/805: write fa [883773,43277] 0 2024-04-25T15:49:49.360 INFO:tasks.workunit.client.1.smithi193.stdout:2/748: read d0/d37/d25/f77 [2812222,22640] 0 2024-04-25T15:49:49.360 INFO:tasks.workunit.client.1.smithi193.stdout:0/765: dwrite d0/d49/d5/d7/dd/d16/d18/f2a [0,4194304] 0 2024-04-25T15:49:49.362 INFO:tasks.workunit.client.1.smithi193.stdout:0/766: write d0/d49/d5/d7/dff/f8e [4567850,30545] 0 2024-04-25T15:49:49.366 INFO:tasks.workunit.client.1.smithi193.stdout:6/737: getdents dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dc1 0 2024-04-25T15:49:49.368 INFO:tasks.workunit.client.1.smithi193.stdout:6/738: chown dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f 668382 1 2024-04-25T15:49:49.370 INFO:tasks.workunit.client.1.smithi193.stdout:8/759: read d2/d3/d27/d69/dae/d4a/d63/da2/fa4 [301520,2514] 0 2024-04-25T15:49:49.371 INFO:tasks.workunit.client.1.smithi193.stdout:8/760: fdatasync d2/d3/d20/d8c/dde/fd2 0 2024-04-25T15:49:49.373 INFO:tasks.workunit.client.1.smithi193.stdout:9/740: write df/d2c/d2f/d4e/d1c/d30/d60/f73 [1315557,71395] 0 2024-04-25T15:49:49.374 INFO:tasks.workunit.client.0.smithi110.stdout:0/876: link d1/d23/d30/d33/d5b/d78/da4/fba d1/d23/f13b 0 2024-04-25T15:49:49.375 INFO:tasks.workunit.client.1.smithi193.stdout:3/780: write d8/d30/d72/f7e [1392862,52808] 0 2024-04-25T15:49:49.379 INFO:tasks.workunit.client.1.smithi193.stdout:7/774: symlink d12/d13/l11d 0 2024-04-25T15:49:49.380 INFO:tasks.workunit.client.0.smithi110.stdout:6/878: creat d1/db/da0/da5/dea/d124/f12f x:0 0 0 2024-04-25T15:49:49.384 INFO:tasks.workunit.client.1.smithi193.stdout:8/761: mknod d2/d3/d27/d69/dae/d29/cf8 0 2024-04-25T15:49:49.385 INFO:tasks.workunit.client.0.smithi110.stdout:6/879: creat d1/db/d24/d2d/d11a/f130 x:0 0 0 2024-04-25T15:49:49.386 INFO:tasks.workunit.client.1.smithi193.stdout:1/680: symlink dc/d4a/d50/d7c/d82/ddc/le3 0 2024-04-25T15:49:49.387 INFO:tasks.workunit.client.0.smithi110.stdout:0/877: rename d1/d6c/d116/d77/lcd to d1/d6c/d116/l13c 0 2024-04-25T15:49:49.387 INFO:tasks.workunit.client.1.smithi193.stdout:9/741: rmdir df/d2c/d2f/d5c/d7e/da2 39 2024-04-25T15:49:49.388 INFO:tasks.workunit.client.0.smithi110.stdout:6/880: readlink d1/dd/d3c/d91/d8d/dce/d102/l129 0 2024-04-25T15:49:49.389 INFO:tasks.workunit.client.0.smithi110.stdout:0/878: mkdir d1/d47/d113/d13d 0 2024-04-25T15:49:49.390 INFO:tasks.workunit.client.1.smithi193.stdout:4/806: fdatasync db/d4b/f71 0 2024-04-25T15:49:49.391 INFO:tasks.workunit.client.1.smithi193.stdout:3/781: chown d8/d13/d1f/d69/dcd/dd9/dde/f91 154 1 2024-04-25T15:49:49.392 INFO:tasks.workunit.client.1.smithi193.stdout:3/782: chown d8/d13/d1f/d69/cb8 857259 1 2024-04-25T15:49:49.392 INFO:tasks.workunit.client.0.smithi110.stdout:6/881: mkdir d1/db/d22/d90/d11d/d131 0 2024-04-25T15:49:49.392 INFO:tasks.workunit.client.1.smithi193.stdout:3/783: chown d8/d13/d14/c75 188847172 1 2024-04-25T15:49:49.393 INFO:tasks.workunit.client.1.smithi193.stdout:3/784: write d8/d30/f47 [2428549,75002] 0 2024-04-25T15:49:49.394 INFO:tasks.workunit.client.0.smithi110.stdout:0/879: rename d1/d23/d30/d33/d35/l55 to d1/d23/d30/d33/dfe/l13e 0 2024-04-25T15:49:49.395 INFO:tasks.workunit.client.0.smithi110.stdout:0/880: chown d1/d23/d30/d33/d5b/d83/f84 92365918 1 2024-04-25T15:49:49.398 INFO:tasks.workunit.client.1.smithi193.stdout:0/767: link d0/f35 d0/d49/d6a/f104 0 2024-04-25T15:49:49.399 INFO:tasks.workunit.client.0.smithi110.stdout:0/881: creat d1/d23/d30/d33/dfe/f13f x:0 0 0 2024-04-25T15:49:49.400 INFO:tasks.workunit.client.0.smithi110.stdout:6/882: getdents d1/dd/db5 0 2024-04-25T15:49:49.402 INFO:tasks.workunit.client.1.smithi193.stdout:1/681: mknod dc/d4a/d50/d7c/d82/ddc/ce4 0 2024-04-25T15:49:49.405 INFO:tasks.workunit.client.1.smithi193.stdout:9/742: chown df/d2c/d2f/d4e/d1c/d30/d60/le2 7473756 1 2024-04-25T15:49:49.406 INFO:tasks.workunit.client.0.smithi110.stdout:6/883: dwrite d1/dd/d3c/d91/d8d/dce/d102/d75/d82/fa6 [0,4194304] 0 2024-04-25T15:49:49.407 INFO:tasks.workunit.client.0.smithi110.stdout:0/882: creat d1/d23/d30/d33/d5b/d78/ddb/d10b/f140 x:0 0 0 2024-04-25T15:49:49.408 INFO:tasks.workunit.client.1.smithi193.stdout:9/743: read df/d2c/d34/d3e/fa5 [480721,28221] 0 2024-04-25T15:49:49.410 INFO:tasks.workunit.client.1.smithi193.stdout:4/807: rmdir db/d13/dce 39 2024-04-25T15:49:49.411 INFO:tasks.workunit.client.1.smithi193.stdout:4/808: readlink db/d13/d1e/d1f/d56/d69/dad/l103 0 2024-04-25T15:49:49.412 INFO:tasks.workunit.client.1.smithi193.stdout:3/785: mkdir d8/d13/d14/da0/df7 0 2024-04-25T15:49:49.415 INFO:tasks.workunit.client.1.smithi193.stdout:5/770: link d17/c2b d17/d18/d30/d38/c103 0 2024-04-25T15:49:49.416 INFO:tasks.workunit.client.1.smithi193.stdout:3/786: dread d8/d13/d14/da0/fb5 [0,4194304] 0 2024-04-25T15:49:49.416 INFO:tasks.workunit.client.1.smithi193.stdout:3/787: stat d8/d13/d1f/d9f/fa5 0 2024-04-25T15:49:49.417 INFO:tasks.workunit.client.0.smithi110.stdout:6/884: symlink d1/db/d22/d90/d11d/d131/l132 0 2024-04-25T15:49:49.419 INFO:tasks.workunit.client.1.smithi193.stdout:8/762: symlink d2/lf9 0 2024-04-25T15:49:49.420 INFO:tasks.workunit.client.1.smithi193.stdout:1/682: mkdir dc/d59/de5 0 2024-04-25T15:49:49.421 INFO:tasks.workunit.client.0.smithi110.stdout:6/885: creat d1/dd/d3c/f133 x:0 0 0 2024-04-25T15:49:49.424 INFO:tasks.workunit.client.1.smithi193.stdout:9/744: unlink df/d2c/d2f/d4e/d1c/l9f 0 2024-04-25T15:49:49.425 INFO:tasks.workunit.client.1.smithi193.stdout:9/745: truncate df/d2c/d2f/d4e/f9c 78271 0 2024-04-25T15:49:49.425 INFO:tasks.workunit.client.0.smithi110.stdout:6/886: mkdir d1/db/d22/d90/d11d/d134 0 2024-04-25T15:49:49.427 INFO:tasks.workunit.client.1.smithi193.stdout:4/809: dread db/d13/d1e/d1f/d22/f2c [0,4194304] 0 2024-04-25T15:49:49.428 INFO:tasks.workunit.client.1.smithi193.stdout:4/810: fdatasync db/f61 0 2024-04-25T15:49:49.428 INFO:tasks.workunit.client.1.smithi193.stdout:0/768: mknod d0/d49/d5/d7/dd/d16/d18/c105 0 2024-04-25T15:49:49.429 INFO:tasks.workunit.client.0.smithi110.stdout:6/887: mkdir d1/db/d22/d33/d63/dc2/dc7/dec/d135 0 2024-04-25T15:49:49.432 INFO:tasks.workunit.client.1.smithi193.stdout:8/763: unlink d2/d3/f37 0 2024-04-25T15:49:49.434 INFO:tasks.workunit.client.0.smithi110.stdout:6/888: dread - d1/db/d22/d90/d76/dad/ff1 zero size 2024-04-25T15:49:49.434 INFO:tasks.workunit.client.1.smithi193.stdout:0/769: dwrite d0/d49/d5/d7/dd/d16/d18/fc5 [0,4194304] 0 2024-04-25T15:49:49.436 INFO:tasks.workunit.client.1.smithi193.stdout:9/746: creat df/d2c/d2f/d5c/da3/fe5 x:0 0 0 2024-04-25T15:49:49.437 INFO:tasks.workunit.client.1.smithi193.stdout:1/683: dread dc/d4a/d50/d5c/d63/f67 [0,4194304] 0 2024-04-25T15:49:49.439 INFO:tasks.workunit.client.0.smithi110.stdout:6/889: rename d1/db/d24/daa/f116 to d1/dd/d3c/d91/d8d/dce/d102/d75/d82/f136 0 2024-04-25T15:49:49.440 INFO:tasks.workunit.client.1.smithi193.stdout:2/749: truncate d0/d42/d51/d4f/f69 1119391 0 2024-04-25T15:49:49.446 INFO:tasks.workunit.client.1.smithi193.stdout:4/811: rename db/f8d to db/d32/d45/dab/f104 0 2024-04-25T15:49:49.446 INFO:tasks.workunit.client.1.smithi193.stdout:7/775: dwrite d12/d31/d37/d92/dd1/ff6 [0,4194304] 0 2024-04-25T15:49:49.452 INFO:tasks.workunit.client.0.smithi110.stdout:6/890: creat d1/db/d22/f137 x:0 0 0 2024-04-25T15:49:49.456 INFO:tasks.workunit.client.0.smithi110.stdout:0/883: write d1/d23/f89 [1239779,50162] 0 2024-04-25T15:49:49.458 INFO:tasks.workunit.client.0.smithi110.stdout:6/891: dread d1/dd/d3c/d91/d8d/dce/da1/fc1 [0,4194304] 0 2024-04-25T15:49:49.463 INFO:tasks.workunit.client.1.smithi193.stdout:7/776: dread d12/f18 [0,4194304] 0 2024-04-25T15:49:49.469 INFO:tasks.workunit.client.1.smithi193.stdout:4/812: rename db/d4b/d51/d54/d77/f79 to db/d13/d9d/f105 0 2024-04-25T15:49:49.472 INFO:tasks.workunit.client.0.smithi110.stdout:6/892: dread d1/dd/f12d [0,4194304] 0 2024-04-25T15:49:49.474 INFO:tasks.workunit.client.0.smithi110.stdout:0/884: link d1/d47/l59 d1/d23/d30/l141 0 2024-04-25T15:49:49.477 INFO:tasks.workunit.client.1.smithi193.stdout:2/750: mknod d0/d37/d25/d5f/d3a/db8/c118 0 2024-04-25T15:49:49.477 INFO:tasks.workunit.client.1.smithi193.stdout:5/771: dwrite d17/d18/f84 [0,4194304] 0 2024-04-25T15:49:49.479 INFO:tasks.workunit.client.0.smithi110.stdout:6/893: symlink d1/l138 0 2024-04-25T15:49:49.479 INFO:tasks.workunit.client.1.smithi193.stdout:8/764: write d2/d3/d27/d69/f9a [517377,110434] 0 2024-04-25T15:49:49.481 INFO:tasks.workunit.client.0.smithi110.stdout:0/885: rename d1/d23/dc5/c105 to d1/d23/d30/d33/d129/db4/c142 0 2024-04-25T15:49:49.484 INFO:tasks.workunit.client.1.smithi193.stdout:9/747: write df/d2c/d2f/d4e/f1a [2373548,70892] 0 2024-04-25T15:49:49.485 INFO:tasks.workunit.client.1.smithi193.stdout:1/684: creat dc/d3c/fe6 x:0 0 0 2024-04-25T15:49:49.486 INFO:tasks.workunit.client.0.smithi110.stdout:6/894: dread d1/db/d22/d33/d43/f6f [0,4194304] 0 2024-04-25T15:49:49.487 INFO:tasks.workunit.client.1.smithi193.stdout:3/788: dwrite d8/d13/f56 [0,4194304] 0 2024-04-25T15:49:49.487 INFO:tasks.workunit.client.1.smithi193.stdout:9/748: write df/d2c/d34/d3e/f6a [5033380,108112] 0 2024-04-25T15:49:49.488 INFO:tasks.workunit.client.1.smithi193.stdout:7/777: dread d12/d1c/d63/f65 [0,4194304] 0 2024-04-25T15:49:49.489 INFO:tasks.workunit.client.1.smithi193.stdout:9/749: chown df/d2c/d2f/d4e/d1c/l2b 2938 1 2024-04-25T15:49:49.490 INFO:tasks.workunit.client.1.smithi193.stdout:2/751: rmdir d0/d37/d25/d5f/d3a/d6d 39 2024-04-25T15:49:49.493 INFO:tasks.workunit.client.0.smithi110.stdout:6/895: dwrite d1/db/d24/d2d/f3a [0,4194304] 0 2024-04-25T15:49:49.493 INFO:tasks.workunit.client.1.smithi193.stdout:5/772: creat d17/d18/d75/d85/d5d/dad/f104 x:0 0 0 2024-04-25T15:49:49.493 INFO:tasks.workunit.client.1.smithi193.stdout:6/739: sync 2024-04-25T15:49:49.496 INFO:tasks.workunit.client.0.smithi110.stdout:6/896: read - d1/db/d22/d33/d43/fd1 zero size 2024-04-25T15:49:49.498 INFO:tasks.workunit.client.1.smithi193.stdout:1/685: write dc/d4a/d50/d5c/d5d/d3d/d55/d8a/fa5 [26563,120647] 0 2024-04-25T15:49:49.500 INFO:tasks.workunit.client.1.smithi193.stdout:9/750: mknod df/d2c/d2f/d4e/ce6 0 2024-04-25T15:49:49.502 INFO:tasks.workunit.client.1.smithi193.stdout:2/752: mknod d0/d9/d3d/d7d/dc3/c119 0 2024-04-25T15:49:49.503 INFO:tasks.workunit.client.1.smithi193.stdout:8/765: dwrite d2/d3/d27/d69/dae/d4a/d53/f58 [0,4194304] 0 2024-04-25T15:49:49.505 INFO:tasks.workunit.client.1.smithi193.stdout:6/740: write dd/d12/d19/d1c/d40/fcb [1011379,27183] 0 2024-04-25T15:49:49.506 INFO:tasks.workunit.client.1.smithi193.stdout:4/813: getdents db/d13/dcb 0 2024-04-25T15:49:49.508 INFO:tasks.workunit.client.1.smithi193.stdout:9/751: mkdir df/d2c/d2f/d4e/d69/de7 0 2024-04-25T15:49:49.508 INFO:tasks.workunit.client.1.smithi193.stdout:5/773: dread d17/f59 [0,4194304] 0 2024-04-25T15:49:49.509 INFO:tasks.workunit.client.1.smithi193.stdout:2/753: unlink d0/d42/d51/d4f/d65/ffd 0 2024-04-25T15:49:49.512 INFO:tasks.workunit.client.1.smithi193.stdout:7/778: link d12/d1c/d63/d70/d68/fb4 d12/d1c/d96/f11e 0 2024-04-25T15:49:49.514 INFO:tasks.workunit.client.1.smithi193.stdout:8/766: symlink d2/d3/d27/d69/dae/dd8/lfa 0 2024-04-25T15:49:49.515 INFO:tasks.workunit.client.1.smithi193.stdout:6/741: mknod dd/d2f/d57/daf/cdf 0 2024-04-25T15:49:49.516 INFO:tasks.workunit.client.1.smithi193.stdout:6/742: write dd/d12/d19/d1c/d40/d3c/d3f/fac [859777,63881] 0 2024-04-25T15:49:49.517 INFO:tasks.workunit.client.1.smithi193.stdout:4/814: mknod db/d4b/d51/d54/d6f/c106 0 2024-04-25T15:49:49.519 INFO:tasks.workunit.client.1.smithi193.stdout:5/774: unlink d17/d18/d30/d38/d71/d96/dc4/da7/l101 0 2024-04-25T15:49:49.520 INFO:tasks.workunit.client.1.smithi193.stdout:3/789: getdents d8/d13/d4b/d8b 0 2024-04-25T15:49:49.525 INFO:tasks.workunit.client.1.smithi193.stdout:1/686: creat dc/d4a/d50/d5c/d5d/d3d/d55/fe7 x:0 0 0 2024-04-25T15:49:49.525 INFO:tasks.workunit.client.1.smithi193.stdout:0/770: sync 2024-04-25T15:49:49.526 INFO:tasks.workunit.client.1.smithi193.stdout:6/743: unlink dd/d12/d19/d1c/d40/d3c/d3f/fdb 0 2024-04-25T15:49:49.526 INFO:tasks.workunit.client.1.smithi193.stdout:9/752: rmdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6 39 2024-04-25T15:49:49.526 INFO:tasks.workunit.client.1.smithi193.stdout:4/815: read - db/d32/d45/dab/fbd zero size 2024-04-25T15:49:49.529 INFO:tasks.workunit.client.1.smithi193.stdout:2/754: symlink d0/d37/d25/d5f/d3a/d6d/d6e/db9/l11a 0 2024-04-25T15:49:49.529 INFO:tasks.workunit.client.1.smithi193.stdout:8/767: link d2/d3/d20/d8c/dde/lb5 d2/da5/lfb 0 2024-04-25T15:49:49.530 INFO:tasks.workunit.client.1.smithi193.stdout:5/775: symlink d17/d18/d30/d38/d71/l105 0 2024-04-25T15:49:49.532 INFO:tasks.workunit.client.1.smithi193.stdout:4/816: dread - db/d4b/fa0 zero size 2024-04-25T15:49:49.533 INFO:tasks.workunit.client.1.smithi193.stdout:9/753: fsync df/d2c/d2f/d4e/f25 0 2024-04-25T15:49:49.534 INFO:tasks.workunit.client.1.smithi193.stdout:6/744: dwrite dd/d2f/d57/f8b [4194304,4194304] 0 2024-04-25T15:49:49.535 INFO:tasks.workunit.client.1.smithi193.stdout:4/817: write db/d4b/d51/d54/d77/ff7 [339568,55954] 0 2024-04-25T15:49:49.536 INFO:tasks.workunit.client.1.smithi193.stdout:8/768: symlink d2/d3/d20/lfc 0 2024-04-25T15:49:49.538 INFO:tasks.workunit.client.1.smithi193.stdout:0/771: creat d0/d49/d5/d61/dd4/f106 x:0 0 0 2024-04-25T15:49:49.542 INFO:tasks.workunit.client.1.smithi193.stdout:2/755: symlink d0/d9/d15/df3/l11b 0 2024-04-25T15:49:49.542 INFO:tasks.workunit.client.1.smithi193.stdout:0/772: stat d0/d49/d5/d7/dd/l3b 0 2024-04-25T15:49:49.542 INFO:tasks.workunit.client.1.smithi193.stdout:9/754: truncate df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/fd4 50644 0 2024-04-25T15:49:49.543 INFO:tasks.workunit.client.1.smithi193.stdout:5/776: mknod d17/d18/d3e/dd1/c106 0 2024-04-25T15:49:49.544 INFO:tasks.workunit.client.1.smithi193.stdout:3/790: dread d8/fb4 [0,4194304] 0 2024-04-25T15:49:49.544 INFO:tasks.workunit.client.1.smithi193.stdout:2/756: truncate d0/d37/fb6 1458041 0 2024-04-25T15:49:49.544 INFO:tasks.workunit.client.1.smithi193.stdout:8/769: rename d2/d3/d27/d69/dae/d4a/d53/c74 to d2/d3/d27/cfd 0 2024-04-25T15:49:49.546 INFO:tasks.workunit.client.1.smithi193.stdout:0/773: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf/f107 x:0 0 0 2024-04-25T15:49:49.546 INFO:tasks.workunit.client.1.smithi193.stdout:4/818: link db/d4b/d51/d54/c7c db/d4b/dea/c107 0 2024-04-25T15:49:49.548 INFO:tasks.workunit.client.1.smithi193.stdout:6/745: getdents dd/d12/d19 0 2024-04-25T15:49:49.549 INFO:tasks.workunit.client.1.smithi193.stdout:2/757: symlink d0/d42/d51/df1/d9f/dc5/l11c 0 2024-04-25T15:49:49.549 INFO:tasks.workunit.client.1.smithi193.stdout:8/770: symlink d2/d3/d27/d69/dae/dd8/lfe 0 2024-04-25T15:49:49.550 INFO:tasks.workunit.client.1.smithi193.stdout:5/777: creat d17/d18/d30/d38/d48/d4f/deb/f107 x:0 0 0 2024-04-25T15:49:49.550 INFO:tasks.workunit.client.1.smithi193.stdout:8/771: readlink d2/d3/d20/d8c/dde/d94/l51 0 2024-04-25T15:49:49.552 INFO:tasks.workunit.client.1.smithi193.stdout:4/819: creat db/d13/d1e/d1f/d56/d57/f108 x:0 0 0 2024-04-25T15:49:49.554 INFO:tasks.workunit.client.1.smithi193.stdout:6/746: truncate dd/f3d 10110115 0 2024-04-25T15:49:49.557 INFO:tasks.workunit.client.1.smithi193.stdout:4/820: fdatasync db/d13/d1e/d1f/fb0 0 2024-04-25T15:49:49.557 INFO:tasks.workunit.client.1.smithi193.stdout:6/747: write dd/d12/d19/d1c/d40/fcb [591883,39617] 0 2024-04-25T15:49:49.560 INFO:tasks.workunit.client.1.smithi193.stdout:5/778: dwrite d17/d18/d75/d85/d5d/dad/f104 [0,4194304] 0 2024-04-25T15:49:49.561 INFO:tasks.workunit.client.1.smithi193.stdout:6/748: chown dd/d12/d19/d1c/c62 1736517 1 2024-04-25T15:49:49.561 INFO:tasks.workunit.client.1.smithi193.stdout:8/772: dwrite d2/d3/d20/db8/dd0/fe3 [0,4194304] 0 2024-04-25T15:49:49.564 INFO:tasks.workunit.client.1.smithi193.stdout:3/791: write d8/d13/d4b/f88 [2512126,56272] 0 2024-04-25T15:49:49.565 INFO:tasks.workunit.client.1.smithi193.stdout:0/774: truncate d0/d49/d5/d7/dd/f58 5246301 0 2024-04-25T15:49:49.570 INFO:tasks.workunit.client.1.smithi193.stdout:0/775: stat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/fe7 0 2024-04-25T15:49:49.570 INFO:tasks.workunit.client.1.smithi193.stdout:8/773: dread d2/d3/d20/f8b [0,4194304] 0 2024-04-25T15:49:49.571 INFO:tasks.workunit.client.1.smithi193.stdout:3/792: getdents d8/d30/d72 0 2024-04-25T15:49:49.571 INFO:tasks.workunit.client.1.smithi193.stdout:6/749: dwrite dd/d12/d19/d45/fc7 [0,4194304] 0 2024-04-25T15:49:49.573 INFO:tasks.workunit.client.1.smithi193.stdout:8/774: symlink d2/da5/lff 0 2024-04-25T15:49:49.573 INFO:tasks.workunit.client.1.smithi193.stdout:6/750: stat dd/d12/d19/d1c/d40/d3c/d3f/ccf 0 2024-04-25T15:49:49.573 INFO:tasks.workunit.client.1.smithi193.stdout:8/775: dread - d2/da5/ff0 zero size 2024-04-25T15:49:49.574 INFO:tasks.workunit.client.1.smithi193.stdout:0/776: symlink d0/d49/d5/d7/dd/d78/l108 0 2024-04-25T15:49:49.576 INFO:tasks.workunit.client.1.smithi193.stdout:6/751: chown dd/d12/d19/d1c/d40/d3c/d3f/f61 16611710 1 2024-04-25T15:49:49.577 INFO:tasks.workunit.client.1.smithi193.stdout:8/776: mkdir d2/d3/d27/d69/dae/d4a/dc1/d100 0 2024-04-25T15:49:49.579 INFO:tasks.workunit.client.1.smithi193.stdout:3/793: dwrite d8/d13/d1f/d69/d77/f50 [4194304,4194304] 0 2024-04-25T15:49:49.580 INFO:tasks.workunit.client.1.smithi193.stdout:3/794: chown d8/l2c 13007 1 2024-04-25T15:49:49.581 INFO:tasks.workunit.client.1.smithi193.stdout:0/777: rename d0/d49/d5/d7/dff/l87 to d0/d49/d5/d7/dd/d16/d26/d5e/d99/deb/l109 0 2024-04-25T15:49:49.581 INFO:tasks.workunit.client.1.smithi193.stdout:6/752: mkdir dd/d12/d19/d1c/d40/d3c/d3f/de0 0 2024-04-25T15:49:49.586 INFO:tasks.workunit.client.1.smithi193.stdout:0/778: unlink d0/d49/d5/d7/dd/d16/d26/df5/f4c 0 2024-04-25T15:49:49.590 INFO:tasks.workunit.client.1.smithi193.stdout:8/777: dwrite d2/d3/d20/db8/dd0/ff2 [0,4194304] 0 2024-04-25T15:49:49.595 INFO:tasks.workunit.client.1.smithi193.stdout:6/753: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0/fe1 x:0 0 0 2024-04-25T15:49:49.598 INFO:tasks.workunit.client.1.smithi193.stdout:6/754: read - dd/d12/d19/db2/fc2 zero size 2024-04-25T15:49:49.600 INFO:tasks.workunit.client.1.smithi193.stdout:3/795: dwrite d8/d13/d1f/d9f/fdf [0,4194304] 0 2024-04-25T15:49:49.601 INFO:tasks.workunit.client.1.smithi193.stdout:6/755: chown dd/d2f/d57/f7d 83234223 1 2024-04-25T15:49:49.603 INFO:tasks.workunit.client.1.smithi193.stdout:8/778: rename d2/caf to d2/d3/d27/d69/dae/d29/c101 0 2024-04-25T15:49:49.606 INFO:tasks.workunit.client.1.smithi193.stdout:8/779: fdatasync d2/d3/d27/d69/dae/d4a/d63/da2/fb9 0 2024-04-25T15:49:49.607 INFO:tasks.workunit.client.1.smithi193.stdout:0/779: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/f10a x:0 0 0 2024-04-25T15:49:49.608 INFO:tasks.workunit.client.1.smithi193.stdout:4/821: dwrite db/d4b/d51/d54/d77/faa [0,4194304] 0 2024-04-25T15:49:49.610 INFO:tasks.workunit.client.1.smithi193.stdout:6/756: mkdir dd/d12/d19/d1c/d40/dce/de2 0 2024-04-25T15:49:49.611 INFO:tasks.workunit.client.1.smithi193.stdout:4/822: chown db/d32 248188031 1 2024-04-25T15:49:49.611 INFO:tasks.workunit.client.1.smithi193.stdout:8/780: rmdir d2/d3/d27/d69/dae 39 2024-04-25T15:49:49.617 INFO:tasks.workunit.client.1.smithi193.stdout:7/779: sync 2024-04-25T15:49:49.618 INFO:tasks.workunit.client.1.smithi193.stdout:5/779: dwrite d17/d18/d3e/f3f [0,4194304] 0 2024-04-25T15:49:49.620 INFO:tasks.workunit.client.1.smithi193.stdout:6/757: truncate dd/d12/d19/d1c/d40/d3c/d3f/f61 4777280 0 2024-04-25T15:49:49.621 INFO:tasks.workunit.client.1.smithi193.stdout:5/780: dread - d17/d18/f102 zero size 2024-04-25T15:49:49.626 INFO:tasks.workunit.client.1.smithi193.stdout:6/758: symlink dd/db5/dc9/le3 0 2024-04-25T15:49:49.627 INFO:tasks.workunit.client.1.smithi193.stdout:5/781: symlink d17/d18/d30/d34/d52/daf/l108 0 2024-04-25T15:49:49.628 INFO:tasks.workunit.client.1.smithi193.stdout:4/823: dwrite db/d4b/d51/d54/d77/ff5 [0,4194304] 0 2024-04-25T15:49:49.629 INFO:tasks.workunit.client.1.smithi193.stdout:8/781: dwrite d2/d3/d20/db8/dd0/ff3 [0,4194304] 0 2024-04-25T15:49:49.634 INFO:tasks.workunit.client.1.smithi193.stdout:4/824: creat db/d13/d9d/f109 x:0 0 0 2024-04-25T15:49:49.643 INFO:tasks.workunit.client.1.smithi193.stdout:8/782: dwrite d2/d3/d27/d69/dae/dd8/ff4 [0,4194304] 0 2024-04-25T15:49:49.644 INFO:tasks.workunit.client.1.smithi193.stdout:0/780: write d0/d49/d5/d7/dd/d46/d48/d62/dd9/ff8 [191934,23239] 0 2024-04-25T15:49:49.646 INFO:tasks.workunit.client.1.smithi193.stdout:5/782: write d17/d18/d75/f9c [2768640,27246] 0 2024-04-25T15:49:49.649 INFO:tasks.workunit.client.1.smithi193.stdout:5/783: chown f11 137342 1 2024-04-25T15:49:49.649 INFO:tasks.workunit.client.1.smithi193.stdout:6/759: dread dd/d12/d19/d1c/d40/d3c/d3f/f83 [0,4194304] 0 2024-04-25T15:49:49.650 INFO:tasks.workunit.client.1.smithi193.stdout:3/796: dread d8/d13/d4b/f57 [0,4194304] 0 2024-04-25T15:49:49.652 INFO:tasks.workunit.client.1.smithi193.stdout:6/760: stat dd/d2f/fa1 0 2024-04-25T15:49:49.653 INFO:tasks.workunit.client.1.smithi193.stdout:1/687: sync 2024-04-25T15:49:49.654 INFO:tasks.workunit.client.1.smithi193.stdout:8/783: fsync d2/f11 0 2024-04-25T15:49:49.654 INFO:tasks.workunit.client.1.smithi193.stdout:0/781: read - d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf/f107 zero size 2024-04-25T15:49:49.654 INFO:tasks.workunit.client.1.smithi193.stdout:3/797: fdatasync d8/fc 0 2024-04-25T15:49:49.656 INFO:tasks.workunit.client.1.smithi193.stdout:1/688: write dc/d59/f8e [2182907,101508] 0 2024-04-25T15:49:49.658 INFO:tasks.workunit.client.1.smithi193.stdout:5/784: mknod d17/d18/d30/d38/dab/c109 0 2024-04-25T15:49:49.660 INFO:tasks.workunit.client.1.smithi193.stdout:4/825: dwrite db/d13/d1e/d1f/f28 [0,4194304] 0 2024-04-25T15:49:49.661 INFO:tasks.workunit.client.1.smithi193.stdout:1/689: write dc/d4a/d50/d7c/fca [1157172,15416] 0 2024-04-25T15:49:49.662 INFO:tasks.workunit.client.1.smithi193.stdout:6/761: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/fe4 x:0 0 0 2024-04-25T15:49:49.664 INFO:tasks.workunit.client.1.smithi193.stdout:0/782: creat d0/d49/d5/d7/dd/d46/d48/d62/f10b x:0 0 0 2024-04-25T15:49:49.668 INFO:tasks.workunit.client.1.smithi193.stdout:5/785: creat d17/f10a x:0 0 0 2024-04-25T15:49:49.668 INFO:tasks.workunit.client.1.smithi193.stdout:0/783: readlink d0/d49/d5/d7/dd/d46/d48/l75 0 2024-04-25T15:49:49.669 INFO:tasks.workunit.client.1.smithi193.stdout:6/762: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dd7/le5 0 2024-04-25T15:49:49.669 INFO:tasks.workunit.client.1.smithi193.stdout:7/780: dread d12/f36 [0,4194304] 0 2024-04-25T15:49:49.671 INFO:tasks.workunit.client.1.smithi193.stdout:4/826: link db/d13/d1e/d1f/l52 db/d13/ddd/l10a 0 2024-04-25T15:49:49.674 INFO:tasks.workunit.client.1.smithi193.stdout:4/827: write db/fd0 [1306672,5349] 0 2024-04-25T15:49:49.675 INFO:tasks.workunit.client.1.smithi193.stdout:8/784: write d2/d3/d27/d69/dae/d4a/d63/da2/fa4 [1070846,79567] 0 2024-04-25T15:49:49.675 INFO:tasks.workunit.client.1.smithi193.stdout:7/781: rmdir d12/d13/dcc 39 2024-04-25T15:49:49.677 INFO:tasks.workunit.client.1.smithi193.stdout:7/782: chown d12/d1c/f8a 109640099 1 2024-04-25T15:49:49.677 INFO:tasks.workunit.client.1.smithi193.stdout:4/828: mkdir db/d32/d40/d10b 0 2024-04-25T15:49:49.678 INFO:tasks.workunit.client.1.smithi193.stdout:5/786: dwrite d17/d18/d30/d38/d48/d4f/deb/f107 [0,4194304] 0 2024-04-25T15:49:49.679 INFO:tasks.workunit.client.1.smithi193.stdout:4/829: chown db/f38 17775 1 2024-04-25T15:49:49.680 INFO:tasks.workunit.client.1.smithi193.stdout:7/783: mkdir d12/d31/d37/d62/d11f 0 2024-04-25T15:49:49.682 INFO:tasks.workunit.client.1.smithi193.stdout:6/763: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/f7b [977277,38482] 0 2024-04-25T15:49:49.682 INFO:tasks.workunit.client.1.smithi193.stdout:1/690: dread dc/d4a/d50/d5c/d5d/d3d/d55/f73 [0,4194304] 0 2024-04-25T15:49:49.683 INFO:tasks.workunit.client.1.smithi193.stdout:4/830: fsync db/d13/d1e/d1f/fb3 0 2024-04-25T15:49:49.684 INFO:tasks.workunit.client.1.smithi193.stdout:5/787: symlink d17/d18/d30/d38/d48/d4f/l10b 0 2024-04-25T15:49:49.688 INFO:tasks.workunit.client.1.smithi193.stdout:1/691: creat dc/d4a/fe8 x:0 0 0 2024-04-25T15:49:49.689 INFO:tasks.workunit.client.1.smithi193.stdout:7/784: symlink d12/d31/d37/d104/d11a/l120 0 2024-04-25T15:49:49.689 INFO:tasks.workunit.client.1.smithi193.stdout:8/785: dwrite d2/d3/d20/fe8 [0,4194304] 0 2024-04-25T15:49:49.690 INFO:tasks.workunit.client.1.smithi193.stdout:4/831: creat db/d13/d1e/d1f/f10c x:0 0 0 2024-04-25T15:49:49.691 INFO:tasks.workunit.client.1.smithi193.stdout:0/784: dwrite d0/f17 [0,4194304] 0 2024-04-25T15:49:49.691 INFO:tasks.workunit.client.1.smithi193.stdout:5/788: truncate d17/d83/fba 88171 0 2024-04-25T15:49:49.692 INFO:tasks.workunit.client.1.smithi193.stdout:6/764: creat dd/d2f/fe6 x:0 0 0 2024-04-25T15:49:49.694 INFO:tasks.workunit.client.1.smithi193.stdout:4/832: stat db/d13/d9d/ca6 0 2024-04-25T15:49:49.695 INFO:tasks.workunit.client.1.smithi193.stdout:7/785: chown d12/d13/dcc/lfe 2339081 1 2024-04-25T15:49:49.697 INFO:tasks.workunit.client.1.smithi193.stdout:1/692: rename dc/l24 to dc/d4a/d50/le9 0 2024-04-25T15:49:49.699 INFO:tasks.workunit.client.1.smithi193.stdout:5/789: dwrite d17/f97 [0,4194304] 0 2024-04-25T15:49:49.700 INFO:tasks.workunit.client.1.smithi193.stdout:5/790: stat d17/d18/d30/lf3 0 2024-04-25T15:49:49.701 INFO:tasks.workunit.client.1.smithi193.stdout:6/765: truncate dd/f31 811812 0 2024-04-25T15:49:49.702 INFO:tasks.workunit.client.1.smithi193.stdout:4/833: creat db/d13/d1e/d1f/d56/d69/f10d x:0 0 0 2024-04-25T15:49:49.702 INFO:tasks.workunit.client.1.smithi193.stdout:7/786: symlink d12/d31/d37/d92/dd1/l121 0 2024-04-25T15:49:49.705 INFO:tasks.workunit.client.1.smithi193.stdout:0/785: mkdir d0/d49/d5/d7/dd/d46/d48/d10c 0 2024-04-25T15:49:49.707 INFO:tasks.workunit.client.1.smithi193.stdout:4/834: mknod db/d13/d1e/d1f/d56/d69/d70/c10e 0 2024-04-25T15:49:49.707 INFO:tasks.workunit.client.1.smithi193.stdout:5/791: mkdir d17/d18/d75/d85/d10c 0 2024-04-25T15:49:49.708 INFO:tasks.workunit.client.1.smithi193.stdout:6/766: symlink dd/d12/d19/d1c/d40/dda/le7 0 2024-04-25T15:49:49.712 INFO:tasks.workunit.client.1.smithi193.stdout:8/786: dwrite d2/d3/f16 [0,4194304] 0 2024-04-25T15:49:49.712 INFO:tasks.workunit.client.1.smithi193.stdout:6/767: dread - dd/d12/d19/d1c/d40/d3c/fd2 zero size 2024-04-25T15:49:49.713 INFO:tasks.workunit.client.1.smithi193.stdout:8/787: chown d2/d3/d27/l8d 1 1 2024-04-25T15:49:49.715 INFO:tasks.workunit.client.1.smithi193.stdout:0/786: rename d0/d49/d5/d7/dd/d16/d26/df5/cf4 to d0/d49/d5/d7/dd/d16/d18/d28/c10d 0 2024-04-25T15:49:49.717 INFO:tasks.workunit.client.1.smithi193.stdout:4/835: link db/d32/d45/f60 db/d32/d88/f10f 0 2024-04-25T15:49:49.717 INFO:tasks.workunit.client.1.smithi193.stdout:0/787: fsync d0/d49/d5/d7/dd/d46/d48/d62/f92 0 2024-04-25T15:49:49.719 INFO:tasks.workunit.client.1.smithi193.stdout:6/768: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/de8 0 2024-04-25T15:49:49.719 INFO:tasks.workunit.client.1.smithi193.stdout:4/836: dread - db/d32/d40/fe4 zero size 2024-04-25T15:49:49.719 INFO:tasks.workunit.client.1.smithi193.stdout:5/792: dwrite d17/d18/d75/f9c [0,4194304] 0 2024-04-25T15:49:49.721 INFO:tasks.workunit.client.1.smithi193.stdout:8/788: mknod d2/d3/d27/d69/dae/d4a/d63/da2/c102 0 2024-04-25T15:49:49.722 INFO:tasks.workunit.client.1.smithi193.stdout:4/837: mkdir db/d84/d110 0 2024-04-25T15:49:49.722 INFO:tasks.workunit.client.1.smithi193.stdout:8/789: read - d2/d18/ddb/feb zero size 2024-04-25T15:49:49.723 INFO:tasks.workunit.client.1.smithi193.stdout:5/793: chown d17/d53/lb4 180 1 2024-04-25T15:49:49.729 INFO:tasks.workunit.client.1.smithi193.stdout:6/769: rename dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/fe4 to dd/d12/d19/d1c/d40/dda/fe9 0 2024-04-25T15:49:49.731 INFO:tasks.workunit.client.1.smithi193.stdout:0/788: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/d93/fea [0,4194304] 0 2024-04-25T15:49:49.732 INFO:tasks.workunit.client.1.smithi193.stdout:6/770: mkdir dd/dea 0 2024-04-25T15:49:49.733 INFO:tasks.workunit.client.1.smithi193.stdout:8/790: dwrite d2/d3/d3a/f7c [0,4194304] 0 2024-04-25T15:49:49.736 INFO:tasks.workunit.client.1.smithi193.stdout:8/791: getdents d2/d3/d27/d69/dae/d4a/dc1/d100 0 2024-04-25T15:49:49.739 INFO:tasks.workunit.client.1.smithi193.stdout:6/771: dwrite dd/d2f/fe6 [0,4194304] 0 2024-04-25T15:49:49.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:49 smithi193 ceph-mon[33747]: pgmap v41: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 111 MiB/s rd, 273 MiB/s wr, 559 op/s 2024-04-25T15:49:49.740 INFO:tasks.workunit.client.1.smithi193.stdout:6/772: write dd/d12/d19/d1c/d40/d3c/d3f/f54 [4973797,4093] 0 2024-04-25T15:49:49.758 INFO:tasks.workunit.client.1.smithi193.stdout:6/773: readlink dd/d12/d19/d1c/d40/d3c/d3f/l77 0 2024-04-25T15:49:49.761 INFO:tasks.workunit.client.1.smithi193.stdout:6/774: read dd/f1b [2445799,24421] 0 2024-04-25T15:49:49.761 INFO:tasks.workunit.client.1.smithi193.stdout:8/792: creat d2/d3/d27/d69/dae/f103 x:0 0 0 2024-04-25T15:49:49.768 INFO:tasks.workunit.client.1.smithi193.stdout:9/755: sync 2024-04-25T15:49:49.768 INFO:tasks.workunit.client.1.smithi193.stdout:2/758: sync 2024-04-25T15:49:49.769 INFO:tasks.workunit.client.1.smithi193.stdout:9/756: chown df/l10 823514775 1 2024-04-25T15:49:49.769 INFO:tasks.workunit.client.1.smithi193.stdout:2/759: read - d0/d9/d15/d31/fd9 zero size 2024-04-25T15:49:49.775 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:49 smithi110 ceph-mon[26344]: pgmap v41: 65 pgs: 65 active+clean; 3.4 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 111 MiB/s rd, 273 MiB/s wr, 559 op/s 2024-04-25T15:49:49.775 INFO:tasks.workunit.client.1.smithi193.stdout:4/838: dread db/d32/f6e [0,4194304] 0 2024-04-25T15:49:49.778 INFO:tasks.workunit.client.1.smithi193.stdout:5/794: dread d17/d18/f21 [8388608,4194304] 0 2024-04-25T15:49:49.781 INFO:tasks.workunit.client.1.smithi193.stdout:9/757: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/f40 [0,4194304] 0 2024-04-25T15:49:49.781 INFO:tasks.workunit.client.1.smithi193.stdout:0/789: dread d0/d49/d5/d7/dd/d16/d18/d28/f3e [0,4194304] 0 2024-04-25T15:49:49.783 INFO:tasks.workunit.client.1.smithi193.stdout:0/790: write d0/d49/d5/d61/dd4/f106 [960414,49490] 0 2024-04-25T15:49:49.842 INFO:tasks.workunit.client.1.smithi193.stdout:6/775: rename dd/d12/d19/d1c/d40/d3c/d3f/d4f/l81 to dd/d2f/d57/leb 0 2024-04-25T15:49:49.843 INFO:tasks.workunit.client.1.smithi193.stdout:2/760: symlink d0/d37/dbb/df8/l11d 0 2024-04-25T15:49:49.846 INFO:tasks.workunit.client.1.smithi193.stdout:4/839: mknod db/d32/de8/c111 0 2024-04-25T15:49:49.847 INFO:tasks.workunit.client.1.smithi193.stdout:5/795: creat d17/d18/d30/d38/d48/d4f/f10d x:0 0 0 2024-04-25T15:49:49.849 INFO:tasks.workunit.client.1.smithi193.stdout:0/791: creat d0/d49/d6a/f10e x:0 0 0 2024-04-25T15:49:49.870 INFO:tasks.workunit.client.1.smithi193.stdout:3/798: sync 2024-04-25T15:49:49.881 INFO:tasks.workunit.client.1.smithi193.stdout:8/793: mknod d2/d3/d27/d69/c104 0 2024-04-25T15:49:49.882 INFO:tasks.workunit.client.1.smithi193.stdout:8/794: readlink d2/d3/d20/d8c/dde/l7e 0 2024-04-25T15:49:49.883 INFO:tasks.workunit.client.1.smithi193.stdout:2/761: read d0/d9/d3d/d7d/fab [571622,54607] 0 2024-04-25T15:49:49.883 INFO:tasks.workunit.client.1.smithi193.stdout:4/840: creat db/d13/d1e/d1f/f112 x:0 0 0 2024-04-25T15:49:49.891 INFO:tasks.workunit.client.1.smithi193.stdout:2/762: mknod d0/d37/dbb/df8/c11e 0 2024-04-25T15:49:49.894 INFO:tasks.workunit.client.1.smithi193.stdout:0/792: truncate d0/d49/d5/f2d 1741867 0 2024-04-25T15:49:49.895 INFO:tasks.workunit.client.1.smithi193.stdout:6/776: dread dd/d12/f5d [0,4194304] 0 2024-04-25T15:49:49.895 INFO:tasks.workunit.client.1.smithi193.stdout:3/799: fsync d8/d13/d1f/f22 0 2024-04-25T15:49:49.896 INFO:tasks.workunit.client.1.smithi193.stdout:9/758: link ca df/d2c/d34/ce8 0 2024-04-25T15:49:49.897 INFO:tasks.workunit.client.1.smithi193.stdout:6/777: chown dd/d2f/f56 420 1 2024-04-25T15:49:49.897 INFO:tasks.workunit.client.1.smithi193.stdout:2/763: dwrite d0/d37/d25/d5f/d3a/d6d/d6e/f3e [0,4194304] 0 2024-04-25T15:49:49.898 INFO:tasks.workunit.client.1.smithi193.stdout:4/841: getdents db/d84/df3/d100 0 2024-04-25T15:49:49.900 INFO:tasks.workunit.client.0.smithi110.stdout:3/938: sync 2024-04-25T15:49:49.902 INFO:tasks.workunit.client.1.smithi193.stdout:4/842: chown db/d13/d1e/d1f/d22/l31 370944941 1 2024-04-25T15:49:49.902 INFO:tasks.workunit.client.1.smithi193.stdout:0/793: dread d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f3f [0,4194304] 0 2024-04-25T15:49:49.904 INFO:tasks.workunit.client.1.smithi193.stdout:2/764: rename d0/d9/f39 to d0/df/da5/f11f 0 2024-04-25T15:49:49.904 INFO:tasks.workunit.client.1.smithi193.stdout:1/693: sync 2024-04-25T15:49:49.904 INFO:tasks.workunit.client.1.smithi193.stdout:7/787: sync 2024-04-25T15:49:49.905 INFO:tasks.workunit.client.0.smithi110.stdout:3/939: dread - d0/d2e/d88/dbf/d99/dad/fe9 zero size 2024-04-25T15:49:49.905 INFO:tasks.workunit.client.1.smithi193.stdout:1/694: fdatasync dc/d4a/d50/fa7 0 2024-04-25T15:49:49.906 INFO:tasks.workunit.client.1.smithi193.stdout:0/794: dread - d0/d49/d5/d7/dd/d16/d26/d2e/d3d/ffe zero size 2024-04-25T15:49:49.907 INFO:tasks.workunit.client.1.smithi193.stdout:6/778: dread dd/d12/d19/d1c/d40/d3c/d3f/f73 [0,4194304] 0 2024-04-25T15:49:49.908 INFO:tasks.workunit.client.1.smithi193.stdout:0/795: stat d0/d49/d5/d7/dd/d16/d26/d2e/f34 0 2024-04-25T15:49:49.908 INFO:tasks.workunit.client.1.smithi193.stdout:4/843: rename db/d4b/d51 to db/d13/dcb/d113 0 2024-04-25T15:49:49.909 INFO:tasks.workunit.client.1.smithi193.stdout:6/779: chown dd/d12/d19/d45/c98 554625 1 2024-04-25T15:49:49.909 INFO:tasks.workunit.client.1.smithi193.stdout:0/796: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60 449067626 1 2024-04-25T15:49:49.910 INFO:tasks.workunit.client.0.smithi110.stdout:3/940: mkdir d0/d2e/d88/dbf/d99/d13d 0 2024-04-25T15:49:49.910 INFO:tasks.workunit.client.0.smithi110.stdout:3/941: readlink d0/lab 0 2024-04-25T15:49:49.912 INFO:tasks.workunit.client.1.smithi193.stdout:7/788: creat d12/d31/d37/d62/d6e/da5/f122 x:0 0 0 2024-04-25T15:49:49.917 INFO:tasks.workunit.client.0.smithi110.stdout:3/942: link d0/d2e/c5b d0/d2e/d88/dbf/d99/d13d/c13e 0 2024-04-25T15:49:49.919 INFO:tasks.workunit.client.1.smithi193.stdout:7/789: fsync d12/d1c/d63/d70/f42 0 2024-04-25T15:49:49.920 INFO:tasks.workunit.client.1.smithi193.stdout:1/695: link dc/d4a/d50/d7c/d7e/f88 dc/d4a/d50/d7c/d82/ddc/fea 0 2024-04-25T15:49:49.920 INFO:tasks.workunit.client.1.smithi193.stdout:3/800: dread d8/d4d/f62 [0,4194304] 0 2024-04-25T15:49:49.922 INFO:tasks.workunit.client.1.smithi193.stdout:4/844: creat db/d32/d40/dee/f114 x:0 0 0 2024-04-25T15:49:49.923 INFO:tasks.workunit.client.1.smithi193.stdout:0/797: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/de0/f10f x:0 0 0 2024-04-25T15:49:49.924 INFO:tasks.workunit.client.1.smithi193.stdout:4/845: chown db/d13/dcb/d113/d54/d77/l80 2755090 1 2024-04-25T15:49:49.924 INFO:tasks.workunit.client.0.smithi110.stdout:3/943: creat d0/d16/dcb/f13f x:0 0 0 2024-04-25T15:49:49.926 INFO:tasks.workunit.client.0.smithi110.stdout:3/944: chown d0/d16/dcb/d2f/d4c/db2/lcd 36046 1 2024-04-25T15:49:49.927 INFO:tasks.workunit.client.1.smithi193.stdout:6/780: dread dd/f42 [0,4194304] 0 2024-04-25T15:49:49.928 INFO:tasks.workunit.client.1.smithi193.stdout:1/696: dwrite dc/d4a/d50/d5c/d5d/d3d/f83 [0,4194304] 0 2024-04-25T15:49:49.932 INFO:tasks.workunit.client.0.smithi110.stdout:3/945: dwrite d0/d2e/d88/dbf/d99/dad/f13b [0,4194304] 0 2024-04-25T15:49:49.934 INFO:tasks.workunit.client.1.smithi193.stdout:4/846: rename db/d13/d1e/l92 to db/d32/d88/da8/l115 0 2024-04-25T15:49:49.935 INFO:tasks.workunit.client.1.smithi193.stdout:4/847: readlink db/d13/d1e/lfc 0 2024-04-25T15:49:49.940 INFO:tasks.workunit.client.1.smithi193.stdout:8/795: write d2/d3/ff [1572765,99062] 0 2024-04-25T15:49:49.940 INFO:tasks.workunit.client.1.smithi193.stdout:1/697: chown dc/d3c/cba 707462492 1 2024-04-25T15:49:49.940 INFO:tasks.workunit.client.1.smithi193.stdout:6/781: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/cec 0 2024-04-25T15:49:49.941 INFO:tasks.workunit.client.0.smithi110.stdout:3/946: stat d0/d7/d40/c5a 0 2024-04-25T15:49:49.941 INFO:tasks.workunit.client.1.smithi193.stdout:1/698: chown dc/dc9 124 1 2024-04-25T15:49:49.943 INFO:tasks.workunit.client.1.smithi193.stdout:4/848: creat db/d84/f116 x:0 0 0 2024-04-25T15:49:49.945 INFO:tasks.workunit.client.1.smithi193.stdout:4/849: chown db/d13/d1e/d1f/d56/fd1 1099400464 1 2024-04-25T15:49:49.947 INFO:tasks.workunit.client.1.smithi193.stdout:8/796: unlink d2/fab 0 2024-04-25T15:49:49.950 INFO:tasks.workunit.client.1.smithi193.stdout:5/796: dwrite d17/d18/d30/d38/d71/d96/dc4/f9a [0,4194304] 0 2024-04-25T15:49:49.954 INFO:tasks.workunit.client.1.smithi193.stdout:1/699: dwrite dc/d4a/fe8 [0,4194304] 0 2024-04-25T15:49:49.956 INFO:tasks.workunit.client.1.smithi193.stdout:6/782: symlink dd/d12/d19/d1c/d40/d3c/d3f/led 0 2024-04-25T15:49:49.956 INFO:tasks.workunit.client.1.smithi193.stdout:9/759: dwrite df/d2c/d2f/d4e/f37 [0,4194304] 0 2024-04-25T15:49:49.956 INFO:tasks.workunit.client.1.smithi193.stdout:1/700: truncate dc/d3c/fe6 698925 0 2024-04-25T15:49:49.957 INFO:tasks.workunit.client.1.smithi193.stdout:8/797: chown d2/d3/d3a/f76 2792 1 2024-04-25T15:49:49.958 INFO:tasks.workunit.client.1.smithi193.stdout:9/760: stat df/d2c/d2f/d4e/fc2 0 2024-04-25T15:49:49.964 INFO:tasks.workunit.client.1.smithi193.stdout:5/797: read d17/f1f [1270744,6382] 0 2024-04-25T15:49:49.965 INFO:tasks.workunit.client.1.smithi193.stdout:6/783: fsync dd/d12/f1e 0 2024-04-25T15:49:49.967 INFO:tasks.workunit.client.1.smithi193.stdout:0/798: rmdir d0/d49 39 2024-04-25T15:49:49.967 INFO:tasks.workunit.client.1.smithi193.stdout:8/798: chown d2/d3/d3a/fda 0 1 2024-04-25T15:49:49.970 INFO:tasks.workunit.client.1.smithi193.stdout:2/765: creat d0/d9/d15/df3/f120 x:0 0 0 2024-04-25T15:49:49.971 INFO:tasks.workunit.client.1.smithi193.stdout:3/801: write d8/d13/d1f/d69/dcd/dd9/dde/f91 [4172502,16340] 0 2024-04-25T15:49:49.971 INFO:tasks.workunit.client.1.smithi193.stdout:7/790: sync 2024-04-25T15:49:49.973 INFO:tasks.workunit.client.1.smithi193.stdout:6/784: creat dd/d12/d19/d45/fee x:0 0 0 2024-04-25T15:49:49.975 INFO:tasks.workunit.client.1.smithi193.stdout:8/799: mkdir d2/d3/d27/d69/dbf/d105 0 2024-04-25T15:49:49.976 INFO:tasks.workunit.client.1.smithi193.stdout:9/761: rename df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d8b/l80 to df/d2c/d2f/d4e/d1c/d30/d3b/le9 0 2024-04-25T15:49:49.977 INFO:tasks.workunit.client.1.smithi193.stdout:5/798: mknod d17/d18/d3e/dd1/ddc/c10e 0 2024-04-25T15:49:49.979 INFO:tasks.workunit.client.1.smithi193.stdout:4/850: rmdir db 39 2024-04-25T15:49:49.979 INFO:tasks.workunit.client.1.smithi193.stdout:9/762: dread - df/d2c/d2f/d4e/fc2 zero size 2024-04-25T15:49:49.979 INFO:tasks.workunit.client.1.smithi193.stdout:2/766: dwrite d0/df/dbf/fd5 [0,4194304] 0 2024-04-25T15:49:49.981 INFO:tasks.workunit.client.1.smithi193.stdout:5/799: fsync d17/d18/d75/f9c 0 2024-04-25T15:49:49.981 INFO:tasks.workunit.client.1.smithi193.stdout:3/802: symlink d8/d30/d45/dd0/lf8 0 2024-04-25T15:49:49.983 INFO:tasks.workunit.client.0.smithi110.stdout:9/864: sync 2024-04-25T15:49:49.983 INFO:tasks.workunit.client.0.smithi110.stdout:5/942: sync 2024-04-25T15:49:49.983 INFO:tasks.workunit.client.0.smithi110.stdout:2/862: sync 2024-04-25T15:49:49.983 INFO:tasks.workunit.client.0.smithi110.stdout:6/897: sync 2024-04-25T15:49:49.983 INFO:tasks.workunit.client.0.smithi110.stdout:4/875: sync 2024-04-25T15:49:49.983 INFO:tasks.workunit.client.0.smithi110.stdout:0/886: sync 2024-04-25T15:49:49.985 INFO:tasks.workunit.client.0.smithi110.stdout:3/947: dwrite d0/f101 [0,4194304] 0 2024-04-25T15:49:49.986 INFO:tasks.workunit.client.1.smithi193.stdout:1/701: dread f6 [0,4194304] 0 2024-04-25T15:49:49.987 INFO:tasks.workunit.client.1.smithi193.stdout:6/785: mkdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0/def 0 2024-04-25T15:49:49.987 INFO:tasks.workunit.client.1.smithi193.stdout:8/800: dread d2/d3/d20/fe8 [0,4194304] 0 2024-04-25T15:49:49.988 INFO:tasks.workunit.client.0.smithi110.stdout:0/887: dread - d1/d23/d30/d33/f121 zero size 2024-04-25T15:49:49.990 INFO:tasks.workunit.client.0.smithi110.stdout:9/865: rmdir d9/d13/d5b/d61 39 2024-04-25T15:49:49.990 INFO:tasks.workunit.client.0.smithi110.stdout:0/888: chown d1/d23/d30/d33/d5b/d78/da4/fba 160797 1 2024-04-25T15:49:49.991 INFO:tasks.workunit.client.1.smithi193.stdout:7/791: mknod d12/d31/d39/c123 0 2024-04-25T15:49:49.992 INFO:tasks.workunit.client.1.smithi193.stdout:9/763: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/f2a 11098887 1 2024-04-25T15:49:49.992 INFO:tasks.workunit.client.0.smithi110.stdout:4/876: creat d13/d24/d2d/d5e/f12d x:0 0 0 2024-04-25T15:49:49.994 INFO:tasks.workunit.client.0.smithi110.stdout:2/863: creat de/d40/d70/f115 x:0 0 0 2024-04-25T15:49:49.995 INFO:tasks.workunit.client.0.smithi110.stdout:2/864: read de/d40/d6e/feb [374424,106093] 0 2024-04-25T15:49:49.997 INFO:tasks.workunit.client.1.smithi193.stdout:5/800: creat d17/d18/d30/d38/d71/d96/dc4/da7/f10f x:0 0 0 2024-04-25T15:49:49.997 INFO:tasks.workunit.client.0.smithi110.stdout:6/898: dread - d1/db/d22/d33/d63/fa9 zero size 2024-04-25T15:49:49.998 INFO:tasks.workunit.client.1.smithi193.stdout:1/702: write dc/d4a/d50/d5c/d5d/d3d/d55/d8a/fc8 [43018,34741] 0 2024-04-25T15:49:49.998 INFO:tasks.workunit.client.0.smithi110.stdout:6/899: chown d1/db/d24/daa/cd7 156 1 2024-04-25T15:49:50.000 INFO:tasks.workunit.client.1.smithi193.stdout:1/703: stat dc/d4a 0 2024-04-25T15:49:50.000 INFO:tasks.workunit.client.1.smithi193.stdout:3/803: dwrite d8/d13/fc8 [0,4194304] 0 2024-04-25T15:49:50.001 INFO:tasks.workunit.client.1.smithi193.stdout:2/767: dwrite d0/d37/d25/d5f/d3a/d6d/d85/f99 [0,4194304] 0 2024-04-25T15:49:50.003 INFO:tasks.workunit.client.0.smithi110.stdout:3/948: mkdir d0/d16/dcb/d2f/d4c/d140 0 2024-04-25T15:49:50.008 INFO:tasks.workunit.client.1.smithi193.stdout:1/704: write dc/d4a/d50/d5c/d5d/d3d/d55/fe7 [100192,50562] 0 2024-04-25T15:49:50.015 INFO:tasks.workunit.client.0.smithi110.stdout:4/877: dread d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/d92/f9a [0,4194304] 0 2024-04-25T15:49:50.015 INFO:tasks.workunit.client.1.smithi193.stdout:4/851: mkdir db/d13/d1e/d1f/d117 0 2024-04-25T15:49:50.015 INFO:tasks.workunit.client.1.smithi193.stdout:7/792: rmdir d12/d13/dc4/dfa 39 2024-04-25T15:49:50.015 INFO:tasks.workunit.client.1.smithi193.stdout:7/793: chown d12/d31/d37/d92/dd1 0 1 2024-04-25T15:49:50.016 INFO:tasks.workunit.client.1.smithi193.stdout:3/804: dread d8/d13/d4b/f57 [0,4194304] 0 2024-04-25T15:49:50.016 INFO:tasks.workunit.client.1.smithi193.stdout:1/705: dread f6 [0,4194304] 0 2024-04-25T15:49:50.017 INFO:tasks.workunit.client.1.smithi193.stdout:6/786: write dd/f1b [2693528,128047] 0 2024-04-25T15:49:50.019 INFO:tasks.workunit.client.0.smithi110.stdout:6/900: creat d1/db/d22/d33/d63/dc2/dc7/f139 x:0 0 0 2024-04-25T15:49:50.020 INFO:tasks.workunit.client.1.smithi193.stdout:0/799: link d0/d49/l4a d0/d49/d5/d7/dd/d16/d18/l110 0 2024-04-25T15:49:50.021 INFO:tasks.workunit.client.0.smithi110.stdout:6/901: fdatasync d1/dd/f64 0 2024-04-25T15:49:50.021 INFO:tasks.workunit.client.0.smithi110.stdout:9/866: dread d9/f39 [0,4194304] 0 2024-04-25T15:49:50.022 INFO:tasks.workunit.client.0.smithi110.stdout:0/889: symlink d1/d23/d30/d33/d129/l143 0 2024-04-25T15:49:50.022 INFO:tasks.workunit.client.1.smithi193.stdout:9/764: dread df/d2c/d34/d3e/f6a [0,4194304] 0 2024-04-25T15:49:50.023 INFO:tasks.workunit.client.1.smithi193.stdout:3/805: dwrite d8/d13/f64 [0,4194304] 0 2024-04-25T15:49:50.024 INFO:tasks.workunit.client.0.smithi110.stdout:6/902: truncate d1/fdf 1747292 0 2024-04-25T15:49:50.024 INFO:tasks.workunit.client.0.smithi110.stdout:5/943: dwrite d4/d1d/d1f/d35/d3d/dc8/f115 [0,4194304] 0 2024-04-25T15:49:50.024 INFO:tasks.workunit.client.0.smithi110.stdout:4/878: mknod d13/d24/d2d/d36/d51/c12e 0 2024-04-25T15:49:50.026 INFO:tasks.workunit.client.1.smithi193.stdout:4/852: truncate db/d32/d40/f93 4284396 0 2024-04-25T15:49:50.030 INFO:tasks.workunit.client.1.smithi193.stdout:7/794: truncate d12/d13/f71 199629 0 2024-04-25T15:49:50.033 INFO:tasks.workunit.client.1.smithi193.stdout:1/706: rename dc/d4a/d50/d5c/d5d/d3d to dc/d59/dc0/deb 0 2024-04-25T15:49:50.035 INFO:tasks.workunit.client.1.smithi193.stdout:6/787: unlink dd/d12/d19/f51 0 2024-04-25T15:49:50.036 INFO:tasks.workunit.client.0.smithi110.stdout:9/867: truncate d9/d13/d7c/fb7 4147755 0 2024-04-25T15:49:50.037 INFO:tasks.workunit.client.1.smithi193.stdout:5/801: dwrite d17/d18/d30/d34/d52/daf/fb9 [0,4194304] 0 2024-04-25T15:49:50.037 INFO:tasks.workunit.client.0.smithi110.stdout:6/903: read d1/db/d22/d90/f2b [3010940,80061] 0 2024-04-25T15:49:50.039 INFO:tasks.workunit.client.1.smithi193.stdout:8/801: dread d2/d3/d20/d8c/da6/fe6 [0,4194304] 0 2024-04-25T15:49:50.043 INFO:tasks.workunit.client.0.smithi110.stdout:4/879: rename d13/d24/f128 to d13/d24/d2d/db5/f12f 0 2024-04-25T15:49:50.044 INFO:tasks.workunit.client.1.smithi193.stdout:2/768: truncate d0/d37/f67 2335406 0 2024-04-25T15:49:50.045 INFO:tasks.workunit.client.1.smithi193.stdout:2/769: dread - d0/d9/d15/d31/fd9 zero size 2024-04-25T15:49:50.046 INFO:tasks.workunit.client.0.smithi110.stdout:2/865: creat de/f116 x:0 0 0 2024-04-25T15:49:50.049 INFO:tasks.workunit.client.0.smithi110.stdout:0/890: mkdir d1/d23/d30/d33/d5b/dbb/d144 0 2024-04-25T15:49:50.051 INFO:tasks.workunit.client.1.smithi193.stdout:0/800: creat d0/d49/d5/d7/dd/d16/d26/d2e/f111 x:0 0 0 2024-04-25T15:49:50.055 INFO:tasks.workunit.client.1.smithi193.stdout:3/806: mkdir d8/d13/d1f/d9f/df9 0 2024-04-25T15:49:50.056 INFO:tasks.workunit.client.0.smithi110.stdout:5/944: write d4/d15/d51/d90/fa3 [656418,102791] 0 2024-04-25T15:49:50.057 INFO:tasks.workunit.client.1.smithi193.stdout:4/853: creat db/d84/f118 x:0 0 0 2024-04-25T15:49:50.058 INFO:tasks.workunit.client.1.smithi193.stdout:4/854: stat db/d13/d1e/d1f/d56/fe3 0 2024-04-25T15:49:50.059 INFO:tasks.workunit.client.0.smithi110.stdout:2/866: dread de/fa2 [0,4194304] 0 2024-04-25T15:49:50.059 INFO:tasks.workunit.client.1.smithi193.stdout:7/795: fdatasync d12/d31/d39/f93 0 2024-04-25T15:49:50.063 INFO:tasks.workunit.client.0.smithi110.stdout:4/880: fsync d13/d24/d2d/d55/faa 0 2024-04-25T15:49:50.066 INFO:tasks.workunit.client.1.smithi193.stdout:1/707: truncate dc/d4a/d4d/f5e 1018677 0 2024-04-25T15:49:50.068 INFO:tasks.workunit.client.1.smithi193.stdout:5/802: readlink d17/l7f 0 2024-04-25T15:49:50.070 INFO:tasks.workunit.client.0.smithi110.stdout:5/945: creat d4/d1d/d1f/d35/d3d/dc8/f134 x:0 0 0 2024-04-25T15:49:50.071 INFO:tasks.workunit.client.1.smithi193.stdout:8/802: mkdir d2/d3/d27/d69/dbf/d106 0 2024-04-25T15:49:50.071 INFO:tasks.workunit.client.1.smithi193.stdout:5/803: dread d17/d18/d75/d85/d5d/dad/f104 [0,4194304] 0 2024-04-25T15:49:50.072 INFO:tasks.workunit.client.0.smithi110.stdout:6/904: symlink d1/db/d22/d90/d76/dee/l13a 0 2024-04-25T15:49:50.075 INFO:tasks.workunit.client.1.smithi193.stdout:9/765: symlink df/d2c/lea 0 2024-04-25T15:49:50.076 INFO:tasks.workunit.client.0.smithi110.stdout:4/881: creat d13/d24/da6/f130 x:0 0 0 2024-04-25T15:49:50.077 INFO:tasks.workunit.client.1.smithi193.stdout:0/801: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d93/c112 0 2024-04-25T15:49:50.078 INFO:tasks.workunit.client.1.smithi193.stdout:3/807: symlink d8/d13/d4b/d8b/de3/lfa 0 2024-04-25T15:49:50.082 INFO:tasks.workunit.client.1.smithi193.stdout:7/796: truncate d12/d1c/d63/f65 727992 0 2024-04-25T15:49:50.082 INFO:tasks.workunit.client.0.smithi110.stdout:2/867: unlink de/d40/d5b/d5c/l75 0 2024-04-25T15:49:50.085 INFO:tasks.workunit.client.1.smithi193.stdout:6/788: stat dd/d12/d19/d1c/d40/d3c/d3f/fc3 0 2024-04-25T15:49:50.085 INFO:tasks.workunit.client.0.smithi110.stdout:9/868: link d9/d44/d5e/cbc d9/db/df/c10e 0 2024-04-25T15:49:50.088 INFO:tasks.workunit.client.0.smithi110.stdout:5/946: creat d4/d1d/d1f/d35/d3d/d93/dc4/f135 x:0 0 0 2024-04-25T15:49:50.091 INFO:tasks.workunit.client.0.smithi110.stdout:9/869: dwrite d9/db/df/d2c/d3c/fe0 [0,4194304] 0 2024-04-25T15:49:50.097 INFO:tasks.workunit.client.0.smithi110.stdout:6/905: creat d1/dd/d3c/d91/d8d/dce/d102/f13b x:0 0 0 2024-04-25T15:49:50.100 INFO:tasks.workunit.client.1.smithi193.stdout:0/802: mknod d0/d49/d5/d7/dd/d16/d26/d2e/d93/c113 0 2024-04-25T15:49:50.102 INFO:tasks.workunit.client.0.smithi110.stdout:2/868: link de/d40/d6e/d52/dda/ffb de/d40/dcd/ddf/f117 0 2024-04-25T15:49:50.104 INFO:tasks.workunit.client.0.smithi110.stdout:9/870: symlink d9/d2d/l10f 0 2024-04-25T15:49:50.106 INFO:tasks.workunit.client.1.smithi193.stdout:4/855: fsync db/d32/d45/dab/f104 0 2024-04-25T15:49:50.106 INFO:tasks.workunit.client.0.smithi110.stdout:9/871: write d9/db/df/f4f [5053014,44605] 0 2024-04-25T15:49:50.108 INFO:tasks.workunit.client.1.smithi193.stdout:3/808: dread d8/ff [0,4194304] 0 2024-04-25T15:49:50.108 INFO:tasks.workunit.client.0.smithi110.stdout:6/906: creat d1/db/d22/d90/d76/dad/f13c x:0 0 0 2024-04-25T15:49:50.109 INFO:tasks.workunit.client.0.smithi110.stdout:4/882: dwrite d13/d24/d2d/d44/da4/fb3 [0,4194304] 0 2024-04-25T15:49:50.111 INFO:tasks.workunit.client.0.smithi110.stdout:5/947: rename d4/d1d/d1f/d35/f6a to d4/d15/d51/d90/ded/d106/f136 0 2024-04-25T15:49:50.113 INFO:tasks.workunit.client.1.smithi193.stdout:7/797: symlink d12/d1c/da3/l124 0 2024-04-25T15:49:50.119 INFO:tasks.workunit.client.1.smithi193.stdout:9/766: dread df/d2c/d2f/d4e/d1c/f23 [0,4194304] 0 2024-04-25T15:49:50.120 INFO:tasks.workunit.client.1.smithi193.stdout:1/708: mknod dc/d4a/d4d/d7a/d85/d92/cec 0 2024-04-25T15:49:50.124 INFO:tasks.workunit.client.1.smithi193.stdout:3/809: dread d8/d13/d14/f59 [0,4194304] 0 2024-04-25T15:49:50.125 INFO:tasks.workunit.client.0.smithi110.stdout:6/907: truncate d1/db/da0/fa7 160898 0 2024-04-25T15:49:50.126 INFO:tasks.workunit.client.1.smithi193.stdout:3/810: write d8/d30/d45/f70 [4210680,70900] 0 2024-04-25T15:49:50.128 INFO:tasks.workunit.client.0.smithi110.stdout:4/883: truncate d13/d24/d2d/d44/d90/fc5 783722 0 2024-04-25T15:49:50.129 INFO:tasks.workunit.client.0.smithi110.stdout:6/908: stat d1/dd/d3c/f7c 0 2024-04-25T15:49:50.129 INFO:tasks.workunit.client.0.smithi110.stdout:4/884: fsync d13/d24/f2a 0 2024-04-25T15:49:50.130 INFO:tasks.workunit.client.1.smithi193.stdout:0/803: truncate d0/d49/d5/d7/dd/f100 2271718 0 2024-04-25T15:49:50.134 INFO:tasks.workunit.client.1.smithi193.stdout:2/770: truncate d0/d9/d3d/d7d/fd2 3891981 0 2024-04-25T15:49:50.134 INFO:tasks.workunit.client.1.smithi193.stdout:6/789: dread dd/d12/d19/f43 [4194304,4194304] 0 2024-04-25T15:49:50.134 INFO:tasks.workunit.client.1.smithi193.stdout:2/771: fdatasync d0/d9/d15/f113 0 2024-04-25T15:49:50.135 INFO:tasks.workunit.client.1.smithi193.stdout:4/856: mknod db/d13/d1e/d1f/d56/d69/dad/def/c119 0 2024-04-25T15:49:50.136 INFO:tasks.workunit.client.1.smithi193.stdout:6/790: write dd/d12/d19/d1c/d40/d3c/d3f/fac [671903,28403] 0 2024-04-25T15:49:50.137 INFO:tasks.workunit.client.1.smithi193.stdout:7/798: unlink d12/d31/d37/d62/d6e/da5/lbb 0 2024-04-25T15:49:50.137 INFO:tasks.workunit.client.0.smithi110.stdout:9/872: link d9/d13/d5b/d61/db3/dd3/lfb d9/d2d/dc7/l110 0 2024-04-25T15:49:50.139 INFO:tasks.workunit.client.0.smithi110.stdout:9/873: write d9/db/d7d/f10c [924768,40621] 0 2024-04-25T15:49:50.139 INFO:tasks.workunit.client.1.smithi193.stdout:1/709: creat dc/d59/dc0/deb/d55/d8a/db7/fed x:0 0 0 2024-04-25T15:49:50.140 INFO:tasks.workunit.client.0.smithi110.stdout:6/909: mknod d1/db/d22/d90/df7/c13d 0 2024-04-25T15:49:50.140 INFO:tasks.workunit.client.0.smithi110.stdout:2/869: dread de/d40/d6e/d50/db7/faa [0,4194304] 0 2024-04-25T15:49:50.141 INFO:tasks.workunit.client.0.smithi110.stdout:9/874: fsync d9/db/df/d2c/f36 0 2024-04-25T15:49:50.142 INFO:tasks.workunit.client.1.smithi193.stdout:8/803: creat d2/d18/f107 x:0 0 0 2024-04-25T15:49:50.143 INFO:tasks.workunit.client.1.smithi193.stdout:8/804: read - d2/da5/ff0 zero size 2024-04-25T15:49:50.143 INFO:tasks.workunit.client.1.smithi193.stdout:5/804: creat d17/f110 x:0 0 0 2024-04-25T15:49:50.147 INFO:tasks.workunit.client.0.smithi110.stdout:6/910: truncate d1/db/d22/d90/fa3 520209 0 2024-04-25T15:49:50.149 INFO:tasks.workunit.client.0.smithi110.stdout:2/870: symlink de/l118 0 2024-04-25T15:49:50.149 INFO:tasks.workunit.client.0.smithi110.stdout:5/948: dread d4/d1d/d1f/d35/f4c [0,4194304] 0 2024-04-25T15:49:50.151 INFO:tasks.workunit.client.1.smithi193.stdout:4/857: stat db/lfa 0 2024-04-25T15:49:50.151 INFO:tasks.workunit.client.0.smithi110.stdout:2/871: write de/d6f/fdc [2029510,94039] 0 2024-04-25T15:49:50.152 INFO:tasks.workunit.client.1.smithi193.stdout:4/858: readlink db/d13/d1e/d1f/d56/d69/d70/l97 0 2024-04-25T15:49:50.153 INFO:tasks.workunit.client.0.smithi110.stdout:4/885: dread d13/d24/d2d/d36/d51/f8c [0,4194304] 0 2024-04-25T15:49:50.157 INFO:tasks.workunit.client.0.smithi110.stdout:9/875: dread d9/d13/d7c/fb7 [0,4194304] 0 2024-04-25T15:49:50.160 INFO:tasks.workunit.client.1.smithi193.stdout:7/799: creat d12/d31/d37/d104/f125 x:0 0 0 2024-04-25T15:49:50.161 INFO:tasks.workunit.client.1.smithi193.stdout:7/800: chown d12/d13/f41 1452 1 2024-04-25T15:49:50.164 INFO:tasks.workunit.client.1.smithi193.stdout:1/710: creat dc/d4a/d50/d7c/d82/fee x:0 0 0 2024-04-25T15:49:50.164 INFO:tasks.workunit.client.1.smithi193.stdout:1/711: stat dc/d4a/d4d/d80 0 2024-04-25T15:49:50.165 INFO:tasks.workunit.client.0.smithi110.stdout:4/886: fsync d13/d24/d2d/d44/d65/d99/daf/d56/d88/f2c 0 2024-04-25T15:49:50.167 INFO:tasks.workunit.client.1.smithi193.stdout:8/805: mkdir d2/d3/d27/d108 0 2024-04-25T15:49:50.168 INFO:tasks.workunit.client.0.smithi110.stdout:9/876: symlink d9/db/df/d4d/l111 0 2024-04-25T15:49:50.168 INFO:tasks.workunit.client.0.smithi110.stdout:6/911: read d1/db/d22/d90/fa3 [448859,39053] 0 2024-04-25T15:49:50.169 INFO:tasks.workunit.client.1.smithi193.stdout:8/806: dread - d2/d3/d27/d69/dae/f103 zero size 2024-04-25T15:49:50.169 INFO:tasks.workunit.client.0.smithi110.stdout:6/912: chown d1/dd/f64 29743 1 2024-04-25T15:49:50.170 INFO:tasks.workunit.client.1.smithi193.stdout:1/712: dwrite dc/d4a/d50/d5c/d5d/f51 [0,4194304] 0 2024-04-25T15:49:50.170 INFO:tasks.workunit.client.1.smithi193.stdout:3/811: symlink d8/d13/d14/lfb 0 2024-04-25T15:49:50.174 INFO:tasks.workunit.client.0.smithi110.stdout:4/887: rename d13/d24/d2d/d36/d51 to d13/d24/d26/dcd/d131 0 2024-04-25T15:49:50.175 INFO:tasks.workunit.client.1.smithi193.stdout:4/859: fdatasync db/d13/d1e/d1f/d56/d69/d70/f9c 0 2024-04-25T15:49:50.177 INFO:tasks.workunit.client.1.smithi193.stdout:9/767: truncate df/d2c/d34/f75 801675 0 2024-04-25T15:49:50.177 INFO:tasks.workunit.client.1.smithi193.stdout:2/772: dwrite d0/d42/f61 [0,4194304] 0 2024-04-25T15:49:50.178 INFO:tasks.workunit.client.0.smithi110.stdout:5/949: write d4/d1d/d1f/f6e [4597577,62523] 0 2024-04-25T15:49:50.178 INFO:tasks.workunit.client.1.smithi193.stdout:9/768: chown df/d2c/d2f/d4e/fa0 83413559 1 2024-04-25T15:49:50.180 INFO:tasks.workunit.client.1.smithi193.stdout:9/769: fsync df/d2c/d2f/d4e/f1a 0 2024-04-25T15:49:50.180 INFO:tasks.workunit.client.0.smithi110.stdout:6/913: mkdir d1/dd/d13e 0 2024-04-25T15:49:50.185 INFO:tasks.workunit.client.0.smithi110.stdout:2/872: write de/d40/f36 [757171,33296] 0 2024-04-25T15:49:50.187 INFO:tasks.workunit.client.0.smithi110.stdout:5/950: rename d4/d1d/d1f/l33 to d4/d15/d51/d90/ded/l137 0 2024-04-25T15:49:50.188 INFO:tasks.workunit.client.0.smithi110.stdout:5/951: read - d4/d1d/d1f/d35/d3d/d93/ff1 zero size 2024-04-25T15:49:50.190 INFO:tasks.workunit.client.0.smithi110.stdout:6/914: mkdir d1/db/d22/d33/d63/d13f 0 2024-04-25T15:49:50.191 INFO:tasks.workunit.client.1.smithi193.stdout:8/807: unlink d2/d3/l42 0 2024-04-25T15:49:50.192 INFO:tasks.workunit.client.1.smithi193.stdout:8/808: dread - d2/d3/d27/d69/dae/d4a/d63/da2/fb9 zero size 2024-04-25T15:49:50.193 INFO:tasks.workunit.client.1.smithi193.stdout:3/812: rmdir d8/d13/d14/d23 39 2024-04-25T15:49:50.194 INFO:tasks.workunit.client.1.smithi193.stdout:8/809: chown d2/d3/d27/d69/f9a 14803268 1 2024-04-25T15:49:50.196 INFO:tasks.workunit.client.1.smithi193.stdout:4/860: dread db/f38 [0,4194304] 0 2024-04-25T15:49:50.196 INFO:tasks.workunit.client.1.smithi193.stdout:0/804: link d0/d49/c8c d0/d49/d5/d7/dd/d16/d26/d5e/c114 0 2024-04-25T15:49:50.203 INFO:tasks.workunit.client.0.smithi110.stdout:4/888: write d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/ff8 [2374841,122396] 0 2024-04-25T15:49:50.209 INFO:tasks.workunit.client.1.smithi193.stdout:7/801: truncate d12/d31/d37/d62/d6e/da5/dd7/fe5 247036 0 2024-04-25T15:49:50.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/802: write d12/d1c/d63/d70/f34 [8715246,84548] 0 2024-04-25T15:49:50.211 INFO:tasks.workunit.client.0.smithi110.stdout:2/873: write de/d40/d5b/d5c/d5f/db2/f95 [1278603,44511] 0 2024-04-25T15:49:50.213 INFO:tasks.workunit.client.0.smithi110.stdout:6/915: dwrite d1/db/d24/daa/fdb [0,4194304] 0 2024-04-25T15:49:50.213 INFO:tasks.workunit.client.0.smithi110.stdout:4/889: dread d13/d24/d2d/f79 [0,4194304] 0 2024-04-25T15:49:50.215 INFO:tasks.workunit.client.0.smithi110.stdout:5/952: getdents d4/d15/d51/d90/ded/d106 0 2024-04-25T15:49:50.216 INFO:tasks.workunit.client.1.smithi193.stdout:9/770: mkdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/deb 0 2024-04-25T15:49:50.217 INFO:tasks.workunit.client.1.smithi193.stdout:9/771: fsync df/d2c/d2f/fc4 0 2024-04-25T15:49:50.218 INFO:tasks.workunit.client.1.smithi193.stdout:3/813: mkdir d8/d13/d1f/d69/dcd/dd9/dde/dfc 0 2024-04-25T15:49:50.220 INFO:tasks.workunit.client.0.smithi110.stdout:4/890: rename fa to d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d129/f132 0 2024-04-25T15:49:50.220 INFO:tasks.workunit.client.0.smithi110.stdout:2/874: dread de/d40/d5b/d5c/d5f/db2/fcb [0,4194304] 0 2024-04-25T15:49:50.221 INFO:tasks.workunit.client.1.smithi193.stdout:8/810: mknod d2/d3/d27/d69/dbf/c109 0 2024-04-25T15:49:50.224 INFO:tasks.workunit.client.0.smithi110.stdout:5/953: creat d4/d1d/d1f/d35/d7f/d92/d9b/f138 x:0 0 0 2024-04-25T15:49:50.225 INFO:tasks.workunit.client.1.smithi193.stdout:4/861: rmdir db/d13 39 2024-04-25T15:49:50.225 INFO:tasks.workunit.client.1.smithi193.stdout:3/814: dwrite d8/d13/d1f/f9e [0,4194304] 0 2024-04-25T15:49:50.226 INFO:tasks.workunit.client.0.smithi110.stdout:4/891: dread - d13/d24/d2d/d44/da4/ff9 zero size 2024-04-25T15:49:50.227 INFO:tasks.workunit.client.0.smithi110.stdout:2/875: fsync de/d40/d6e/d52/f112 0 2024-04-25T15:49:50.234 INFO:tasks.workunit.client.0.smithi110.stdout:5/954: symlink d4/d1d/l139 0 2024-04-25T15:49:50.235 INFO:tasks.workunit.client.0.smithi110.stdout:2/876: dwrite de/d40/d4f/d9f/fa3 [0,4194304] 0 2024-04-25T15:49:50.235 INFO:tasks.workunit.client.0.smithi110.stdout:4/892: creat d13/d24/d26/f133 x:0 0 0 2024-04-25T15:49:50.239 INFO:tasks.workunit.client.1.smithi193.stdout:6/791: getdents dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dd7 0 2024-04-25T15:49:50.241 INFO:tasks.workunit.client.1.smithi193.stdout:6/792: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/f60 [3769896,41079] 0 2024-04-25T15:49:50.242 INFO:tasks.workunit.client.0.smithi110.stdout:6/916: getdents d1/db/da0/da5/dea 0 2024-04-25T15:49:50.243 INFO:tasks.workunit.client.0.smithi110.stdout:6/917: fsync d1/db/d22/d33/d63/fd4 0 2024-04-25T15:49:50.246 INFO:tasks.workunit.client.0.smithi110.stdout:5/955: mkdir d4/d15/d51/d90/ded/d13a 0 2024-04-25T15:49:50.248 INFO:tasks.workunit.client.0.smithi110.stdout:4/893: dread d13/d24/d2d/f37 [8388608,4194304] 0 2024-04-25T15:49:50.248 INFO:tasks.workunit.client.0.smithi110.stdout:4/894: chown c10 22935 1 2024-04-25T15:49:50.251 INFO:tasks.workunit.client.0.smithi110.stdout:6/918: write d1/dd/d3c/d91/d8d/dce/d102/d75/d82/f136 [254676,76374] 0 2024-04-25T15:49:50.255 INFO:tasks.workunit.client.0.smithi110.stdout:4/895: rename d13/d24/d2d/d44/d65/d99/daf/l110 to d13/dba/l134 0 2024-04-25T15:49:50.257 INFO:tasks.workunit.client.1.smithi193.stdout:3/815: fdatasync d8/d13/d14/d27/f73 0 2024-04-25T15:49:50.257 INFO:tasks.workunit.client.1.smithi193.stdout:2/773: link d0/l4 d0/d42/l121 0 2024-04-25T15:49:50.259 INFO:tasks.workunit.client.0.smithi110.stdout:4/896: mkdir d13/d24/d2d/d5e/d122/d135 0 2024-04-25T15:49:50.262 INFO:tasks.workunit.client.1.smithi193.stdout:6/793: read dd/d12/d19/d45/fc7 [3511487,13859] 0 2024-04-25T15:49:50.262 INFO:tasks.workunit.client.1.smithi193.stdout:5/805: link d17/f2a d17/d18/d30/d38/d71/dd7/de7/f111 0 2024-04-25T15:49:50.262 INFO:tasks.workunit.client.0.smithi110.stdout:6/919: dread d1/db/d22/d90/d76/f81 [0,4194304] 0 2024-04-25T15:49:50.263 INFO:tasks.workunit.client.1.smithi193.stdout:1/713: getdents dc/d4a/d50/d5c 0 2024-04-25T15:49:50.263 INFO:tasks.workunit.client.1.smithi193.stdout:6/794: read - dd/d12/d19/d1c/d40/d3c/fd2 zero size 2024-04-25T15:49:50.264 INFO:tasks.workunit.client.0.smithi110.stdout:4/897: truncate d13/d24/d2d/dd9/dbb/dd2/ff3 122928 0 2024-04-25T15:49:50.268 INFO:tasks.workunit.client.1.smithi193.stdout:0/805: creat d0/d49/d5/d7/dd/d16/d18/f115 x:0 0 0 2024-04-25T15:49:50.268 INFO:tasks.workunit.client.1.smithi193.stdout:7/803: dwrite d12/d31/d37/d92/dd1/fd5 [0,4194304] 0 2024-04-25T15:49:50.269 INFO:tasks.workunit.client.1.smithi193.stdout:0/806: dread - d0/d49/d6a/f10e zero size 2024-04-25T15:49:50.270 INFO:tasks.workunit.client.1.smithi193.stdout:7/804: chown d12/d31/d37/d104/f125 411835530 1 2024-04-25T15:49:50.270 INFO:tasks.workunit.client.1.smithi193.stdout:3/816: symlink d8/d13/d1f/d69/dcd/dd9/lfd 0 2024-04-25T15:49:50.272 INFO:tasks.workunit.client.1.smithi193.stdout:3/817: chown d8/d13/d14/d27/c38 124 1 2024-04-25T15:49:50.276 INFO:tasks.workunit.client.1.smithi193.stdout:9/772: rename df/d2c/d34/f75 to df/d2c/d2f/d5c/fec 0 2024-04-25T15:49:50.278 INFO:tasks.workunit.client.1.smithi193.stdout:5/806: truncate d17/d83/f95 1082900 0 2024-04-25T15:49:50.279 INFO:tasks.workunit.client.1.smithi193.stdout:1/714: readlink dc/d3c/l4f 0 2024-04-25T15:49:50.280 INFO:tasks.workunit.client.1.smithi193.stdout:6/795: truncate dd/d12/d19/d1c/f26 392232 0 2024-04-25T15:49:50.282 INFO:tasks.workunit.client.0.smithi110.stdout:4/898: truncate d13/d24/d2d/d44/d90/fc1 4415430 0 2024-04-25T15:49:50.282 INFO:tasks.workunit.client.0.smithi110.stdout:4/899: chown d13/d24/f9f 4185 1 2024-04-25T15:49:50.283 INFO:tasks.workunit.client.1.smithi193.stdout:7/805: mknod d12/d1c/d63/d70/d68/d8d/c126 0 2024-04-25T15:49:50.284 INFO:tasks.workunit.client.1.smithi193.stdout:0/807: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/l116 0 2024-04-25T15:49:50.284 INFO:tasks.workunit.client.0.smithi110.stdout:4/900: read d13/d24/d2d/d44/d65/d99/daf/d56/f118 [3210464,76817] 0 2024-04-25T15:49:50.285 INFO:tasks.workunit.client.1.smithi193.stdout:4/862: dwrite db/d13/d1e/d1f/d56/d57/fa4 [0,4194304] 0 2024-04-25T15:49:50.285 INFO:tasks.workunit.client.0.smithi110.stdout:4/901: read d13/d24/f9f [686786,95531] 0 2024-04-25T15:49:50.286 INFO:tasks.workunit.client.0.smithi110.stdout:4/902: fsync d13/d24/d2d/d5e/f7b 0 2024-04-25T15:49:50.286 INFO:tasks.workunit.client.1.smithi193.stdout:9/773: unlink df/d2c/d2f/d4e/d1c/fbf 0 2024-04-25T15:49:50.287 INFO:tasks.workunit.client.1.smithi193.stdout:6/796: dread dd/d12/d19/d1c/d40/d3c/d3f/d4f/f60 [4194304,4194304] 0 2024-04-25T15:49:50.288 INFO:tasks.workunit.client.1.smithi193.stdout:1/715: fdatasync dc/f2b 0 2024-04-25T15:49:50.291 INFO:tasks.workunit.client.1.smithi193.stdout:2/774: dwrite d0/df/dbf/f9e [0,4194304] 0 2024-04-25T15:49:50.294 INFO:tasks.workunit.client.1.smithi193.stdout:2/775: chown d0/d9/d15/d10b 221665 1 2024-04-25T15:49:50.298 INFO:tasks.workunit.client.1.smithi193.stdout:4/863: write db/d32/d45/ff9 [949027,122856] 0 2024-04-25T15:49:50.299 INFO:tasks.workunit.client.0.smithi110.stdout:4/903: rmdir d13/d24/d2d/d36/de0 0 2024-04-25T15:49:50.299 INFO:tasks.workunit.client.1.smithi193.stdout:7/806: dwrite d12/d31/d37/d104/f4a [0,4194304] 0 2024-04-25T15:49:50.300 INFO:tasks.workunit.client.0.smithi110.stdout:4/904: chown d13/d24/d26/l2e 702 1 2024-04-25T15:49:50.306 INFO:tasks.workunit.client.1.smithi193.stdout:7/807: dread - d12/d31/d37/d62/d6e/da5/fb6 zero size 2024-04-25T15:49:50.306 INFO:tasks.workunit.client.1.smithi193.stdout:1/716: dwrite dc/d4a/d50/d7c/fca [0,4194304] 0 2024-04-25T15:49:50.308 INFO:tasks.workunit.client.1.smithi193.stdout:3/818: unlink d8/d13/d14/d23/c52 0 2024-04-25T15:49:50.309 INFO:tasks.workunit.client.1.smithi193.stdout:7/808: chown d12/d13/ff9 7 1 2024-04-25T15:49:50.310 INFO:tasks.workunit.client.1.smithi193.stdout:3/819: read d8/d13/d14/f59 [2270901,95558] 0 2024-04-25T15:49:50.312 INFO:tasks.workunit.client.1.smithi193.stdout:1/717: read - dc/d4a/d50/d7c/d82/fee zero size 2024-04-25T15:49:50.313 INFO:tasks.workunit.client.1.smithi193.stdout:0/808: mkdir d0/d49/d5/d7/dd/d16/d26/d117 0 2024-04-25T15:49:50.315 INFO:tasks.workunit.client.1.smithi193.stdout:6/797: truncate dd/d12/d19/d45/f55 339729 0 2024-04-25T15:49:50.316 INFO:tasks.workunit.client.1.smithi193.stdout:3/820: chown d8/d13/d14/f80 35384 1 2024-04-25T15:49:50.317 INFO:tasks.workunit.client.0.smithi110.stdout:4/905: dread d13/d24/d2d/dd9/fc2 [0,4194304] 0 2024-04-25T15:49:50.318 INFO:tasks.workunit.client.1.smithi193.stdout:7/809: unlink d12/d31/d39/d60/f64 0 2024-04-25T15:49:50.318 INFO:tasks.workunit.client.1.smithi193.stdout:9/774: dread df/d2c/d2f/d4e/d1c/d30/d3b/f47 [0,4194304] 0 2024-04-25T15:49:50.319 INFO:tasks.workunit.client.0.smithi110.stdout:4/906: stat d13/d24/d2d/d44/d65/d99/daf/d56/d88 0 2024-04-25T15:49:50.319 INFO:tasks.workunit.client.1.smithi193.stdout:6/798: read dd/d12/d19/d1c/f20 [99800,15729] 0 2024-04-25T15:49:50.319 INFO:tasks.workunit.client.1.smithi193.stdout:4/864: dwrite db/d13/d1e/f6d [8388608,4194304] 0 2024-04-25T15:49:50.321 INFO:tasks.workunit.client.1.smithi193.stdout:1/718: rmdir dc/d4a/d4d/d7a/da6 39 2024-04-25T15:49:50.323 INFO:tasks.workunit.client.0.smithi110.stdout:4/907: mkdir d13/d24/d2d/d5e/d136 0 2024-04-25T15:49:50.327 INFO:tasks.workunit.client.1.smithi193.stdout:8/811: sync 2024-04-25T15:49:50.328 INFO:tasks.workunit.client.1.smithi193.stdout:7/810: mkdir d12/d1c/d96/d127 0 2024-04-25T15:49:50.330 INFO:tasks.workunit.client.0.smithi110.stdout:4/908: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d113/f137 x:0 0 0 2024-04-25T15:49:50.331 INFO:tasks.workunit.client.1.smithi193.stdout:6/799: unlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/l65 0 2024-04-25T15:49:50.332 INFO:tasks.workunit.client.1.smithi193.stdout:8/812: dwrite d2/d3/d20/d8c/dde/d94/fcb [0,4194304] 0 2024-04-25T15:49:50.334 INFO:tasks.workunit.client.1.smithi193.stdout:4/865: unlink db/d13/d1e/d1f/d56/d69/d70/d8a/f8c 0 2024-04-25T15:49:50.335 INFO:tasks.workunit.client.1.smithi193.stdout:5/807: truncate d17/d18/d30/d34/f46 3582121 0 2024-04-25T15:49:50.337 INFO:tasks.workunit.client.1.smithi193.stdout:2/776: getdents d0/d9/d15/d31/d50 0 2024-04-25T15:49:50.342 INFO:tasks.workunit.client.1.smithi193.stdout:1/719: readlink dc/d4a/d50/d7c/dda/ldb 0 2024-04-25T15:49:50.348 INFO:tasks.workunit.client.1.smithi193.stdout:1/720: dwrite dc/d4a/fde [0,4194304] 0 2024-04-25T15:49:50.357 INFO:tasks.workunit.client.1.smithi193.stdout:3/821: dwrite d8/fc [0,4194304] 0 2024-04-25T15:49:50.357 INFO:tasks.workunit.client.1.smithi193.stdout:0/809: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/fa8 [0,4194304] 0 2024-04-25T15:49:50.358 INFO:tasks.workunit.client.1.smithi193.stdout:3/822: chown d8/d13/f32 1 1 2024-04-25T15:49:50.359 INFO:tasks.workunit.client.1.smithi193.stdout:6/800: dread - dd/d12/d19/fc4 zero size 2024-04-25T15:49:50.365 INFO:tasks.workunit.client.1.smithi193.stdout:4/866: rmdir db/d13/d1e/d1f/d56 39 2024-04-25T15:49:50.367 INFO:tasks.workunit.client.1.smithi193.stdout:2/777: creat d0/d37/d25/d5f/d3a/d6d/d6e/f122 x:0 0 0 2024-04-25T15:49:50.375 INFO:tasks.workunit.client.1.smithi193.stdout:7/811: write d12/fb8 [2780920,52843] 0 2024-04-25T15:49:50.381 INFO:tasks.workunit.client.1.smithi193.stdout:7/812: dwrite d12/d31/d37/d104/f74 [0,4194304] 0 2024-04-25T15:49:50.385 INFO:tasks.workunit.client.1.smithi193.stdout:3/823: mknod d8/d13/d4b/d8b/d9a/cfe 0 2024-04-25T15:49:50.387 INFO:tasks.workunit.client.1.smithi193.stdout:8/813: mkdir d2/d10a 0 2024-04-25T15:49:50.398 INFO:tasks.workunit.client.1.smithi193.stdout:9/775: rename df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/c32 to df/d2c/d2f/d4e/d1c/d30/ced 0 2024-04-25T15:49:50.400 INFO:tasks.workunit.client.1.smithi193.stdout:0/810: write d0/d49/f2c [660427,13240] 0 2024-04-25T15:49:50.403 INFO:tasks.workunit.client.1.smithi193.stdout:0/811: chown d0/d49/d5/d7/dd/l39 4324396 1 2024-04-25T15:49:50.403 INFO:tasks.workunit.client.1.smithi193.stdout:1/721: dwrite dc/d4a/d4d/f99 [4194304,4194304] 0 2024-04-25T15:49:50.409 INFO:tasks.workunit.client.1.smithi193.stdout:0/812: dwrite d0/d49/d5/d7/dd/d16/d18/f103 [0,4194304] 0 2024-04-25T15:49:50.412 INFO:tasks.workunit.client.1.smithi193.stdout:8/814: mknod d2/d3/d20/d8c/dde/d94/c10b 0 2024-04-25T15:49:50.415 INFO:tasks.workunit.client.1.smithi193.stdout:4/867: creat db/d13/d1e/d1f/d56/d69/d70/d8a/f11a x:0 0 0 2024-04-25T15:49:50.418 INFO:tasks.workunit.client.1.smithi193.stdout:6/801: rename dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/fc5 to dd/dea/ff0 0 2024-04-25T15:49:50.418 INFO:tasks.workunit.client.1.smithi193.stdout:9/776: rmdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf 39 2024-04-25T15:49:50.421 INFO:tasks.workunit.client.1.smithi193.stdout:1/722: creat dc/d4a/d50/fef x:0 0 0 2024-04-25T15:49:50.425 INFO:tasks.workunit.client.1.smithi193.stdout:8/815: dread - d2/f6e zero size 2024-04-25T15:49:50.426 INFO:tasks.workunit.client.0.smithi110.stdout:4/909: truncate d13/d24/d2d/d44/d65/d99/dbc/fdb 3315294 0 2024-04-25T15:49:50.429 INFO:tasks.workunit.client.0.smithi110.stdout:4/910: write d13/d24/d26/dcd/d131/f8c [1713595,85123] 0 2024-04-25T15:49:50.429 INFO:tasks.workunit.client.1.smithi193.stdout:4/868: dread db/d13/d1e/d1f/d56/d57/f9b [0,4194304] 0 2024-04-25T15:49:50.430 INFO:tasks.workunit.client.1.smithi193.stdout:2/778: rename d0/d9/d15 to d0/d9/d15/d31/d123 22 2024-04-25T15:49:50.430 INFO:tasks.workunit.client.1.smithi193.stdout:4/869: read - db/d13/d1e/d1f/d56/fe3 zero size 2024-04-25T15:49:50.431 INFO:tasks.workunit.client.1.smithi193.stdout:2/779: chown d0/d9/d15/c70 7674232 1 2024-04-25T15:49:50.432 INFO:tasks.workunit.client.1.smithi193.stdout:4/870: chown db/d13/d1e/d1f/d56/d69/d70/f76 1560272690 1 2024-04-25T15:49:50.432 INFO:tasks.workunit.client.1.smithi193.stdout:6/802: unlink dd/d12/d19/d1c/cb3 0 2024-04-25T15:49:50.434 INFO:tasks.workunit.client.1.smithi193.stdout:9/777: truncate df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/f70 1481471 0 2024-04-25T15:49:50.436 INFO:tasks.workunit.client.1.smithi193.stdout:3/824: mknod d8/d13/d1f/d69/d77/cff 0 2024-04-25T15:49:50.436 INFO:tasks.workunit.client.1.smithi193.stdout:3/825: dread - d8/d13/d14/d51/f5f zero size 2024-04-25T15:49:50.437 INFO:tasks.workunit.client.1.smithi193.stdout:0/813: unlink d0/d49/d5/d7/dd/d16/d18/d28/f94 0 2024-04-25T15:49:50.439 INFO:tasks.workunit.client.1.smithi193.stdout:5/808: sync 2024-04-25T15:49:50.439 INFO:tasks.workunit.client.1.smithi193.stdout:5/809: write d17/f10a [1042913,27248] 0 2024-04-25T15:49:50.440 INFO:tasks.workunit.client.1.smithi193.stdout:7/813: rename d12/d31/d37/d92/ccb to d12/d31/d37/d92/de8/d10d/c128 0 2024-04-25T15:49:50.440 INFO:tasks.workunit.client.1.smithi193.stdout:0/814: dread d0/d49/d5/d7/dd/d46/f4d [0,4194304] 0 2024-04-25T15:49:50.441 INFO:tasks.workunit.client.1.smithi193.stdout:4/871: creat db/d32/d45/dab/f11b x:0 0 0 2024-04-25T15:49:50.442 INFO:tasks.workunit.client.0.smithi110.stdout:4/911: dread f6 [0,4194304] 0 2024-04-25T15:49:50.444 INFO:tasks.workunit.client.1.smithi193.stdout:7/814: dread d12/d1c/f28 [0,4194304] 0 2024-04-25T15:49:50.538 INFO:tasks.workunit.client.1.smithi193.stdout:1/723: creat dc/d4a/d4d/d7a/da6/ff0 x:0 0 0 2024-04-25T15:49:50.538 INFO:tasks.workunit.client.0.smithi110.stdout:4/912: write d13/d24/d2d/db5/f12f [1662074,121872] 0 2024-04-25T15:49:50.546 INFO:tasks.workunit.client.1.smithi193.stdout:2/780: mkdir d0/d9/d3d/d114/d124 0 2024-04-25T15:49:50.547 INFO:tasks.workunit.client.1.smithi193.stdout:2/781: fsync d0/d9/d15/d31/fd9 0 2024-04-25T15:49:50.549 INFO:tasks.workunit.client.1.smithi193.stdout:4/872: creat db/d13/d9d/f11c x:0 0 0 2024-04-25T15:49:50.550 INFO:tasks.workunit.client.1.smithi193.stdout:4/873: write db/f61 [4183635,28342] 0 2024-04-25T15:49:50.552 INFO:tasks.workunit.client.1.smithi193.stdout:4/874: write db/d13/d1e/d1f/d56/fe3 [90644,115003] 0 2024-04-25T15:49:50.554 INFO:tasks.workunit.client.1.smithi193.stdout:6/803: write dd/d12/f35 [3060787,107672] 0 2024-04-25T15:49:50.554 INFO:tasks.workunit.client.1.smithi193.stdout:9/778: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/f2a [0,4194304] 0 2024-04-25T15:49:50.555 INFO:tasks.workunit.client.1.smithi193.stdout:1/724: rename dc/d4a/d50/d7c/d82/ddc to dc/d4a/d4d/d7a/dd0/de2/df1 0 2024-04-25T15:49:50.556 INFO:tasks.workunit.client.1.smithi193.stdout:8/816: creat d2/d3/d27/d69/dae/f10c x:0 0 0 2024-04-25T15:49:50.558 INFO:tasks.workunit.client.1.smithi193.stdout:5/810: dread f16 [0,4194304] 0 2024-04-25T15:49:50.561 INFO:tasks.workunit.client.1.smithi193.stdout:3/826: dwrite d8/d13/f32 [0,4194304] 0 2024-04-25T15:49:50.564 INFO:tasks.workunit.client.0.smithi110.stdout:4/913: dwrite d13/d24/d2d/d44/da4/fca [0,4194304] 0 2024-04-25T15:49:50.573 INFO:tasks.workunit.client.0.smithi110.stdout:4/914: creat d13/d24/d2d/d5e/d136/f138 x:0 0 0 2024-04-25T15:49:50.575 INFO:tasks.workunit.client.0.smithi110.stdout:4/915: dread d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d117/f123 [0,4194304] 0 2024-04-25T15:49:50.578 INFO:tasks.workunit.client.0.smithi110.stdout:4/916: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/f139 x:0 0 0 2024-04-25T15:49:50.580 INFO:tasks.workunit.client.0.smithi110.stdout:4/917: symlink d13/d24/l13a 0 2024-04-25T15:49:50.584 INFO:tasks.workunit.client.1.smithi193.stdout:7/815: rmdir d12/d13/dcc/d100 0 2024-04-25T15:49:50.588 INFO:tasks.workunit.client.1.smithi193.stdout:1/725: link dc/d4a/d50/d5c/d5d/fa0 dc/d4a/d4d/d80/ff2 0 2024-04-25T15:49:50.589 INFO:tasks.workunit.client.1.smithi193.stdout:9/779: read df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/f98 [38994,33390] 0 2024-04-25T15:49:50.590 INFO:tasks.workunit.client.0.smithi110.stdout:4/918: write d13/d24/d26/f49 [5129096,9410] 0 2024-04-25T15:49:50.592 INFO:tasks.workunit.client.1.smithi193.stdout:4/875: dwrite db/d32/f6e [0,4194304] 0 2024-04-25T15:49:50.593 INFO:tasks.workunit.client.1.smithi193.stdout:0/815: dwrite d0/d49/d5/d7/dd/d16/f30 [0,4194304] 0 2024-04-25T15:49:50.594 INFO:tasks.workunit.client.0.smithi110.stdout:4/919: creat d13/d24/d2d/d5e/d122/d135/f13b x:0 0 0 2024-04-25T15:49:50.595 INFO:tasks.workunit.client.1.smithi193.stdout:3/827: dwrite d8/d13/d14/fd3 [0,4194304] 0 2024-04-25T15:49:50.598 INFO:tasks.workunit.client.1.smithi193.stdout:9/780: write f9 [788124,127076] 0 2024-04-25T15:49:50.598 INFO:tasks.workunit.client.0.smithi110.stdout:4/920: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d13c 0 2024-04-25T15:49:50.601 INFO:tasks.workunit.client.1.smithi193.stdout:4/876: dread - db/d32/d40/dee/f114 zero size 2024-04-25T15:49:50.602 INFO:tasks.workunit.client.0.smithi110.stdout:4/921: mkdir d13/dba/dd8/d13d 0 2024-04-25T15:49:50.604 INFO:tasks.workunit.client.1.smithi193.stdout:3/828: fsync d8/d13/fc8 0 2024-04-25T15:49:50.605 INFO:tasks.workunit.client.1.smithi193.stdout:5/811: dwrite d17/f57 [0,4194304] 0 2024-04-25T15:49:50.605 INFO:tasks.workunit.client.1.smithi193.stdout:4/877: stat db/d32/d40/cd9 0 2024-04-25T15:49:50.606 INFO:tasks.workunit.client.0.smithi110.stdout:4/922: dread d13/d24/d2d/d44/d90/fc5 [0,4194304] 0 2024-04-25T15:49:50.606 INFO:tasks.workunit.client.1.smithi193.stdout:2/782: dwrite d0/d37/d4d/dbe/df7/fe4 [0,4194304] 0 2024-04-25T15:49:50.607 INFO:tasks.workunit.client.1.smithi193.stdout:0/816: dread - d0/d49/d5/d7/dd/d16/ff1 zero size 2024-04-25T15:49:50.607 INFO:tasks.workunit.client.0.smithi110.stdout:4/923: dread - d13/d24/d2d/f11b zero size 2024-04-25T15:49:50.607 INFO:tasks.workunit.client.0.smithi110.stdout:4/924: chown d13/d24/d2d 126810854 1 2024-04-25T15:49:50.608 INFO:tasks.workunit.client.0.smithi110.stdout:4/925: stat d13/d24/d2d/d44/d65/d99/dbc/c126 0 2024-04-25T15:49:50.609 INFO:tasks.workunit.client.0.smithi110.stdout:4/926: chown d13/dba/ffb 121 1 2024-04-25T15:49:50.611 INFO:tasks.workunit.client.1.smithi193.stdout:6/804: dwrite dd/d12/d19/d1c/d40/d3c/f68 [0,4194304] 0 2024-04-25T15:49:50.614 INFO:tasks.workunit.client.1.smithi193.stdout:5/812: symlink d17/d18/d3e/db6/l112 0 2024-04-25T15:49:50.616 INFO:tasks.workunit.client.1.smithi193.stdout:0/817: dread d0/d49/d5/d7/fe8 [0,4194304] 0 2024-04-25T15:49:50.618 INFO:tasks.workunit.client.1.smithi193.stdout:5/813: dread - d17/f110 zero size 2024-04-25T15:49:50.619 INFO:tasks.workunit.client.1.smithi193.stdout:4/878: creat db/d84/df3/f11d x:0 0 0 2024-04-25T15:49:50.621 INFO:tasks.workunit.client.1.smithi193.stdout:5/814: stat d17/f1f 0 2024-04-25T15:49:50.622 INFO:tasks.workunit.client.0.smithi110.stdout:4/927: creat d13/d24/d2d/d44/d65/d99/daf/f13e x:0 0 0 2024-04-25T15:49:50.623 INFO:tasks.workunit.client.0.smithi110.stdout:4/928: readlink d13/l18 0 2024-04-25T15:49:50.624 INFO:tasks.workunit.client.1.smithi193.stdout:9/781: dwrite df/d2c/d2f/d5c/fb4 [0,4194304] 0 2024-04-25T15:49:50.624 INFO:tasks.workunit.client.1.smithi193.stdout:4/879: fsync db/d32/d45/fda 0 2024-04-25T15:49:50.624 INFO:tasks.workunit.client.1.smithi193.stdout:5/815: readlink d17/d18/d3e/db6/l112 0 2024-04-25T15:49:50.626 INFO:tasks.workunit.client.1.smithi193.stdout:2/783: dwrite d0/d37/d25/d5f/f106 [0,4194304] 0 2024-04-25T15:49:50.630 INFO:tasks.workunit.client.0.smithi110.stdout:4/929: dread d13/d24/d2d/d44/da4/fb3 [0,4194304] 0 2024-04-25T15:49:50.631 INFO:tasks.workunit.client.1.smithi193.stdout:3/829: symlink d8/d13/d1f/d69/d77/d71/dd2/l100 0 2024-04-25T15:49:50.632 INFO:tasks.workunit.client.0.smithi110.stdout:3/949: sync 2024-04-25T15:49:50.632 INFO:tasks.workunit.client.0.smithi110.stdout:9/877: sync 2024-04-25T15:49:50.632 INFO:tasks.workunit.client.0.smithi110.stdout:0/891: sync 2024-04-25T15:49:50.633 INFO:tasks.workunit.client.1.smithi193.stdout:1/726: link dc/d4a/d50/le9 dc/d59/dc0/deb/d55/d8a/lf3 0 2024-04-25T15:49:50.634 INFO:tasks.workunit.client.0.smithi110.stdout:4/930: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d13c/d13f 0 2024-04-25T15:49:50.634 INFO:tasks.workunit.client.0.smithi110.stdout:9/878: fsync d9/d13/d5b/d61/f9d 0 2024-04-25T15:49:50.634 INFO:tasks.workunit.client.1.smithi193.stdout:7/816: dread d12/d31/d39/f53 [0,4194304] 0 2024-04-25T15:49:50.635 INFO:tasks.workunit.client.1.smithi193.stdout:5/816: write d17/d18/d30/d38/dab/fdb [1574348,57254] 0 2024-04-25T15:49:50.636 INFO:tasks.workunit.client.0.smithi110.stdout:3/950: truncate d0/d16/dcb/d2f/d4c/ffd 4330297 0 2024-04-25T15:49:50.636 INFO:tasks.workunit.client.0.smithi110.stdout:0/892: mknod d1/d23/dc5/d108/c145 0 2024-04-25T15:49:50.636 INFO:tasks.workunit.client.1.smithi193.stdout:6/805: read dd/d12/d19/f7f [3285323,1174] 0 2024-04-25T15:49:50.636 INFO:tasks.workunit.client.1.smithi193.stdout:1/727: chown dc/d3c/l46 13372044 1 2024-04-25T15:49:50.638 INFO:tasks.workunit.client.1.smithi193.stdout:6/806: stat dd/f3d 0 2024-04-25T15:49:50.639 INFO:tasks.workunit.client.1.smithi193.stdout:1/728: write dc/d4a/d50/d7c/d82/fee [775250,17903] 0 2024-04-25T15:49:50.640 INFO:tasks.workunit.client.1.smithi193.stdout:8/817: sync 2024-04-25T15:49:50.640 INFO:tasks.workunit.client.1.smithi193.stdout:3/830: read d8/d13/f61 [553375,47876] 0 2024-04-25T15:49:50.641 INFO:tasks.workunit.client.0.smithi110.stdout:4/931: creat d13/d24/d2d/d5e/d122/d135/f140 x:0 0 0 2024-04-25T15:49:50.641 INFO:tasks.workunit.client.1.smithi193.stdout:4/880: mkdir db/d13/dd2/d11e 0 2024-04-25T15:49:50.641 INFO:tasks.workunit.client.1.smithi193.stdout:8/818: fdatasync d2/d3/ff 0 2024-04-25T15:49:50.642 INFO:tasks.workunit.client.1.smithi193.stdout:7/817: dread d12/d31/d39/db0/fb5 [0,4194304] 0 2024-04-25T15:49:50.643 INFO:tasks.workunit.client.1.smithi193.stdout:8/819: chown d2/d3/d20/db8 98057217 1 2024-04-25T15:49:50.644 INFO:tasks.workunit.client.1.smithi193.stdout:9/782: link df/d2d/fd6 df/d2c/d2f/d4e/d69/de7/fee 0 2024-04-25T15:49:50.644 INFO:tasks.workunit.client.1.smithi193.stdout:8/820: stat d2/d3/d27/d69/dbf/f40 0 2024-04-25T15:49:50.646 INFO:tasks.workunit.client.1.smithi193.stdout:0/818: link d0/d49/d5/d7/dd/d16/d26/fb0 d0/d49/d5/d7/dd/d16/d26/d9e/f118 0 2024-04-25T15:49:50.649 INFO:tasks.workunit.client.0.smithi110.stdout:4/932: dread d13/d24/d2d/d44/d65/d99/fee [0,4194304] 0 2024-04-25T15:49:50.650 INFO:tasks.workunit.client.0.smithi110.stdout:0/893: creat d1/d23/f146 x:0 0 0 2024-04-25T15:49:50.651 INFO:tasks.workunit.client.1.smithi193.stdout:1/729: read dc/d4a/d50/d5c/f91 [1810314,127869] 0 2024-04-25T15:49:50.652 INFO:tasks.workunit.client.1.smithi193.stdout:7/818: dwrite d12/d1c/d63/d70/d68/fe1 [0,4194304] 0 2024-04-25T15:49:50.652 INFO:tasks.workunit.client.1.smithi193.stdout:7/819: readlink la 0 2024-04-25T15:49:50.655 INFO:tasks.workunit.client.0.smithi110.stdout:4/933: mkdir d13/d24/d2d/d44/d65/d99/d141 0 2024-04-25T15:49:50.655 INFO:tasks.workunit.client.1.smithi193.stdout:5/817: dread d17/f26 [0,4194304] 0 2024-04-25T15:49:50.661 INFO:tasks.workunit.client.1.smithi193.stdout:5/818: dwrite d17/d18/d30/d3b/fed [0,4194304] 0 2024-04-25T15:49:50.662 INFO:tasks.workunit.client.1.smithi193.stdout:2/784: dwrite d0/d37/d4d/dbe/df7/f112 [0,4194304] 0 2024-04-25T15:49:50.662 INFO:tasks.workunit.client.0.smithi110.stdout:4/934: dwrite d13/d24/d26/dcd/d131/f8c [0,4194304] 0 2024-04-25T15:49:50.664 INFO:tasks.workunit.client.1.smithi193.stdout:6/807: mkdir dd/d12/d19/d1c/d40/dce/de2/df1 0 2024-04-25T15:49:50.665 INFO:tasks.workunit.client.0.smithi110.stdout:0/894: creat d1/d23/d30/d33/d5b/dbb/f147 x:0 0 0 2024-04-25T15:49:50.665 INFO:tasks.workunit.client.0.smithi110.stdout:9/879: write d9/db/df/f6c [1203435,26487] 0 2024-04-25T15:49:50.666 INFO:tasks.workunit.client.0.smithi110.stdout:5/956: sync 2024-04-25T15:49:50.667 INFO:tasks.workunit.client.0.smithi110.stdout:2/877: sync 2024-04-25T15:49:50.667 INFO:tasks.workunit.client.0.smithi110.stdout:6/920: sync 2024-04-25T15:49:50.668 INFO:tasks.workunit.client.0.smithi110.stdout:9/880: readlink d9/d44/d5e/l64 0 2024-04-25T15:49:50.668 INFO:tasks.workunit.client.0.smithi110.stdout:0/895: chown d1/d23/d30/d33/d5b/d78/d107/f104 2077043 1 2024-04-25T15:49:50.669 INFO:tasks.workunit.client.0.smithi110.stdout:6/921: fsync d1/db/d119/f7a 0 2024-04-25T15:49:50.669 INFO:tasks.workunit.client.0.smithi110.stdout:4/935: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d129/d142 0 2024-04-25T15:49:50.671 INFO:tasks.workunit.client.0.smithi110.stdout:9/881: readlink d9/db/df/d2c/d3c/l3e 0 2024-04-25T15:49:50.675 INFO:tasks.workunit.client.1.smithi193.stdout:0/819: mkdir d0/d49/d5/d7/dd/d78/d119 0 2024-04-25T15:49:50.677 INFO:tasks.workunit.client.0.smithi110.stdout:2/878: dread de/d40/d86/fb6 [0,4194304] 0 2024-04-25T15:49:50.680 INFO:tasks.workunit.client.0.smithi110.stdout:5/957: truncate d4/d1d/f65 1343275 0 2024-04-25T15:49:50.681 INFO:tasks.workunit.client.1.smithi193.stdout:1/730: symlink dc/d4a/d4d/d80/lf4 0 2024-04-25T15:49:50.681 INFO:tasks.workunit.client.0.smithi110.stdout:3/951: write d0/d2e/f30 [985598,288] 0 2024-04-25T15:49:50.682 INFO:tasks.workunit.client.1.smithi193.stdout:1/731: write dc/d3c/fe6 [805889,59974] 0 2024-04-25T15:49:50.682 INFO:tasks.workunit.client.0.smithi110.stdout:6/922: mkdir d1/db/da0/da5/dea/d124/d140 0 2024-04-25T15:49:50.683 INFO:tasks.workunit.client.0.smithi110.stdout:3/952: read d0/d16/f117 [796907,31259] 0 2024-04-25T15:49:50.683 INFO:tasks.workunit.client.1.smithi193.stdout:4/881: mkdir db/d13/d1e/d1f/d117/d11f 0 2024-04-25T15:49:50.684 INFO:tasks.workunit.client.1.smithi193.stdout:1/732: dread dc/d59/dc0/deb/d55/fe7 [0,4194304] 0 2024-04-25T15:49:50.689 INFO:tasks.workunit.client.0.smithi110.stdout:3/953: dread d0/d2e/dfc/f11f [0,4194304] 0 2024-04-25T15:49:50.692 INFO:tasks.workunit.client.0.smithi110.stdout:3/954: dread d0/d16/dcb/db0/fef [0,4194304] 0 2024-04-25T15:49:50.695 INFO:tasks.workunit.client.0.smithi110.stdout:5/958: read d4/f44 [3077529,85994] 0 2024-04-25T15:49:50.695 INFO:tasks.workunit.client.1.smithi193.stdout:5/819: creat d17/d18/d30/d38/d48/d4f/deb/f113 x:0 0 0 2024-04-25T15:49:50.696 INFO:tasks.workunit.client.1.smithi193.stdout:5/820: readlink d17/d18/d30/d38/l93 0 2024-04-25T15:49:50.698 INFO:tasks.workunit.client.1.smithi193.stdout:2/785: symlink d0/d37/d25/d5f/d3a/d6d/d6e/l125 0 2024-04-25T15:49:50.699 INFO:tasks.workunit.client.0.smithi110.stdout:2/879: dread de/d40/d6e/f9d [0,4194304] 0 2024-04-25T15:49:50.700 INFO:tasks.workunit.client.1.smithi193.stdout:6/808: write dd/dea/ff0 [626521,96786] 0 2024-04-25T15:49:50.701 INFO:tasks.workunit.client.0.smithi110.stdout:4/936: write d13/d24/d26/fea [630245,127254] 0 2024-04-25T15:49:50.703 INFO:tasks.workunit.client.0.smithi110.stdout:6/923: unlink d1/dd/d3c/d91/d8d/dce/d102/f13b 0 2024-04-25T15:49:50.704 INFO:tasks.workunit.client.0.smithi110.stdout:2/880: dread de/d40/d6e/d52/d6d/d7d/f85 [0,4194304] 0 2024-04-25T15:49:50.706 INFO:tasks.workunit.client.0.smithi110.stdout:9/882: mknod d9/d13/d5b/df7/c112 0 2024-04-25T15:49:50.707 INFO:tasks.workunit.client.1.smithi193.stdout:8/821: truncate d2/d3/d20/d8c/dde/fd2 1410908 0 2024-04-25T15:49:50.708 INFO:tasks.workunit.client.1.smithi193.stdout:7/820: dwrite d12/d1c/d45/f6a [0,4194304] 0 2024-04-25T15:49:50.709 INFO:tasks.workunit.client.0.smithi110.stdout:0/896: symlink d1/d23/dc5/d130/l148 0 2024-04-25T15:49:50.709 INFO:tasks.workunit.client.1.smithi193.stdout:8/822: write d2/d3/d27/d69/dae/f103 [694551,16835] 0 2024-04-25T15:49:50.710 INFO:tasks.workunit.client.1.smithi193.stdout:0/820: symlink d0/d49/d5/d61/dd4/l11a 0 2024-04-25T15:49:50.711 INFO:tasks.workunit.client.1.smithi193.stdout:8/823: truncate d2/d3/f5f 5003618 0 2024-04-25T15:49:50.713 INFO:tasks.workunit.client.0.smithi110.stdout:5/959: dread d4/fb2 [0,4194304] 0 2024-04-25T15:49:50.713 INFO:tasks.workunit.client.0.smithi110.stdout:0/897: dread d1/d23/d30/d33/f88 [0,4194304] 0 2024-04-25T15:49:50.714 INFO:tasks.workunit.client.0.smithi110.stdout:3/955: symlink d0/d2e/l141 0 2024-04-25T15:49:50.715 INFO:tasks.workunit.client.0.smithi110.stdout:3/956: chown d0/d16/f18 21 1 2024-04-25T15:49:50.715 INFO:tasks.workunit.client.0.smithi110.stdout:0/898: readlink d1/d23/d30/d33/d35/l11f 0 2024-04-25T15:49:50.716 INFO:tasks.workunit.client.0.smithi110.stdout:0/899: chown d1/d6c/d116/f82 126908836 1 2024-04-25T15:49:50.716 INFO:tasks.workunit.client.0.smithi110.stdout:0/900: stat d1/d23/d30/d33/d35/f3a 0 2024-04-25T15:49:50.718 INFO:tasks.workunit.client.1.smithi193.stdout:4/882: mkdir db/d13/d9d/d120 0 2024-04-25T15:49:50.718 INFO:tasks.workunit.client.1.smithi193.stdout:4/883: stat db/l36 0 2024-04-25T15:49:50.718 INFO:tasks.workunit.client.0.smithi110.stdout:0/901: truncate d1/d23/d9e/ff9 426881 0 2024-04-25T15:49:50.718 INFO:tasks.workunit.client.0.smithi110.stdout:0/902: chown d1/d47/d54 26977488 1 2024-04-25T15:49:50.719 INFO:tasks.workunit.client.1.smithi193.stdout:4/884: chown db/d13/d1e/d1f/d56/d69/dad 45418024 1 2024-04-25T15:49:50.721 INFO:tasks.workunit.client.1.smithi193.stdout:3/831: link d8/d13/d14/d27/fb6 d8/d13/d1f/d69/d77/d71/dd2/f101 0 2024-04-25T15:49:50.723 INFO:tasks.workunit.client.1.smithi193.stdout:9/783: link df/d2c/d34/d3e/d4d/fd3 df/d2c/d34/fef 0 2024-04-25T15:49:50.723 INFO:tasks.workunit.client.0.smithi110.stdout:6/924: readlink d1/db/l100 0 2024-04-25T15:49:50.725 INFO:tasks.workunit.client.1.smithi193.stdout:5/821: symlink d17/d18/d30/d34/l114 0 2024-04-25T15:49:50.728 INFO:tasks.workunit.client.1.smithi193.stdout:2/786: read d0/d42/d51/d4f/f69 [40592,80958] 0 2024-04-25T15:49:50.729 INFO:tasks.workunit.client.0.smithi110.stdout:9/883: symlink d9/d13/d5b/d61/db3/db4/dc5/de4/l113 0 2024-04-25T15:49:50.729 INFO:tasks.workunit.client.0.smithi110.stdout:9/884: readlink d9/lea 0 2024-04-25T15:49:50.730 INFO:tasks.workunit.client.0.smithi110.stdout:9/885: chown d9/d44/f7f 1574 1 2024-04-25T15:49:50.733 INFO:tasks.workunit.client.1.smithi193.stdout:7/821: mkdir d12/d1c/d63/d70/d68/d8f/d129 0 2024-04-25T15:49:50.735 INFO:tasks.workunit.client.1.smithi193.stdout:8/824: rmdir d2/d18 39 2024-04-25T15:49:50.737 INFO:tasks.workunit.client.1.smithi193.stdout:0/821: creat d0/d49/d5/d7/dd/d16/d26/df5/f11b x:0 0 0 2024-04-25T15:49:50.738 INFO:tasks.workunit.client.0.smithi110.stdout:3/957: chown d0/d16/dcb/d50/l80 224514464 1 2024-04-25T15:49:50.741 INFO:tasks.workunit.client.1.smithi193.stdout:4/885: write db/d32/f5f [2800349,82106] 0 2024-04-25T15:49:50.743 INFO:tasks.workunit.client.0.smithi110.stdout:0/903: rename d1/d23/d30/d33/f4b to d1/d47/d113/f149 0 2024-04-25T15:49:50.744 INFO:tasks.workunit.client.0.smithi110.stdout:2/881: dwrite de/d40/d6e/d52/d6d/fb3 [0,4194304] 0 2024-04-25T15:49:50.744 INFO:tasks.workunit.client.0.smithi110.stdout:0/904: fdatasync d1/d47/f61 0 2024-04-25T15:49:50.748 INFO:tasks.workunit.client.1.smithi193.stdout:5/822: mkdir d17/d18/d30/d38/d71/d96/dc4/da7/d115 0 2024-04-25T15:49:50.749 INFO:tasks.workunit.client.0.smithi110.stdout:9/886: symlink d9/d44/d5e/d80/dbb/l114 0 2024-04-25T15:49:50.749 INFO:tasks.workunit.client.1.smithi193.stdout:2/787: truncate d0/d37/d4d/fad 397392 0 2024-04-25T15:49:50.749 INFO:tasks.workunit.client.0.smithi110.stdout:9/887: read d9/d44/fab [250635,62775] 0 2024-04-25T15:49:50.751 INFO:tasks.workunit.client.1.smithi193.stdout:2/788: chown d0/d37/d25/d5f 227 1 2024-04-25T15:49:50.751 INFO:tasks.workunit.client.0.smithi110.stdout:5/960: mkdir d4/d1d/d1f/d13b 0 2024-04-25T15:49:50.752 INFO:tasks.workunit.client.1.smithi193.stdout:2/789: chown d0/d42/d51/df1/ce5 122231 1 2024-04-25T15:49:50.753 INFO:tasks.workunit.client.1.smithi193.stdout:8/825: unlink d2/d3/d27/d69/dbf/c55 0 2024-04-25T15:49:50.753 INFO:tasks.workunit.client.0.smithi110.stdout:3/958: mknod d0/d2e/dfc/d10f/c142 0 2024-04-25T15:49:50.755 INFO:tasks.workunit.client.1.smithi193.stdout:4/886: dread db/d13/dcb/fdb [0,4194304] 0 2024-04-25T15:49:50.755 INFO:tasks.workunit.client.1.smithi193.stdout:8/826: read d2/d3/d27/d69/f9a [480318,39996] 0 2024-04-25T15:49:50.756 INFO:tasks.workunit.client.1.smithi193.stdout:4/887: truncate db/d13/d9d/f11c 765529 0 2024-04-25T15:49:50.756 INFO:tasks.workunit.client.0.smithi110.stdout:2/882: symlink de/d40/d70/dc6/l119 0 2024-04-25T15:49:50.759 INFO:tasks.workunit.client.1.smithi193.stdout:3/832: rename d8/d13/d1f/d69/d77/cce to d8/d13/d4b/d8b/d9a/c102 0 2024-04-25T15:49:50.760 INFO:tasks.workunit.client.0.smithi110.stdout:6/925: mkdir d1/dd/d3c/d91/d8d/dce/d102/d141 0 2024-04-25T15:49:50.764 INFO:tasks.workunit.client.1.smithi193.stdout:6/809: link dd/d12/d19/d1c/c62 dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/cf2 0 2024-04-25T15:49:50.766 INFO:tasks.workunit.client.0.smithi110.stdout:3/959: mknod d0/d16/dcb/db0/c143 0 2024-04-25T15:49:50.770 INFO:tasks.workunit.client.0.smithi110.stdout:4/937: link d13/dba/l134 d13/dba/dd8/l143 0 2024-04-25T15:49:50.772 INFO:tasks.workunit.client.0.smithi110.stdout:3/960: dwrite d0/d7/f68 [0,4194304] 0 2024-04-25T15:49:50.773 INFO:tasks.workunit.client.0.smithi110.stdout:6/926: fdatasync d1/db/d22/d33/de4/fe7 0 2024-04-25T15:49:50.778 INFO:tasks.workunit.client.0.smithi110.stdout:4/938: dwrite d13/d24/da6/f130 [0,4194304] 0 2024-04-25T15:49:50.782 INFO:tasks.workunit.client.1.smithi193.stdout:8/827: dwrite d2/d18/f107 [0,4194304] 0 2024-04-25T15:49:50.787 INFO:tasks.workunit.client.0.smithi110.stdout:2/883: symlink de/d40/d6e/d52/d6d/l11a 0 2024-04-25T15:49:50.790 INFO:tasks.workunit.client.1.smithi193.stdout:3/833: creat d8/d30/d45/f103 x:0 0 0 2024-04-25T15:49:50.791 INFO:tasks.workunit.client.0.smithi110.stdout:0/905: dwrite d1/d23/d30/d33/d5b/d78/d107/fc1 [0,4194304] 0 2024-04-25T15:49:50.793 INFO:tasks.workunit.client.1.smithi193.stdout:5/823: dwrite d17/d18/f21 [0,4194304] 0 2024-04-25T15:49:50.794 INFO:tasks.workunit.client.1.smithi193.stdout:2/790: truncate d0/d37/dbb/df8/f95 4067223 0 2024-04-25T15:49:50.799 INFO:tasks.workunit.client.1.smithi193.stdout:0/822: rename d0/d49/d5/d7/l9d to d0/d49/d5/d7/l11c 0 2024-04-25T15:49:50.801 INFO:tasks.workunit.client.0.smithi110.stdout:5/961: rename d4/d1d/d1f/d35/d7f to d4/d15/d13c 0 2024-04-25T15:49:50.801 INFO:tasks.workunit.client.1.smithi193.stdout:1/733: link dc/l13 dc/d4a/d50/d5c/d63/lf5 0 2024-04-25T15:49:50.805 INFO:tasks.workunit.client.0.smithi110.stdout:2/884: mknod de/d40/d6e/d50/db7/c11b 0 2024-04-25T15:49:50.806 INFO:tasks.workunit.client.0.smithi110.stdout:2/885: read - de/d40/d4f/d9f/f100 zero size 2024-04-25T15:49:50.807 INFO:tasks.workunit.client.0.smithi110.stdout:3/961: symlink d0/d16/dcb/d2f/d4c/db2/l144 0 2024-04-25T15:49:50.807 INFO:tasks.workunit.client.0.smithi110.stdout:3/962: chown d0/d2e/f39 678581 1 2024-04-25T15:49:50.809 INFO:tasks.workunit.client.0.smithi110.stdout:9/888: getdents d9/d13/d5b/d61/db3 0 2024-04-25T15:49:50.809 INFO:tasks.workunit.client.1.smithi193.stdout:8/828: mkdir d2/d3/d20/d8c/dde/dbb/d10d 0 2024-04-25T15:49:50.809 INFO:tasks.workunit.client.1.smithi193.stdout:9/784: getdents df/d2c/d2f/d5c 0 2024-04-25T15:49:50.809 INFO:tasks.workunit.client.1.smithi193.stdout:4/888: creat db/d13/d1e/d1f/d56/d57/df2/f121 x:0 0 0 2024-04-25T15:49:50.810 INFO:tasks.workunit.client.1.smithi193.stdout:4/889: chown db/d32/c39 0 1 2024-04-25T15:49:50.811 INFO:tasks.workunit.client.1.smithi193.stdout:6/810: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/de8/lf3 0 2024-04-25T15:49:50.812 INFO:tasks.workunit.client.1.smithi193.stdout:9/785: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d8b/c90 30159195 1 2024-04-25T15:49:50.813 INFO:tasks.workunit.client.0.smithi110.stdout:0/906: rename d1/d23/d30/d33/d129 to d1/d23/d30/d33/d35/d14a 0 2024-04-25T15:49:50.814 INFO:tasks.workunit.client.1.smithi193.stdout:3/834: fdatasync d8/d13/d14/da0/fb5 0 2024-04-25T15:49:50.816 INFO:tasks.workunit.client.1.smithi193.stdout:5/824: creat d17/d18/d30/d38/d71/dd7/f116 x:0 0 0 2024-04-25T15:49:50.817 INFO:tasks.workunit.client.0.smithi110.stdout:6/927: creat d1/db/d24/d2d/f142 x:0 0 0 2024-04-25T15:49:50.818 INFO:tasks.workunit.client.1.smithi193.stdout:2/791: chown d0/d37/d25/d5f/d98/fec 407433 1 2024-04-25T15:49:50.819 INFO:tasks.workunit.client.0.smithi110.stdout:5/962: dread d4/d1d/d1f/f8f [0,4194304] 0 2024-04-25T15:49:50.819 INFO:tasks.workunit.client.0.smithi110.stdout:3/963: unlink d0/d7/d7d/l9c 0 2024-04-25T15:49:50.819 INFO:tasks.workunit.client.0.smithi110.stdout:6/928: dread - d1/db/d24/d2d/d11a/f130 zero size 2024-04-25T15:49:50.822 INFO:tasks.workunit.client.1.smithi193.stdout:2/792: dread d0/d42/f61 [0,4194304] 0 2024-04-25T15:49:50.822 INFO:tasks.workunit.client.0.smithi110.stdout:0/907: dwrite d1/d23/d30/d33/d5b/d78/ddb/ff8 [0,4194304] 0 2024-04-25T15:49:50.823 INFO:tasks.workunit.client.0.smithi110.stdout:6/929: write d1/db/d22/d90/d9e/f9f [1863599,130079] 0 2024-04-25T15:49:50.824 INFO:tasks.workunit.client.1.smithi193.stdout:2/793: write d0/d37/d25/d5f/d3a/d6d/d6e/db9/fe8 [4537322,112907] 0 2024-04-25T15:49:50.828 INFO:tasks.workunit.client.0.smithi110.stdout:2/886: rename de/d40/d5b to de/d40/d70/d11c 0 2024-04-25T15:49:50.834 INFO:tasks.workunit.client.1.smithi193.stdout:1/734: dread dc/f3f [0,4194304] 0 2024-04-25T15:49:50.836 INFO:tasks.workunit.client.0.smithi110.stdout:3/964: fdatasync d0/d16/dcb/d2f/d33/f9e 0 2024-04-25T15:49:50.842 INFO:tasks.workunit.client.0.smithi110.stdout:4/939: write d13/d24/d26/dcd/d131/f10b [2892919,48941] 0 2024-04-25T15:49:50.845 INFO:tasks.workunit.client.0.smithi110.stdout:0/908: rename d1/d23/d30/d33/d5b/d78/ddb/de7/c123 to d1/d23/d30/d33/d5b/d78/d107/dee/c14b 0 2024-04-25T15:49:50.847 INFO:tasks.workunit.client.0.smithi110.stdout:2/887: creat de/d40/d70/d11c/d5c/d5f/db2/dad/f11d x:0 0 0 2024-04-25T15:49:50.851 INFO:tasks.workunit.client.0.smithi110.stdout:9/889: write d9/db/df/d25/f37 [8657465,123379] 0 2024-04-25T15:49:50.851 INFO:tasks.workunit.client.0.smithi110.stdout:3/965: mkdir d0/d2e/dfc/d10f/d145 0 2024-04-25T15:49:50.856 INFO:tasks.workunit.client.0.smithi110.stdout:4/940: truncate d13/d24/d26/dcd/d131/fcf 111133 0 2024-04-25T15:49:50.859 INFO:tasks.workunit.client.0.smithi110.stdout:0/909: creat d1/d23/dc5/f14c x:0 0 0 2024-04-25T15:49:50.860 INFO:tasks.workunit.client.0.smithi110.stdout:5/963: dwrite d4/f10 [0,4194304] 0 2024-04-25T15:49:50.863 INFO:tasks.workunit.client.0.smithi110.stdout:9/890: mknod d9/db/df/d4d/c115 0 2024-04-25T15:49:50.864 INFO:tasks.workunit.client.0.smithi110.stdout:6/930: write d1/dd/d3c/d91/d8d/f101 [486685,93140] 0 2024-04-25T15:49:50.865 INFO:tasks.workunit.client.0.smithi110.stdout:4/941: mkdir d13/dba/dd8/d144 0 2024-04-25T15:49:50.866 INFO:tasks.workunit.client.0.smithi110.stdout:6/931: truncate d1/db/d24/d2d/fba 1435314 0 2024-04-25T15:49:50.868 INFO:tasks.workunit.client.0.smithi110.stdout:2/888: rename de/d40/d6e/d50/l60 to de/d40/d70/l11e 0 2024-04-25T15:49:50.870 INFO:tasks.workunit.client.0.smithi110.stdout:0/910: symlink d1/d23/d30/d33/d5b/l14d 0 2024-04-25T15:49:50.872 INFO:tasks.workunit.client.0.smithi110.stdout:4/942: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d129/d145 0 2024-04-25T15:49:50.882 INFO:tasks.workunit.client.1.smithi193.stdout:0/823: sync 2024-04-25T15:49:50.898 INFO:tasks.workunit.client.1.smithi193.stdout:5/825: creat d17/d53/f117 x:0 0 0 2024-04-25T15:49:50.903 INFO:tasks.workunit.client.1.smithi193.stdout:2/794: creat d0/d37/d25/d5f/d3a/d6d/d85/f126 x:0 0 0 2024-04-25T15:49:50.907 INFO:tasks.workunit.client.1.smithi193.stdout:7/822: rename d12/d13/f71 to d12/d31/d39/f12a 0 2024-04-25T15:49:50.907 INFO:tasks.workunit.client.1.smithi193.stdout:8/829: truncate d2/d3/d3a/f5e 259431 0 2024-04-25T15:49:50.909 INFO:tasks.workunit.client.1.smithi193.stdout:9/786: symlink df/d2c/lf0 0 2024-04-25T15:49:50.913 INFO:tasks.workunit.client.1.smithi193.stdout:0/824: symlink d0/d49/l11d 0 2024-04-25T15:49:50.915 INFO:tasks.workunit.client.1.smithi193.stdout:8/830: fsync d2/f48 0 2024-04-25T15:49:50.918 INFO:tasks.workunit.client.1.smithi193.stdout:5/826: fdatasync d17/d18/d30/d34/f46 0 2024-04-25T15:49:50.919 INFO:tasks.workunit.client.1.smithi193.stdout:5/827: readlink d17/lea 0 2024-04-25T15:49:50.920 INFO:tasks.workunit.client.1.smithi193.stdout:7/823: dread f2 [0,4194304] 0 2024-04-25T15:49:50.920 INFO:tasks.workunit.client.1.smithi193.stdout:9/787: truncate df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/f84 228436 0 2024-04-25T15:49:50.921 INFO:tasks.workunit.client.0.smithi110.stdout:6/932: symlink d1/d12/db8/l143 0 2024-04-25T15:49:50.922 INFO:tasks.workunit.client.1.smithi193.stdout:0/825: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf/d11e 0 2024-04-25T15:49:50.922 INFO:tasks.workunit.client.0.smithi110.stdout:3/966: rename d0/d7/l48 to d0/d2e/d88/dbf/d99/l146 0 2024-04-25T15:49:50.922 INFO:tasks.workunit.client.1.smithi193.stdout:0/826: fdatasync d0/d49/d5/d7/dd/d16/d26/d5e/fc8 0 2024-04-25T15:49:50.923 INFO:tasks.workunit.client.0.smithi110.stdout:0/911: creat d1/d23/d30/d33/d5b/d78/da4/f14e x:0 0 0 2024-04-25T15:49:50.923 INFO:tasks.workunit.client.0.smithi110.stdout:6/933: chown d1/d48/c70 846065928 1 2024-04-25T15:49:50.925 INFO:tasks.workunit.client.1.smithi193.stdout:6/811: getdents dd/d12/d2d 0 2024-04-25T15:49:50.926 INFO:tasks.workunit.client.0.smithi110.stdout:2/889: creat de/d40/dd7/df4/f11f x:0 0 0 2024-04-25T15:49:50.926 INFO:tasks.workunit.client.1.smithi193.stdout:2/795: dread d0/d37/d25/d5f/d3a/f58 [0,4194304] 0 2024-04-25T15:49:50.928 INFO:tasks.workunit.client.1.smithi193.stdout:0/827: dwrite d0/d49/d5/d7/dd/d16/d18/f22 [0,4194304] 0 2024-04-25T15:49:50.930 INFO:tasks.workunit.client.1.smithi193.stdout:5/828: mkdir d17/d18/d30/d34/d118 0 2024-04-25T15:49:50.932 INFO:tasks.workunit.client.1.smithi193.stdout:9/788: read df/d2c/d2f/d4e/d1c/d30/f7a [1815113,39159] 0 2024-04-25T15:49:50.933 INFO:tasks.workunit.client.1.smithi193.stdout:9/789: write df/d2c/d34/faa [3475302,130952] 0 2024-04-25T15:49:50.942 INFO:tasks.workunit.client.1.smithi193.stdout:6/812: rename dd/fe to dd/d12/d19/d1c/d40/dda/ff4 0 2024-04-25T15:49:50.943 INFO:tasks.workunit.client.1.smithi193.stdout:2/796: unlink d0/d42/d51/d4f/f69 0 2024-04-25T15:49:50.943 INFO:tasks.workunit.client.1.smithi193.stdout:2/797: chown d0/d37/d25/fa9 14711 1 2024-04-25T15:49:50.946 INFO:tasks.workunit.client.1.smithi193.stdout:7/824: link d12/d31/d37/d104/f125 d12/d31/d37/d104/d11a/f12b 0 2024-04-25T15:49:50.946 INFO:tasks.workunit.client.1.smithi193.stdout:4/890: write db/d13/d1e/d1f/d56/d57/f9b [1223799,33976] 0 2024-04-25T15:49:50.947 INFO:tasks.workunit.client.1.smithi193.stdout:9/790: rmdir df/d2c/d34 39 2024-04-25T15:49:50.948 INFO:tasks.workunit.client.1.smithi193.stdout:3/835: write d8/d30/d45/fc3 [788449,129469] 0 2024-04-25T15:49:50.950 INFO:tasks.workunit.client.1.smithi193.stdout:6/813: unlink dd/d12/d2d/cb8 0 2024-04-25T15:49:50.950 INFO:tasks.workunit.client.1.smithi193.stdout:6/814: chown dd/l14 57 1 2024-04-25T15:49:50.953 INFO:tasks.workunit.client.1.smithi193.stdout:5/829: mkdir d17/d18/d30/d119 0 2024-04-25T15:49:50.954 INFO:tasks.workunit.client.1.smithi193.stdout:1/735: write dc/f1b [2887144,96797] 0 2024-04-25T15:49:50.954 INFO:tasks.workunit.client.1.smithi193.stdout:5/830: fsync d17/d18/f84 0 2024-04-25T15:49:50.959 INFO:tasks.workunit.client.1.smithi193.stdout:7/825: getdents d12/d31/d37/d62/d6e/df2 0 2024-04-25T15:49:50.961 INFO:tasks.workunit.client.1.smithi193.stdout:1/736: rename dc/d4a/d4d/d7a/d85/c9f to dc/d4a/d4d/d80/cf6 0 2024-04-25T15:49:50.962 INFO:tasks.workunit.client.1.smithi193.stdout:9/791: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/deb/ff1 x:0 0 0 2024-04-25T15:49:50.964 INFO:tasks.workunit.client.1.smithi193.stdout:9/792: read df/d2c/d2f/fc4 [628864,44501] 0 2024-04-25T15:49:50.964 INFO:tasks.workunit.client.1.smithi193.stdout:6/815: creat dd/d12/d19/d1c/d40/d3c/d3f/de0/ff5 x:0 0 0 2024-04-25T15:49:50.964 INFO:tasks.workunit.client.1.smithi193.stdout:9/793: chown df/d2c/d2f/f77 1295846645 1 2024-04-25T15:49:50.965 INFO:tasks.workunit.client.1.smithi193.stdout:2/798: creat d0/d9/d15/d31/f127 x:0 0 0 2024-04-25T15:49:50.966 INFO:tasks.workunit.client.1.smithi193.stdout:2/799: dread - d0/d9/d15/df3/f120 zero size 2024-04-25T15:49:50.967 INFO:tasks.workunit.client.1.smithi193.stdout:2/800: fdatasync d0/d37/d25/d5f/d3a/d6d/d6e/db9/fe8 0 2024-04-25T15:49:50.969 INFO:tasks.workunit.client.1.smithi193.stdout:2/801: read d0/d42/d51/f40 [866995,80954] 0 2024-04-25T15:49:50.971 INFO:tasks.workunit.client.1.smithi193.stdout:6/816: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dc1/cf6 0 2024-04-25T15:49:50.972 INFO:tasks.workunit.client.1.smithi193.stdout:5/831: link d17/d18/d3e/db6/l112 d17/d18/d30/d34/d118/l11a 0 2024-04-25T15:49:50.973 INFO:tasks.workunit.client.1.smithi193.stdout:8/831: truncate d2/d3/d20/db8/dd0/ff2 2924524 0 2024-04-25T15:49:50.975 INFO:tasks.workunit.client.1.smithi193.stdout:2/802: dread d0/d37/d25/f77 [0,4194304] 0 2024-04-25T15:49:50.975 INFO:tasks.workunit.client.1.smithi193.stdout:1/737: creat dc/d4a/d4d/d7a/dd0/dd2/ff7 x:0 0 0 2024-04-25T15:49:50.976 INFO:tasks.workunit.client.1.smithi193.stdout:1/738: read - dc/d59/dc0/deb/fcd zero size 2024-04-25T15:49:50.977 INFO:tasks.workunit.client.1.smithi193.stdout:2/803: truncate d0/df/dbf/f100 433029 0 2024-04-25T15:49:50.981 INFO:tasks.workunit.client.1.smithi193.stdout:5/832: rename d17/d18/d75/d85/c7a to d17/d18/d30/d38/d48/d4f/c11b 0 2024-04-25T15:49:50.981 INFO:tasks.workunit.client.1.smithi193.stdout:8/832: truncate d2/d3/d27/ff6 479579 0 2024-04-25T15:49:50.982 INFO:tasks.workunit.client.1.smithi193.stdout:8/833: chown d2/da5/ff0 65657763 1 2024-04-25T15:49:50.988 INFO:tasks.workunit.client.1.smithi193.stdout:8/834: read d2/d3/d27/d69/dbf/f40 [6725318,87754] 0 2024-04-25T15:49:50.988 INFO:tasks.workunit.client.1.smithi193.stdout:0/828: dwrite d0/d49/d5/d7/dd/d46/d48/d62/f8f [0,4194304] 0 2024-04-25T15:49:50.990 INFO:tasks.workunit.client.1.smithi193.stdout:3/836: dwrite d8/d30/f36 [0,4194304] 0 2024-04-25T15:49:51.000 INFO:tasks.workunit.client.1.smithi193.stdout:2/804: write d0/d9/d15/f35 [94072,27757] 0 2024-04-25T15:49:51.001 INFO:tasks.workunit.client.1.smithi193.stdout:1/739: truncate dc/d59/f8e 1051489 0 2024-04-25T15:49:51.002 INFO:tasks.workunit.client.1.smithi193.stdout:3/837: write d8/d13/d4b/d8b/de3/fee [28400,21034] 0 2024-04-25T15:49:51.003 INFO:tasks.workunit.client.1.smithi193.stdout:7/826: dwrite d12/d1c/faf [0,4194304] 0 2024-04-25T15:49:51.003 INFO:tasks.workunit.client.1.smithi193.stdout:5/833: truncate d17/d18/d30/d38/d71/d96/dc4/f70 1119166 0 2024-04-25T15:49:51.005 INFO:tasks.workunit.client.1.smithi193.stdout:9/794: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/f94 [0,4194304] 0 2024-04-25T15:49:51.010 INFO:tasks.workunit.client.1.smithi193.stdout:0/829: unlink d0/d49/d5/d7/dd/d16/d18/d28/c10d 0 2024-04-25T15:49:51.013 INFO:tasks.workunit.client.1.smithi193.stdout:2/805: symlink d0/d42/d51/df1/l128 0 2024-04-25T15:49:51.014 INFO:tasks.workunit.client.1.smithi193.stdout:4/891: sync 2024-04-25T15:49:51.015 INFO:tasks.workunit.client.1.smithi193.stdout:1/740: creat dc/d4a/d50/dab/ff8 x:0 0 0 2024-04-25T15:49:51.015 INFO:tasks.workunit.client.1.smithi193.stdout:5/834: chown d17/d18/d3e/dd1/ddc/c10e 18 1 2024-04-25T15:49:51.017 INFO:tasks.workunit.client.1.smithi193.stdout:6/817: dwrite dd/d12/d19/f43 [0,4194304] 0 2024-04-25T15:49:51.019 INFO:tasks.workunit.client.1.smithi193.stdout:3/838: chown d8/d13/d14/l3d 2377 1 2024-04-25T15:49:51.019 INFO:tasks.workunit.client.1.smithi193.stdout:0/830: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/lb7 155 1 2024-04-25T15:49:51.019 INFO:tasks.workunit.client.1.smithi193.stdout:4/892: readlink db/d13/l19 0 2024-04-25T15:49:51.021 INFO:tasks.workunit.client.1.smithi193.stdout:7/827: symlink d12/d13/dcc/df4/l12c 0 2024-04-25T15:49:51.022 INFO:tasks.workunit.client.1.smithi193.stdout:6/818: dread - dd/d12/d19/fc4 zero size 2024-04-25T15:49:51.023 INFO:tasks.workunit.client.1.smithi193.stdout:5/835: chown d17/d18/d30/d38/d71/d96/dc4/ccc 106569162 1 2024-04-25T15:49:51.023 INFO:tasks.workunit.client.1.smithi193.stdout:8/835: dwrite d2/d3/d27/d69/dae/d29/f60 [0,4194304] 0 2024-04-25T15:49:51.025 INFO:tasks.workunit.client.1.smithi193.stdout:2/806: creat d0/d42/d51/d4f/d65/d6a/f129 x:0 0 0 2024-04-25T15:49:51.027 INFO:tasks.workunit.client.1.smithi193.stdout:4/893: fdatasync db/d13/d1e/d1f/d22/f7a 0 2024-04-25T15:49:51.032 INFO:tasks.workunit.client.1.smithi193.stdout:8/836: rmdir d2/d3 39 2024-04-25T15:49:51.034 INFO:tasks.workunit.client.1.smithi193.stdout:5/836: write d17/d18/d30/d38/d48/d4f/deb/f113 [333564,112251] 0 2024-04-25T15:49:51.034 INFO:tasks.workunit.client.1.smithi193.stdout:0/831: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d93/l11f 0 2024-04-25T15:49:51.034 INFO:tasks.workunit.client.1.smithi193.stdout:6/819: truncate dd/f91 621617 0 2024-04-25T15:49:51.035 INFO:tasks.workunit.client.1.smithi193.stdout:9/795: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/f59 [0,4194304] 0 2024-04-25T15:49:51.037 INFO:tasks.workunit.client.1.smithi193.stdout:1/741: dwrite dc/d4a/d50/d7c/d7e/f8c [0,4194304] 0 2024-04-25T15:49:51.037 INFO:tasks.workunit.client.1.smithi193.stdout:7/828: mkdir d12/d13/dc4/dfa/dff/d12d 0 2024-04-25T15:49:51.037 INFO:tasks.workunit.client.1.smithi193.stdout:4/894: write db/d13/d1e/d1f/d56/d69/d70/f9c [148924,108633] 0 2024-04-25T15:49:51.040 INFO:tasks.workunit.client.1.smithi193.stdout:9/796: readlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/l72 0 2024-04-25T15:49:51.057 INFO:tasks.workunit.client.1.smithi193.stdout:4/895: chown db/d32/d45/l83 2371 1 2024-04-25T15:49:51.057 INFO:tasks.workunit.client.1.smithi193.stdout:0/832: fsync d0/d49/d5/d7/dd/d16/d26/d5e/d99/f9a 0 2024-04-25T15:49:51.057 INFO:tasks.workunit.client.1.smithi193.stdout:1/742: chown dc/d4a/d50/d5c/d5d/ddd 30323 1 2024-04-25T15:49:51.058 INFO:tasks.workunit.client.1.smithi193.stdout:7/829: mkdir d12/d1c/d96/d12e 0 2024-04-25T15:49:51.058 INFO:tasks.workunit.client.1.smithi193.stdout:5/837: creat d17/d18/d30/d34/f11c x:0 0 0 2024-04-25T15:49:51.059 INFO:tasks.workunit.client.1.smithi193.stdout:8/837: fsync d2/d3/d20/db8/dd0/fe3 0 2024-04-25T15:49:51.060 INFO:tasks.workunit.client.1.smithi193.stdout:3/839: truncate d8/d13/d14/d51/f6c 3837310 0 2024-04-25T15:49:51.062 INFO:tasks.workunit.client.1.smithi193.stdout:7/830: readlink d12/d1c/l51 0 2024-04-25T15:49:51.062 INFO:tasks.workunit.client.1.smithi193.stdout:1/743: creat dc/d4a/ff9 x:0 0 0 2024-04-25T15:49:51.062 INFO:tasks.workunit.client.1.smithi193.stdout:2/807: dwrite d0/d42/d51/df1/d9f/dc5/ff5 [0,4194304] 0 2024-04-25T15:49:51.062 INFO:tasks.workunit.client.1.smithi193.stdout:4/896: rmdir db/d32/d45 39 2024-04-25T15:49:51.068 INFO:tasks.workunit.client.1.smithi193.stdout:2/808: chown d0/d37/d25/d5f/d3a/d6d/d6e/d8e/fe6 47644 1 2024-04-25T15:49:51.068 INFO:tasks.workunit.client.1.smithi193.stdout:0/833: mkdir d0/d49/d5/d7/dd/d16/d120 0 2024-04-25T15:49:51.069 INFO:tasks.workunit.client.1.smithi193.stdout:5/838: dread d17/d53/fc1 [0,4194304] 0 2024-04-25T15:49:51.071 INFO:tasks.workunit.client.1.smithi193.stdout:1/744: creat dc/d4a/d50/d7c/d82/ffa x:0 0 0 2024-04-25T15:49:51.073 INFO:tasks.workunit.client.1.smithi193.stdout:0/834: stat d0/d49/d5/d7/dd/d46/d48/d62 0 2024-04-25T15:49:51.073 INFO:tasks.workunit.client.0.smithi110.stdout:5/964: rename d4/d2c/f27 to d4/d15/d51/d90/ded/f13d 0 2024-04-25T15:49:51.073 INFO:tasks.workunit.client.0.smithi110.stdout:3/967: truncate d0/d7/d40/f7e 643352 0 2024-04-25T15:49:51.073 INFO:tasks.workunit.client.1.smithi193.stdout:3/840: truncate d8/d13/d1f/d69/dcd/dd9/fe2 1494640 0 2024-04-25T15:49:51.074 INFO:tasks.workunit.client.1.smithi193.stdout:8/838: rename d2/d3/d27/d69/dbf/f84 to d2/d3/d27/d69/dae/d29/f10e 0 2024-04-25T15:49:51.075 INFO:tasks.workunit.client.1.smithi193.stdout:1/745: write dc/d59/dc0/deb/d55/fe0 [893870,116837] 0 2024-04-25T15:49:51.075 INFO:tasks.workunit.client.1.smithi193.stdout:4/897: dread db/d13/d1e/d1f/d56/d69/d70/f76 [0,4194304] 0 2024-04-25T15:49:51.076 INFO:tasks.workunit.client.0.smithi110.stdout:4/943: mkdir d13/d24/d2d/d44/d65/d99/d146 0 2024-04-25T15:49:51.076 INFO:tasks.workunit.client.0.smithi110.stdout:6/934: dwrite d1/db/d22/d33/d63/fd4 [0,4194304] 0 2024-04-25T15:49:51.077 INFO:tasks.workunit.client.0.smithi110.stdout:5/965: chown d4/d1d/d1f/d35/d3d/d93 23275821 1 2024-04-25T15:49:51.078 INFO:tasks.workunit.client.1.smithi193.stdout:4/898: write db/d13/d1e/d1f/f64 [2054767,99065] 0 2024-04-25T15:49:51.079 INFO:tasks.workunit.client.1.smithi193.stdout:0/835: read d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f59 [2160216,62152] 0 2024-04-25T15:49:51.080 INFO:tasks.workunit.client.1.smithi193.stdout:4/899: dread db/d13/d9d/f11c [0,4194304] 0 2024-04-25T15:49:51.080 INFO:tasks.workunit.client.1.smithi193.stdout:3/841: read d8/d13/d1f/dbf/f8c [1685486,107066] 0 2024-04-25T15:49:51.082 INFO:tasks.workunit.client.0.smithi110.stdout:0/912: dwrite d1/d23/f42 [0,4194304] 0 2024-04-25T15:49:51.083 INFO:tasks.workunit.client.0.smithi110.stdout:0/913: chown d1/d6c/d116/d77 1758888 1 2024-04-25T15:49:51.086 INFO:tasks.workunit.client.0.smithi110.stdout:5/966: read d4/d15/d51/d90/ded/d106/f136 [30516,55276] 0 2024-04-25T15:49:51.086 INFO:tasks.workunit.client.0.smithi110.stdout:3/968: mknod d0/d16/dcb/d2f/d4c/d63/c147 0 2024-04-25T15:49:51.087 INFO:tasks.workunit.client.0.smithi110.stdout:2/890: creat de/d6f/f120 x:0 0 0 2024-04-25T15:49:51.088 INFO:tasks.workunit.client.0.smithi110.stdout:5/967: chown d4/d1d/l9d 1 1 2024-04-25T15:49:51.089 INFO:tasks.workunit.client.0.smithi110.stdout:6/935: mknod d1/dd/d3c/d91/d8d/dce/d102/d75/d82/c144 0 2024-04-25T15:49:51.089 INFO:tasks.workunit.client.0.smithi110.stdout:3/969: stat d0/d2e/d88/dbf/d99/dad/daf 0 2024-04-25T15:49:51.092 INFO:tasks.workunit.client.0.smithi110.stdout:2/891: dread de/d40/d6e/f9d [0,4194304] 0 2024-04-25T15:49:51.093 INFO:tasks.workunit.client.0.smithi110.stdout:4/944: rename d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcb to d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/ded/d129/d142/d147 0 2024-04-25T15:49:51.094 INFO:tasks.workunit.client.0.smithi110.stdout:5/968: read - d4/d1d/d1f/f94 zero size 2024-04-25T15:49:51.095 INFO:tasks.workunit.client.0.smithi110.stdout:2/892: dread de/d40/d70/d11c/d5c/dbe/f103 [0,4194304] 0 2024-04-25T15:49:51.096 INFO:tasks.workunit.client.0.smithi110.stdout:0/914: fdatasync d1/d47/d54/f9c 0 2024-04-25T15:49:51.097 INFO:tasks.workunit.client.0.smithi110.stdout:3/970: mkdir d0/d2e/d88/dbf/da5/d148 0 2024-04-25T15:49:51.102 INFO:tasks.workunit.client.0.smithi110.stdout:3/971: dread - d0/d7/dd0/d10b/f118 zero size 2024-04-25T15:49:51.103 INFO:tasks.workunit.client.0.smithi110.stdout:0/915: dwrite d1/d20/f73 [4194304,4194304] 0 2024-04-25T15:49:51.105 INFO:tasks.workunit.client.0.smithi110.stdout:2/893: getdents de/d40/d6e/d52/d6d/d7d/dd4 0 2024-04-25T15:49:51.106 INFO:tasks.workunit.client.0.smithi110.stdout:2/894: chown de/d40/c109 1 1 2024-04-25T15:49:51.107 INFO:tasks.workunit.client.0.smithi110.stdout:5/969: creat d4/d1d/d1f/d35/f13e x:0 0 0 2024-04-25T15:49:51.109 INFO:tasks.workunit.client.0.smithi110.stdout:0/916: symlink d1/d23/d30/d33/d5b/d83/l14f 0 2024-04-25T15:49:51.112 INFO:tasks.workunit.client.1.smithi193.stdout:5/839: rename d17/f57 to d17/d18/d30/d38/d71/d96/f11d 0 2024-04-25T15:49:51.112 INFO:tasks.workunit.client.1.smithi193.stdout:2/809: mkdir d0/d37/d25/d5f/d3a/d12a 0 2024-04-25T15:49:51.114 INFO:tasks.workunit.client.1.smithi193.stdout:4/900: readlink db/d13/d1e/d1f/d22/l31 0 2024-04-25T15:49:51.114 INFO:tasks.workunit.client.1.smithi193.stdout:0/836: readlink d0/d49/d5/d7/dd/d16/lf7 0 2024-04-25T15:49:51.114 INFO:tasks.workunit.client.1.smithi193.stdout:2/810: chown d0/d37/d25/dda 3 1 2024-04-25T15:49:51.115 INFO:tasks.workunit.client.1.smithi193.stdout:7/831: dread d12/d31/fc7 [0,4194304] 0 2024-04-25T15:49:51.115 INFO:tasks.workunit.client.1.smithi193.stdout:3/842: truncate d8/d13/d14/f80 4609849 0 2024-04-25T15:49:51.117 INFO:tasks.workunit.client.1.smithi193.stdout:2/811: write d0/d42/d51/d4f/fa7 [1168176,73787] 0 2024-04-25T15:49:51.117 INFO:tasks.workunit.client.0.smithi110.stdout:2/895: dwrite de/d6f/ff3 [0,4194304] 0 2024-04-25T15:49:51.123 INFO:tasks.workunit.client.1.smithi193.stdout:1/746: mkdir dc/d4a/d50/d5c/d5d/dfb 0 2024-04-25T15:49:51.123 INFO:tasks.workunit.client.0.smithi110.stdout:0/917: dwrite d1/d6c/d116/fd1 [0,4194304] 0 2024-04-25T15:49:51.124 INFO:tasks.workunit.client.1.smithi193.stdout:9/797: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/f43 [0,4194304] 0 2024-04-25T15:49:51.125 INFO:tasks.workunit.client.1.smithi193.stdout:4/901: chown db/d13/d9d/f105 61906391 1 2024-04-25T15:49:51.126 INFO:tasks.workunit.client.0.smithi110.stdout:3/972: dread d0/d16/dcb/d2f/d4c/d63/fc0 [0,4194304] 0 2024-04-25T15:49:51.130 INFO:tasks.workunit.client.0.smithi110.stdout:0/918: chown d1/d47/d54/f6d 71706295 1 2024-04-25T15:49:51.131 INFO:tasks.workunit.client.1.smithi193.stdout:3/843: readlink d8/d13/l90 0 2024-04-25T15:49:51.132 INFO:tasks.workunit.client.1.smithi193.stdout:3/844: readlink d8/d13/d1f/d69/d77/d71/dd2/l100 0 2024-04-25T15:49:51.132 INFO:tasks.workunit.client.1.smithi193.stdout:3/845: chown d8/d13/l1e 9 1 2024-04-25T15:49:51.133 INFO:tasks.workunit.client.1.smithi193.stdout:5/840: rename d17/d18/d9b/fc3 to d17/d18/d30/d34/d52/daf/f11e 0 2024-04-25T15:49:51.134 INFO:tasks.workunit.client.0.smithi110.stdout:2/896: dread de/d40/d70/d11c/d5c/d5f/db2/f95 [0,4194304] 0 2024-04-25T15:49:51.135 INFO:tasks.workunit.client.0.smithi110.stdout:3/973: creat d0/d16/dcb/d2f/d4c/f149 x:0 0 0 2024-04-25T15:49:51.135 INFO:tasks.workunit.client.0.smithi110.stdout:6/936: write d1/db/d22/d33/f56 [184510,16604] 0 2024-04-25T15:49:51.139 INFO:tasks.workunit.client.1.smithi193.stdout:0/837: creat d0/d49/d5/d7/dd/d16/d26/d5e/f121 x:0 0 0 2024-04-25T15:49:51.142 INFO:tasks.workunit.client.1.smithi193.stdout:2/812: unlink d0/l2 0 2024-04-25T15:49:51.144 INFO:tasks.workunit.client.0.smithi110.stdout:3/974: dread d0/d16/dcb/d2f/d4c/db2/fe4 [0,4194304] 0 2024-04-25T15:49:51.144 INFO:tasks.workunit.client.0.smithi110.stdout:2/897: rename de/f116 to de/d40/d70/d11c/d5c/d5f/db2/f121 0 2024-04-25T15:49:51.145 INFO:tasks.workunit.client.1.smithi193.stdout:1/747: dread dc/f47 [0,4194304] 0 2024-04-25T15:49:51.145 INFO:tasks.workunit.client.1.smithi193.stdout:5/841: dwrite d17/d18/d30/d38/d71/d96/dc4/da7/f10f [0,4194304] 0 2024-04-25T15:49:51.146 INFO:tasks.workunit.client.0.smithi110.stdout:4/945: dwrite d13/d24/d2d/d44/fb4 [0,4194304] 0 2024-04-25T15:49:51.146 INFO:tasks.workunit.client.1.smithi193.stdout:1/748: chown dc/f18 500574 1 2024-04-25T15:49:51.147 INFO:tasks.workunit.client.0.smithi110.stdout:5/970: write d4/d1d/d1f/f32 [1484199,125698] 0 2024-04-25T15:49:51.147 INFO:tasks.workunit.client.0.smithi110.stdout:3/975: write d0/d16/dcb/d2f/d4c/fa0 [2112943,118191] 0 2024-04-25T15:49:51.149 INFO:tasks.workunit.client.1.smithi193.stdout:6/820: sync 2024-04-25T15:49:51.149 INFO:tasks.workunit.client.1.smithi193.stdout:1/749: write dc/d4a/d4d/d80/f98 [3392511,42437] 0 2024-04-25T15:49:51.150 INFO:tasks.workunit.client.0.smithi110.stdout:5/971: dread - d4/d1d/d1f/d35/d3d/d93/ff1 zero size 2024-04-25T15:49:51.151 INFO:tasks.workunit.client.1.smithi193.stdout:0/838: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/f34 [0,4194304] 0 2024-04-25T15:49:51.154 INFO:tasks.workunit.client.1.smithi193.stdout:4/902: rename db/d13/l3f to db/d32/de8/l122 0 2024-04-25T15:49:51.155 INFO:tasks.workunit.client.1.smithi193.stdout:0/839: dread - d0/d49/d5/d7/dd/d16/d26/df5/f11b zero size 2024-04-25T15:49:51.156 INFO:tasks.workunit.client.1.smithi193.stdout:4/903: chown db/d32/d40/l5e 10503847 1 2024-04-25T15:49:51.159 INFO:tasks.workunit.client.0.smithi110.stdout:4/946: symlink d13/d24/d26/dcd/l148 0 2024-04-25T15:49:51.161 INFO:tasks.workunit.client.1.smithi193.stdout:3/846: dread d8/d13/d14/d23/de8/fea [0,4194304] 0 2024-04-25T15:49:51.162 INFO:tasks.workunit.client.1.smithi193.stdout:3/847: fsync d8/d30/d45/f103 0 2024-04-25T15:49:51.163 INFO:tasks.workunit.client.0.smithi110.stdout:3/976: mknod d0/d2e/c14a 0 2024-04-25T15:49:51.165 INFO:tasks.workunit.client.1.smithi193.stdout:2/813: symlink d0/d9/d3d/d114/l12b 0 2024-04-25T15:49:51.166 INFO:tasks.workunit.client.0.smithi110.stdout:0/919: write d1/d23/d30/d33/d5b/dbb/da8/fe6 [2012103,77310] 0 2024-04-25T15:49:51.166 INFO:tasks.workunit.client.1.smithi193.stdout:2/814: truncate d0/d9/d15/d31/f127 1017443 0 2024-04-25T15:49:51.169 INFO:tasks.workunit.client.0.smithi110.stdout:6/937: link d1/dd/d3c/d91/d8d/dce/d102/l129 d1/db/d22/d90/d11d/d134/l145 0 2024-04-25T15:49:51.171 INFO:tasks.workunit.client.0.smithi110.stdout:2/898: link de/d40/d6e/f93 de/d40/dde/f122 0 2024-04-25T15:49:51.172 INFO:tasks.workunit.client.1.smithi193.stdout:9/798: mknod df/d2c/d2f/cf2 0 2024-04-25T15:49:51.174 INFO:tasks.workunit.client.1.smithi193.stdout:8/839: sync 2024-04-25T15:49:51.176 INFO:tasks.workunit.client.1.smithi193.stdout:8/840: chown d2/d3/d27/d69/dae/d4a/l62 3455298 1 2024-04-25T15:49:51.176 INFO:tasks.workunit.client.0.smithi110.stdout:3/977: chown d0/d16/c12a 1896899167 1 2024-04-25T15:49:51.176 INFO:tasks.workunit.client.1.smithi193.stdout:8/841: chown d2/d3/d27/d69/dae/f5a 761 1 2024-04-25T15:49:51.178 INFO:tasks.workunit.client.1.smithi193.stdout:8/842: write d2/d3/d27/d69/dae/f49 [814091,109390] 0 2024-04-25T15:49:51.179 INFO:tasks.workunit.client.1.smithi193.stdout:8/843: read - d2/d3/d27/d69/dae/dd6/fd1 zero size 2024-04-25T15:49:51.180 INFO:tasks.workunit.client.1.smithi193.stdout:4/904: rename db/d4b/f71 to db/d13/ddd/f123 0 2024-04-25T15:49:51.180 INFO:tasks.workunit.client.0.smithi110.stdout:2/899: symlink de/d6f/dbb/l123 0 2024-04-25T15:49:51.182 INFO:tasks.workunit.client.1.smithi193.stdout:0/840: creat d0/d49/d5/d7/dd/d16/d26/d5e/d99/deb/f122 x:0 0 0 2024-04-25T15:49:51.183 INFO:tasks.workunit.client.0.smithi110.stdout:3/978: mkdir d0/d7/dd0/d10b/d14b 0 2024-04-25T15:49:51.185 INFO:tasks.workunit.client.0.smithi110.stdout:5/972: rename d4/d15/d51/d99/c111 to d4/d15/d51/d90/c13f 0 2024-04-25T15:49:51.186 INFO:tasks.workunit.client.1.smithi193.stdout:2/815: symlink d0/l12c 0 2024-04-25T15:49:51.186 INFO:tasks.workunit.client.1.smithi193.stdout:2/816: stat d0/d42/d51/d4f/ff9 0 2024-04-25T15:49:51.188 INFO:tasks.workunit.client.1.smithi193.stdout:9/799: dread df/d2c/d2f/d4e/f1a [0,4194304] 0 2024-04-25T15:49:51.188 INFO:tasks.workunit.client.1.smithi193.stdout:7/832: link d12/d13/c15 d12/d1c/dd6/dda/c12f 0 2024-04-25T15:49:51.190 INFO:tasks.workunit.client.1.smithi193.stdout:1/750: mkdir dc/dfc 0 2024-04-25T15:49:51.191 INFO:tasks.workunit.client.1.smithi193.stdout:9/800: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/fbb [0,4194304] 0 2024-04-25T15:49:51.192 INFO:tasks.workunit.client.0.smithi110.stdout:4/947: creat d13/f149 x:0 0 0 2024-04-25T15:49:51.193 INFO:tasks.workunit.client.0.smithi110.stdout:2/900: symlink de/d40/d70/d11c/l124 0 2024-04-25T15:49:51.194 INFO:tasks.workunit.client.1.smithi193.stdout:6/821: mkdir dd/d12/d19/d1c/d40/dce/de2/df1/df7 0 2024-04-25T15:49:51.197 INFO:tasks.workunit.client.1.smithi193.stdout:9/801: dwrite df/d2c/d2f/d4e/d1c/d30/d60/f73 [0,4194304] 0 2024-04-25T15:49:51.198 INFO:tasks.workunit.client.0.smithi110.stdout:0/920: truncate d1/d47/d54/f74 450009 0 2024-04-25T15:49:51.201 INFO:tasks.workunit.client.0.smithi110.stdout:4/948: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d117/d14a 0 2024-04-25T15:49:51.202 INFO:tasks.workunit.client.0.smithi110.stdout:4/949: chown d13/d24/d26/dcd/d131/fb6 23 1 2024-04-25T15:49:51.204 INFO:tasks.workunit.client.0.smithi110.stdout:4/950: write d13/d24/d2d/d44/fb4 [2023468,64224] 0 2024-04-25T15:49:51.204 INFO:tasks.workunit.client.0.smithi110.stdout:3/979: write d0/d16/f3a [5116306,12838] 0 2024-04-25T15:49:51.206 INFO:tasks.workunit.client.0.smithi110.stdout:3/980: fdatasync d0/d16/dcb/d2f/d4c/fa0 0 2024-04-25T15:49:51.207 INFO:tasks.workunit.client.1.smithi193.stdout:3/848: creat d8/df0/f104 x:0 0 0 2024-04-25T15:49:51.208 INFO:tasks.workunit.client.0.smithi110.stdout:5/973: mknod d4/d15/d51/d90/c140 0 2024-04-25T15:49:51.210 INFO:tasks.workunit.client.1.smithi193.stdout:2/817: mkdir d0/df/da5/d12d 0 2024-04-25T15:49:51.212 INFO:tasks.workunit.client.0.smithi110.stdout:2/901: write de/d40/d6e/feb [1337406,81533] 0 2024-04-25T15:49:51.213 INFO:tasks.workunit.client.1.smithi193.stdout:5/842: creat d17/d18/f11f x:0 0 0 2024-04-25T15:49:51.215 INFO:tasks.workunit.client.1.smithi193.stdout:3/849: dread d8/d30/d45/fbe [0,4194304] 0 2024-04-25T15:49:51.215 INFO:tasks.workunit.client.0.smithi110.stdout:3/981: dwrite d0/d7/dd0/ddf/fe3 [0,4194304] 0 2024-04-25T15:49:51.215 INFO:tasks.workunit.client.0.smithi110.stdout:6/938: getdents d1/dd/d3c/d91/d8d/dce/d102/d75/d82 0 2024-04-25T15:49:51.218 INFO:tasks.workunit.client.1.smithi193.stdout:7/833: mknod d12/d1c/dd6/d101/c130 0 2024-04-25T15:49:51.220 INFO:tasks.workunit.client.1.smithi193.stdout:1/751: rmdir dc/d59/dc0/deb 39 2024-04-25T15:49:51.221 INFO:tasks.workunit.client.1.smithi193.stdout:1/752: fdatasync dc/d4a/d4d/d7a/dd0/dd2/ff7 0 2024-04-25T15:49:51.222 INFO:tasks.workunit.client.0.smithi110.stdout:4/951: mknod d13/d24/d2d/d36/c14b 0 2024-04-25T15:49:51.222 INFO:tasks.workunit.client.1.smithi193.stdout:1/753: truncate dc/d59/fcb 793866 0 2024-04-25T15:49:51.226 INFO:tasks.workunit.client.0.smithi110.stdout:3/982: dread d0/d2e/d88/dbf/fbe [0,4194304] 0 2024-04-25T15:49:51.227 INFO:tasks.workunit.client.0.smithi110.stdout:3/983: chown d0/d7/d40/l42 899969571 1 2024-04-25T15:49:51.227 INFO:tasks.workunit.client.0.smithi110.stdout:9/891: sync 2024-04-25T15:49:51.228 INFO:tasks.workunit.client.0.smithi110.stdout:3/984: chown d0/d2e/d88/dbf/cdd 7351783 1 2024-04-25T15:49:51.229 INFO:tasks.workunit.client.1.smithi193.stdout:6/822: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dc1/lf8 0 2024-04-25T15:49:51.230 INFO:tasks.workunit.client.0.smithi110.stdout:0/921: mkdir d1/d6c/d7f/d79/d150 0 2024-04-25T15:49:51.232 INFO:tasks.workunit.client.1.smithi193.stdout:9/802: mknod df/d2c/d2f/d4e/d1c/cf3 0 2024-04-25T15:49:51.233 INFO:tasks.workunit.client.1.smithi193.stdout:9/803: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/f2e 8524796 1 2024-04-25T15:49:51.233 INFO:tasks.workunit.client.1.smithi193.stdout:9/804: readlink df/l10 0 2024-04-25T15:49:51.234 INFO:tasks.workunit.client.0.smithi110.stdout:5/974: rename d4/d1d/d1f/d35/d3d/d52/d75/dfa/d105/d10a/dd4/la9 to d4/d15/d13c/d92/l141 0 2024-04-25T15:49:51.235 INFO:tasks.workunit.client.1.smithi193.stdout:8/844: symlink d2/d3/d3a/l10f 0 2024-04-25T15:49:51.236 INFO:tasks.workunit.client.0.smithi110.stdout:0/922: dwrite d1/d23/d30/d33/d5b/d83/f84 [0,4194304] 0 2024-04-25T15:49:51.238 INFO:tasks.workunit.client.1.smithi193.stdout:4/905: unlink db/d4b/dea/c107 0 2024-04-25T15:49:51.239 INFO:tasks.workunit.client.1.smithi193.stdout:4/906: chown db/d13/d1e/d1f/d56/fd1 72995 1 2024-04-25T15:49:51.243 INFO:tasks.workunit.client.0.smithi110.stdout:0/923: dread d1/d23/d30/d33/d5b/d78/ddb/ff8 [0,4194304] 0 2024-04-25T15:49:51.245 INFO:tasks.workunit.client.1.smithi193.stdout:5/843: creat d17/d18/d30/d38/f120 x:0 0 0 2024-04-25T15:49:51.246 INFO:tasks.workunit.client.1.smithi193.stdout:7/834: symlink d12/d31/d37/d62/l131 0 2024-04-25T15:49:51.247 INFO:tasks.workunit.client.0.smithi110.stdout:4/952: symlink d13/d24/d2d/db5/dd7/l14c 0 2024-04-25T15:49:51.247 INFO:tasks.workunit.client.1.smithi193.stdout:5/844: dread - d17/d18/d30/d34/d52/daf/f11e zero size 2024-04-25T15:49:51.248 INFO:tasks.workunit.client.1.smithi193.stdout:1/754: fsync dc/f4b 0 2024-04-25T15:49:51.248 INFO:tasks.workunit.client.0.smithi110.stdout:4/953: chown d13/d24/d2d/d44/d65/d99/daf/d56/f76 181 1 2024-04-25T15:49:51.249 INFO:tasks.workunit.client.1.smithi193.stdout:8/845: dread d2/d3/d27/d69/dae/d4a/d63/da2/fa4 [0,4194304] 0 2024-04-25T15:49:51.251 INFO:tasks.workunit.client.0.smithi110.stdout:5/975: creat d4/d1d/d1f/d35/d3d/d52/d75/dfa/d105/f142 x:0 0 0 2024-04-25T15:49:51.256 INFO:tasks.workunit.client.1.smithi193.stdout:0/841: rename d0/fc to d0/d49/d5/d7/dd/d16/f123 0 2024-04-25T15:49:51.257 INFO:tasks.workunit.client.0.smithi110.stdout:0/924: fsync d1/d23/d30/d33/d5b/f9b 0 2024-04-25T15:49:51.258 INFO:tasks.workunit.client.1.smithi193.stdout:0/842: write d0/d49/d5/d7/dd/d16/d26/d2e/f34 [4001608,13903] 0 2024-04-25T15:49:51.260 INFO:tasks.workunit.client.1.smithi193.stdout:0/843: write d0/d49/d5/d7/dd/d16/d26/d9e/f118 [4348526,89986] 0 2024-04-25T15:49:51.260 INFO:tasks.workunit.client.0.smithi110.stdout:9/892: mknod d9/d13/c116 0 2024-04-25T15:49:51.261 INFO:tasks.workunit.client.1.smithi193.stdout:5/845: write d17/d18/d30/d38/d71/d96/f11d [1907004,13207] 0 2024-04-25T15:49:51.262 INFO:tasks.workunit.client.0.smithi110.stdout:3/985: truncate d0/d7/d40/f7e 415792 0 2024-04-25T15:49:51.262 INFO:tasks.workunit.client.1.smithi193.stdout:7/835: dread d12/d1c/d63/d109/fea [0,4194304] 0 2024-04-25T15:49:51.263 INFO:tasks.workunit.client.1.smithi193.stdout:9/805: mkdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/df4 0 2024-04-25T15:49:51.264 INFO:tasks.workunit.client.1.smithi193.stdout:9/806: stat df/d2c/d2f/d4e/d69/d6b/c93 0 2024-04-25T15:49:51.267 INFO:tasks.workunit.client.0.smithi110.stdout:5/976: mkdir d4/d15/d13c/d92/d9b/d143 0 2024-04-25T15:49:51.268 INFO:tasks.workunit.client.1.smithi193.stdout:8/846: truncate d2/d18/db1/fb3 2428242 0 2024-04-25T15:49:51.269 INFO:tasks.workunit.client.1.smithi193.stdout:8/847: stat d2/d3/d27/d69/dae/d4a/d63/da2/de4 0 2024-04-25T15:49:51.272 INFO:tasks.workunit.client.1.smithi193.stdout:3/850: rename d8/d30/d45/dd0/ld8 to d8/d4d/l105 0 2024-04-25T15:49:51.275 INFO:tasks.workunit.client.1.smithi193.stdout:6/823: creat dd/d12/d19/d1c/d40/d3c/ff9 x:0 0 0 2024-04-25T15:49:51.276 INFO:tasks.workunit.client.0.smithi110.stdout:9/893: unlink d9/d44/d5e/la6 0 2024-04-25T15:49:51.278 INFO:tasks.workunit.client.0.smithi110.stdout:3/986: symlink d0/d16/d8c/l14c 0 2024-04-25T15:49:51.279 INFO:tasks.workunit.client.1.smithi193.stdout:5/846: symlink d17/d18/d9b/l121 0 2024-04-25T15:49:51.280 INFO:tasks.workunit.client.1.smithi193.stdout:7/836: creat d12/d13/dc4/dfa/dff/f132 x:0 0 0 2024-04-25T15:49:51.281 INFO:tasks.workunit.client.1.smithi193.stdout:7/837: write d12/d13/dc4/fdb [79250,71844] 0 2024-04-25T15:49:51.281 INFO:tasks.workunit.client.0.smithi110.stdout:5/977: rename d4/d15/d51/l120 to d4/d15/d13c/d92/d9b/dcb/l144 0 2024-04-25T15:49:51.282 INFO:tasks.workunit.client.1.smithi193.stdout:2/818: sync 2024-04-25T15:49:51.282 INFO:tasks.workunit.client.1.smithi193.stdout:0/844: truncate d0/d49/d5/d7/dd/d16/d18/d28/f3e 3199327 0 2024-04-25T15:49:51.283 INFO:tasks.workunit.client.1.smithi193.stdout:0/845: chown d0/fe2 957 1 2024-04-25T15:49:51.284 INFO:tasks.workunit.client.1.smithi193.stdout:2/819: chown d0/d37/d25/d5f/d3a/d6d/d6e/db9/l11a 525753928 1 2024-04-25T15:49:51.285 INFO:tasks.workunit.client.0.smithi110.stdout:0/925: getdents d1/d47/d132 0 2024-04-25T15:49:51.286 INFO:tasks.workunit.client.0.smithi110.stdout:0/926: chown d1/d20/ld0 1086807202 1 2024-04-25T15:49:51.286 INFO:tasks.workunit.client.1.smithi193.stdout:1/755: creat dc/d4a/d50/d5c/d5d/ddd/ffd x:0 0 0 2024-04-25T15:49:51.288 INFO:tasks.workunit.client.0.smithi110.stdout:9/894: mknod d9/d44/d5e/d80/dbb/c117 0 2024-04-25T15:49:51.289 INFO:tasks.workunit.client.0.smithi110.stdout:6/939: dwrite f0 [0,4194304] 0 2024-04-25T15:49:51.292 INFO:tasks.workunit.client.1.smithi193.stdout:2/820: dwrite d0/d37/d25/d5f/d3a/d6d/d6e/f96 [0,4194304] 0 2024-04-25T15:49:51.293 INFO:tasks.workunit.client.1.smithi193.stdout:9/807: dwrite df/d2c/d34/d3e/d4d/d5e/f6c [0,4194304] 0 2024-04-25T15:49:51.294 INFO:tasks.workunit.client.0.smithi110.stdout:4/954: write d13/d24/d2d/d55/ffc [353560,18299] 0 2024-04-25T15:49:51.295 INFO:tasks.workunit.client.1.smithi193.stdout:3/851: creat d8/d13/d14/d51/f106 x:0 0 0 2024-04-25T15:49:51.296 INFO:tasks.workunit.client.0.smithi110.stdout:2/902: link de/d40/d70/d11c/d5c/d5f/db2/dad/ff1 de/d40/d70/d11c/d5c/f125 0 2024-04-25T15:49:51.301 INFO:tasks.workunit.client.1.smithi193.stdout:3/852: write d8/d13/f64 [3955278,118672] 0 2024-04-25T15:49:51.303 INFO:tasks.workunit.client.1.smithi193.stdout:8/848: dwrite d2/d18/db1/fbd [4194304,4194304] 0 2024-04-25T15:49:51.305 INFO:tasks.workunit.client.0.smithi110.stdout:9/895: rename d9/f39 to d9/db/df/d2c/d3c/d56/da0/f118 0 2024-04-25T15:49:51.311 INFO:tasks.workunit.client.0.smithi110.stdout:2/903: symlink de/d40/dcd/ddf/l126 0 2024-04-25T15:49:51.312 INFO:tasks.workunit.client.1.smithi193.stdout:7/838: write d12/d1c/d63/d109/fea [1527889,72515] 0 2024-04-25T15:49:51.316 INFO:tasks.workunit.client.1.smithi193.stdout:7/839: write d12/d1c/f7e [26516,46952] 0 2024-04-25T15:49:51.321 INFO:tasks.workunit.client.0.smithi110.stdout:4/955: rename d13/d24/d2d/l5b to d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/l14d 0 2024-04-25T15:49:51.323 INFO:tasks.workunit.client.1.smithi193.stdout:1/756: mkdir dc/d4a/d50/d5c/dfe 0 2024-04-25T15:49:51.323 INFO:tasks.workunit.client.1.smithi193.stdout:6/824: dread dd/d12/d19/f79 [0,4194304] 0 2024-04-25T15:49:51.324 INFO:tasks.workunit.client.1.smithi193.stdout:4/907: link db/d13/dcb/d113/d54/c72 db/d13/dd2/d11e/c124 0 2024-04-25T15:49:51.325 INFO:tasks.workunit.client.0.smithi110.stdout:3/987: dread d0/d16/dcb/d2f/fd4 [0,4194304] 0 2024-04-25T15:49:51.325 INFO:tasks.workunit.client.1.smithi193.stdout:4/908: chown db/d13/d1e/d1f/fb3 58 1 2024-04-25T15:49:51.326 INFO:tasks.workunit.client.1.smithi193.stdout:4/909: chown db/d13/d9d/d120 336 1 2024-04-25T15:49:51.327 INFO:tasks.workunit.client.0.smithi110.stdout:4/956: fdatasync d13/d24/d2d/d36/fb0 0 2024-04-25T15:49:51.335 INFO:tasks.workunit.client.0.smithi110.stdout:2/904: creat de/d40/d70/d11c/d5c/d5f/db2/d110/f127 x:0 0 0 2024-04-25T15:49:51.336 INFO:tasks.workunit.client.1.smithi193.stdout:3/853: rmdir d8/d13/d14/d27 39 2024-04-25T15:49:51.336 INFO:tasks.workunit.client.1.smithi193.stdout:3/854: dread - d8/d30/d72/db2/fdc zero size 2024-04-25T15:49:51.337 INFO:tasks.workunit.client.0.smithi110.stdout:0/927: write d1/d23/d30/d33/d5b/d83/fe9 [2855843,47119] 0 2024-04-25T15:49:51.337 INFO:tasks.workunit.client.0.smithi110.stdout:9/896: creat d9/d44/f119 x:0 0 0 2024-04-25T15:49:51.337 INFO:tasks.workunit.client.1.smithi193.stdout:8/849: symlink d2/d3/d20/d8c/dde/dbb/l110 0 2024-04-25T15:49:51.338 INFO:tasks.workunit.client.0.smithi110.stdout:4/957: stat d13/d24/d2d/d5e/d122/d135/f13b 0 2024-04-25T15:49:51.338 INFO:tasks.workunit.client.1.smithi193.stdout:8/850: write d2/d3/d20/d8c/dde/d94/fcb [2489464,31554] 0 2024-04-25T15:49:51.340 INFO:tasks.workunit.client.0.smithi110.stdout:5/978: dwrite d4/d1d/d1f/f49 [4194304,4194304] 0 2024-04-25T15:49:51.342 INFO:tasks.workunit.client.0.smithi110.stdout:6/940: dread d1/db/d22/d90/f72 [0,4194304] 0 2024-04-25T15:49:51.343 INFO:tasks.workunit.client.0.smithi110.stdout:3/988: dwrite d0/d2e/f71 [4194304,4194304] 0 2024-04-25T15:49:51.348 INFO:tasks.workunit.client.1.smithi193.stdout:0/846: write d0/d49/d5/d7/fe8 [470918,66393] 0 2024-04-25T15:49:51.349 INFO:tasks.workunit.client.1.smithi193.stdout:5/847: dread d17/d18/d75/f88 [0,4194304] 0 2024-04-25T15:49:51.350 INFO:tasks.workunit.client.1.smithi193.stdout:6/825: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/cfa 0 2024-04-25T15:49:51.351 INFO:tasks.workunit.client.0.smithi110.stdout:3/989: dwrite d0/d7/d7d/fd3 [0,4194304] 0 2024-04-25T15:49:51.351 INFO:tasks.workunit.client.1.smithi193.stdout:0/847: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/l79 16370 1 2024-04-25T15:49:51.352 INFO:tasks.workunit.client.1.smithi193.stdout:5/848: write d17/d18/d30/d38/d48/d4f/deb/f113 [361553,16492] 0 2024-04-25T15:49:51.353 INFO:tasks.workunit.client.1.smithi193.stdout:4/910: creat db/d13/d1e/d1f/d56/d69/d70/f125 x:0 0 0 2024-04-25T15:49:51.354 INFO:tasks.workunit.client.0.smithi110.stdout:3/990: truncate d0/d16/dcb/d2f/d4c/d63/f136 648067 0 2024-04-25T15:49:51.354 INFO:tasks.workunit.client.1.smithi193.stdout:0/848: dread - d0/d49/d5/d7/dd/d16/d18/d28/f102 zero size 2024-04-25T15:49:51.358 INFO:tasks.workunit.client.1.smithi193.stdout:2/821: mkdir d0/d9/d12e 0 2024-04-25T15:49:51.358 INFO:tasks.workunit.client.1.smithi193.stdout:2/822: chown d0/df/da5/lb7 1009060252 1 2024-04-25T15:49:51.363 INFO:tasks.workunit.client.0.smithi110.stdout:3/991: dwrite d0/d16/dcb/d2f/d4c/d63/f136 [0,4194304] 0 2024-04-25T15:49:51.364 INFO:tasks.workunit.client.0.smithi110.stdout:3/992: chown d0/d16/dcb/db0/fc2 49 1 2024-04-25T15:49:51.368 INFO:tasks.workunit.client.1.smithi193.stdout:3/855: creat d8/d13/d4b/d8b/de3/f107 x:0 0 0 2024-04-25T15:49:51.373 INFO:tasks.workunit.client.1.smithi193.stdout:8/851: unlink d2/d18/db1/ccc 0 2024-04-25T15:49:51.378 INFO:tasks.workunit.client.1.smithi193.stdout:6/826: fdatasync dd/d2f/f72 0 2024-04-25T15:49:51.379 INFO:tasks.workunit.client.1.smithi193.stdout:6/827: write dd/d12/d19/d1c/d40/d3c/d3f/de0/ff5 [346701,126893] 0 2024-04-25T15:49:51.384 INFO:tasks.workunit.client.1.smithi193.stdout:4/911: creat db/d13/d1e/d1f/d56/f126 x:0 0 0 2024-04-25T15:49:51.389 INFO:tasks.workunit.client.1.smithi193.stdout:9/808: creat df/d2c/ff5 x:0 0 0 2024-04-25T15:49:51.391 INFO:tasks.workunit.client.1.smithi193.stdout:3/856: mkdir d8/d13/d14/d23/d108 0 2024-04-25T15:49:51.395 INFO:tasks.workunit.client.1.smithi193.stdout:8/852: creat d2/d18/dbc/dd3/f111 x:0 0 0 2024-04-25T15:49:51.397 INFO:tasks.workunit.client.1.smithi193.stdout:1/757: mkdir dc/d59/dc0/deb/db0/dff 0 2024-04-25T15:49:51.401 INFO:tasks.workunit.client.1.smithi193.stdout:6/828: truncate dd/d12/d19/d1c/d40/f23 4433001 0 2024-04-25T15:49:51.403 INFO:tasks.workunit.client.1.smithi193.stdout:5/849: truncate d17/d18/d3e/f65 705387 0 2024-04-25T15:49:51.404 INFO:tasks.workunit.client.1.smithi193.stdout:7/840: dwrite d12/d31/d39/db0/fb5 [0,4194304] 0 2024-04-25T15:49:51.412 INFO:tasks.workunit.client.1.smithi193.stdout:5/850: dwrite d17/d18/d30/d38/d48/d4f/fd6 [0,4194304] 0 2024-04-25T15:49:51.416 INFO:tasks.workunit.client.1.smithi193.stdout:1/758: unlink dc/d4a/d50/d7c/c84 0 2024-04-25T15:49:51.422 INFO:tasks.workunit.client.1.smithi193.stdout:1/759: dwrite dc/d4a/ff9 [0,4194304] 0 2024-04-25T15:49:51.426 INFO:tasks.workunit.client.1.smithi193.stdout:7/841: rmdir d12/d13/dcc 39 2024-04-25T15:49:51.431 INFO:tasks.workunit.client.1.smithi193.stdout:2/823: rename d0/d9/d15/d31/lc8 to d0/df/da5/l12f 0 2024-04-25T15:49:51.436 INFO:tasks.workunit.client.1.smithi193.stdout:8/853: mkdir d2/d112 0 2024-04-25T15:49:51.440 INFO:tasks.workunit.client.1.smithi193.stdout:0/849: dwrite d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/fbc [0,4194304] 0 2024-04-25T15:49:51.441 INFO:tasks.workunit.client.1.smithi193.stdout:7/842: creat d12/d31/d37/d92/dbf/f133 x:0 0 0 2024-04-25T15:49:51.443 INFO:tasks.workunit.client.1.smithi193.stdout:5/851: dread d17/d18/f21 [4194304,4194304] 0 2024-04-25T15:49:51.448 INFO:tasks.workunit.client.1.smithi193.stdout:8/854: mkdir d2/d3/d27/d69/dae/dd8/d113 0 2024-04-25T15:49:51.452 INFO:tasks.workunit.client.1.smithi193.stdout:4/912: write db/d32/f7f [1004333,106848] 0 2024-04-25T15:49:51.452 INFO:tasks.workunit.client.1.smithi193.stdout:2/824: mkdir d0/d9/d3d/d114/d130 0 2024-04-25T15:49:51.455 INFO:tasks.workunit.client.1.smithi193.stdout:0/850: rename d0/d49/d5/d7/dd/d46/d48/d62/lf2 to d0/d49/d5/d61/dd4/l124 0 2024-04-25T15:49:51.455 INFO:tasks.workunit.client.0.smithi110.stdout:2/905: mknod de/d40/d6e/d52/df7/c128 0 2024-04-25T15:49:51.460 INFO:tasks.workunit.client.1.smithi193.stdout:9/809: dwrite df/d2c/d2f/d5c/fec [0,4194304] 0 2024-04-25T15:49:51.460 INFO:tasks.workunit.client.1.smithi193.stdout:8/855: chown d2/d3/d3a/f5e 3 1 2024-04-25T15:49:51.462 INFO:tasks.workunit.client.1.smithi193.stdout:6/829: write dd/d12/d19/d1c/d40/fbe [219205,64825] 0 2024-04-25T15:49:51.462 INFO:tasks.workunit.client.1.smithi193.stdout:8/856: dread - d2/d18/dbc/fce zero size 2024-04-25T15:49:51.462 INFO:tasks.workunit.client.0.smithi110.stdout:4/958: chown d13/d24/d2d/dd9/dbb/dd2/ff3 451 1 2024-04-25T15:49:51.463 INFO:tasks.workunit.client.1.smithi193.stdout:8/857: write d2/d3/d3a/f54 [1218245,41933] 0 2024-04-25T15:49:51.468 INFO:tasks.workunit.client.1.smithi193.stdout:4/913: read - db/d32/d45/fcd zero size 2024-04-25T15:49:51.469 INFO:tasks.workunit.client.0.smithi110.stdout:2/906: dread de/d40/d6e/f8e [0,4194304] 0 2024-04-25T15:49:51.470 INFO:tasks.workunit.client.0.smithi110.stdout:5/979: rename d4/d2c/d83/ff8 to d4/d15/d51/d90/ded/d13a/f145 0 2024-04-25T15:49:51.472 INFO:tasks.workunit.client.1.smithi193.stdout:2/825: chown d0/d42/d51/d4f/d65/ca6 31484586 1 2024-04-25T15:49:51.474 INFO:tasks.workunit.client.1.smithi193.stdout:0/851: mkdir d0/d49/d5/d7/dd/d16/d18/d28/d125 0 2024-04-25T15:49:51.475 INFO:tasks.workunit.client.1.smithi193.stdout:0/852: fsync d0/d49/fe 0 2024-04-25T15:49:51.477 INFO:tasks.workunit.client.0.smithi110.stdout:3/993: mknod d0/d16/dcb/d2f/d4c/d63/c14d 0 2024-04-25T15:49:51.478 INFO:tasks.workunit.client.0.smithi110.stdout:3/994: chown d0/d16/dcb/d2f/d4c/d63/fed 7996737 1 2024-04-25T15:49:51.480 INFO:tasks.workunit.client.1.smithi193.stdout:4/914: dread db/d32/fb4 [0,4194304] 0 2024-04-25T15:49:51.481 INFO:tasks.workunit.client.1.smithi193.stdout:4/915: chown db/d32/de8 1001629 1 2024-04-25T15:49:51.482 INFO:tasks.workunit.client.0.smithi110.stdout:9/897: mknod d9/d44/d5e/db5/de7/c11a 0 2024-04-25T15:49:51.482 INFO:tasks.workunit.client.1.smithi193.stdout:0/853: dwrite d0/d49/d5/d7/dd/d16/d26/d9e/f118 [0,4194304] 0 2024-04-25T15:49:51.482 INFO:tasks.workunit.client.1.smithi193.stdout:4/916: write db/d32/f5f [4277643,87188] 0 2024-04-25T15:49:51.484 INFO:tasks.workunit.client.1.smithi193.stdout:4/917: dread - db/d13/d9d/f109 zero size 2024-04-25T15:49:51.486 INFO:tasks.workunit.client.1.smithi193.stdout:6/830: rmdir dd/d12/d19/d1c/d40/d3c 39 2024-04-25T15:49:51.489 INFO:tasks.workunit.client.1.smithi193.stdout:8/858: symlink d2/d3/d27/d69/dae/d4a/l114 0 2024-04-25T15:49:51.490 INFO:tasks.workunit.client.1.smithi193.stdout:7/843: creat d12/d13/dc4/f134 x:0 0 0 2024-04-25T15:49:51.494 INFO:tasks.workunit.client.1.smithi193.stdout:7/844: dread d12/d31/d39/f53 [0,4194304] 0 2024-04-25T15:49:51.494 INFO:tasks.workunit.client.0.smithi110.stdout:5/980: symlink d4/d1d/d1f/d35/d3d/d52/d131/l146 0 2024-04-25T15:49:51.499 INFO:tasks.workunit.client.0.smithi110.stdout:6/941: write d1/db/d22/d33/f3b [1399845,113254] 0 2024-04-25T15:49:51.501 INFO:tasks.workunit.client.1.smithi193.stdout:0/854: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf/f126 x:0 0 0 2024-04-25T15:49:51.503 INFO:tasks.workunit.client.1.smithi193.stdout:0/855: read d0/d49/faa [3931,34203] 0 2024-04-25T15:49:51.503 INFO:tasks.workunit.client.1.smithi193.stdout:4/918: creat db/d13/d1e/d1f/d56/d57/df2/f127 x:0 0 0 2024-04-25T15:49:51.503 INFO:tasks.workunit.client.0.smithi110.stdout:0/928: dwrite d1/d23/d30/d33/d35/f6b [0,4194304] 0 2024-04-25T15:49:51.504 INFO:tasks.workunit.client.1.smithi193.stdout:0/856: chown d0/d49/d5/d7/dd/d16/d26/f55 1 1 2024-04-25T15:49:51.506 INFO:tasks.workunit.client.1.smithi193.stdout:6/831: mknod dd/d12/d19/d1c/d40/dce/cfb 0 2024-04-25T15:49:51.506 INFO:tasks.workunit.client.1.smithi193.stdout:5/852: write d17/d18/d30/d38/d71/d96/dc4/f70 [266213,77531] 0 2024-04-25T15:49:51.514 INFO:tasks.workunit.client.1.smithi193.stdout:3/857: sync 2024-04-25T15:49:51.515 INFO:tasks.workunit.client.1.smithi193.stdout:3/858: chown d8/c37 398065472 1 2024-04-25T15:49:51.526 INFO:tasks.workunit.client.0.smithi110.stdout:9/898: symlink d9/d13/d5b/d61/d67/l11b 0 2024-04-25T15:49:51.526 INFO:tasks.workunit.client.0.smithi110.stdout:6/942: rename d1/db/d24/d2d/f34 to d1/db/d22/d90/df7/f146 0 2024-04-25T15:49:51.527 INFO:tasks.workunit.client.0.smithi110.stdout:4/959: write d13/d24/d2d/d44/d65/d99/daf/d56/f76 [6684482,14978] 0 2024-04-25T15:49:51.527 INFO:tasks.workunit.client.0.smithi110.stdout:3/995: write d0/d16/dcb/d2f/d4c/d63/fc0 [3541412,127915] 0 2024-04-25T15:49:51.527 INFO:tasks.workunit.client.0.smithi110.stdout:2/907: dread de/d40/d6e/d52/f62 [0,4194304] 0 2024-04-25T15:49:51.528 INFO:tasks.workunit.client.0.smithi110.stdout:0/929: chown d1/d23/d30/d33/f88 4827366 1 2024-04-25T15:49:51.529 INFO:tasks.workunit.client.1.smithi193.stdout:9/810: dwrite df/d2d/fd6 [0,4194304] 0 2024-04-25T15:49:51.530 INFO:tasks.workunit.client.1.smithi193.stdout:5/853: fsync d17/d18/d30/f91 0 2024-04-25T15:49:51.530 INFO:tasks.workunit.client.1.smithi193.stdout:8/859: symlink d2/d3/d27/d69/dae/l115 0 2024-04-25T15:49:51.530 INFO:tasks.workunit.client.1.smithi193.stdout:2/826: link d0/df/da5/cf2 d0/d37/d25/d5f/d98/c131 0 2024-04-25T15:49:51.531 INFO:tasks.workunit.client.1.smithi193.stdout:7/845: creat d12/d1c/d63/d70/d68/d8d/d106/f135 x:0 0 0 2024-04-25T15:49:51.532 INFO:tasks.workunit.client.1.smithi193.stdout:9/811: read df/d2d/f99 [266986,86866] 0 2024-04-25T15:49:51.532 INFO:tasks.workunit.client.1.smithi193.stdout:7/846: fsync d12/f10b 0 2024-04-25T15:49:51.534 INFO:tasks.workunit.client.1.smithi193.stdout:7/847: read d12/d31/d37/d92/fee [1172106,4905] 0 2024-04-25T15:49:51.535 INFO:tasks.workunit.client.1.smithi193.stdout:7/848: write d12/d1c/faf [283322,90901] 0 2024-04-25T15:49:51.536 INFO:tasks.workunit.client.1.smithi193.stdout:3/859: mknod d8/d13/d14/d51/c109 0 2024-04-25T15:49:51.543 INFO:tasks.workunit.client.1.smithi193.stdout:6/832: truncate dd/d12/d19/d1c/d40/d3c/d3f/fc3 534081 0 2024-04-25T15:49:51.548 INFO:tasks.workunit.client.1.smithi193.stdout:3/860: mkdir d8/d13/d4b/d8b/d10a 0 2024-04-25T15:49:51.548 INFO:tasks.workunit.client.1.smithi193.stdout:5/854: write d17/d18/d3e/fa5 [898267,52367] 0 2024-04-25T15:49:51.549 INFO:tasks.workunit.client.1.smithi193.stdout:4/919: link db/d13/dcb/d113/d54/d77/l80 db/l128 0 2024-04-25T15:49:51.550 INFO:tasks.workunit.client.1.smithi193.stdout:2/827: write d0/d9/d3d/fa3 [1219014,2234] 0 2024-04-25T15:49:51.551 INFO:tasks.workunit.client.1.smithi193.stdout:8/860: rename d2/d3/d3a/f61 to d2/d3/d20/f116 0 2024-04-25T15:49:51.552 INFO:tasks.workunit.client.1.smithi193.stdout:9/812: dread df/d2c/d2f/d4e/f19 [0,4194304] 0 2024-04-25T15:49:51.553 INFO:tasks.workunit.client.1.smithi193.stdout:6/833: mknod dd/d12/d19/d1c/d40/dce/de2/df1/cfc 0 2024-04-25T15:49:51.555 INFO:tasks.workunit.client.1.smithi193.stdout:3/861: symlink d8/d13/d4b/d8b/da2/l10b 0 2024-04-25T15:49:51.557 INFO:tasks.workunit.client.1.smithi193.stdout:4/920: symlink db/d13/d1e/l129 0 2024-04-25T15:49:51.557 INFO:tasks.workunit.client.1.smithi193.stdout:4/921: stat db/d13/dcb/d113/d54/d6f/c106 0 2024-04-25T15:49:51.557 INFO:tasks.workunit.client.1.smithi193.stdout:4/922: chown db/l78 50668160 1 2024-04-25T15:49:51.559 INFO:tasks.workunit.client.1.smithi193.stdout:5/855: rmdir d17/d18/d30/d38 39 2024-04-25T15:49:51.562 INFO:tasks.workunit.client.1.smithi193.stdout:7/849: write d12/d31/d37/d92/dbf/fe9 [56384,110570] 0 2024-04-25T15:49:51.564 INFO:tasks.workunit.client.1.smithi193.stdout:9/813: chown df/d2c/d2f/d5c/d7e/da2/dc6 463276828 1 2024-04-25T15:49:51.564 INFO:tasks.workunit.client.1.smithi193.stdout:7/850: chown d12/d1c/d63/fce 0 1 2024-04-25T15:49:51.565 INFO:tasks.workunit.client.1.smithi193.stdout:3/862: fdatasync d8/d13/d14/d51/f5f 0 2024-04-25T15:49:51.567 INFO:tasks.workunit.client.1.smithi193.stdout:3/863: chown d8/d13/d1f/d69/d77 371 1 2024-04-25T15:49:51.567 INFO:tasks.workunit.client.1.smithi193.stdout:2/828: write d0/d9/d15/f1b [2195842,127894] 0 2024-04-25T15:49:51.567 INFO:tasks.workunit.client.1.smithi193.stdout:6/834: dread dd/d12/d19/f7f [0,4194304] 0 2024-04-25T15:49:51.569 INFO:tasks.workunit.client.1.smithi193.stdout:7/851: creat d12/d1c/d63/d70/d68/d8d/f136 x:0 0 0 2024-04-25T15:49:51.569 INFO:tasks.workunit.client.1.smithi193.stdout:3/864: read - d8/d30/d72/db2/fdc zero size 2024-04-25T15:49:51.570 INFO:tasks.workunit.client.1.smithi193.stdout:7/852: stat d12/f36 0 2024-04-25T15:49:51.570 INFO:tasks.workunit.client.1.smithi193.stdout:9/814: dread - df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f8a zero size 2024-04-25T15:49:51.571 INFO:tasks.workunit.client.1.smithi193.stdout:2/829: rmdir d0/df/dbf 39 2024-04-25T15:49:51.576 INFO:tasks.workunit.client.1.smithi193.stdout:5/856: rename d17/d18/d3e/cae to d17/d18/d30/d38/d71/dd7/c122 0 2024-04-25T15:49:51.576 INFO:tasks.workunit.client.1.smithi193.stdout:4/923: dwrite db/d13/d1e/d1f/d56/fd1 [0,4194304] 0 2024-04-25T15:49:51.578 INFO:tasks.workunit.client.1.smithi193.stdout:3/865: write d8/d4d/f87 [4219457,86138] 0 2024-04-25T15:49:51.579 INFO:tasks.workunit.client.1.smithi193.stdout:8/861: dread d2/d3/d27/f7a [0,4194304] 0 2024-04-25T15:49:51.583 INFO:tasks.workunit.client.1.smithi193.stdout:4/924: unlink db/d4b/le2 0 2024-04-25T15:49:51.583 INFO:tasks.workunit.client.1.smithi193.stdout:9/815: chown df/d2c/d2f/d4e/d1c/d30/d3b/le9 1 1 2024-04-25T15:49:51.584 INFO:tasks.workunit.client.1.smithi193.stdout:8/862: fdatasync d2/d3/d27/d69/dae/f49 0 2024-04-25T15:49:51.585 INFO:tasks.workunit.client.1.smithi193.stdout:9/816: fdatasync df/d2c/d2f/d4e/d69/de7/fee 0 2024-04-25T15:49:51.587 INFO:tasks.workunit.client.1.smithi193.stdout:7/853: getdents d12/d31/d37/d92/dbf 0 2024-04-25T15:49:51.591 INFO:tasks.workunit.client.1.smithi193.stdout:8/863: mkdir d2/d3/d20/d8c/d117 0 2024-04-25T15:49:51.592 INFO:tasks.workunit.client.1.smithi193.stdout:9/817: fdatasync df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f8a 0 2024-04-25T15:49:51.592 INFO:tasks.workunit.client.1.smithi193.stdout:8/864: chown d2/d3/d20/db8/cba 809 1 2024-04-25T15:49:51.593 INFO:tasks.workunit.client.1.smithi193.stdout:4/925: mkdir db/d13/dd2/d12a 0 2024-04-25T15:49:51.594 INFO:tasks.workunit.client.1.smithi193.stdout:5/857: dread d17/d18/d3e/f4c [0,4194304] 0 2024-04-25T15:49:51.594 INFO:tasks.workunit.client.1.smithi193.stdout:7/854: mknod d12/d31/d37/d104/d9d/c137 0 2024-04-25T15:49:51.594 INFO:tasks.workunit.client.1.smithi193.stdout:3/866: link f2 d8/d13/d4b/d8b/d10a/f10c 0 2024-04-25T15:49:51.595 INFO:tasks.workunit.client.1.smithi193.stdout:3/867: write d8/d4d/f87 [404694,13145] 0 2024-04-25T15:49:51.596 INFO:tasks.workunit.client.1.smithi193.stdout:6/835: rename dd/d12/d19/d1c/d40/d3c/l8d to dd/d12/lfd 0 2024-04-25T15:49:51.598 INFO:tasks.workunit.client.1.smithi193.stdout:2/830: dread d0/d42/d51/d4f/d65/d6a/f3b [0,4194304] 0 2024-04-25T15:49:51.598 INFO:tasks.workunit.client.1.smithi193.stdout:4/926: rmdir db/d13/dcb/d113/d54/d6f 39 2024-04-25T15:49:51.599 INFO:tasks.workunit.client.1.smithi193.stdout:7/855: readlink d12/d1c/d63/d70/d68/d8f/lbc 0 2024-04-25T15:49:51.601 INFO:tasks.workunit.client.1.smithi193.stdout:5/858: rename d17/d18/d75/d85/d5d/f68 to d17/d18/d30/d34/d52/daf/f123 0 2024-04-25T15:49:51.601 INFO:tasks.workunit.client.1.smithi193.stdout:9/818: mknod df/cf6 0 2024-04-25T15:49:51.604 INFO:tasks.workunit.client.1.smithi193.stdout:7/856: mkdir d12/d1c/d63/d70/d138 0 2024-04-25T15:49:51.605 INFO:tasks.workunit.client.1.smithi193.stdout:2/831: dwrite d0/d9/d15/f1b [4194304,4194304] 0 2024-04-25T15:49:51.606 INFO:tasks.workunit.client.1.smithi193.stdout:3/868: rename d8/d30/d45/f70 to d8/d13/d1f/d69/dcd/dd9/dde/f10d 0 2024-04-25T15:49:51.607 INFO:tasks.workunit.client.1.smithi193.stdout:5/859: mkdir d17/d53/d124 0 2024-04-25T15:49:51.611 INFO:tasks.workunit.client.1.smithi193.stdout:6/836: dwrite dd/d2f/d57/daf/fcc [0,4194304] 0 2024-04-25T15:49:51.613 INFO:tasks.workunit.client.1.smithi193.stdout:2/832: write d0/df/dbf/f9e [2194500,90800] 0 2024-04-25T15:49:51.616 INFO:tasks.workunit.client.1.smithi193.stdout:1/760: sync 2024-04-25T15:49:51.618 INFO:tasks.workunit.client.1.smithi193.stdout:6/837: symlink dd/db5/lfe 0 2024-04-25T15:49:51.623 INFO:tasks.workunit.client.1.smithi193.stdout:4/927: rename db/d32/d45/dab to db/d13/dd2/d12b 0 2024-04-25T15:49:51.623 INFO:tasks.workunit.client.1.smithi193.stdout:5/860: dwrite d17/d18/d30/d38/d48/d4f/f10d [0,4194304] 0 2024-04-25T15:49:51.625 INFO:tasks.workunit.client.1.smithi193.stdout:1/761: mknod dc/d4a/d4d/d7a/dd0/de2/c100 0 2024-04-25T15:49:51.626 INFO:tasks.workunit.client.1.smithi193.stdout:9/819: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/fbb [1356884,8810] 0 2024-04-25T15:49:51.628 INFO:tasks.workunit.client.1.smithi193.stdout:7/857: rename d12/d31/d37/c11c to d12/d31/d39/d60/c139 0 2024-04-25T15:49:51.632 INFO:tasks.workunit.client.1.smithi193.stdout:6/838: dwrite dd/d12/d19/d45/fee [0,4194304] 0 2024-04-25T15:49:51.633 INFO:tasks.workunit.client.1.smithi193.stdout:2/833: link d0/lfe d0/df/dbf/l132 0 2024-04-25T15:49:51.634 INFO:tasks.workunit.client.1.smithi193.stdout:7/858: mknod d12/d31/dc3/c13a 0 2024-04-25T15:49:51.640 INFO:tasks.workunit.client.1.smithi193.stdout:1/762: creat dc/d4a/d50/d5c/d5d/f101 x:0 0 0 2024-04-25T15:49:51.641 INFO:tasks.workunit.client.1.smithi193.stdout:9/820: dwrite df/d2c/d2f/d4e/fa0 [4194304,4194304] 0 2024-04-25T15:49:51.641 INFO:tasks.workunit.client.1.smithi193.stdout:6/839: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/de8/cff 0 2024-04-25T15:49:51.642 INFO:tasks.workunit.client.1.smithi193.stdout:3/869: dread d8/d13/d1f/d69/dcd/dd9/dde/f10d [0,4194304] 0 2024-04-25T15:49:51.643 INFO:tasks.workunit.client.1.smithi193.stdout:9/821: chown df/d2c/d2f/d5c 54608 1 2024-04-25T15:49:51.643 INFO:tasks.workunit.client.1.smithi193.stdout:7/859: creat d12/d13/dc4/dfa/dff/f13b x:0 0 0 2024-04-25T15:49:51.643 INFO:tasks.workunit.client.1.smithi193.stdout:3/870: stat d8/d13/d14/d23 0 2024-04-25T15:49:51.645 INFO:tasks.workunit.client.1.smithi193.stdout:7/860: fsync d12/d31/d37/d92/dbf/f10e 0 2024-04-25T15:49:51.645 INFO:tasks.workunit.client.1.smithi193.stdout:3/871: fsync d8/d4d/f62 0 2024-04-25T15:49:51.647 INFO:tasks.workunit.client.1.smithi193.stdout:1/763: getdents dc/d4a/d4d/d7a/d85 0 2024-04-25T15:49:51.648 INFO:tasks.workunit.client.1.smithi193.stdout:7/861: chown d12/d1c/d45/l73 2377 1 2024-04-25T15:49:51.650 INFO:tasks.workunit.client.0.smithi110.stdout:9/899: mknod d9/db/df/d2c/d3c/d56/da0/c11c 0 2024-04-25T15:49:51.650 INFO:tasks.workunit.client.0.smithi110.stdout:4/960: fdatasync ff 0 2024-04-25T15:49:51.650 INFO:tasks.workunit.client.0.smithi110.stdout:6/943: symlink d1/db/d22/d90/d76/l147 0 2024-04-25T15:49:51.651 INFO:tasks.workunit.client.1.smithi193.stdout:1/764: write dc/d59/dc0/deb/f97 [2917004,51057] 0 2024-04-25T15:49:51.652 INFO:tasks.workunit.client.0.smithi110.stdout:3/996: fsync d0/d16/dcb/d2f/d4c/db2/f123 0 2024-04-25T15:49:51.653 INFO:tasks.workunit.client.0.smithi110.stdout:0/930: write d1/d23/d30/d33/d5b/f9b [144881,47019] 0 2024-04-25T15:49:51.654 INFO:tasks.workunit.client.1.smithi193.stdout:6/840: dwrite dd/d12/d2d/fc8 [0,4194304] 0 2024-04-25T15:49:51.657 INFO:tasks.workunit.client.0.smithi110.stdout:4/961: read d13/d24/d26/dcd/d131/fb6 [1461326,90125] 0 2024-04-25T15:49:51.657 INFO:tasks.workunit.client.1.smithi193.stdout:8/865: truncate d2/d3/d20/d8c/dde/d94/fcb 1261343 0 2024-04-25T15:49:51.658 INFO:tasks.workunit.client.1.smithi193.stdout:9/822: dwrite df/d2c/d34/d3e/f6a [4194304,4194304] 0 2024-04-25T15:49:51.659 INFO:tasks.workunit.client.0.smithi110.stdout:4/962: dread - d13/d24/d2d/d5e/d122/d135/f13b zero size 2024-04-25T15:49:51.659 INFO:tasks.workunit.client.1.smithi193.stdout:1/765: rmdir dc/d59 39 2024-04-25T15:49:51.660 INFO:tasks.workunit.client.1.smithi193.stdout:8/866: write d2/d18/ddb/feb [5002,118657] 0 2024-04-25T15:49:51.660 INFO:tasks.workunit.client.1.smithi193.stdout:6/841: stat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dd7/le5 0 2024-04-25T15:49:51.664 INFO:tasks.workunit.client.1.smithi193.stdout:6/842: truncate dd/d12/d19/d1c/d40/d3c/d3f/de0/ff5 1173792 0 2024-04-25T15:49:51.664 INFO:tasks.workunit.client.1.smithi193.stdout:6/843: fsync dd/d2f/d57/f8b 0 2024-04-25T15:49:51.667 INFO:tasks.workunit.client.0.smithi110.stdout:4/963: dwrite d13/d24/d26/f49 [4194304,4194304] 0 2024-04-25T15:49:51.669 INFO:tasks.workunit.client.1.smithi193.stdout:7/862: rename d12/d1c/d63/d70/faa to d12/d1c/d63/d70/d68/d8f/d129/f13c 0 2024-04-25T15:49:51.672 INFO:tasks.workunit.client.0.smithi110.stdout:6/944: symlink d1/db/d22/d90/l148 0 2024-04-25T15:49:51.674 INFO:tasks.workunit.client.1.smithi193.stdout:1/766: symlink dc/d4a/d50/d5c/d5d/ddd/l102 0 2024-04-25T15:49:51.675 INFO:tasks.workunit.client.0.smithi110.stdout:3/997: mknod d0/d16/dcb/d2f/d4c/d108/c14e 0 2024-04-25T15:49:51.676 INFO:tasks.workunit.client.1.smithi193.stdout:1/767: truncate dc/d3c/f79 1200559 0 2024-04-25T15:49:51.678 INFO:tasks.workunit.client.1.smithi193.stdout:5/861: write d17/d18/d75/d85/f73 [2145775,47480] 0 2024-04-25T15:49:51.678 INFO:tasks.workunit.client.0.smithi110.stdout:6/945: dwrite d1/dd/d3c/d91/d8d/dce/f73 [0,4194304] 0 2024-04-25T15:49:51.680 INFO:tasks.workunit.client.1.smithi193.stdout:1/768: dread dc/f1e [0,4194304] 0 2024-04-25T15:49:51.680 INFO:tasks.workunit.client.1.smithi193.stdout:9/823: rename df/d2c/d2f/d4e/f1a to df/d2c/d2f/d4e/ff7 0 2024-04-25T15:49:51.681 INFO:tasks.workunit.client.1.smithi193.stdout:9/824: stat df/d2c/d2f/d4e/l9d 0 2024-04-25T15:49:51.686 INFO:tasks.workunit.client.0.smithi110.stdout:2/908: dread de/d40/d70/d11c/d5c/d5f/f63 [0,4194304] 0 2024-04-25T15:49:51.688 INFO:tasks.workunit.client.0.smithi110.stdout:2/909: dread de/d40/d70/f84 [0,4194304] 0 2024-04-25T15:49:51.688 INFO:tasks.workunit.client.0.smithi110.stdout:2/910: stat de/d40/d86/cec 0 2024-04-25T15:49:51.704 INFO:tasks.workunit.client.1.smithi193.stdout:0/857: sync 2024-04-25T15:49:51.719 INFO:tasks.workunit.client.1.smithi193.stdout:0/858: write d0/d49/f2c [849199,49119] 0 2024-04-25T15:49:51.721 INFO:tasks.workunit.client.1.smithi193.stdout:2/834: dwrite d0/d9/d15/d31/d50/f9a [4194304,4194304] 0 2024-04-25T15:49:51.721 INFO:tasks.workunit.client.1.smithi193.stdout:2/835: readlink d0/d37/dbb/df8/l107 0 2024-04-25T15:49:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:51 smithi193 ceph-mon[33747]: pgmap v42: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 87 MiB/s rd, 207 MiB/s wr, 436 op/s 2024-04-25T15:49:51.740 INFO:tasks.workunit.client.0.smithi110.stdout:0/931: fsync d1/d6c/d116/fc9 0 2024-04-25T15:49:51.741 INFO:tasks.workunit.client.0.smithi110.stdout:0/932: chown d1/d47/f95 2 1 2024-04-25T15:49:51.744 INFO:tasks.workunit.client.0.smithi110.stdout:4/964: mkdir d13/d24/d2d/d5e/d122/d14e 0 2024-04-25T15:49:51.746 INFO:tasks.workunit.client.1.smithi193.stdout:3/872: truncate d8/d13/d1f/d69/fcf 6632337 0 2024-04-25T15:49:51.747 INFO:tasks.workunit.client.1.smithi193.stdout:3/873: write d8/d13/d1f/d69/dcd/dd9/dde/f91 [2353258,126459] 0 2024-04-25T15:49:51.751 INFO:tasks.workunit.client.0.smithi110.stdout:9/900: rename d9/d44/d5e/d80 to d9/db/df/d11d 0 2024-04-25T15:49:51.751 INFO:tasks.workunit.client.0.smithi110.stdout:9/901: chown d9/db/df/f68 56510794 1 2024-04-25T15:49:51.752 INFO:tasks.workunit.client.1.smithi193.stdout:8/867: mkdir d2/d3/d27/d108/d118 0 2024-04-25T15:49:51.756 INFO:tasks.workunit.client.0.smithi110.stdout:2/911: rmdir de/d40/d6e/d50 39 2024-04-25T15:49:51.757 INFO:tasks.workunit.client.1.smithi193.stdout:5/862: creat d17/d18/d30/d34/d52/dee/f125 x:0 0 0 2024-04-25T15:49:51.763 INFO:tasks.workunit.client.0.smithi110.stdout:9/902: rmdir d9/d13/d5b/d61/db3 39 2024-04-25T15:49:51.765 INFO:tasks.workunit.client.1.smithi193.stdout:0/859: creat d0/d49/d5/d61/dd4/f127 x:0 0 0 2024-04-25T15:49:51.768 INFO:tasks.workunit.client.0.smithi110.stdout:6/946: fsync d1/dd/f21 0 2024-04-25T15:49:51.774 INFO:tasks.workunit.client.1.smithi193.stdout:4/928: sync 2024-04-25T15:49:51.776 INFO:tasks.workunit.client.0.smithi110.stdout:9/903: creat d9/d13/d5b/d61/d67/dd7/df5/f11e x:0 0 0 2024-04-25T15:49:51.778 INFO:tasks.workunit.client.1.smithi193.stdout:8/868: rename d2/d3/d20/d8c/dde/dbb/cdc to d2/d3/d27/d69/dae/d4a/d53/c119 0 2024-04-25T15:49:51.778 INFO:tasks.workunit.client.0.smithi110.stdout:3/998: dwrite d0/d16/dcb/d50/fca [0,4194304] 0 2024-04-25T15:49:51.779 INFO:tasks.workunit.client.0.smithi110.stdout:3/999: read d0/d7/d7d/fd3 [3248627,8243] 0 2024-04-25T15:49:51.780 INFO:tasks.workunit.client.0.smithi110.stdout:4/965: write d13/d24/d2d/d44/d90/fc1 [3217703,117647] 0 2024-04-25T15:49:51.783 INFO:tasks.workunit.client.0.smithi110.stdout:6/947: unlink d1/db/d22/d33/f56 0 2024-04-25T15:49:51.784 INFO:tasks.workunit.client.1.smithi193.stdout:9/825: creat df/d2c/d2f/d4e/d69/de7/ff8 x:0 0 0 2024-04-25T15:49:51.785 INFO:tasks.workunit.client.1.smithi193.stdout:3/874: dwrite d8/d30/d45/fbe [0,4194304] 0 2024-04-25T15:49:51.787 INFO:tasks.workunit.client.1.smithi193.stdout:0/860: creat d0/d49/d5/d7/dd/d16/d26/d5e/d99/f128 x:0 0 0 2024-04-25T15:49:51.789 INFO:tasks.workunit.client.1.smithi193.stdout:2/836: mknod d0/d37/c133 0 2024-04-25T15:49:51.790 INFO:tasks.workunit.client.1.smithi193.stdout:2/837: chown d0/d37/f92 429401281 1 2024-04-25T15:49:51.794 INFO:tasks.workunit.client.1.smithi193.stdout:0/861: write d0/d49/d5/d7/dd/d46/d48/d62/dd9/ff8 [618759,12041] 0 2024-04-25T15:49:51.794 INFO:tasks.workunit.client.1.smithi193.stdout:2/838: chown d0/d37/d25/d5f/d3a/d6d/d6e/db9/l116 13790 1 2024-04-25T15:49:51.794 INFO:tasks.workunit.client.1.smithi193.stdout:4/929: write db/d13/dcb/d113/d54/d77/fc9 [490907,97187] 0 2024-04-25T15:49:51.797 INFO:tasks.workunit.client.0.smithi110.stdout:4/966: fdatasync d13/d24/d2d/d44/da4/fb3 0 2024-04-25T15:49:51.799 INFO:tasks.workunit.client.1.smithi193.stdout:5/863: mkdir d17/d18/d30/d119/d126 0 2024-04-25T15:49:51.800 INFO:tasks.workunit.client.0.smithi110.stdout:6/948: mkdir d1/dd/d3c/d91/d8d/dce/d149 0 2024-04-25T15:49:51.801 INFO:tasks.workunit.client.1.smithi193.stdout:8/869: mkdir d2/d3/d27/d108/d11a 0 2024-04-25T15:49:51.802 INFO:tasks.workunit.client.0.smithi110.stdout:2/912: mknod de/d40/d6e/d52/d6d/d99/c129 0 2024-04-25T15:49:51.803 INFO:tasks.workunit.client.1.smithi193.stdout:1/769: rmdir dc/dfc 0 2024-04-25T15:49:51.804 INFO:tasks.workunit.client.0.smithi110.stdout:0/933: getdents d1/d23/d30/d33/d5b/dbb/da8 0 2024-04-25T15:49:51.805 INFO:tasks.workunit.client.0.smithi110.stdout:9/904: fsync d9/d13/fa9 0 2024-04-25T15:49:51.806 INFO:tasks.workunit.client.0.smithi110.stdout:9/905: dread - d9/db/d58/f104 zero size 2024-04-25T15:49:51.807 INFO:tasks.workunit.client.1.smithi193.stdout:9/826: unlink df/d2c/d2f/d4e/fc2 0 2024-04-25T15:49:51.808 INFO:tasks.workunit.client.0.smithi110.stdout:9/906: read d9/db/df/d2c/d3c/fac [3933834,121502] 0 2024-04-25T15:49:51.809 INFO:tasks.workunit.client.1.smithi193.stdout:6/844: sync 2024-04-25T15:49:51.809 INFO:tasks.workunit.client.0.smithi110.stdout:6/949: creat d1/dd/f14a x:0 0 0 2024-04-25T15:49:51.810 INFO:tasks.workunit.client.0.smithi110.stdout:9/907: fsync d9/d13/d5b/d61/d67/dd7/df5/f107 0 2024-04-25T15:49:51.810 INFO:tasks.workunit.client.0.smithi110.stdout:6/950: chown d1/db/d24/d2d/l54 677973 1 2024-04-25T15:49:51.811 INFO:tasks.workunit.client.0.smithi110.stdout:9/908: write d9/db/df/d2c/f36 [5976659,73249] 0 2024-04-25T15:49:51.811 INFO:tasks.workunit.client.0.smithi110.stdout:2/913: rmdir de/d40/d6e/d52/d6d 39 2024-04-25T15:49:51.813 INFO:tasks.workunit.client.0.smithi110.stdout:9/909: write d9/db/df/d2c/d3c/ffc [865936,104835] 0 2024-04-25T15:49:51.813 INFO:tasks.workunit.client.1.smithi193.stdout:7/863: getdents d12/d13/dc4 0 2024-04-25T15:49:51.818 INFO:tasks.workunit.client.1.smithi193.stdout:2/839: read d0/d37/f43 [255767,67285] 0 2024-04-25T15:49:51.821 INFO:tasks.workunit.client.0.smithi110.stdout:2/914: dread de/d6f/fdc [0,4194304] 0 2024-04-25T15:49:51.822 INFO:tasks.workunit.client.0.smithi110.stdout:2/915: dread - de/d40/d4f/d9f/f100 zero size 2024-04-25T15:49:51.822 INFO:tasks.workunit.client.0.smithi110.stdout:9/910: truncate d9/d2d/dc7/fe1 505140 0 2024-04-25T15:49:51.823 INFO:tasks.workunit.client.1.smithi193.stdout:8/870: symlink d2/d3/d27/d69/dae/d4a/d63/da2/l11b 0 2024-04-25T15:49:51.824 INFO:tasks.workunit.client.1.smithi193.stdout:8/871: chown d2/d3/d27/d69/dbf/d106 3957 1 2024-04-25T15:49:51.825 INFO:tasks.workunit.client.0.smithi110.stdout:6/951: write d1/f9 [829934,121103] 0 2024-04-25T15:49:51.827 INFO:tasks.workunit.client.1.smithi193.stdout:9/827: mkdir df/d2c/d2f/d4e/d69/df9 0 2024-04-25T15:49:51.833 INFO:tasks.workunit.client.0.smithi110.stdout:6/952: rename d1/db/d24/fb9 to d1/d12/daf/f14b 0 2024-04-25T15:49:51.835 INFO:tasks.workunit.client.0.smithi110.stdout:2/916: write de/f2d [2030576,94880] 0 2024-04-25T15:49:51.836 INFO:tasks.workunit.client.0.smithi110.stdout:6/953: fsync d1/db/d22/d33/d63/ffe 0 2024-04-25T15:49:51.841 INFO:tasks.workunit.client.0.smithi110.stdout:9/911: dread d9/d13/d84/f9b [0,4194304] 0 2024-04-25T15:49:51.847 INFO:tasks.workunit.client.0.smithi110.stdout:2/917: getdents de/d40/d6e/d50 0 2024-04-25T15:49:51.848 INFO:tasks.workunit.client.0.smithi110.stdout:2/918: write de/f2d [2642658,98345] 0 2024-04-25T15:49:51.848 INFO:tasks.workunit.client.0.smithi110.stdout:2/919: stat f8 0 2024-04-25T15:49:51.851 INFO:tasks.workunit.client.0.smithi110.stdout:6/954: link d1/dd/d3c/d91/d8d/dce/d102/d75/d82/l8f d1/dd/d3c/d91/d8d/dce/d102/l14c 0 2024-04-25T15:49:51.854 INFO:tasks.workunit.client.0.smithi110.stdout:2/920: chown de/d40/d6e/l71 733412 1 2024-04-25T15:49:51.858 INFO:tasks.workunit.client.0.smithi110.stdout:9/912: write d9/d44/f7f [291155,105697] 0 2024-04-25T15:49:51.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:51 smithi110 ceph-mon[26344]: pgmap v42: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 87 MiB/s rd, 207 MiB/s wr, 436 op/s 2024-04-25T15:49:51.863 INFO:tasks.workunit.client.0.smithi110.stdout:9/913: mkdir d9/d13/d5b/d11f 0 2024-04-25T15:49:51.864 INFO:tasks.workunit.client.0.smithi110.stdout:9/914: readlink d9/d13/d5b/d61/d67/dd7/led 0 2024-04-25T15:49:51.866 INFO:tasks.workunit.client.0.smithi110.stdout:6/955: dread d1/db/d22/d33/d63/dc2/f107 [0,4194304] 0 2024-04-25T15:49:51.869 INFO:tasks.workunit.client.0.smithi110.stdout:9/915: dread d9/db/df/d11d/dbb/fcd [0,4194304] 0 2024-04-25T15:49:51.869 INFO:tasks.workunit.client.0.smithi110.stdout:6/956: dread - d1/dd/d3c/d91/d8d/dce/d102/f10e zero size 2024-04-25T15:49:51.872 INFO:tasks.workunit.client.0.smithi110.stdout:2/921: dread de/d40/d70/d11c/d5c/d5f/db2/fa0 [0,4194304] 0 2024-04-25T15:49:51.873 INFO:tasks.workunit.client.0.smithi110.stdout:6/957: truncate d1/db/f8c 1179156 0 2024-04-25T15:49:51.881 INFO:tasks.workunit.client.1.smithi193.stdout:3/875: rename d8/d4d/l105 to d8/d13/d1f/d69/dcd/l10e 0 2024-04-25T15:49:51.881 INFO:tasks.workunit.client.0.smithi110.stdout:2/922: symlink de/d40/dde/l12a 0 2024-04-25T15:49:51.882 INFO:tasks.workunit.client.1.smithi193.stdout:3/876: stat d8/d13/d14/d23/d108 0 2024-04-25T15:49:51.882 INFO:tasks.workunit.client.1.smithi193.stdout:3/877: chown d8/d13/d4b/d8b/d9a/lc0 9 1 2024-04-25T15:49:51.883 INFO:tasks.workunit.client.1.smithi193.stdout:7/864: fsync d12/d1c/d63/d70/f25 0 2024-04-25T15:49:51.884 INFO:tasks.workunit.client.1.smithi193.stdout:7/865: chown d12/d31/d37/d104/f94 0 1 2024-04-25T15:49:51.885 INFO:tasks.workunit.client.0.smithi110.stdout:2/923: creat de/d40/d70/d11c/d5c/dbe/f12b x:0 0 0 2024-04-25T15:49:51.886 INFO:tasks.workunit.client.1.smithi193.stdout:2/840: unlink d0/d9/d15/d31/f127 0 2024-04-25T15:49:51.887 INFO:tasks.workunit.client.1.smithi193.stdout:4/930: symlink db/d84/d110/l12c 0 2024-04-25T15:49:51.888 INFO:tasks.workunit.client.1.smithi193.stdout:4/931: fdatasync db/f3b 0 2024-04-25T15:49:51.892 INFO:tasks.workunit.client.1.smithi193.stdout:1/770: dread - dc/d59/dc0/deb/d55/fcc zero size 2024-04-25T15:49:51.893 INFO:tasks.workunit.client.1.smithi193.stdout:1/771: chown dc/d59/dc0/deb/db0 13905 1 2024-04-25T15:49:51.894 INFO:tasks.workunit.client.1.smithi193.stdout:1/772: write dc/d4a/d4d/d7a/dd0/dd2/ff7 [1043772,78921] 0 2024-04-25T15:49:51.894 INFO:tasks.workunit.client.1.smithi193.stdout:1/773: chown dc/d4a/d4d/d7a/da6/le1 20 1 2024-04-25T15:49:51.895 INFO:tasks.workunit.client.1.smithi193.stdout:6/845: rename dd/db5/dc9/le3 to dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/l100 0 2024-04-25T15:49:51.896 INFO:tasks.workunit.client.1.smithi193.stdout:0/862: creat d0/d49/d5/d7/f129 x:0 0 0 2024-04-25T15:49:51.898 INFO:tasks.workunit.client.1.smithi193.stdout:3/878: unlink d8/d30/cc1 0 2024-04-25T15:49:51.901 INFO:tasks.workunit.client.1.smithi193.stdout:7/866: rmdir d12/d31/d39/d60 39 2024-04-25T15:49:51.901 INFO:tasks.workunit.client.1.smithi193.stdout:7/867: chown d12/d31/d37/d104/f74 413 1 2024-04-25T15:49:51.903 INFO:tasks.workunit.client.1.smithi193.stdout:2/841: rmdir d0/d37/d25/d5f/d3a/d6d 39 2024-04-25T15:49:51.906 INFO:tasks.workunit.client.1.smithi193.stdout:8/872: getdents d2/d3/d27/d69/dbf/d106 0 2024-04-25T15:49:51.907 INFO:tasks.workunit.client.1.smithi193.stdout:9/828: mkdir df/d2c/d2f/dfa 0 2024-04-25T15:49:51.908 INFO:tasks.workunit.client.1.smithi193.stdout:9/829: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/lcb 1165786 1 2024-04-25T15:49:51.910 INFO:tasks.workunit.client.1.smithi193.stdout:6/846: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dd7/f101 x:0 0 0 2024-04-25T15:49:51.912 INFO:tasks.workunit.client.1.smithi193.stdout:9/830: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/fbb [0,4194304] 0 2024-04-25T15:49:51.914 INFO:tasks.workunit.client.0.smithi110.stdout:9/916: dwrite d9/db/df/d2c/d3c/d56/da0/f118 [0,4194304] 0 2024-04-25T15:49:51.915 INFO:tasks.workunit.client.1.smithi193.stdout:2/842: mknod d0/d9/d15/df3/c134 0 2024-04-25T15:49:51.918 INFO:tasks.workunit.client.1.smithi193.stdout:1/774: symlink dc/d4a/d50/d5c/d5d/l103 0 2024-04-25T15:49:51.922 INFO:tasks.workunit.client.0.smithi110.stdout:6/958: dwrite d1/dd/d3c/f5d [8388608,4194304] 0 2024-04-25T15:49:51.923 INFO:tasks.workunit.client.0.smithi110.stdout:9/917: write d9/db/df/d25/f4c [1588149,101673] 0 2024-04-25T15:49:51.926 INFO:tasks.workunit.client.1.smithi193.stdout:6/847: symlink dd/db5/l102 0 2024-04-25T15:49:51.927 INFO:tasks.workunit.client.1.smithi193.stdout:0/863: dread d0/d49/d5/d7/dd/d16/d18/d28/f3e [0,4194304] 0 2024-04-25T15:49:51.929 INFO:tasks.workunit.client.1.smithi193.stdout:6/848: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa [826716,122609] 0 2024-04-25T15:49:51.932 INFO:tasks.workunit.client.0.smithi110.stdout:9/918: rename f6 to d9/d44/d5e/db5/de7/f120 0 2024-04-25T15:49:51.933 INFO:tasks.workunit.client.0.smithi110.stdout:2/924: dwrite de/d40/d70/f84 [0,4194304] 0 2024-04-25T15:49:51.934 INFO:tasks.workunit.client.1.smithi193.stdout:7/868: symlink d12/d31/d37/l13d 0 2024-04-25T15:49:51.936 INFO:tasks.workunit.client.0.smithi110.stdout:6/959: creat d1/db/d22/d90/df9/f14d x:0 0 0 2024-04-25T15:49:51.937 INFO:tasks.workunit.client.1.smithi193.stdout:2/843: fsync d0/d37/f92 0 2024-04-25T15:49:51.938 INFO:tasks.workunit.client.1.smithi193.stdout:9/831: dread df/d2c/d2f/d4e/d1c/f76 [4194304,4194304] 0 2024-04-25T15:49:51.938 INFO:tasks.workunit.client.1.smithi193.stdout:5/864: dwrite d17/d18/d30/d38/d71/dd7/de7/f111 [4194304,4194304] 0 2024-04-25T15:49:51.940 INFO:tasks.workunit.client.1.smithi193.stdout:9/832: fsync df/d2c/d34/faa 0 2024-04-25T15:49:51.943 INFO:tasks.workunit.client.0.smithi110.stdout:6/960: dwrite d1/db/d24/d2d/f3a [4194304,4194304] 0 2024-04-25T15:49:51.943 INFO:tasks.workunit.client.1.smithi193.stdout:1/775: unlink dc/d4a/d50/l56 0 2024-04-25T15:49:51.943 INFO:tasks.workunit.client.1.smithi193.stdout:4/932: write db/d13/f21 [6828853,102106] 0 2024-04-25T15:49:51.946 INFO:tasks.workunit.client.0.smithi110.stdout:2/925: chown de/d40/f3e 879767 1 2024-04-25T15:49:51.948 INFO:tasks.workunit.client.1.smithi193.stdout:3/879: creat d8/d13/f10f x:0 0 0 2024-04-25T15:49:51.951 INFO:tasks.workunit.client.0.smithi110.stdout:6/961: rename d1/dd/db5/fe3 to d1/db/d22/d90/df7/d11e/f14e 0 2024-04-25T15:49:51.952 INFO:tasks.workunit.client.1.smithi193.stdout:9/833: dwrite df/d2c/d34/fdb [0,4194304] 0 2024-04-25T15:49:51.955 INFO:tasks.workunit.client.1.smithi193.stdout:6/849: symlink dd/d12/d19/d1c/d40/dda/l103 0 2024-04-25T15:49:51.956 INFO:tasks.workunit.client.0.smithi110.stdout:2/926: fsync de/d40/f3e 0 2024-04-25T15:49:51.957 INFO:tasks.workunit.client.0.smithi110.stdout:2/927: stat de/d40/d4f/l65 0 2024-04-25T15:49:51.958 INFO:tasks.workunit.client.1.smithi193.stdout:0/864: truncate d0/d49/d5/d7/dd/d16/d26/d2e/d3d/f3f 1008920 0 2024-04-25T15:49:51.960 INFO:tasks.workunit.client.0.smithi110.stdout:9/919: creat d9/d13/d5b/d61/d67/f121 x:0 0 0 2024-04-25T15:49:51.961 INFO:tasks.workunit.client.1.smithi193.stdout:7/869: dread d12/d31/d37/d62/d6e/f8c [0,4194304] 0 2024-04-25T15:49:51.964 INFO:tasks.workunit.client.0.smithi110.stdout:6/962: symlink d1/db/d22/d33/l14f 0 2024-04-25T15:49:51.969 INFO:tasks.workunit.client.0.smithi110.stdout:2/928: rename de/d40/d70/d11c/d5c/c102 to de/d40/dd7/df4/c12c 0 2024-04-25T15:49:51.970 INFO:tasks.workunit.client.1.smithi193.stdout:8/873: dwrite d2/d3/d20/db8/dd0/ff2 [0,4194304] 0 2024-04-25T15:49:51.970 INFO:tasks.workunit.client.1.smithi193.stdout:4/933: mkdir db/d32/de8/d12d 0 2024-04-25T15:49:51.974 INFO:tasks.workunit.client.0.smithi110.stdout:6/963: rename d1/db/d22/d33/d63/dc2/dc7 to d1/db/d22/d90/d76/dad/d150 0 2024-04-25T15:49:51.976 INFO:tasks.workunit.client.0.smithi110.stdout:2/929: symlink de/d40/d4f/l12d 0 2024-04-25T15:49:51.978 INFO:tasks.workunit.client.1.smithi193.stdout:6/850: symlink dd/db5/dc9/l104 0 2024-04-25T15:49:51.980 INFO:tasks.workunit.client.1.smithi193.stdout:7/870: read d12/d31/d37/d104/f94 [67587,87180] 0 2024-04-25T15:49:51.980 INFO:tasks.workunit.client.1.smithi193.stdout:5/865: symlink d17/d18/d75/l127 0 2024-04-25T15:49:51.981 INFO:tasks.workunit.client.1.smithi193.stdout:1/776: mkdir dc/d59/dc0/deb/db0/dff/d104 0 2024-04-25T15:49:51.982 INFO:tasks.workunit.client.1.smithi193.stdout:4/934: read db/f6a [384817,127181] 0 2024-04-25T15:49:51.983 INFO:tasks.workunit.client.1.smithi193.stdout:1/777: chown dc/d4a/d50/d5c/d5d/dfb 614653772 1 2024-04-25T15:49:51.984 INFO:tasks.workunit.client.1.smithi193.stdout:9/834: creat df/ffb x:0 0 0 2024-04-25T15:49:51.984 INFO:tasks.workunit.client.1.smithi193.stdout:6/851: fsync dd/d12/d19/d45/f55 0 2024-04-25T15:49:51.985 INFO:tasks.workunit.client.1.smithi193.stdout:9/835: stat df/d2c/d2f/d4e/f9c 0 2024-04-25T15:49:51.985 INFO:tasks.workunit.client.1.smithi193.stdout:6/852: readlink dd/d12/d19/d45/l95 0 2024-04-25T15:49:51.987 INFO:tasks.workunit.client.1.smithi193.stdout:7/871: rename d12/d31/d37/d62/d6e/f8c to d12/d1c/d63/d70/f13e 0 2024-04-25T15:49:51.987 INFO:tasks.workunit.client.1.smithi193.stdout:4/935: rmdir db/d13/ddd 39 2024-04-25T15:49:51.988 INFO:tasks.workunit.client.1.smithi193.stdout:1/778: mkdir dc/d4a/d4d/d7a/d85/d92/d105 0 2024-04-25T15:49:51.989 INFO:tasks.workunit.client.1.smithi193.stdout:5/866: creat d17/d18/d30/d38/d71/d96/dc4/da7/d115/f128 x:0 0 0 2024-04-25T15:49:51.990 INFO:tasks.workunit.client.1.smithi193.stdout:2/844: link d0/df/dbf/l132 d0/d42/d51/l135 0 2024-04-25T15:49:51.991 INFO:tasks.workunit.client.1.smithi193.stdout:3/880: getdents d8/d13/d14/d27/d54/dd7 0 2024-04-25T15:49:51.992 INFO:tasks.workunit.client.1.smithi193.stdout:1/779: symlink dc/d4a/d50/dab/db8/l106 0 2024-04-25T15:49:51.994 INFO:tasks.workunit.client.1.smithi193.stdout:5/867: creat d17/d18/d3e/dd1/ddc/f129 x:0 0 0 2024-04-25T15:49:51.994 INFO:tasks.workunit.client.1.smithi193.stdout:9/836: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/ffc x:0 0 0 2024-04-25T15:49:51.997 INFO:tasks.workunit.client.1.smithi193.stdout:1/780: rmdir dc/d4a/d77 39 2024-04-25T15:49:51.998 INFO:tasks.workunit.client.1.smithi193.stdout:4/936: dwrite db/d84/f118 [0,4194304] 0 2024-04-25T15:49:51.998 INFO:tasks.workunit.client.1.smithi193.stdout:2/845: getdents d0/d37/d25/d5f/d3a/d12a 0 2024-04-25T15:49:51.998 INFO:tasks.workunit.client.1.smithi193.stdout:8/874: dread d2/d3/d20/f116 [0,4194304] 0 2024-04-25T15:49:51.998 INFO:tasks.workunit.client.1.smithi193.stdout:8/875: stat d2/d3/d3a/f5e 0 2024-04-25T15:49:51.999 INFO:tasks.workunit.client.0.smithi110.stdout:2/930: write de/d6f/fdc [1446012,95859] 0 2024-04-25T15:49:52.001 INFO:tasks.workunit.client.0.smithi110.stdout:2/931: stat de/d40/d70/d11c/d5c/d5f/db2/da9/cc5 0 2024-04-25T15:49:52.003 INFO:tasks.workunit.client.1.smithi193.stdout:5/868: mknod d17/d18/d30/d38/d71/dd7/de7/c12a 0 2024-04-25T15:49:52.005 INFO:tasks.workunit.client.1.smithi193.stdout:4/937: creat db/d32/d88/f12e x:0 0 0 2024-04-25T15:49:52.006 INFO:tasks.workunit.client.1.smithi193.stdout:4/938: readlink db/d84/df3/l102 0 2024-04-25T15:49:52.007 INFO:tasks.workunit.client.0.smithi110.stdout:2/932: symlink de/d40/dcd/ddf/l12e 0 2024-04-25T15:49:52.014 INFO:tasks.workunit.client.1.smithi193.stdout:5/869: dread - d17/f89 zero size 2024-04-25T15:49:52.014 INFO:tasks.workunit.client.0.smithi110.stdout:9/920: truncate d9/db/df/f68 1989233 0 2024-04-25T15:49:52.016 INFO:tasks.workunit.client.0.smithi110.stdout:2/933: rename de/d40/d70/d104 to de/d40/d6e/d52/d12f 0 2024-04-25T15:49:52.017 INFO:tasks.workunit.client.1.smithi193.stdout:4/939: symlink db/d13/dcb/d113/d54/d77/l12f 0 2024-04-25T15:49:52.020 INFO:tasks.workunit.client.1.smithi193.stdout:0/865: dread d0/d49/d5/d7/dd/f73 [0,4194304] 0 2024-04-25T15:49:52.021 INFO:tasks.workunit.client.0.smithi110.stdout:9/921: mkdir d9/db/df/d2c/d122 0 2024-04-25T15:49:52.022 INFO:tasks.workunit.client.0.smithi110.stdout:9/922: readlink d9/d13/l18 0 2024-04-25T15:49:52.023 INFO:tasks.workunit.client.0.smithi110.stdout:6/964: dwrite d1/dd/f17 [8388608,4194304] 0 2024-04-25T15:49:52.025 INFO:tasks.workunit.client.0.smithi110.stdout:6/965: fdatasync d1/f9 0 2024-04-25T15:49:52.025 INFO:tasks.workunit.client.0.smithi110.stdout:6/966: chown d1/db/c14 13696732 1 2024-04-25T15:49:52.028 INFO:tasks.workunit.client.1.smithi193.stdout:9/837: getdents df/d2c/d2f/d4e/d69 0 2024-04-25T15:49:52.028 INFO:tasks.workunit.client.1.smithi193.stdout:7/872: dread d12/d1c/d63/d70/f13e [0,4194304] 0 2024-04-25T15:49:52.030 INFO:tasks.workunit.client.1.smithi193.stdout:5/870: rename d17/d18/d30/d38/d71/d96 to d17/d18/d30/d3b/d12b 0 2024-04-25T15:49:52.032 INFO:tasks.workunit.client.1.smithi193.stdout:4/940: mknod db/d13/dcb/d113/d54/d77/df6/c130 0 2024-04-25T15:49:52.033 INFO:tasks.workunit.client.0.smithi110.stdout:6/967: creat d1/dd/d3c/d91/d8d/dce/da1/f151 x:0 0 0 2024-04-25T15:49:52.034 INFO:tasks.workunit.client.1.smithi193.stdout:7/873: chown d12/d31/d39/d60/fa8 29 1 2024-04-25T15:49:52.036 INFO:tasks.workunit.client.1.smithi193.stdout:1/781: write dc/f30 [4040381,102189] 0 2024-04-25T15:49:52.036 INFO:tasks.workunit.client.1.smithi193.stdout:4/941: read db/d13/d1e/d1f/d56/d69/fe0 [3009096,54015] 0 2024-04-25T15:49:52.036 INFO:tasks.workunit.client.1.smithi193.stdout:7/874: write d12/f2c [4635465,12049] 0 2024-04-25T15:49:52.037 INFO:tasks.workunit.client.1.smithi193.stdout:5/871: fdatasync d17/d18/d30/f35 0 2024-04-25T15:49:52.039 INFO:tasks.workunit.client.0.smithi110.stdout:9/923: rename d9/d13/d5b/ld5 to d9/d44/d5e/l123 0 2024-04-25T15:49:52.039 INFO:tasks.workunit.client.1.smithi193.stdout:3/881: dwrite f2 [0,4194304] 0 2024-04-25T15:49:52.039 INFO:tasks.workunit.client.1.smithi193.stdout:0/866: dread d0/d49/d5/d7/dd/d16/d26/d5e/fc8 [4194304,4194304] 0 2024-04-25T15:49:52.041 INFO:tasks.workunit.client.1.smithi193.stdout:3/882: chown d8/d13/d4b/d8b/de3/f107 166959 1 2024-04-25T15:49:52.042 INFO:tasks.workunit.client.1.smithi193.stdout:2/846: write d0/d37/d4d/dbe/fd8 [678120,86892] 0 2024-04-25T15:49:52.044 INFO:tasks.workunit.client.0.smithi110.stdout:6/968: creat d1/db/d22/f152 x:0 0 0 2024-04-25T15:49:52.044 INFO:tasks.workunit.client.0.smithi110.stdout:9/924: read d9/d2d/f5d [4096697,54740] 0 2024-04-25T15:49:52.048 INFO:tasks.workunit.client.1.smithi193.stdout:4/942: mknod db/d13/d1e/d1f/d56/c131 0 2024-04-25T15:49:52.048 INFO:tasks.workunit.client.1.smithi193.stdout:5/872: dread d17/d18/d75/d85/d5d/dad/fb5 [0,4194304] 0 2024-04-25T15:49:52.050 INFO:tasks.workunit.client.1.smithi193.stdout:1/782: truncate dc/f47 2669660 0 2024-04-25T15:49:52.052 INFO:tasks.workunit.client.0.smithi110.stdout:9/925: creat d9/d44/d5e/db5/f124 x:0 0 0 2024-04-25T15:49:52.053 INFO:tasks.workunit.client.0.smithi110.stdout:9/926: read - d9/d13/d5b/d61/d67/f121 zero size 2024-04-25T15:49:52.053 INFO:tasks.workunit.client.0.smithi110.stdout:9/927: readlink d9/db/df/l6e 0 2024-04-25T15:49:52.054 INFO:tasks.workunit.client.1.smithi193.stdout:1/783: read - dc/d4a/d4d/d7a/fd7 zero size 2024-04-25T15:49:52.054 INFO:tasks.workunit.client.1.smithi193.stdout:3/883: dwrite d8/d30/d45/fc3 [0,4194304] 0 2024-04-25T15:49:52.054 INFO:tasks.workunit.client.0.smithi110.stdout:6/969: truncate d1/db/da0/da5/f126 562894 0 2024-04-25T15:49:52.055 INFO:tasks.workunit.client.1.smithi193.stdout:8/876: dwrite d2/d3/d20/d8c/dde/fd2 [0,4194304] 0 2024-04-25T15:49:52.058 INFO:tasks.workunit.client.1.smithi193.stdout:9/838: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/ffd x:0 0 0 2024-04-25T15:49:52.058 INFO:tasks.workunit.client.1.smithi193.stdout:8/877: write d2/d18/dbc/dd3/f111 [663485,7677] 0 2024-04-25T15:49:52.059 INFO:tasks.workunit.client.0.smithi110.stdout:6/970: symlink d1/db/d24/daa/l153 0 2024-04-25T15:49:52.060 INFO:tasks.workunit.client.1.smithi193.stdout:6/853: sync 2024-04-25T15:49:52.063 INFO:tasks.workunit.client.1.smithi193.stdout:2/847: write d0/d37/d25/d5f/d3a/d6d/d85/f99 [2041441,22996] 0 2024-04-25T15:49:52.064 INFO:tasks.workunit.client.0.smithi110.stdout:6/971: creat d1/dd/d3c/f154 x:0 0 0 2024-04-25T15:49:52.066 INFO:tasks.workunit.client.1.smithi193.stdout:4/943: mknod db/d13/d1e/d1f/d56/d69/c132 0 2024-04-25T15:49:52.067 INFO:tasks.workunit.client.1.smithi193.stdout:5/873: dread - d17/d18/d30/d38/d48/fa0 zero size 2024-04-25T15:49:52.067 INFO:tasks.workunit.client.0.smithi110.stdout:6/972: fdatasync d1/db/d22/d33/d43/f60 0 2024-04-25T15:49:52.068 INFO:tasks.workunit.client.1.smithi193.stdout:5/874: chown d17/d18/d30/d38/d71 17901 1 2024-04-25T15:49:52.070 INFO:tasks.workunit.client.1.smithi193.stdout:8/878: dread d2/d3/ff [0,4194304] 0 2024-04-25T15:49:52.070 INFO:tasks.workunit.client.0.smithi110.stdout:9/928: dread d9/db/d7d/f8b [0,4194304] 0 2024-04-25T15:49:52.072 INFO:tasks.workunit.client.0.smithi110.stdout:6/973: creat d1/db/d22/d90/d76/dad/d150/f155 x:0 0 0 2024-04-25T15:49:52.073 INFO:tasks.workunit.client.1.smithi193.stdout:0/867: mkdir d0/d49/d5/d7/dd/d16/d18/d28/d125/d12a 0 2024-04-25T15:49:52.074 INFO:tasks.workunit.client.0.smithi110.stdout:9/929: truncate d9/d13/d5b/d61/d67/dd7/f6b 1392465 0 2024-04-25T15:49:52.078 INFO:tasks.workunit.client.1.smithi193.stdout:6/854: rmdir dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0 39 2024-04-25T15:49:52.086 INFO:tasks.workunit.client.1.smithi193.stdout:5/875: mkdir d17/d18/d3e/db6/d12c 0 2024-04-25T15:49:52.087 INFO:tasks.workunit.client.1.smithi193.stdout:9/839: dread df/d2c/d2f/d4e/d1c/d30/d60/f85 [0,4194304] 0 2024-04-25T15:49:52.089 INFO:tasks.workunit.client.1.smithi193.stdout:9/840: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/f94 [2380534,87127] 0 2024-04-25T15:49:52.089 INFO:tasks.workunit.client.1.smithi193.stdout:7/875: link d12/d31/d37/d62/d6e/lc8 d12/d1c/dd6/dda/l13f 0 2024-04-25T15:49:52.090 INFO:tasks.workunit.client.1.smithi193.stdout:6/855: stat dd/d12/d19/d1c/d40/l33 0 2024-04-25T15:49:52.091 INFO:tasks.workunit.client.1.smithi193.stdout:9/841: chown df/d2d/f39 1909 1 2024-04-25T15:49:52.093 INFO:tasks.workunit.client.1.smithi193.stdout:3/884: link d8/d13/d14/d27/d54/dd7/fed d8/d13/d4b/d8b/de3/f110 0 2024-04-25T15:49:52.096 INFO:tasks.workunit.client.1.smithi193.stdout:8/879: dwrite d2/d3/d20/fe8 [0,4194304] 0 2024-04-25T15:49:52.098 INFO:tasks.workunit.client.1.smithi193.stdout:8/880: read - d2/d3/d27/d69/dae/f10c zero size 2024-04-25T15:49:52.099 INFO:tasks.workunit.client.1.smithi193.stdout:1/784: dwrite dc/d4a/d4d/d7a/da6/fb5 [0,4194304] 0 2024-04-25T15:49:52.099 INFO:tasks.workunit.client.1.smithi193.stdout:0/868: creat d0/d49/f12b x:0 0 0 2024-04-25T15:49:52.101 INFO:tasks.workunit.client.1.smithi193.stdout:2/848: getdents d0/df/dbf 0 2024-04-25T15:49:52.101 INFO:tasks.workunit.client.1.smithi193.stdout:3/885: symlink d8/d4d/l111 0 2024-04-25T15:49:52.102 INFO:tasks.workunit.client.1.smithi193.stdout:0/869: dread - d0/d49/d5/d7/dd/d16/d26/d5e/d99/f128 zero size 2024-04-25T15:49:52.105 INFO:tasks.workunit.client.1.smithi193.stdout:0/870: dread - d0/d49/d5/d7/dd/d16/d26/d5e/d99/f128 zero size 2024-04-25T15:49:52.106 INFO:tasks.workunit.client.0.smithi110.stdout:6/974: dwrite d1/dd/f8e [0,4194304] 0 2024-04-25T15:49:52.107 INFO:tasks.workunit.client.0.smithi110.stdout:9/930: dwrite d9/d13/d7c/fb7 [4194304,4194304] 0 2024-04-25T15:49:52.109 INFO:tasks.workunit.client.1.smithi193.stdout:8/881: fdatasync d2/d3/d27/d69/dae/d4a/f4e 0 2024-04-25T15:49:52.110 INFO:tasks.workunit.client.1.smithi193.stdout:3/886: dwrite d8/d13/f32 [4194304,4194304] 0 2024-04-25T15:49:52.111 INFO:tasks.workunit.client.1.smithi193.stdout:4/944: dwrite db/d32/d88/fbf [0,4194304] 0 2024-04-25T15:49:52.119 INFO:tasks.workunit.client.0.smithi110.stdout:9/931: creat d9/db/df/d4d/de2/f125 x:0 0 0 2024-04-25T15:49:52.120 INFO:tasks.workunit.client.1.smithi193.stdout:1/785: truncate dc/d59/dc0/deb/f43 7239968 0 2024-04-25T15:49:52.120 INFO:tasks.workunit.client.1.smithi193.stdout:5/876: write f11 [1114858,109828] 0 2024-04-25T15:49:52.123 INFO:tasks.workunit.client.1.smithi193.stdout:1/786: write dc/d4a/d4d/d80/faa [5227904,4042] 0 2024-04-25T15:49:52.125 INFO:tasks.workunit.client.1.smithi193.stdout:1/787: chown dc/d59/dc0/deb/c52 3877 1 2024-04-25T15:49:52.126 INFO:tasks.workunit.client.1.smithi193.stdout:7/876: dwrite d12/d1c/d63/f80 [0,4194304] 0 2024-04-25T15:49:52.127 INFO:tasks.workunit.client.1.smithi193.stdout:0/871: unlink d0/d49/d5/d7/cc9 0 2024-04-25T15:49:52.129 INFO:tasks.workunit.client.1.smithi193.stdout:9/842: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f3c [5210506,36256] 0 2024-04-25T15:49:52.130 INFO:tasks.workunit.client.1.smithi193.stdout:0/872: read - d0/d49/d5/d7/dd/d16/d26/df5/f11b zero size 2024-04-25T15:49:52.130 INFO:tasks.workunit.client.1.smithi193.stdout:9/843: truncate df/ffb 974172 0 2024-04-25T15:49:52.130 INFO:tasks.workunit.client.0.smithi110.stdout:6/975: creat d1/d12/f156 x:0 0 0 2024-04-25T15:49:52.132 INFO:tasks.workunit.client.1.smithi193.stdout:4/945: chown db/d13/dd2/feb 10469 1 2024-04-25T15:49:52.132 INFO:tasks.workunit.client.1.smithi193.stdout:9/844: fsync df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/deb/ff1 0 2024-04-25T15:49:52.133 INFO:tasks.workunit.client.0.smithi110.stdout:9/932: dread d9/d13/d84/f92 [0,4194304] 0 2024-04-25T15:49:52.134 INFO:tasks.workunit.client.0.smithi110.stdout:9/933: chown d9/d44/d5e/fc1 1 1 2024-04-25T15:49:52.134 INFO:tasks.workunit.client.0.smithi110.stdout:6/976: fdatasync d1/fe 0 2024-04-25T15:49:52.136 INFO:tasks.workunit.client.0.smithi110.stdout:6/977: dread - d1/db/d22/d90/d76/dad/d150/f155 zero size 2024-04-25T15:49:52.137 INFO:tasks.workunit.client.0.smithi110.stdout:6/978: chown d1/db/d24/d2d/d11a 12124443 1 2024-04-25T15:49:52.137 INFO:tasks.workunit.client.0.smithi110.stdout:6/979: chown d1/d48/c70 14787626 1 2024-04-25T15:49:52.140 INFO:tasks.workunit.client.0.smithi110.stdout:6/980: truncate d1/db/d22/d90/df9/f14d 172291 0 2024-04-25T15:49:52.140 INFO:tasks.workunit.client.1.smithi193.stdout:0/873: unlink d0/fda 0 2024-04-25T15:49:52.144 INFO:tasks.workunit.client.1.smithi193.stdout:9/845: creat df/d2c/d2f/d5c/da3/dac/ffe x:0 0 0 2024-04-25T15:49:52.145 INFO:tasks.workunit.client.1.smithi193.stdout:2/849: write d0/d37/d25/d5f/d3a/d6d/fcd [1147465,49436] 0 2024-04-25T15:49:52.147 INFO:tasks.workunit.client.1.smithi193.stdout:1/788: mknod dc/c107 0 2024-04-25T15:49:52.148 INFO:tasks.workunit.client.1.smithi193.stdout:2/850: read d0/d37/d4d/dbe/df7/ffa [1322342,123950] 0 2024-04-25T15:49:52.149 INFO:tasks.workunit.client.1.smithi193.stdout:2/851: dread - d0/d37/d4d/f109 zero size 2024-04-25T15:49:52.149 INFO:tasks.workunit.client.1.smithi193.stdout:3/887: dread d8/d30/d72/fa3 [0,4194304] 0 2024-04-25T15:49:52.150 INFO:tasks.workunit.client.1.smithi193.stdout:3/888: dread - d8/d13/d14/d27/fb6 zero size 2024-04-25T15:49:52.151 INFO:tasks.workunit.client.1.smithi193.stdout:4/946: symlink db/d13/dd2/l133 0 2024-04-25T15:49:52.152 INFO:tasks.workunit.client.1.smithi193.stdout:0/874: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/f12c x:0 0 0 2024-04-25T15:49:52.152 INFO:tasks.workunit.client.0.smithi110.stdout:6/981: link d1/dd/d3c/l80 d1/db/d24/d2d/l157 0 2024-04-25T15:49:52.153 INFO:tasks.workunit.client.0.smithi110.stdout:6/982: readlink d1/d12/l58 0 2024-04-25T15:49:52.153 INFO:tasks.workunit.client.1.smithi193.stdout:6/856: getdents dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0 0 2024-04-25T15:49:52.157 INFO:tasks.workunit.client.0.smithi110.stdout:9/934: dwrite d9/db/df/d11d/f8a [0,4194304] 0 2024-04-25T15:49:52.158 INFO:tasks.workunit.client.1.smithi193.stdout:7/877: link d12/d1c/d63/d70/d68/d8d/f88 d12/d1c/d96/d12e/f140 0 2024-04-25T15:49:52.161 INFO:tasks.workunit.client.1.smithi193.stdout:2/852: creat d0/d37/d25/d5f/d3a/d6d/f136 x:0 0 0 2024-04-25T15:49:52.162 INFO:tasks.workunit.client.0.smithi110.stdout:6/983: dwrite d1/db/d22/d33/d63/dc2/f107 [0,4194304] 0 2024-04-25T15:49:52.167 INFO:tasks.workunit.client.1.smithi193.stdout:3/889: truncate d8/d30/d72/fa3 2673890 0 2024-04-25T15:49:52.167 INFO:tasks.workunit.client.1.smithi193.stdout:0/875: creat d0/d49/d5/d7/dd/d16/d26/d2e/d93/f12d x:0 0 0 2024-04-25T15:49:52.167 INFO:tasks.workunit.client.1.smithi193.stdout:3/890: stat d8 0 2024-04-25T15:49:52.169 INFO:tasks.workunit.client.0.smithi110.stdout:6/984: truncate d1/db/d22/d90/fcb 2024171 0 2024-04-25T15:49:52.171 INFO:tasks.workunit.client.0.smithi110.stdout:6/985: truncate d1/db/d22/d33/d43/f4e 4814556 0 2024-04-25T15:49:52.172 INFO:tasks.workunit.client.1.smithi193.stdout:9/846: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/df4/fff x:0 0 0 2024-04-25T15:49:52.175 INFO:tasks.workunit.client.1.smithi193.stdout:3/891: dwrite d8/d30/d45/fbe [0,4194304] 0 2024-04-25T15:49:52.178 INFO:tasks.workunit.client.1.smithi193.stdout:2/853: mknod d0/d9/d15/d31/d50/c137 0 2024-04-25T15:49:52.178 INFO:tasks.workunit.client.0.smithi110.stdout:9/935: write d9/d13/f1a [457195,6496] 0 2024-04-25T15:49:52.181 INFO:tasks.workunit.client.1.smithi193.stdout:0/876: symlink d0/d49/d5/d7/dd/d46/d48/d62/dd9/l12e 0 2024-04-25T15:49:52.181 INFO:tasks.workunit.client.1.smithi193.stdout:9/847: dread - df/d2c/d34/d3e/d4d/d5e/fb5 zero size 2024-04-25T15:49:52.183 INFO:tasks.workunit.client.1.smithi193.stdout:7/878: dwrite d12/d31/d37/d92/dbf/fc2 [0,4194304] 0 2024-04-25T15:49:52.183 INFO:tasks.workunit.client.1.smithi193.stdout:1/789: rename dc/lad to dc/d59/dc0/deb/d55/l108 0 2024-04-25T15:49:52.186 INFO:tasks.workunit.client.1.smithi193.stdout:7/879: write d12/d13/f14 [6673631,94151] 0 2024-04-25T15:49:52.186 INFO:tasks.workunit.client.1.smithi193.stdout:4/947: getdents db/d32/d40 0 2024-04-25T15:49:52.189 INFO:tasks.workunit.client.1.smithi193.stdout:9/848: read - df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f8a zero size 2024-04-25T15:49:52.192 INFO:tasks.workunit.client.1.smithi193.stdout:8/882: sync 2024-04-25T15:49:52.193 INFO:tasks.workunit.client.0.smithi110.stdout:6/986: truncate d1/db/d24/daa/fdb 415027 0 2024-04-25T15:49:52.195 INFO:tasks.workunit.client.1.smithi193.stdout:0/877: rename d0/d49/d5/d7/dd/d78/d119 to d0/d49/d5/d7/dd/d16/d26/d12f 0 2024-04-25T15:49:52.197 INFO:tasks.workunit.client.0.smithi110.stdout:6/987: creat d1/dd/f158 x:0 0 0 2024-04-25T15:49:52.197 INFO:tasks.workunit.client.1.smithi193.stdout:7/880: creat d12/d31/f141 x:0 0 0 2024-04-25T15:49:52.198 INFO:tasks.workunit.client.1.smithi193.stdout:2/854: dwrite d0/df/f10 [8388608,4194304] 0 2024-04-25T15:49:52.199 INFO:tasks.workunit.client.1.smithi193.stdout:1/790: dwrite dc/d4a/d4d/d7a/f96 [0,4194304] 0 2024-04-25T15:49:52.199 INFO:tasks.workunit.client.1.smithi193.stdout:9/849: creat df/d2c/d34/d3e/d4d/d5e/f100 x:0 0 0 2024-04-25T15:49:52.200 INFO:tasks.workunit.client.0.smithi110.stdout:6/988: mkdir d1/d48/d159 0 2024-04-25T15:49:52.201 INFO:tasks.workunit.client.1.smithi193.stdout:6/857: dwrite dd/d12/d19/d45/fc7 [0,4194304] 0 2024-04-25T15:49:52.202 INFO:tasks.workunit.client.0.smithi110.stdout:5/981: sync 2024-04-25T15:49:52.202 INFO:tasks.workunit.client.1.smithi193.stdout:4/948: rename db/cd to db/d32/de8/d12d/c134 0 2024-04-25T15:49:52.204 INFO:tasks.workunit.client.1.smithi193.stdout:1/791: write dc/d3c/fe6 [1075077,96727] 0 2024-04-25T15:49:52.210 INFO:tasks.workunit.client.1.smithi193.stdout:7/881: creat d12/d31/d37/d62/db3/f142 x:0 0 0 2024-04-25T15:49:52.210 INFO:tasks.workunit.client.0.smithi110.stdout:6/989: dwrite d1/dd/f14a [0,4194304] 0 2024-04-25T15:49:52.217 INFO:tasks.workunit.client.1.smithi193.stdout:9/850: symlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/l101 0 2024-04-25T15:49:52.218 INFO:tasks.workunit.client.0.smithi110.stdout:9/936: write d9/db/df/d2c/d3c/fa4 [1849516,50805] 0 2024-04-25T15:49:52.221 INFO:tasks.workunit.client.0.smithi110.stdout:6/990: rmdir d1/d12 39 2024-04-25T15:49:52.222 INFO:tasks.workunit.client.1.smithi193.stdout:3/892: write d8/d13/d1f/d69/d77/fbb [10909,29049] 0 2024-04-25T15:49:52.223 INFO:tasks.workunit.client.1.smithi193.stdout:4/949: creat db/d13/dcb/d113/d54/d77/f135 x:0 0 0 2024-04-25T15:49:52.224 INFO:tasks.workunit.client.1.smithi193.stdout:4/950: read - db/d13/d1e/d1f/fb3 zero size 2024-04-25T15:49:52.225 INFO:tasks.workunit.client.0.smithi110.stdout:9/937: mknod d9/d13/d5b/d61/db3/c126 0 2024-04-25T15:49:52.231 INFO:tasks.workunit.client.1.smithi193.stdout:3/893: mknod d8/d13/d1f/d69/dcd/dd9/c112 0 2024-04-25T15:49:52.233 INFO:tasks.workunit.client.1.smithi193.stdout:2/855: rename d0/d37/d25/dda/fe0 to d0/d37/d25/d5f/d3a/f138 0 2024-04-25T15:49:52.234 INFO:tasks.workunit.client.1.smithi193.stdout:5/877: sync 2024-04-25T15:49:52.234 INFO:tasks.workunit.client.1.smithi193.stdout:1/792: dread dc/f1b [0,4194304] 0 2024-04-25T15:49:52.238 INFO:tasks.workunit.client.1.smithi193.stdout:9/851: mkdir df/d2c/d2f/d4e/d1c/d30/d102 0 2024-04-25T15:49:52.239 INFO:tasks.workunit.client.0.smithi110.stdout:5/982: dread d4/d1d/d1f/d35/d3d/f58 [4194304,4194304] 0 2024-04-25T15:49:52.241 INFO:tasks.workunit.client.1.smithi193.stdout:6/858: creat dd/d12/d19/d1c/d40/f105 x:0 0 0 2024-04-25T15:49:52.242 INFO:tasks.workunit.client.1.smithi193.stdout:3/894: mknod d8/d13/d4b/d8b/d9a/c113 0 2024-04-25T15:49:52.244 INFO:tasks.workunit.client.1.smithi193.stdout:7/882: creat d12/d31/d39/f143 x:0 0 0 2024-04-25T15:49:52.246 INFO:tasks.workunit.client.0.smithi110.stdout:5/983: symlink d4/d15/d51/d90/ded/def/l147 0 2024-04-25T15:49:52.248 INFO:tasks.workunit.client.1.smithi193.stdout:2/856: mknod d0/d9/d3d/d114/c139 0 2024-04-25T15:49:52.250 INFO:tasks.workunit.client.1.smithi193.stdout:5/878: fdatasync d17/d18/d30/d3b/d12b/fd2 0 2024-04-25T15:49:52.252 INFO:tasks.workunit.client.1.smithi193.stdout:2/857: dread d0/d42/d51/d4f/fc4 [0,4194304] 0 2024-04-25T15:49:52.253 INFO:tasks.workunit.client.0.smithi110.stdout:5/984: creat d4/d1d/d1f/d35/d3d/d52/d75/d113/d123/f148 x:0 0 0 2024-04-25T15:49:52.254 INFO:tasks.workunit.client.1.smithi193.stdout:8/883: dwrite d2/d3/d3a/f5e [0,4194304] 0 2024-04-25T15:49:52.260 INFO:tasks.workunit.client.1.smithi193.stdout:0/878: dwrite d0/d49/faa [0,4194304] 0 2024-04-25T15:49:52.262 INFO:tasks.workunit.client.0.smithi110.stdout:9/938: write d9/db/df/d25/fc4 [475357,62770] 0 2024-04-25T15:49:52.268 INFO:tasks.workunit.client.0.smithi110.stdout:9/939: rename d9/db/df/d2c/d3c/d56/da0/f118 to d9/db/df/d2c/d122/f127 0 2024-04-25T15:49:52.269 INFO:tasks.workunit.client.1.smithi193.stdout:3/895: dread d8/d13/d1f/d69/d77/d71/f86 [0,4194304] 0 2024-04-25T15:49:52.271 INFO:tasks.workunit.client.1.smithi193.stdout:6/859: symlink dd/d12/d19/d1c/l106 0 2024-04-25T15:49:52.273 INFO:tasks.workunit.client.1.smithi193.stdout:2/858: truncate d0/d37/d25/d5f/d3a/d6d/d6e/fe2 897787 0 2024-04-25T15:49:52.273 INFO:tasks.workunit.client.1.smithi193.stdout:4/951: write db/d13/d1e/d1f/fbe [724031,113865] 0 2024-04-25T15:49:52.274 INFO:tasks.workunit.client.0.smithi110.stdout:9/940: creat d9/db/d58/f128 x:0 0 0 2024-04-25T15:49:52.276 INFO:tasks.workunit.client.1.smithi193.stdout:0/879: unlink d0/d49/f12b 0 2024-04-25T15:49:52.279 INFO:tasks.workunit.client.1.smithi193.stdout:9/852: creat df/d2c/d2f/d4e/d1c/d30/f103 x:0 0 0 2024-04-25T15:49:52.283 INFO:tasks.workunit.client.1.smithi193.stdout:6/860: stat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/cdd 0 2024-04-25T15:49:52.284 INFO:tasks.workunit.client.1.smithi193.stdout:1/793: getdents dc/d4a/d4d 0 2024-04-25T15:49:52.284 INFO:tasks.workunit.client.1.smithi193.stdout:7/883: write d12/d31/fd9 [455152,10049] 0 2024-04-25T15:49:52.285 INFO:tasks.workunit.client.0.smithi110.stdout:9/941: rmdir d9/d13/d5b/d61/d67/dde 0 2024-04-25T15:49:52.285 INFO:tasks.workunit.client.1.smithi193.stdout:7/884: stat d12/d31/d37/d62/d6e/lf0 0 2024-04-25T15:49:52.286 INFO:tasks.workunit.client.1.smithi193.stdout:7/885: read d12/d31/d37/d62/d6e/da5/f108 [3083905,2221] 0 2024-04-25T15:49:52.288 INFO:tasks.workunit.client.0.smithi110.stdout:9/942: creat d9/d44/d5e/db5/f129 x:0 0 0 2024-04-25T15:49:52.288 INFO:tasks.workunit.client.1.smithi193.stdout:5/879: link d17/d18/d30/d38/d48/l6e d17/d53/l12d 0 2024-04-25T15:49:52.290 INFO:tasks.workunit.client.0.smithi110.stdout:5/985: dwrite d4/d15/f1a [0,4194304] 0 2024-04-25T15:49:52.292 INFO:tasks.workunit.client.1.smithi193.stdout:3/896: rename d8/d13/d1f/d69/d77/f3a to d8/d30/d45/f114 0 2024-04-25T15:49:52.296 INFO:tasks.workunit.client.1.smithi193.stdout:4/952: truncate db/d13/d1e/d1f/d22/f2a 958584 0 2024-04-25T15:49:52.297 INFO:tasks.workunit.client.0.smithi110.stdout:9/943: mknod d9/d13/d5b/c12a 0 2024-04-25T15:49:52.298 INFO:tasks.workunit.client.0.smithi110.stdout:5/986: dread d4/d1d/d1f/d35/d3d/d52/f6f [0,4194304] 0 2024-04-25T15:49:52.302 INFO:tasks.workunit.client.0.smithi110.stdout:9/944: mknod d9/db/df/d2c/d3c/d56/c12b 0 2024-04-25T15:49:52.303 INFO:tasks.workunit.client.0.smithi110.stdout:9/945: readlink d9/db/df/l5c 0 2024-04-25T15:49:52.305 INFO:tasks.workunit.client.0.smithi110.stdout:5/987: fsync d4/d15/d51/fa4 0 2024-04-25T15:49:52.307 INFO:tasks.workunit.client.0.smithi110.stdout:9/946: rmdir d9/d13/d5b/d61/db3/db4/dc5 39 2024-04-25T15:49:52.308 INFO:tasks.workunit.client.0.smithi110.stdout:9/947: dread d9/db/df/d11d/dbb/fcd [0,4194304] 0 2024-04-25T15:49:52.313 INFO:tasks.workunit.client.0.smithi110.stdout:9/948: dwrite d9/d13/d5b/d61/db3/f103 [0,4194304] 0 2024-04-25T15:49:52.318 INFO:tasks.workunit.client.0.smithi110.stdout:9/949: chown d9/d13/d5b/d61/db3/db4/dc5/f108 26 1 2024-04-25T15:49:52.319 INFO:tasks.workunit.client.0.smithi110.stdout:5/988: getdents d4/d1d/d1f/d35/d3d/d52/d75/d113/d123 0 2024-04-25T15:49:52.320 INFO:tasks.workunit.client.0.smithi110.stdout:5/989: chown d4/d1d/d1f/d35/d3d/d52 3682929 1 2024-04-25T15:49:52.322 INFO:tasks.workunit.client.0.smithi110.stdout:9/950: creat d9/db/df/d25/f12c x:0 0 0 2024-04-25T15:49:52.323 INFO:tasks.workunit.client.0.smithi110.stdout:5/990: getdents d4/d1d/d1f/d35/d3d/d52/ddf 0 2024-04-25T15:49:52.325 INFO:tasks.workunit.client.0.smithi110.stdout:9/951: unlink d9/d13/d5b/d61/db3/dd3/ddb/fec 0 2024-04-25T15:49:52.326 INFO:tasks.workunit.client.0.smithi110.stdout:9/952: readlink d9/d13/d5b/d61/d67/l11b 0 2024-04-25T15:49:52.327 INFO:tasks.workunit.client.0.smithi110.stdout:5/991: mknod d4/d1d/d1f/d35/d3d/d52/ddf/c149 0 2024-04-25T15:49:52.328 INFO:tasks.workunit.client.0.smithi110.stdout:9/953: mkdir d9/d13/d5b/d61/db3/d12d 0 2024-04-25T15:49:52.329 INFO:tasks.workunit.client.0.smithi110.stdout:5/992: chown d4/d2c/d2a/c3f 0 1 2024-04-25T15:49:52.332 INFO:tasks.workunit.client.0.smithi110.stdout:9/954: creat d9/d44/d5e/f12e x:0 0 0 2024-04-25T15:49:52.334 INFO:tasks.workunit.client.0.smithi110.stdout:5/993: link d4/f8 d4/d1d/d1f/d35/d3d/d52/d75/dd3/f14a 0 2024-04-25T15:49:52.335 INFO:tasks.workunit.client.0.smithi110.stdout:5/994: read - d4/d15/d13c/d92/d9b/dcb/fe3 zero size 2024-04-25T15:49:52.338 INFO:tasks.workunit.client.1.smithi193.stdout:7/886: truncate f7 2713634 0 2024-04-25T15:49:52.339 INFO:tasks.workunit.client.1.smithi193.stdout:9/853: mkdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d104 0 2024-04-25T15:49:52.340 INFO:tasks.workunit.client.1.smithi193.stdout:3/897: truncate d8/d13/d14/d27/f73 4078235 0 2024-04-25T15:49:52.340 INFO:tasks.workunit.client.1.smithi193.stdout:6/861: write dd/d12/d19/d1c/f22 [1974716,122395] 0 2024-04-25T15:49:52.342 INFO:tasks.workunit.client.1.smithi193.stdout:9/854: truncate df/d2c/d2f/d4e/d1c/fc7 1130752 0 2024-04-25T15:49:52.345 INFO:tasks.workunit.client.1.smithi193.stdout:4/953: creat db/d13/dd2/d12a/f136 x:0 0 0 2024-04-25T15:49:52.346 INFO:tasks.workunit.client.1.smithi193.stdout:3/898: unlink d8/d13/f61 0 2024-04-25T15:49:52.346 INFO:tasks.workunit.client.1.smithi193.stdout:4/954: write db/d13/dd2/d12b/f11b [643060,19207] 0 2024-04-25T15:49:52.349 INFO:tasks.workunit.client.1.smithi193.stdout:2/859: rename d0/d42/d51/d4f/d65 to d0/d9/d13a 0 2024-04-25T15:49:52.350 INFO:tasks.workunit.client.1.smithi193.stdout:4/955: write db/d13/dd2/d12b/f11b [128459,8064] 0 2024-04-25T15:49:52.350 INFO:tasks.workunit.client.1.smithi193.stdout:6/862: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa [0,4194304] 0 2024-04-25T15:49:52.351 INFO:tasks.workunit.client.1.smithi193.stdout:9/855: symlink df/d2c/d2f/d4e/d69/de7/l105 0 2024-04-25T15:49:52.353 INFO:tasks.workunit.client.1.smithi193.stdout:5/880: dread d17/d18/d75/f99 [0,4194304] 0 2024-04-25T15:49:52.354 INFO:tasks.workunit.client.1.smithi193.stdout:9/856: stat df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/l74 0 2024-04-25T15:49:52.358 INFO:tasks.workunit.client.1.smithi193.stdout:2/860: readlink d0/d9/d3d/l66 0 2024-04-25T15:49:52.358 INFO:tasks.workunit.client.1.smithi193.stdout:2/861: readlink d0/d37/dbb/df8/d110/l117 0 2024-04-25T15:49:52.369 INFO:tasks.workunit.client.1.smithi193.stdout:8/884: sync 2024-04-25T15:49:52.370 INFO:tasks.workunit.client.1.smithi193.stdout:3/899: link d8/d13/d14/f80 d8/d13/d1f/d69/dcd/dd9/f115 0 2024-04-25T15:49:52.372 INFO:tasks.workunit.client.1.smithi193.stdout:4/956: truncate db/d13/d1e/f48 3552874 0 2024-04-25T15:49:52.373 INFO:tasks.workunit.client.1.smithi193.stdout:4/957: fdatasync db/d32/d88/da8/faf 0 2024-04-25T15:49:52.377 INFO:tasks.workunit.client.0.smithi110.stdout:9/955: link d9/d13/l18 d9/db/df/d2c/l12f 0 2024-04-25T15:49:52.378 INFO:tasks.workunit.client.1.smithi193.stdout:4/958: dwrite db/d32/d40/dee/f114 [0,4194304] 0 2024-04-25T15:49:52.380 INFO:tasks.workunit.client.1.smithi193.stdout:0/880: rename d0/d49/d5/d7/fb to d0/f130 0 2024-04-25T15:49:52.382 INFO:tasks.workunit.client.1.smithi193.stdout:0/881: chown d0/d49/d5/d7/dd/d16/d26/f55 14344924 1 2024-04-25T15:49:52.384 INFO:tasks.workunit.client.1.smithi193.stdout:0/882: read d0/d49/d5/d7/dd/d16/d26/d2e/d93/fb9 [23800,94571] 0 2024-04-25T15:49:52.385 INFO:tasks.workunit.client.1.smithi193.stdout:8/885: readlink d2/l41 0 2024-04-25T15:49:52.386 INFO:tasks.workunit.client.0.smithi110.stdout:5/995: link d4/d15/c66 d4/d1d/d1f/d108/c14b 0 2024-04-25T15:49:52.386 INFO:tasks.workunit.client.0.smithi110.stdout:5/996: chown d4/d1d/lb9 46525810 1 2024-04-25T15:49:52.389 INFO:tasks.workunit.client.0.smithi110.stdout:9/956: creat d9/db/df/d2c/f130 x:0 0 0 2024-04-25T15:49:52.390 INFO:tasks.workunit.client.0.smithi110.stdout:5/997: read - d4/d1d/d1f/d35/d3d/d52/fc9 zero size 2024-04-25T15:49:52.391 INFO:tasks.workunit.client.1.smithi193.stdout:5/881: dwrite d17/d18/fa8 [0,4194304] 0 2024-04-25T15:49:52.393 INFO:tasks.workunit.client.0.smithi110.stdout:9/957: creat d9/d13/d84/f131 x:0 0 0 2024-04-25T15:49:52.394 INFO:tasks.workunit.client.0.smithi110.stdout:5/998: read d4/d1d/d1f/f6e [238411,4124] 0 2024-04-25T15:49:52.394 INFO:tasks.workunit.client.1.smithi193.stdout:1/794: sync 2024-04-25T15:49:52.400 INFO:tasks.workunit.client.1.smithi193.stdout:9/857: creat df/f106 x:0 0 0 2024-04-25T15:49:52.402 INFO:tasks.workunit.client.0.smithi110.stdout:9/958: truncate d9/d44/d5e/fc9 718653 0 2024-04-25T15:49:52.404 INFO:tasks.workunit.client.1.smithi193.stdout:7/887: rename d12/d31/d37/d62/db3/cf7 to d12/d1c/d63/d70/d68/d8f/c144 0 2024-04-25T15:49:52.405 INFO:tasks.workunit.client.1.smithi193.stdout:2/862: truncate d0/d42/d51/d4f/fa7 1067414 0 2024-04-25T15:49:52.408 INFO:tasks.workunit.client.0.smithi110.stdout:5/999: mkdir d4/d15/d14c 0 2024-04-25T15:49:52.409 INFO:tasks.workunit.client.0.smithi110.stdout:4/967: sync 2024-04-25T15:49:52.409 INFO:tasks.workunit.client.0.smithi110.stdout:0/934: sync 2024-04-25T15:49:52.409 INFO:tasks.workunit.client.1.smithi193.stdout:0/883: mkdir d0/d49/d5/d61/dd4/d131 0 2024-04-25T15:49:52.409 INFO:tasks.workunit.client.0.smithi110.stdout:9/959: creat d9/db/df/d11d/f132 x:0 0 0 2024-04-25T15:49:52.411 INFO:tasks.workunit.client.0.smithi110.stdout:0/935: chown d1/d23/d30/d33/d5b/d78/ddb/d115/f120 34159 1 2024-04-25T15:49:52.411 INFO:tasks.workunit.client.1.smithi193.stdout:3/900: truncate d8/d13/d14/d27/d54/dd7/fed 530293 0 2024-04-25T15:49:52.411 INFO:tasks.workunit.client.0.smithi110.stdout:0/936: stat d1/d23/dc5/d130 0 2024-04-25T15:49:52.414 INFO:tasks.workunit.client.0.smithi110.stdout:4/968: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/f14f x:0 0 0 2024-04-25T15:49:52.415 INFO:tasks.workunit.client.1.smithi193.stdout:6/863: getdents dd/d12/d19/d45 0 2024-04-25T15:49:52.415 INFO:tasks.workunit.client.1.smithi193.stdout:6/864: chown dd/dea/ff0 1591210557 1 2024-04-25T15:49:52.416 INFO:tasks.workunit.client.1.smithi193.stdout:6/865: stat dd/d2f/d57/f5c 0 2024-04-25T15:49:52.416 INFO:tasks.workunit.client.0.smithi110.stdout:0/937: symlink d1/d23/d30/d33/d5b/d78/ddb/d10b/l151 0 2024-04-25T15:49:52.417 INFO:tasks.workunit.client.0.smithi110.stdout:9/960: dwrite d9/db/d7d/f109 [0,4194304] 0 2024-04-25T15:49:52.417 INFO:tasks.workunit.client.1.smithi193.stdout:5/882: dread - d17/d18/d30/d38/d71/dd7/fe6 zero size 2024-04-25T15:49:52.418 INFO:tasks.workunit.client.1.smithi193.stdout:5/883: chown d17/d18/d75/d85/d5d/dad 969517522 1 2024-04-25T15:49:52.422 INFO:tasks.workunit.client.0.smithi110.stdout:4/969: mkdir d13/d24/d2d/d36/d150 0 2024-04-25T15:49:52.423 INFO:tasks.workunit.client.0.smithi110.stdout:4/970: chown d13/d24/d2d/d44/d65/d99/daf/la8 0 1 2024-04-25T15:49:52.424 INFO:tasks.workunit.client.0.smithi110.stdout:4/971: chown d13/d24/d2d/d44/da4/fb3 73252 1 2024-04-25T15:49:52.425 INFO:tasks.workunit.client.1.smithi193.stdout:1/795: read dc/d4a/d50/d5c/f91 [1513599,73521] 0 2024-04-25T15:49:52.425 INFO:tasks.workunit.client.0.smithi110.stdout:0/938: readlink d1/lcf 0 2024-04-25T15:49:52.426 INFO:tasks.workunit.client.1.smithi193.stdout:1/796: chown dc/d59/dc0/deb/d55/f73 1656 1 2024-04-25T15:49:52.427 INFO:tasks.workunit.client.0.smithi110.stdout:0/939: write d1/d23/d30/d33/f121 [740743,33552] 0 2024-04-25T15:49:52.428 INFO:tasks.workunit.client.1.smithi193.stdout:7/888: mknod d12/d31/d37/d62/db3/c145 0 2024-04-25T15:49:52.431 INFO:tasks.workunit.client.1.smithi193.stdout:2/863: creat d0/d9/d15/d31/d50/f13b x:0 0 0 2024-04-25T15:49:52.432 INFO:tasks.workunit.client.0.smithi110.stdout:9/961: symlink d9/d13/d5b/d61/d67/l133 0 2024-04-25T15:49:52.434 INFO:tasks.workunit.client.0.smithi110.stdout:0/940: mknod d1/d6c/d7f/c152 0 2024-04-25T15:49:52.434 INFO:tasks.workunit.client.0.smithi110.stdout:9/962: read d9/f24 [1836673,12052] 0 2024-04-25T15:49:52.437 INFO:tasks.workunit.client.1.smithi193.stdout:8/886: mkdir d2/d3/d20/d8c/d11c 0 2024-04-25T15:49:52.438 INFO:tasks.workunit.client.0.smithi110.stdout:9/963: symlink d9/db/d7d/l134 0 2024-04-25T15:49:52.439 INFO:tasks.workunit.client.0.smithi110.stdout:9/964: read - d9/d13/d5b/d61/d67/dd7/df5/f11e zero size 2024-04-25T15:49:52.440 INFO:tasks.workunit.client.0.smithi110.stdout:0/941: truncate d1/d23/d30/d33/d5b/f98 874487 0 2024-04-25T15:49:52.441 INFO:tasks.workunit.client.0.smithi110.stdout:4/972: link d13/d24/d26/dcd/d131/c69 d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/df4/c151 0 2024-04-25T15:49:52.442 INFO:tasks.workunit.client.0.smithi110.stdout:4/973: truncate d13/f149 638048 0 2024-04-25T15:49:52.442 INFO:tasks.workunit.client.1.smithi193.stdout:6/866: truncate dd/f74 4779879 0 2024-04-25T15:49:52.443 INFO:tasks.workunit.client.1.smithi193.stdout:0/884: write d0/d49/d5/d7/dd/d16/d26/d5e/d99/fa3 [878155,92440] 0 2024-04-25T15:49:52.444 INFO:tasks.workunit.client.0.smithi110.stdout:2/934: sync 2024-04-25T15:49:52.444 INFO:tasks.workunit.client.0.smithi110.stdout:6/991: sync 2024-04-25T15:49:52.445 INFO:tasks.workunit.client.0.smithi110.stdout:0/942: creat d1/d23/d30/d33/d5b/dbb/da8/f153 x:0 0 0 2024-04-25T15:49:52.445 INFO:tasks.workunit.client.1.smithi193.stdout:9/858: mkdir df/d2c/d2f/d4e/d69/d6b/de3/d107 0 2024-04-25T15:49:52.450 INFO:tasks.workunit.client.0.smithi110.stdout:6/992: creat d1/d48/f15a x:0 0 0 2024-04-25T15:49:52.450 INFO:tasks.workunit.client.1.smithi193.stdout:3/901: dread d8/d13/d14/d27/f5a [0,4194304] 0 2024-04-25T15:49:52.451 INFO:tasks.workunit.client.0.smithi110.stdout:6/993: chown d1/db/d24/d2d/fba 290 1 2024-04-25T15:49:52.452 INFO:tasks.workunit.client.0.smithi110.stdout:9/965: write d9/db/df/d4d/f78 [729006,62926] 0 2024-04-25T15:49:52.459 INFO:tasks.workunit.client.1.smithi193.stdout:2/864: unlink d0/d37/d25/d5f/d3a/d6d/d6e/d8e/fe6 0 2024-04-25T15:49:52.460 INFO:tasks.workunit.client.1.smithi193.stdout:9/859: dread df/d2d/fba [0,4194304] 0 2024-04-25T15:49:52.460 INFO:tasks.workunit.client.1.smithi193.stdout:2/865: fdatasync d0/d42/d51/df1/d9c/fba 0 2024-04-25T15:49:52.460 INFO:tasks.workunit.client.0.smithi110.stdout:4/974: mknod d13/d24/d2d/d44/d65/d99/d146/c152 0 2024-04-25T15:49:52.461 INFO:tasks.workunit.client.0.smithi110.stdout:2/935: write de/d40/dcd/ff5 [377892,43307] 0 2024-04-25T15:49:52.461 INFO:tasks.workunit.client.0.smithi110.stdout:0/943: write d1/f38 [3735155,35575] 0 2024-04-25T15:49:52.462 INFO:tasks.workunit.client.1.smithi193.stdout:9/860: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/deb/ff1 [660860,88799] 0 2024-04-25T15:49:52.464 INFO:tasks.workunit.client.0.smithi110.stdout:9/966: rmdir d9/db/d58 39 2024-04-25T15:49:52.466 INFO:tasks.workunit.client.1.smithi193.stdout:6/867: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0/f107 x:0 0 0 2024-04-25T15:49:52.469 INFO:tasks.workunit.client.1.smithi193.stdout:4/959: getdents db/d13/d1e/d1f/d56/d69/d70 0 2024-04-25T15:49:52.470 INFO:tasks.workunit.client.0.smithi110.stdout:2/936: readlink de/d40/l6b 0 2024-04-25T15:49:52.470 INFO:tasks.workunit.client.0.smithi110.stdout:2/937: stat de/d40/d6e/d50 0 2024-04-25T15:49:52.472 INFO:tasks.workunit.client.1.smithi193.stdout:3/902: rename d8/d13/d4b/f57 to d8/d13/d14/d23/d108/f116 0 2024-04-25T15:49:52.473 INFO:tasks.workunit.client.1.smithi193.stdout:7/889: unlink d12/d1c/dd6/dda/l13f 0 2024-04-25T15:49:52.475 INFO:tasks.workunit.client.0.smithi110.stdout:9/967: symlink d9/db/df/df3/l135 0 2024-04-25T15:49:52.476 INFO:tasks.workunit.client.0.smithi110.stdout:4/975: dwrite f6 [4194304,4194304] 0 2024-04-25T15:49:52.477 INFO:tasks.workunit.client.1.smithi193.stdout:2/866: dread d0/d37/d25/d5f/d3a/d6d/d6e/fe2 [0,4194304] 0 2024-04-25T15:49:52.478 INFO:tasks.workunit.client.0.smithi110.stdout:4/976: readlink d13/d24/d26/dcd/d131/lf7 0 2024-04-25T15:49:52.478 INFO:tasks.workunit.client.1.smithi193.stdout:9/861: unlink df/d2c/d34/fef 0 2024-04-25T15:49:52.480 INFO:tasks.workunit.client.1.smithi193.stdout:8/887: truncate d2/d18/db1/fb3 1425341 0 2024-04-25T15:49:52.480 INFO:tasks.workunit.client.0.smithi110.stdout:6/994: link d1/db/d22/d90/d9e/lae d1/dd/d3c/d91/d8d/dce/l15b 0 2024-04-25T15:49:52.481 INFO:tasks.workunit.client.1.smithi193.stdout:8/888: write d2/d3/d27/d69/dae/d4a/d63/da2/fb9 [274455,68653] 0 2024-04-25T15:49:52.482 INFO:tasks.workunit.client.1.smithi193.stdout:5/884: creat d17/d18/d3e/f12e x:0 0 0 2024-04-25T15:49:52.484 INFO:tasks.workunit.client.1.smithi193.stdout:6/868: rmdir dd/d12/d19/d1c/d40/dce/de2/df1 39 2024-04-25T15:49:52.484 INFO:tasks.workunit.client.1.smithi193.stdout:3/903: dread d8/d13/d1f/d9f/fdf [0,4194304] 0 2024-04-25T15:49:52.486 INFO:tasks.workunit.client.0.smithi110.stdout:9/968: symlink d9/db/df/d2c/d3c/l136 0 2024-04-25T15:49:52.488 INFO:tasks.workunit.client.0.smithi110.stdout:2/938: dwrite de/d40/d6e/d52/d6d/d99/f114 [0,4194304] 0 2024-04-25T15:49:52.491 INFO:tasks.workunit.client.1.smithi193.stdout:1/797: rename dc/d4a/d50/d5c/d63/dbe to dc/d4a/d4d/d109 0 2024-04-25T15:49:52.494 INFO:tasks.workunit.client.0.smithi110.stdout:9/969: creat d9/db/df/d11d/f137 x:0 0 0 2024-04-25T15:49:52.496 INFO:tasks.workunit.client.0.smithi110.stdout:0/944: dread d1/d23/d30/d33/d5b/d83/f68 [0,4194304] 0 2024-04-25T15:49:52.501 INFO:tasks.workunit.client.1.smithi193.stdout:2/867: unlink d0/d42/fef 0 2024-04-25T15:49:52.501 INFO:tasks.workunit.client.0.smithi110.stdout:4/977: dread d13/dba/dd8/f11e [0,4194304] 0 2024-04-25T15:49:52.502 INFO:tasks.workunit.client.0.smithi110.stdout:9/970: creat d9/db/d7d/f138 x:0 0 0 2024-04-25T15:49:52.505 INFO:tasks.workunit.client.0.smithi110.stdout:2/939: dwrite de/d40/d6e/f21 [0,4194304] 0 2024-04-25T15:49:52.507 INFO:tasks.workunit.client.1.smithi193.stdout:5/885: mkdir d17/d18/d75/d85/d5d/d12f 0 2024-04-25T15:49:52.510 INFO:tasks.workunit.client.0.smithi110.stdout:0/945: rmdir d1/d23/d30/d33/d5b/d78/ddb 39 2024-04-25T15:49:52.512 INFO:tasks.workunit.client.0.smithi110.stdout:9/971: dwrite d9/d13/d5b/d61/db3/dd3/ddb/fdd [0,4194304] 0 2024-04-25T15:49:52.514 INFO:tasks.workunit.client.0.smithi110.stdout:9/972: write d9/d13/d5b/d61/f9d [5240773,8664] 0 2024-04-25T15:49:52.517 INFO:tasks.workunit.client.1.smithi193.stdout:8/889: dread d2/d3/d3a/f4c [0,4194304] 0 2024-04-25T15:49:52.519 INFO:tasks.workunit.client.1.smithi193.stdout:8/890: chown d2/d3/d27/d69/dae/dd6 1 1 2024-04-25T15:49:52.519 INFO:tasks.workunit.client.1.smithi193.stdout:3/904: dwrite d8/d13/d1f/d9f/fa5 [0,4194304] 0 2024-04-25T15:49:52.520 INFO:tasks.workunit.client.0.smithi110.stdout:0/946: dwrite d1/d23/d30/d33/dfe/f13f [0,4194304] 0 2024-04-25T15:49:52.521 INFO:tasks.workunit.client.0.smithi110.stdout:2/940: creat de/d40/d6e/d52/dda/f130 x:0 0 0 2024-04-25T15:49:52.537 INFO:tasks.workunit.client.0.smithi110.stdout:4/978: dwrite d13/d24/d2d/d36/feb [0,4194304] 0 2024-04-25T15:49:52.538 INFO:tasks.workunit.client.1.smithi193.stdout:5/886: dread d17/d18/d30/d3b/d12b/dc4/f9a [0,4194304] 0 2024-04-25T15:49:52.538 INFO:tasks.workunit.client.1.smithi193.stdout:5/887: readlink d17/d18/d30/d34/l114 0 2024-04-25T15:49:52.539 INFO:tasks.workunit.client.1.smithi193.stdout:0/885: sync 2024-04-25T15:49:52.542 INFO:tasks.workunit.client.0.smithi110.stdout:0/947: write d1/d23/f13b [301188,78733] 0 2024-04-25T15:49:52.546 INFO:tasks.workunit.client.0.smithi110.stdout:9/973: getdents d9/d13/d5b/d61/db3/dd3 0 2024-04-25T15:49:52.547 INFO:tasks.workunit.client.1.smithi193.stdout:1/798: chown dc/d4a/d4d/d7a/d85/cd6 1 1 2024-04-25T15:49:52.548 INFO:tasks.workunit.client.1.smithi193.stdout:7/890: dwrite f7 [0,4194304] 0 2024-04-25T15:49:52.549 INFO:tasks.workunit.client.0.smithi110.stdout:2/941: getdents de/d40/dcd 0 2024-04-25T15:49:52.550 INFO:tasks.workunit.client.1.smithi193.stdout:2/868: fsync d0/d37/dbb/df8/f95 0 2024-04-25T15:49:52.551 INFO:tasks.workunit.client.0.smithi110.stdout:9/974: dread d9/db/df/d2c/d122/f127 [0,4194304] 0 2024-04-25T15:49:52.552 INFO:tasks.workunit.client.0.smithi110.stdout:9/975: write d9/d13/d5b/d61/d67/dd7/f10d [266310,19955] 0 2024-04-25T15:49:52.553 INFO:tasks.workunit.client.1.smithi193.stdout:6/869: truncate dd/d12/d19/d45/fc7 3959013 0 2024-04-25T15:49:52.553 INFO:tasks.workunit.client.1.smithi193.stdout:4/960: getdents db/d4b/dea 0 2024-04-25T15:49:52.555 INFO:tasks.workunit.client.1.smithi193.stdout:3/905: fdatasync d8/d13/d1f/d69/fcf 0 2024-04-25T15:49:52.558 INFO:tasks.workunit.client.1.smithi193.stdout:9/862: rename df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/f2a to df/f108 0 2024-04-25T15:49:52.559 INFO:tasks.workunit.client.1.smithi193.stdout:1/799: creat dc/d59/f10a x:0 0 0 2024-04-25T15:49:52.560 INFO:tasks.workunit.client.0.smithi110.stdout:0/948: write d1/d6c/f94 [2046925,56600] 0 2024-04-25T15:49:52.560 INFO:tasks.workunit.client.0.smithi110.stdout:9/976: dwrite d9/db/df/d11d/f137 [0,4194304] 0 2024-04-25T15:49:52.563 INFO:tasks.workunit.client.1.smithi193.stdout:0/886: mknod d0/d49/d5/d7/dd/d16/d120/c132 0 2024-04-25T15:49:52.565 INFO:tasks.workunit.client.1.smithi193.stdout:8/891: creat d2/f11d x:0 0 0 2024-04-25T15:49:52.566 INFO:tasks.workunit.client.0.smithi110.stdout:4/979: truncate d13/d24/d26/dcd/d131/f10b 1840839 0 2024-04-25T15:49:52.566 INFO:tasks.workunit.client.1.smithi193.stdout:8/892: read - d2/d18/dbc/fce zero size 2024-04-25T15:49:52.567 INFO:tasks.workunit.client.1.smithi193.stdout:8/893: stat d2/d3/c1c 0 2024-04-25T15:49:52.568 INFO:tasks.workunit.client.1.smithi193.stdout:8/894: chown d2/d3/d27/fd7 361 1 2024-04-25T15:49:52.569 INFO:tasks.workunit.client.1.smithi193.stdout:3/906: creat d8/d4d/f117 x:0 0 0 2024-04-25T15:49:52.570 INFO:tasks.workunit.client.1.smithi193.stdout:3/907: write d8/d30/d45/fbe [1431103,81523] 0 2024-04-25T15:49:52.572 INFO:tasks.workunit.client.1.smithi193.stdout:4/961: rename db/d13/d1e/d1f/fb0 to db/d13/dcb/d113/d54/d77/f137 0 2024-04-25T15:49:52.575 INFO:tasks.workunit.client.1.smithi193.stdout:9/863: unlink df/d2c/d2f/d4e/d69/d6b/c93 0 2024-04-25T15:49:52.577 INFO:tasks.workunit.client.1.smithi193.stdout:7/891: symlink d12/d31/d37/d62/d6e/df2/l146 0 2024-04-25T15:49:52.580 INFO:tasks.workunit.client.1.smithi193.stdout:6/870: dwrite dd/d12/d19/d45/f55 [0,4194304] 0 2024-04-25T15:49:52.581 INFO:tasks.workunit.client.1.smithi193.stdout:1/800: truncate f6 202639 0 2024-04-25T15:49:52.581 INFO:tasks.workunit.client.0.smithi110.stdout:0/949: link d1/f32 d1/d6c/d7f/d79/f154 0 2024-04-25T15:49:52.584 INFO:tasks.workunit.client.0.smithi110.stdout:9/977: getdents d9/d44/d5e 0 2024-04-25T15:49:52.585 INFO:tasks.workunit.client.1.smithi193.stdout:2/869: symlink d0/d9/d3d/d114/d124/l13c 0 2024-04-25T15:49:52.585 INFO:tasks.workunit.client.0.smithi110.stdout:4/980: dread d13/d24/d2d/d36/f40 [0,4194304] 0 2024-04-25T15:49:52.589 INFO:tasks.workunit.client.0.smithi110.stdout:0/950: fsync d1/d23/f11e 0 2024-04-25T15:49:52.590 INFO:tasks.workunit.client.1.smithi193.stdout:5/888: creat d17/d18/d30/d38/d71/f130 x:0 0 0 2024-04-25T15:49:52.591 INFO:tasks.workunit.client.1.smithi193.stdout:5/889: chown d17/d18/d30/d3b/d12b/fd2 985 1 2024-04-25T15:49:52.593 INFO:tasks.workunit.client.0.smithi110.stdout:9/978: dwrite d9/db/df/d2c/f130 [0,4194304] 0 2024-04-25T15:49:52.596 INFO:tasks.workunit.client.0.smithi110.stdout:9/979: dread d9/db/d7d/f8b [0,4194304] 0 2024-04-25T15:49:52.601 INFO:tasks.workunit.client.1.smithi193.stdout:3/908: symlink d8/d4d/l118 0 2024-04-25T15:49:52.601 INFO:tasks.workunit.client.1.smithi193.stdout:3/909: fsync d8/d4d/f87 0 2024-04-25T15:49:52.602 INFO:tasks.workunit.client.1.smithi193.stdout:4/962: symlink db/d13/d1e/d1f/d56/d69/d70/l138 0 2024-04-25T15:49:52.602 INFO:tasks.workunit.client.1.smithi193.stdout:4/963: stat db/lfa 0 2024-04-25T15:49:52.604 INFO:tasks.workunit.client.1.smithi193.stdout:9/864: mkdir df/d2c/d34/d3e/d4d/d5e/d109 0 2024-04-25T15:49:52.609 INFO:tasks.workunit.client.1.smithi193.stdout:9/865: dread df/d2c/d34/d3e/d4d/fc0 [0,4194304] 0 2024-04-25T15:49:52.611 INFO:tasks.workunit.client.0.smithi110.stdout:4/981: getdents d13/d24/d26 0 2024-04-25T15:49:52.612 INFO:tasks.workunit.client.1.smithi193.stdout:0/887: mkdir d0/d49/d6a/dad/d133 0 2024-04-25T15:49:52.612 INFO:tasks.workunit.client.0.smithi110.stdout:4/982: read d13/d24/d2d/d44/d65/d99/fee [215812,18502] 0 2024-04-25T15:49:52.613 INFO:tasks.workunit.client.0.smithi110.stdout:0/951: getdents d1/d47/d54 0 2024-04-25T15:49:52.614 INFO:tasks.workunit.client.1.smithi193.stdout:5/890: creat d17/d18/d30/d38/d48/d4f/deb/f131 x:0 0 0 2024-04-25T15:49:52.615 INFO:tasks.workunit.client.0.smithi110.stdout:9/980: creat d9/d13/d5b/f139 x:0 0 0 2024-04-25T15:49:52.616 INFO:tasks.workunit.client.1.smithi193.stdout:5/891: write d17/d18/d9b/fa9 [4610455,93979] 0 2024-04-25T15:49:52.616 INFO:tasks.workunit.client.1.smithi193.stdout:8/895: mknod d2/d3/d3a/c11e 0 2024-04-25T15:49:52.616 INFO:tasks.workunit.client.1.smithi193.stdout:5/892: chown d17/d18/d30/f91 6294278 1 2024-04-25T15:49:52.617 INFO:tasks.workunit.client.1.smithi193.stdout:8/896: write d2/d3/d27/d69/dae/d29/f60 [1955085,122714] 0 2024-04-25T15:49:52.620 INFO:tasks.workunit.client.0.smithi110.stdout:9/981: creat d9/d44/d5e/f13a x:0 0 0 2024-04-25T15:49:52.623 INFO:tasks.workunit.client.0.smithi110.stdout:4/983: creat d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/df4/f153 x:0 0 0 2024-04-25T15:49:52.627 INFO:tasks.workunit.client.1.smithi193.stdout:4/964: write db/d13/dd2/d12b/f104 [3713973,104340] 0 2024-04-25T15:49:52.627 INFO:tasks.workunit.client.1.smithi193.stdout:0/888: dread - d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/fe7 zero size 2024-04-25T15:49:52.628 INFO:tasks.workunit.client.0.smithi110.stdout:9/982: dread d9/f34 [0,4194304] 0 2024-04-25T15:49:52.631 INFO:tasks.workunit.client.1.smithi193.stdout:5/893: dread d17/d18/d75/f99 [0,4194304] 0 2024-04-25T15:49:52.631 INFO:tasks.workunit.client.1.smithi193.stdout:8/897: truncate d2/d3/d3a/f76 3691499 0 2024-04-25T15:49:52.632 INFO:tasks.workunit.client.0.smithi110.stdout:0/952: link d1/d47/l59 d1/d23/d30/d33/d5b/l155 0 2024-04-25T15:49:52.633 INFO:tasks.workunit.client.1.smithi193.stdout:8/898: chown d2/d3/d20/db8 49353164 1 2024-04-25T15:49:52.634 INFO:tasks.workunit.client.1.smithi193.stdout:6/871: creat dd/d2f/f108 x:0 0 0 2024-04-25T15:49:52.635 INFO:tasks.workunit.client.0.smithi110.stdout:4/984: dwrite d13/d24/d2d/d36/ff6 [0,4194304] 0 2024-04-25T15:49:52.635 INFO:tasks.workunit.client.1.smithi193.stdout:3/910: dread d8/d13/d4b/f6e [0,4194304] 0 2024-04-25T15:49:52.637 INFO:tasks.workunit.client.1.smithi193.stdout:1/801: dwrite dc/d4a/d50/d5c/fa2 [0,4194304] 0 2024-04-25T15:49:52.641 INFO:tasks.workunit.client.0.smithi110.stdout:0/953: dwrite d1/d23/d30/d33/d5b/d78/da4/fba [0,4194304] 0 2024-04-25T15:49:52.642 INFO:tasks.workunit.client.1.smithi193.stdout:7/892: getdents d12/d1c/d63/d109 0 2024-04-25T15:49:52.645 INFO:tasks.workunit.client.1.smithi193.stdout:0/889: creat d0/d49/d5/d7/dd/d16/d26/d9e/f134 x:0 0 0 2024-04-25T15:49:52.646 INFO:tasks.workunit.client.0.smithi110.stdout:9/983: dwrite d9/d13/f100 [0,4194304] 0 2024-04-25T15:49:52.647 INFO:tasks.workunit.client.1.smithi193.stdout:5/894: dwrite d17/d18/d75/ff8 [0,4194304] 0 2024-04-25T15:49:52.648 INFO:tasks.workunit.client.1.smithi193.stdout:0/890: fdatasync d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/de0/f10f 0 2024-04-25T15:49:52.651 INFO:tasks.workunit.client.1.smithi193.stdout:2/870: getdents d0/d37/d25/d5f/d3a/d6d/d6e 0 2024-04-25T15:49:52.653 INFO:tasks.workunit.client.1.smithi193.stdout:3/911: fdatasync d8/d13/d4b/d8b/fab 0 2024-04-25T15:49:52.657 INFO:tasks.workunit.client.0.smithi110.stdout:4/985: getdents d13 0 2024-04-25T15:49:52.660 INFO:tasks.workunit.client.0.smithi110.stdout:9/984: truncate d9/d44/d5e/db5/de7/f120 1651244 0 2024-04-25T15:49:52.660 INFO:tasks.workunit.client.1.smithi193.stdout:5/895: mkdir d17/d18/d75/d85/d5d/dad/d132 0 2024-04-25T15:49:52.661 INFO:tasks.workunit.client.1.smithi193.stdout:4/965: dwrite db/d13/d1e/d1f/d22/f7a [0,4194304] 0 2024-04-25T15:49:52.662 INFO:tasks.workunit.client.0.smithi110.stdout:4/986: mkdir d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/df4/d154 0 2024-04-25T15:49:52.666 INFO:tasks.workunit.client.1.smithi193.stdout:3/912: creat d8/d13/d4b/d8b/de3/f119 x:0 0 0 2024-04-25T15:49:52.668 INFO:tasks.workunit.client.0.smithi110.stdout:4/987: rename d13/d24/d2d/d5e/f12d to d13/d24/d2d/d44/d65/d99/daf/d56/d88/dcc/dfa/da2/d117/f155 0 2024-04-25T15:49:52.670 INFO:tasks.workunit.client.0.smithi110.stdout:9/985: getdents d9/d44/d5e 0 2024-04-25T15:49:52.671 INFO:tasks.workunit.client.1.smithi193.stdout:0/891: symlink d0/d49/l135 0 2024-04-25T15:49:52.671 INFO:tasks.workunit.client.0.smithi110.stdout:9/986: stat d9/db/df/d4d/l111 0 2024-04-25T15:49:52.674 INFO:tasks.workunit.client.1.smithi193.stdout:4/966: dread - db/d13/d9d/fb7 zero size 2024-04-25T15:49:52.676 INFO:tasks.workunit.client.0.smithi110.stdout:9/987: mkdir d9/d13/d5b/d13b 0 2024-04-25T15:49:52.677 INFO:tasks.workunit.client.0.smithi110.stdout:4/988: truncate d13/d24/d2d/d44/d65/d99/daf/d56/f82 2882425 0 2024-04-25T15:49:52.680 INFO:tasks.workunit.client.1.smithi193.stdout:7/893: link d12/d1c/da3/cad d12/d1c/da3/c147 0 2024-04-25T15:49:52.680 INFO:tasks.workunit.client.0.smithi110.stdout:9/988: creat d9/d13/d5b/d61/db3/db4/dc5/de4/f13c x:0 0 0 2024-04-25T15:49:52.681 INFO:tasks.workunit.client.0.smithi110.stdout:4/989: creat d13/d24/d2d/dd9/de1/f156 x:0 0 0 2024-04-25T15:49:52.681 INFO:tasks.workunit.client.1.smithi193.stdout:7/894: write d12/d1c/d63/f65 [1398491,1609] 0 2024-04-25T15:49:52.682 INFO:tasks.workunit.client.1.smithi193.stdout:0/892: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/de0/d136 0 2024-04-25T15:49:52.682 INFO:tasks.workunit.client.1.smithi193.stdout:8/899: link d2/d3/d20/cee d2/d3/d27/d69/dae/c11f 0 2024-04-25T15:49:52.682 INFO:tasks.workunit.client.0.smithi110.stdout:9/989: chown d9/db/df/d2c/d122/f127 5510 1 2024-04-25T15:49:52.683 INFO:tasks.workunit.client.1.smithi193.stdout:4/967: symlink db/d32/d45/l139 0 2024-04-25T15:49:52.684 INFO:tasks.workunit.client.0.smithi110.stdout:0/954: dwrite d1/d23/f44 [0,4194304] 0 2024-04-25T15:49:52.685 INFO:tasks.workunit.client.1.smithi193.stdout:6/872: getdents dd/d12/d19/d1c/d40/d3c/d3f 0 2024-04-25T15:49:52.686 INFO:tasks.workunit.client.1.smithi193.stdout:1/802: getdents dc/d59/dc0/deb/d55/d9c 0 2024-04-25T15:49:52.691 INFO:tasks.workunit.client.0.smithi110.stdout:4/990: fsync d13/d24/d2d/d44/d65/d99/daf/d56/d88/f2c 0 2024-04-25T15:49:52.695 INFO:tasks.workunit.client.1.smithi193.stdout:1/803: dwrite dc/d4a/d50/d5c/d5d/f51 [0,4194304] 0 2024-04-25T15:49:52.696 INFO:tasks.workunit.client.0.smithi110.stdout:0/955: dread d1/fd [0,4194304] 0 2024-04-25T15:49:52.697 INFO:tasks.workunit.client.1.smithi193.stdout:2/871: dwrite d0/d37/d25/d5f/d3a/d6d/d85/f8d [0,4194304] 0 2024-04-25T15:49:52.697 INFO:tasks.workunit.client.1.smithi193.stdout:8/900: symlink d2/d18/db1/l120 0 2024-04-25T15:49:52.699 INFO:tasks.workunit.client.1.smithi193.stdout:8/901: write d2/d3/d27/fd7 [443776,130077] 0 2024-04-25T15:49:52.702 INFO:tasks.workunit.client.1.smithi193.stdout:2/872: dread d0/d42/d51/d4f/fc4 [0,4194304] 0 2024-04-25T15:49:52.704 INFO:tasks.workunit.client.1.smithi193.stdout:0/893: rename d0/d49/d5/d7/dd/d16/d26/d5e/c114 to d0/d49/d5/d61/dd4/d131/c137 0 2024-04-25T15:49:52.710 INFO:tasks.workunit.client.0.smithi110.stdout:9/990: write d9/db/df/d2c/d3c/fac [4936249,93873] 0 2024-04-25T15:49:52.711 INFO:tasks.workunit.client.0.smithi110.stdout:0/956: symlink d1/d23/d30/d33/d5b/d78/ddb/l156 0 2024-04-25T15:49:52.711 INFO:tasks.workunit.client.0.smithi110.stdout:9/991: chown d9/d13/d5b/cf6 1262298300 1 2024-04-25T15:49:52.716 INFO:tasks.workunit.client.0.smithi110.stdout:0/957: symlink d1/d23/d30/d33/d5b/d83/de1/l157 0 2024-04-25T15:49:52.717 INFO:tasks.workunit.client.0.smithi110.stdout:4/991: write d13/d24/d2d/d44/d65/d99/daf/d56/d88/f20 [4593274,94317] 0 2024-04-25T15:49:52.718 INFO:tasks.workunit.client.0.smithi110.stdout:9/992: dwrite d9/db/d7d/f138 [0,4194304] 0 2024-04-25T15:49:52.720 INFO:tasks.workunit.client.1.smithi193.stdout:1/804: rename dc/d59/f10a to dc/d59/dc0/deb/d55/d9c/f10b 0 2024-04-25T15:49:52.721 INFO:tasks.workunit.client.1.smithi193.stdout:2/873: truncate d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4/feb 177558 0 2024-04-25T15:49:52.725 INFO:tasks.workunit.client.1.smithi193.stdout:3/913: getdents d8/d13/d1f/d69/d77/d71 0 2024-04-25T15:49:52.726 INFO:tasks.workunit.client.0.smithi110.stdout:9/993: rename d9/d44/d5e/f12e to d9/d2d/dc7/f13d 0 2024-04-25T15:49:52.727 INFO:tasks.workunit.client.1.smithi193.stdout:3/914: dread - d8/d13/d4b/d8b/de3/f107 zero size 2024-04-25T15:49:52.728 INFO:tasks.workunit.client.1.smithi193.stdout:7/895: link d12/d31/fc7 d12/d31/d37/f148 0 2024-04-25T15:49:52.729 INFO:tasks.workunit.client.0.smithi110.stdout:4/992: mknod d13/d24/d2d/c157 0 2024-04-25T15:49:52.732 INFO:tasks.workunit.client.1.smithi193.stdout:6/873: rename dd/d12/d2d/la2 to dd/d12/d19/d1c/d40/dce/de2/df1/l109 0 2024-04-25T15:49:52.733 INFO:tasks.workunit.client.1.smithi193.stdout:2/874: creat d0/d42/d51/d4f/f13d x:0 0 0 2024-04-25T15:49:52.735 INFO:tasks.workunit.client.0.smithi110.stdout:4/993: link d13/d24/d2d/d44/da4/fca d13/d24/d2d/dd9/de1/f158 0 2024-04-25T15:49:52.736 INFO:tasks.workunit.client.0.smithi110.stdout:4/994: fsync d13/d24/d2d/f11b 0 2024-04-25T15:49:52.737 INFO:tasks.workunit.client.1.smithi193.stdout:4/968: link db/d13/dd2/fd5 db/d84/d110/f13a 0 2024-04-25T15:49:52.738 INFO:tasks.workunit.client.0.smithi110.stdout:4/995: creat d13/d24/d2d/dd9/f159 x:0 0 0 2024-04-25T15:49:52.740 INFO:tasks.workunit.client.1.smithi193.stdout:2/875: truncate d0/df/dbf/fbc 945622 0 2024-04-25T15:49:52.741 INFO:tasks.workunit.client.1.smithi193.stdout:3/915: link d8/d30/f42 d8/d13/d14/d27/d54/dd7/f11a 0 2024-04-25T15:49:52.742 INFO:tasks.workunit.client.1.smithi193.stdout:9/866: sync 2024-04-25T15:49:52.744 INFO:tasks.workunit.client.1.smithi193.stdout:4/969: creat db/d13/d1e/f13b x:0 0 0 2024-04-25T15:49:52.744 INFO:tasks.workunit.client.1.smithi193.stdout:4/970: stat f2 0 2024-04-25T15:49:52.746 INFO:tasks.workunit.client.1.smithi193.stdout:3/916: dread d8/d13/d1f/d9f/fa5 [0,4194304] 0 2024-04-25T15:49:52.750 INFO:tasks.workunit.client.0.smithi110.stdout:0/958: dread d1/d23/d30/d33/d35/d14a/db4/db9/fd3 [0,4194304] 0 2024-04-25T15:49:52.750 INFO:tasks.workunit.client.0.smithi110.stdout:4/996: dread d13/d24/d2d/d36/fb0 [0,4194304] 0 2024-04-25T15:49:52.750 INFO:tasks.workunit.client.1.smithi193.stdout:0/894: getdents d0/d49/d5/d7/dd/d16/d26/d5e 0 2024-04-25T15:49:52.751 INFO:tasks.workunit.client.0.smithi110.stdout:0/959: fdatasync d1/d47/d54/f71 0 2024-04-25T15:49:52.751 INFO:tasks.workunit.client.1.smithi193.stdout:0/895: readlink d0/d49/d5/d7/dd/d46/d48/d62/dd9/l12e 0 2024-04-25T15:49:52.751 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:52 smithi110 ceph-mon[26344]: pgmap v43: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 80 MiB/s rd, 185 MiB/s wr, 411 op/s 2024-04-25T15:49:52.752 INFO:tasks.workunit.client.0.smithi110.stdout:4/997: fsync d13/d24/d2d/d5e/d122/d135/f140 0 2024-04-25T15:49:52.755 INFO:tasks.workunit.client.0.smithi110.stdout:9/994: dread d9/d2d/f30 [0,4194304] 0 2024-04-25T15:49:52.758 INFO:tasks.workunit.client.0.smithi110.stdout:0/960: dwrite d1/d23/d30/d33/d5b/f9b [0,4194304] 0 2024-04-25T15:49:52.760 INFO:tasks.workunit.client.0.smithi110.stdout:4/998: symlink d13/d24/d2d/d5e/l15a 0 2024-04-25T15:49:52.761 INFO:tasks.workunit.client.1.smithi193.stdout:7/896: write d12/d31/d37/d62/d6e/fbe [388360,52350] 0 2024-04-25T15:49:52.762 INFO:tasks.workunit.client.1.smithi193.stdout:7/897: write d12/d1c/d63/d70/f34 [2998483,50975] 0 2024-04-25T15:49:52.764 INFO:tasks.workunit.client.1.smithi193.stdout:1/805: rename dc/f3f to dc/d59/dc0/deb/d55/d8a/f10c 0 2024-04-25T15:49:52.765 INFO:tasks.workunit.client.0.smithi110.stdout:9/995: mknod d9/db/df/d2c/d3c/d56/c13e 0 2024-04-25T15:49:52.766 INFO:tasks.workunit.client.1.smithi193.stdout:7/898: dread d12/d1c/dd6/d101/f115 [0,4194304] 0 2024-04-25T15:49:52.767 INFO:tasks.workunit.client.1.smithi193.stdout:3/917: mkdir d8/d13/d1f/d69/dcd/dd9/d11b 0 2024-04-25T15:49:52.769 INFO:tasks.workunit.client.1.smithi193.stdout:7/899: write d12/d1c/d63/d109/fea [2784015,106222] 0 2024-04-25T15:49:52.771 INFO:tasks.workunit.client.1.smithi193.stdout:1/806: dwrite dc/d4a/d50/fef [0,4194304] 0 2024-04-25T15:49:52.773 INFO:tasks.workunit.client.1.smithi193.stdout:1/807: dread - dc/d59/dc0/deb/fcd zero size 2024-04-25T15:49:52.773 INFO:tasks.workunit.client.0.smithi110.stdout:9/996: mknod d9/d13/d5b/d61/db3/dd3/c13f 0 2024-04-25T15:49:52.774 INFO:tasks.workunit.client.0.smithi110.stdout:9/997: chown d9/d13/d5b/d61/d67/dd7/led 9476143 1 2024-04-25T15:49:52.781 INFO:tasks.workunit.client.1.smithi193.stdout:0/896: creat d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/f138 x:0 0 0 2024-04-25T15:49:52.781 INFO:tasks.workunit.client.1.smithi193.stdout:2/876: write d0/d37/dbb/df8/fca [557822,94594] 0 2024-04-25T15:49:52.782 INFO:tasks.workunit.client.1.smithi193.stdout:9/867: mknod df/d2c/d2f/d4e/d1c/d30/c10a 0 2024-04-25T15:49:52.782 INFO:tasks.workunit.client.0.smithi110.stdout:0/961: getdents d1/d23/d30/d33/d5b/d78/d107/dee 0 2024-04-25T15:49:52.785 INFO:tasks.workunit.client.1.smithi193.stdout:4/971: dread f2 [0,4194304] 0 2024-04-25T15:49:52.785 INFO:tasks.workunit.client.0.smithi110.stdout:4/999: getdents d13/d24/d2d/dd9/dbb 0 2024-04-25T15:49:52.787 INFO:tasks.workunit.client.1.smithi193.stdout:2/877: write d0/d9/d15/d31/fd9 [456596,78085] 0 2024-04-25T15:49:52.787 INFO:tasks.workunit.client.1.smithi193.stdout:6/874: getdents dd/d2f/d57 0 2024-04-25T15:49:52.788 INFO:tasks.workunit.client.0.smithi110.stdout:9/998: link d9/db/d7d/f8b d9/d2d/dc7/f140 0 2024-04-25T15:49:52.788 INFO:tasks.workunit.client.1.smithi193.stdout:7/900: dwrite d12/d1c/d63/d70/d68/d8d/f9c [0,4194304] 0 2024-04-25T15:49:52.793 INFO:tasks.workunit.client.0.smithi110.stdout:0/962: rename d1/d23/d30/d33/d5b/d78/d107/f104 to d1/d20/f158 0 2024-04-25T15:49:52.794 INFO:tasks.workunit.client.0.smithi110.stdout:9/999: dwrite d9/db/df/f21 [0,4194304] 0 2024-04-25T15:49:52.795 INFO:tasks.workunit.client.1.smithi193.stdout:2/878: dread d0/d37/f43 [0,4194304] 0 2024-04-25T15:49:52.796 INFO:tasks.workunit.client.1.smithi193.stdout:4/972: dwrite db/d13/d1e/d1f/f64 [0,4194304] 0 2024-04-25T15:49:52.796 INFO:tasks.workunit.client.1.smithi193.stdout:1/808: creat dc/d4a/d4d/d7a/dd0/de2/df1/f10d x:0 0 0 2024-04-25T15:49:52.799 INFO:tasks.workunit.client.1.smithi193.stdout:1/809: write dc/d4a/d50/d7c/fd9 [100307,103717] 0 2024-04-25T15:49:52.799 INFO:tasks.workunit.client.1.smithi193.stdout:0/897: rmdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60 39 2024-04-25T15:49:52.800 INFO:tasks.workunit.client.1.smithi193.stdout:4/973: chown db/d13/dcb/d113/d54/d77/ff5 6469081 1 2024-04-25T15:49:52.802 INFO:tasks.workunit.client.1.smithi193.stdout:7/901: creat d12/d31/d37/d104/f149 x:0 0 0 2024-04-25T15:49:52.804 INFO:tasks.workunit.client.1.smithi193.stdout:2/879: unlink d0/d37/d25/fa9 0 2024-04-25T15:49:52.805 INFO:tasks.workunit.client.1.smithi193.stdout:7/902: write d12/d31/d39/d60/fa8 [3435438,51799] 0 2024-04-25T15:49:52.806 INFO:tasks.workunit.client.1.smithi193.stdout:0/898: creat d0/d49/d5/d7/dd/d16/d18/d28/d56/f139 x:0 0 0 2024-04-25T15:49:52.806 INFO:tasks.workunit.client.1.smithi193.stdout:7/903: fdatasync d12/f10b 0 2024-04-25T15:49:52.809 INFO:tasks.workunit.client.1.smithi193.stdout:4/974: link db/f38 db/d13/d9d/f13c 0 2024-04-25T15:49:52.809 INFO:tasks.workunit.client.1.smithi193.stdout:5/896: sync 2024-04-25T15:49:52.809 INFO:tasks.workunit.client.1.smithi193.stdout:4/975: chown db/d84/df3/d100 378279244 1 2024-04-25T15:49:52.810 INFO:tasks.workunit.client.1.smithi193.stdout:5/897: dread - d17/d18/d3e/f12e zero size 2024-04-25T15:49:52.810 INFO:tasks.workunit.client.1.smithi193.stdout:4/976: chown db/d13/d1e/d1f/d56/d69/d70 20303079 1 2024-04-25T15:49:52.811 INFO:tasks.workunit.client.1.smithi193.stdout:5/898: dread - d17/d18/f11f zero size 2024-04-25T15:49:52.812 INFO:tasks.workunit.client.1.smithi193.stdout:7/904: mkdir d12/d13/dc4/dfa/d14a 0 2024-04-25T15:49:52.812 INFO:tasks.workunit.client.1.smithi193.stdout:7/905: fdatasync d12/d1c/d96/f102 0 2024-04-25T15:49:52.818 INFO:tasks.workunit.client.1.smithi193.stdout:1/810: rename dc/d4a/d4d/d7a/da6/ff0 to dc/d4a/d4d/d7a/f10e 0 2024-04-25T15:49:52.819 INFO:tasks.workunit.client.1.smithi193.stdout:2/880: link d0/d37/d25/d5f/f108 d0/d9/d15/d10b/f13e 0 2024-04-25T15:49:52.819 INFO:tasks.workunit.client.1.smithi193.stdout:0/899: unlink d0/lc6 0 2024-04-25T15:49:52.820 INFO:tasks.workunit.client.1.smithi193.stdout:4/977: fsync db/d32/d40/f99 0 2024-04-25T15:49:52.821 INFO:tasks.workunit.client.1.smithi193.stdout:2/881: stat d0/d9/d13a 0 2024-04-25T15:49:52.824 INFO:tasks.workunit.client.1.smithi193.stdout:0/900: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e 1 1 2024-04-25T15:49:52.828 INFO:tasks.workunit.client.1.smithi193.stdout:9/868: dwrite df/d2c/d2f/d5c/d7e/fc1 [0,4194304] 0 2024-04-25T15:49:52.831 INFO:tasks.workunit.client.1.smithi193.stdout:6/875: dread dd/d12/d19/d1c/d40/f23 [0,4194304] 0 2024-04-25T15:49:52.832 INFO:tasks.workunit.client.1.smithi193.stdout:4/978: dwrite db/d32/d40/dee/f114 [4194304,4194304] 0 2024-04-25T15:49:52.832 INFO:tasks.workunit.client.1.smithi193.stdout:6/876: chown dd/db5 948 1 2024-04-25T15:49:52.833 INFO:tasks.workunit.client.1.smithi193.stdout:2/882: dwrite d0/d37/dbb/df8/f86 [4194304,4194304] 0 2024-04-25T15:49:52.837 INFO:tasks.workunit.client.1.smithi193.stdout:5/899: dwrite d17/d18/f2e [0,4194304] 0 2024-04-25T15:49:52.846 INFO:tasks.workunit.client.1.smithi193.stdout:0/901: mkdir d0/d49/d5/d7/dd/d16/d26/df5/d13a 0 2024-04-25T15:49:52.847 INFO:tasks.workunit.client.1.smithi193.stdout:4/979: rmdir db/d13/d1e/d1f/d117 39 2024-04-25T15:49:52.848 INFO:tasks.workunit.client.1.smithi193.stdout:6/877: mknod dd/d2f/d57/c10a 0 2024-04-25T15:49:52.850 INFO:tasks.workunit.client.1.smithi193.stdout:2/883: symlink d0/d37/d25/d5f/d3a/d6d/d6e/l13f 0 2024-04-25T15:49:52.850 INFO:tasks.workunit.client.1.smithi193.stdout:2/884: chown d0/d37/d25/d5f/d3a/d6d/d6e/d8e 447 1 2024-04-25T15:49:52.850 INFO:tasks.workunit.client.1.smithi193.stdout:5/900: symlink d17/d18/d30/d38/d71/dd7/de7/l133 0 2024-04-25T15:49:52.855 INFO:tasks.workunit.client.1.smithi193.stdout:4/980: creat db/d32/d40/dec/f13d x:0 0 0 2024-04-25T15:49:52.856 INFO:tasks.workunit.client.1.smithi193.stdout:4/981: chown db/f38 1101059 1 2024-04-25T15:49:52.857 INFO:tasks.workunit.client.1.smithi193.stdout:2/885: fdatasync d0/fed 0 2024-04-25T15:49:52.858 INFO:tasks.workunit.client.1.smithi193.stdout:1/811: creat dc/d4a/d50/d5c/d5d/f10f x:0 0 0 2024-04-25T15:49:52.859 INFO:tasks.workunit.client.1.smithi193.stdout:0/902: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/l13b 0 2024-04-25T15:49:52.860 INFO:tasks.workunit.client.1.smithi193.stdout:1/812: chown dc/d59/dc0/deb 310506 1 2024-04-25T15:49:52.862 INFO:tasks.workunit.client.1.smithi193.stdout:5/901: mknod d17/d18/d3e/dd1/c134 0 2024-04-25T15:49:52.863 INFO:tasks.workunit.client.1.smithi193.stdout:2/886: creat d0/df/dbf/f140 x:0 0 0 2024-04-25T15:49:52.864 INFO:tasks.workunit.client.1.smithi193.stdout:1/813: fdatasync dc/d59/dc0/deb/d55/d9c/fd5 0 2024-04-25T15:49:52.866 INFO:tasks.workunit.client.1.smithi193.stdout:5/902: mkdir d17/d18/d75/d85/d5d/dad/dd8/d135 0 2024-04-25T15:49:52.867 INFO:tasks.workunit.client.1.smithi193.stdout:5/903: write d17/d18/d30/d38/d48/d4f/f10d [4995012,8103] 0 2024-04-25T15:49:52.870 INFO:tasks.workunit.client.1.smithi193.stdout:0/903: truncate d0/d49/d5/d7/dd/d16/d26/d5e/d99/f9a 1044384 0 2024-04-25T15:49:52.871 INFO:tasks.workunit.client.1.smithi193.stdout:1/814: fsync dc/d59/dc0/deb/d55/f73 0 2024-04-25T15:49:52.872 INFO:tasks.workunit.client.1.smithi193.stdout:4/982: dread f1 [0,4194304] 0 2024-04-25T15:49:52.874 INFO:tasks.workunit.client.1.smithi193.stdout:5/904: rmdir d17/d18/d30/d38/d48/d4f/deb 39 2024-04-25T15:49:52.875 INFO:tasks.workunit.client.1.smithi193.stdout:5/905: write d17/d18/d30/d38/d48/d4f/f10d [1490930,39668] 0 2024-04-25T15:49:52.876 INFO:tasks.workunit.client.1.smithi193.stdout:2/887: creat d0/d9/d15/d31/f141 x:0 0 0 2024-04-25T15:49:52.877 INFO:tasks.workunit.client.1.smithi193.stdout:0/904: write d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/f138 [277376,82098] 0 2024-04-25T15:49:52.879 INFO:tasks.workunit.client.1.smithi193.stdout:4/983: mkdir db/d13/d1e/d1f/d56/d69/dad/d13e 0 2024-04-25T15:49:52.879 INFO:tasks.workunit.client.1.smithi193.stdout:0/905: readlink d0/d49/d5/d7/dd/d16/d18/la7 0 2024-04-25T15:49:52.880 INFO:tasks.workunit.client.1.smithi193.stdout:4/984: write db/d13/d9d/f109 [587541,79244] 0 2024-04-25T15:49:52.882 INFO:tasks.workunit.client.1.smithi193.stdout:0/906: write d0/d49/d5/d7/dd/d16/d26/d5e/fc8 [501653,92623] 0 2024-04-25T15:49:52.883 INFO:tasks.workunit.client.1.smithi193.stdout:1/815: mkdir dc/d59/dc0/deb/d55/d110 0 2024-04-25T15:49:52.884 INFO:tasks.workunit.client.1.smithi193.stdout:0/907: dread - d0/d49/d5/d7/dd/d16/d26/d5e/f121 zero size 2024-04-25T15:49:52.886 INFO:tasks.workunit.client.1.smithi193.stdout:8/902: sync 2024-04-25T15:49:52.886 INFO:tasks.workunit.client.1.smithi193.stdout:1/816: chown dc/d4a/d50/dab/lbd 112 1 2024-04-25T15:49:52.887 INFO:tasks.workunit.client.1.smithi193.stdout:2/888: truncate d0/d37/d25/d5f/d3a/d6d/d6e/d8e/dd4/feb 201815 0 2024-04-25T15:49:52.888 INFO:tasks.workunit.client.1.smithi193.stdout:0/908: dread d0/d49/faa [0,4194304] 0 2024-04-25T15:49:52.889 INFO:tasks.workunit.client.1.smithi193.stdout:2/889: write d0/d42/d51/d4f/ff9 [2258147,37841] 0 2024-04-25T15:49:52.889 INFO:tasks.workunit.client.1.smithi193.stdout:7/906: truncate d12/f2e 2573714 0 2024-04-25T15:49:52.890 INFO:tasks.workunit.client.1.smithi193.stdout:2/890: stat d0/d42/d51/f46 0 2024-04-25T15:49:52.891 INFO:tasks.workunit.client.1.smithi193.stdout:2/891: fdatasync d0/d9/d15/f35 0 2024-04-25T15:49:52.891 INFO:tasks.workunit.client.1.smithi193.stdout:7/907: write d12/d1c/d63/d109/fea [1392677,67961] 0 2024-04-25T15:49:52.892 INFO:tasks.workunit.client.1.smithi193.stdout:6/878: write dd/d12/f5d [3140820,119892] 0 2024-04-25T15:49:52.895 INFO:tasks.workunit.client.1.smithi193.stdout:0/909: mkdir d0/d49/d5/d7/dd/d46/d48/d62/d13c 0 2024-04-25T15:49:52.897 INFO:tasks.workunit.client.1.smithi193.stdout:1/817: mkdir dc/d59/dc0/deb/d111 0 2024-04-25T15:49:52.899 INFO:tasks.workunit.client.1.smithi193.stdout:2/892: truncate d0/d42/d51/f40 7700854 0 2024-04-25T15:49:52.900 INFO:tasks.workunit.client.1.smithi193.stdout:0/910: symlink d0/d49/d5/d7/dd/d16/d26/d5e/d99/l13d 0 2024-04-25T15:49:52.901 INFO:tasks.workunit.client.1.smithi193.stdout:1/818: rmdir dc/d4a/d50/d5c/d63 39 2024-04-25T15:49:52.902 INFO:tasks.workunit.client.1.smithi193.stdout:1/819: dread - dc/d59/dc0/deb/d55/fcc zero size 2024-04-25T15:49:52.903 INFO:tasks.workunit.client.1.smithi193.stdout:7/908: link d12/d31/fd9 d12/d1c/d63/db7/f14b 0 2024-04-25T15:49:52.905 INFO:tasks.workunit.client.1.smithi193.stdout:2/893: read d0/d42/d51/df1/fea [1845477,55145] 0 2024-04-25T15:49:52.906 INFO:tasks.workunit.client.1.smithi193.stdout:7/909: symlink d12/d31/d37/d104/l14c 0 2024-04-25T15:49:52.907 INFO:tasks.workunit.client.1.smithi193.stdout:5/906: write d17/d18/d30/d3b/d12b/fc7 [1878246,13265] 0 2024-04-25T15:49:52.907 INFO:tasks.workunit.client.0.smithi110.stdout:6/995: sync 2024-04-25T15:49:52.909 INFO:tasks.workunit.client.1.smithi193.stdout:4/985: write db/d13/d1e/d1f/d56/d69/d70/d8a/fb8 [278613,130036] 0 2024-04-25T15:49:52.921 INFO:tasks.workunit.client.1.smithi193.stdout:6/879: truncate dd/d2f/fe6 1315338 0 2024-04-25T15:49:52.923 INFO:tasks.workunit.client.0.smithi110.stdout:6/996: dwrite d1/db/d22/d90/f72 [0,4194304] 0 2024-04-25T15:49:52.923 INFO:tasks.workunit.client.1.smithi193.stdout:5/907: dwrite d17/f97 [0,4194304] 0 2024-04-25T15:49:52.923 INFO:tasks.workunit.client.1.smithi193.stdout:0/911: dread d0/d49/d5/d7/dd/f3c [0,4194304] 0 2024-04-25T15:49:52.928 INFO:tasks.workunit.client.1.smithi193.stdout:7/910: dwrite d12/d31/d39/db0/fd3 [0,4194304] 0 2024-04-25T15:49:52.932 INFO:tasks.workunit.client.1.smithi193.stdout:6/880: write dd/d12/d19/d45/fee [1704022,101360] 0 2024-04-25T15:49:52.932 INFO:tasks.workunit.client.1.smithi193.stdout:3/918: sync 2024-04-25T15:49:52.935 INFO:tasks.workunit.client.1.smithi193.stdout:8/903: dwrite d2/d3/d27/fdf [4194304,4194304] 0 2024-04-25T15:49:52.935 INFO:tasks.workunit.client.1.smithi193.stdout:6/881: mkdir dd/d12/d19/d1c/d40/dda/d10b 0 2024-04-25T15:49:52.935 INFO:tasks.workunit.client.1.smithi193.stdout:5/908: write d17/d18/d30/d3b/f60 [2784364,124766] 0 2024-04-25T15:49:52.938 INFO:tasks.workunit.client.1.smithi193.stdout:0/912: fdatasync d0/d49/d5/d7/f129 0 2024-04-25T15:49:52.940 INFO:tasks.workunit.client.1.smithi193.stdout:4/986: dread db/d13/d1e/d1f/d22/f6b [0,4194304] 0 2024-04-25T15:49:52.941 INFO:tasks.workunit.client.1.smithi193.stdout:2/894: dread d0/fc [0,4194304] 0 2024-04-25T15:49:52.946 INFO:tasks.workunit.client.1.smithi193.stdout:7/911: write d12/d31/d37/d62/d6e/da5/fb6 [372778,2436] 0 2024-04-25T15:49:52.949 INFO:tasks.workunit.client.1.smithi193.stdout:4/987: truncate db/fd0 2205049 0 2024-04-25T15:49:52.950 INFO:tasks.workunit.client.1.smithi193.stdout:8/904: mknod d2/d18/dbc/dd3/c121 0 2024-04-25T15:49:52.952 INFO:tasks.workunit.client.1.smithi193.stdout:0/913: creat d0/d49/d5/d61/f13e x:0 0 0 2024-04-25T15:49:52.956 INFO:tasks.workunit.client.1.smithi193.stdout:1/820: dwrite dc/d4a/d4d/f5e [0,4194304] 0 2024-04-25T15:49:52.958 INFO:tasks.workunit.client.0.smithi110.stdout:6/997: dwrite d1/d12/db8/fe1 [0,4194304] 0 2024-04-25T15:49:52.958 INFO:tasks.workunit.client.0.smithi110.stdout:6/998: chown d1/db/da0/da5/dea 2 1 2024-04-25T15:49:52.959 INFO:tasks.workunit.client.1.smithi193.stdout:3/919: dwrite d8/d13/f10f [0,4194304] 0 2024-04-25T15:49:52.959 INFO:tasks.workunit.client.1.smithi193.stdout:6/882: unlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/c9a 0 2024-04-25T15:49:52.960 INFO:tasks.workunit.client.1.smithi193.stdout:7/912: creat d12/d31/d37/d104/f14d x:0 0 0 2024-04-25T15:49:52.962 INFO:tasks.workunit.client.1.smithi193.stdout:4/988: mkdir db/d32/d88/da8/d13f 0 2024-04-25T15:49:52.962 INFO:tasks.workunit.client.1.smithi193.stdout:3/920: write d8/d13/d1f/f9e [68395,11865] 0 2024-04-25T15:49:52.962 INFO:tasks.workunit.client.1.smithi193.stdout:5/909: dwrite d17/d18/f11f [0,4194304] 0 2024-04-25T15:49:52.968 INFO:tasks.workunit.client.1.smithi193.stdout:6/883: mkdir dd/d12/d2d/d10c 0 2024-04-25T15:49:52.969 INFO:tasks.workunit.client.1.smithi193.stdout:4/989: fsync db/d13/d1e/d1f/d22/fca 0 2024-04-25T15:49:52.970 INFO:tasks.workunit.client.1.smithi193.stdout:5/910: unlink d17/d18/d75/ce2 0 2024-04-25T15:49:52.972 INFO:tasks.workunit.client.1.smithi193.stdout:3/921: creat d8/d30/d72/db2/f11c x:0 0 0 2024-04-25T15:49:52.973 INFO:tasks.workunit.client.1.smithi193.stdout:1/821: creat dc/d4a/d77/f112 x:0 0 0 2024-04-25T15:49:52.976 INFO:tasks.workunit.client.1.smithi193.stdout:0/914: creat d0/d49/d5/d7/dd/d46/d48/d62/f13f x:0 0 0 2024-04-25T15:49:52.976 INFO:tasks.workunit.client.1.smithi193.stdout:0/915: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/de0 3036 1 2024-04-25T15:49:52.979 INFO:tasks.workunit.client.1.smithi193.stdout:4/990: truncate db/f6a 1680273 0 2024-04-25T15:49:52.980 INFO:tasks.workunit.client.1.smithi193.stdout:4/991: write db/d13/d1e/d1f/fbe [21868,42956] 0 2024-04-25T15:49:52.981 INFO:tasks.workunit.client.1.smithi193.stdout:1/822: fdatasync dc/f87 0 2024-04-25T15:49:52.982 INFO:tasks.workunit.client.1.smithi193.stdout:3/922: chown d8/d13/d1f/d69/d77/l46 15718183 1 2024-04-25T15:49:52.983 INFO:tasks.workunit.client.1.smithi193.stdout:3/923: read d8/d13/d4b/f6e [289131,17478] 0 2024-04-25T15:49:52.983 INFO:tasks.workunit.client.1.smithi193.stdout:0/916: fsync d0/d49/d5/d7/dd/d16/d26/d5e/f69 0 2024-04-25T15:49:52.984 INFO:tasks.workunit.client.1.smithi193.stdout:3/924: readlink d8/d13/d1f/dbf/l9d 0 2024-04-25T15:49:52.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:52 smithi193 ceph-mon[33747]: pgmap v43: 65 pgs: 65 active+clean; 3.5 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 80 MiB/s rd, 185 MiB/s wr, 411 op/s 2024-04-25T15:49:52.991 INFO:tasks.workunit.client.1.smithi193.stdout:3/925: fsync d8/d30/fe9 0 2024-04-25T15:49:52.991 INFO:tasks.workunit.client.0.smithi110.stdout:2/942: sync 2024-04-25T15:49:52.992 INFO:tasks.workunit.client.1.smithi193.stdout:0/917: mkdir d0/d49/d5/d7/dd/d16/d18/d140 0 2024-04-25T15:49:52.994 INFO:tasks.workunit.client.0.smithi110.stdout:2/943: creat de/d40/dd7/df4/f131 x:0 0 0 2024-04-25T15:49:52.995 INFO:tasks.workunit.client.1.smithi193.stdout:3/926: symlink d8/d13/d4b/d8b/d9a/l11d 0 2024-04-25T15:49:52.996 INFO:tasks.workunit.client.0.smithi110.stdout:2/944: chown de/d40/d6e/d50 110967 1 2024-04-25T15:49:52.996 INFO:tasks.workunit.client.1.smithi193.stdout:4/992: dwrite db/d32/d40/fdf [0,4194304] 0 2024-04-25T15:49:52.997 INFO:tasks.workunit.client.0.smithi110.stdout:6/999: dwrite d1/dd/f12d [0,4194304] 0 2024-04-25T15:49:52.998 INFO:tasks.workunit.client.1.smithi193.stdout:1/823: dwrite dc/d4a/d50/d5c/f91 [0,4194304] 0 2024-04-25T15:49:52.998 INFO:tasks.workunit.client.1.smithi193.stdout:3/927: chown d8/d13/d14/d51/f5f 1172 1 2024-04-25T15:49:52.999 INFO:tasks.workunit.client.0.smithi110.stdout:2/945: creat de/d40/dd7/f132 x:0 0 0 2024-04-25T15:49:53.000 INFO:tasks.workunit.client.1.smithi193.stdout:3/928: readlink d8/d13/d1f/d9f/lda 0 2024-04-25T15:49:53.002 INFO:tasks.workunit.client.1.smithi193.stdout:0/918: creat d0/d49/d5/d7/dd/d16/d26/d5e/f141 x:0 0 0 2024-04-25T15:49:53.003 INFO:tasks.workunit.client.1.smithi193.stdout:9/869: sync 2024-04-25T15:49:53.004 INFO:tasks.workunit.client.1.smithi193.stdout:4/993: symlink db/d84/l140 0 2024-04-25T15:49:53.006 INFO:tasks.workunit.client.1.smithi193.stdout:2/895: dwrite d0/d42/d51/d4f/fa7 [0,4194304] 0 2024-04-25T15:49:53.011 INFO:tasks.workunit.client.1.smithi193.stdout:0/919: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d93/d142 0 2024-04-25T15:49:53.011 INFO:tasks.workunit.client.1.smithi193.stdout:6/884: write dd/d2f/d57/f69 [1023252,57792] 0 2024-04-25T15:49:53.013 INFO:tasks.workunit.client.1.smithi193.stdout:5/911: truncate d17/d18/d30/d3b/f60 2273994 0 2024-04-25T15:49:53.013 INFO:tasks.workunit.client.1.smithi193.stdout:2/896: chown d0/d37/d25/d5f/f108 22 1 2024-04-25T15:49:53.016 INFO:tasks.workunit.client.1.smithi193.stdout:9/870: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/ffd 11 1 2024-04-25T15:49:53.018 INFO:tasks.workunit.client.1.smithi193.stdout:1/824: link dc/d4a/d4d/d7a/dd0/de2/df1/le3 dc/d59/dc0/l113 0 2024-04-25T15:49:53.019 INFO:tasks.workunit.client.1.smithi193.stdout:2/897: symlink d0/d37/d25/d5f/d3a/d6d/d6e/db9/l142 0 2024-04-25T15:49:53.021 INFO:tasks.workunit.client.1.smithi193.stdout:1/825: read dc/d4a/d4d/d7a/d85/fb6 [1171303,116327] 0 2024-04-25T15:49:53.024 INFO:tasks.workunit.client.1.smithi193.stdout:5/912: creat d17/d18/d30/d34/d52/dee/df7/f136 x:0 0 0 2024-04-25T15:49:53.024 INFO:tasks.workunit.client.1.smithi193.stdout:5/913: readlink d17/d18/d9b/l121 0 2024-04-25T15:49:53.025 INFO:tasks.workunit.client.1.smithi193.stdout:9/871: mkdir df/d2c/d34/d10b 0 2024-04-25T15:49:53.026 INFO:tasks.workunit.client.1.smithi193.stdout:1/826: creat dc/d4a/d77/f114 x:0 0 0 2024-04-25T15:49:53.028 INFO:tasks.workunit.client.1.smithi193.stdout:2/898: creat d0/d37/d4d/dbe/df7/f143 x:0 0 0 2024-04-25T15:49:53.028 INFO:tasks.workunit.client.1.smithi193.stdout:1/827: symlink dc/d4a/l115 0 2024-04-25T15:49:53.030 INFO:tasks.workunit.client.1.smithi193.stdout:9/872: rename df/d2c/d34/d3e/cae to df/d2c/d2f/d4e/d1c/d30/d102/c10c 0 2024-04-25T15:49:53.030 INFO:tasks.workunit.client.1.smithi193.stdout:2/899: creat d0/d42/d51/d4f/f144 x:0 0 0 2024-04-25T15:49:53.031 INFO:tasks.workunit.client.1.smithi193.stdout:3/929: write d8/d13/d14/f3e [2767460,1424] 0 2024-04-25T15:49:53.032 INFO:tasks.workunit.client.1.smithi193.stdout:2/900: rmdir d0/d44 39 2024-04-25T15:49:53.033 INFO:tasks.workunit.client.1.smithi193.stdout:9/873: dread df/d2c/d2f/d4e/d1c/fc7 [0,4194304] 0 2024-04-25T15:49:53.034 INFO:tasks.workunit.client.1.smithi193.stdout:2/901: chown d0/d9/d3d/l66 770 1 2024-04-25T15:49:53.036 INFO:tasks.workunit.client.1.smithi193.stdout:4/994: truncate db/d13/d1e/d1f/d56/d57/fa4 3414828 0 2024-04-25T15:49:53.038 INFO:tasks.workunit.client.0.smithi110.stdout:2/946: dread de/d40/d6e/d50/db7/fe3 [0,4194304] 0 2024-04-25T15:49:53.038 INFO:tasks.workunit.client.1.smithi193.stdout:1/828: link dc/d59/dc0/deb/f95 dc/dc9/f116 0 2024-04-25T15:49:53.039 INFO:tasks.workunit.client.1.smithi193.stdout:5/914: dread d17/d18/d30/d38/d48/d4f/fd6 [0,4194304] 0 2024-04-25T15:49:53.039 INFO:tasks.workunit.client.1.smithi193.stdout:2/902: rmdir d0/df/dbf 39 2024-04-25T15:49:53.039 INFO:tasks.workunit.client.1.smithi193.stdout:1/829: write dc/d4a/d50/d7c/d7e/f8c [895709,120106] 0 2024-04-25T15:49:53.040 INFO:tasks.workunit.client.1.smithi193.stdout:6/885: truncate dd/d2f/d57/daf/fcc 4111179 0 2024-04-25T15:49:53.041 INFO:tasks.workunit.client.1.smithi193.stdout:0/920: write d0/d49/d5/d7/dd/d16/d26/d5e/f69 [653807,58962] 0 2024-04-25T15:49:53.043 INFO:tasks.workunit.client.0.smithi110.stdout:2/947: unlink de/d40/d4f/d105/f106 0 2024-04-25T15:49:53.046 INFO:tasks.workunit.client.1.smithi193.stdout:3/930: mknod d8/d13/d1f/d69/dcd/dd9/d11b/c11e 0 2024-04-25T15:49:53.047 INFO:tasks.workunit.client.0.smithi110.stdout:2/948: rename de/d40/d6e/f21 to de/d40/d70/dfa/f133 0 2024-04-25T15:49:53.054 INFO:tasks.workunit.client.0.smithi110.stdout:2/949: dwrite de/d40/d4f/d9f/f100 [0,4194304] 0 2024-04-25T15:49:53.054 INFO:tasks.workunit.client.1.smithi193.stdout:2/903: dwrite d0/d42/d51/df1/d9c/fba [0,4194304] 0 2024-04-25T15:49:53.055 INFO:tasks.workunit.client.1.smithi193.stdout:0/921: dwrite d0/f35 [0,4194304] 0 2024-04-25T15:49:53.055 INFO:tasks.workunit.client.1.smithi193.stdout:6/886: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0/f107 [0,4194304] 0 2024-04-25T15:49:53.057 INFO:tasks.workunit.client.1.smithi193.stdout:6/887: stat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/f87 0 2024-04-25T15:49:53.060 INFO:tasks.workunit.client.0.smithi110.stdout:2/950: creat de/d40/d6e/d50/db7/f134 x:0 0 0 2024-04-25T15:49:53.061 INFO:tasks.workunit.client.1.smithi193.stdout:1/830: dread dc/f30 [4194304,4194304] 0 2024-04-25T15:49:53.066 INFO:tasks.workunit.client.1.smithi193.stdout:0/922: dwrite d0/d49/d5/d7/dd/d16/d26/d5e/fa8 [0,4194304] 0 2024-04-25T15:49:53.073 INFO:tasks.workunit.client.0.smithi110.stdout:0/963: sync 2024-04-25T15:49:53.073 INFO:tasks.workunit.client.0.smithi110.stdout:2/951: unlink de/d40/l4d 0 2024-04-25T15:49:53.084 INFO:tasks.workunit.client.0.smithi110.stdout:0/964: rmdir d1/d23/d81/df4 39 2024-04-25T15:49:53.086 INFO:tasks.workunit.client.1.smithi193.stdout:9/874: truncate df/d2c/d34/fdb 1193356 0 2024-04-25T15:49:53.087 INFO:tasks.workunit.client.0.smithi110.stdout:0/965: creat d1/d23/d30/d33/d35/f159 x:0 0 0 2024-04-25T15:49:53.090 INFO:tasks.workunit.client.1.smithi193.stdout:0/923: rename d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/dbf to d0/d49/d5/d7/dd/d16/d26/d2e/d143 0 2024-04-25T15:49:53.091 INFO:tasks.workunit.client.0.smithi110.stdout:0/966: dread d1/d23/d30/d33/d35/d14a/db4/db9/fd3 [0,4194304] 0 2024-04-25T15:49:53.092 INFO:tasks.workunit.client.1.smithi193.stdout:2/904: mkdir d0/d42/d51/d145 0 2024-04-25T15:49:53.092 INFO:tasks.workunit.client.0.smithi110.stdout:0/967: fsync d1/d23/d30/d33/d35/f6b 0 2024-04-25T15:49:53.093 INFO:tasks.workunit.client.1.smithi193.stdout:6/888: mkdir dd/d12/d19/d1c/d10d 0 2024-04-25T15:49:53.094 INFO:tasks.workunit.client.1.smithi193.stdout:1/831: creat dc/d59/dc0/deb/d55/d110/f117 x:0 0 0 2024-04-25T15:49:53.095 INFO:tasks.workunit.client.1.smithi193.stdout:6/889: dread - dd/d12/d19/d45/f7e zero size 2024-04-25T15:49:53.095 INFO:tasks.workunit.client.0.smithi110.stdout:2/952: rmdir de 39 2024-04-25T15:49:53.096 INFO:tasks.workunit.client.1.smithi193.stdout:4/995: dwrite db/d13/d1e/f48 [0,4194304] 0 2024-04-25T15:49:53.097 INFO:tasks.workunit.client.1.smithi193.stdout:3/931: creat d8/d13/d14/f11f x:0 0 0 2024-04-25T15:49:53.100 INFO:tasks.workunit.client.0.smithi110.stdout:2/953: chown de/d40/d6e/d52/d6d/fb3 679520614 1 2024-04-25T15:49:53.108 INFO:tasks.workunit.client.1.smithi193.stdout:2/905: rename d0/d42/d51/d4f/c105 to d0/d37/d25/dda/c146 0 2024-04-25T15:49:53.109 INFO:tasks.workunit.client.1.smithi193.stdout:1/832: rmdir dc/d59/dc0/deb 39 2024-04-25T15:49:53.109 INFO:tasks.workunit.client.1.smithi193.stdout:9/875: write df/d2c/d2f/f77 [2357375,82109] 0 2024-04-25T15:49:53.112 INFO:tasks.workunit.client.1.smithi193.stdout:7/913: sync 2024-04-25T15:49:53.112 INFO:tasks.workunit.client.1.smithi193.stdout:8/905: sync 2024-04-25T15:49:53.113 INFO:tasks.workunit.client.1.smithi193.stdout:4/996: unlink db/d13/c23 0 2024-04-25T15:49:53.114 INFO:tasks.workunit.client.1.smithi193.stdout:0/924: truncate d0/d49/d5/d7/dd/d16/d18/f7c 927905 0 2024-04-25T15:49:53.115 INFO:tasks.workunit.client.1.smithi193.stdout:8/906: write d2/d3/d27/fd7 [80371,76084] 0 2024-04-25T15:49:53.116 INFO:tasks.workunit.client.1.smithi193.stdout:2/906: dwrite d0/d9/d15/d31/d50/f9a [8388608,4194304] 0 2024-04-25T15:49:53.116 INFO:tasks.workunit.client.1.smithi193.stdout:9/876: fsync df/d2c/fca 0 2024-04-25T15:49:53.117 INFO:tasks.workunit.client.0.smithi110.stdout:0/968: dwrite d1/d47/d113/f11c [0,4194304] 0 2024-04-25T15:49:53.119 INFO:tasks.workunit.client.0.smithi110.stdout:0/969: write d1/d47/d113/f11c [4134049,47466] 0 2024-04-25T15:49:53.121 INFO:tasks.workunit.client.1.smithi193.stdout:3/932: mkdir d8/d30/d72/de1/d120 0 2024-04-25T15:49:53.121 INFO:tasks.workunit.client.1.smithi193.stdout:6/890: write dd/d12/d19/d1c/d40/fa6 [75696,75839] 0 2024-04-25T15:49:53.126 INFO:tasks.workunit.client.1.smithi193.stdout:9/877: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/ffc 260145 1 2024-04-25T15:49:53.129 INFO:tasks.workunit.client.1.smithi193.stdout:4/997: dwrite db/d13/d1e/f6d [4194304,4194304] 0 2024-04-25T15:49:53.130 INFO:tasks.workunit.client.0.smithi110.stdout:2/954: dread de/d40/d6e/d52/d6d/fb3 [0,4194304] 0 2024-04-25T15:49:53.131 INFO:tasks.workunit.client.1.smithi193.stdout:6/891: write dd/d12/d19/d1c/d40/fbe [572786,62720] 0 2024-04-25T15:49:53.131 INFO:tasks.workunit.client.1.smithi193.stdout:1/833: unlink dc/d3c/f8b 0 2024-04-25T15:49:53.131 INFO:tasks.workunit.client.0.smithi110.stdout:0/970: fdatasync d1/d20/f158 0 2024-04-25T15:49:53.132 INFO:tasks.workunit.client.1.smithi193.stdout:7/914: dwrite d12/d1c/d63/f80 [4194304,4194304] 0 2024-04-25T15:49:53.132 INFO:tasks.workunit.client.1.smithi193.stdout:3/933: read d8/d13/d4b/d8b/de3/f110 [529010,10386] 0 2024-04-25T15:49:53.137 INFO:tasks.workunit.client.0.smithi110.stdout:2/955: dread de/d40/d70/d11c/d5c/d5f/db2/da9/dac/fe1 [0,4194304] 0 2024-04-25T15:49:53.139 INFO:tasks.workunit.client.1.smithi193.stdout:8/907: dwrite d2/d3/d20/db8/dd0/ff2 [0,4194304] 0 2024-04-25T15:49:53.139 INFO:tasks.workunit.client.0.smithi110.stdout:0/971: creat d1/d23/d30/d33/d35/f15a x:0 0 0 2024-04-25T15:49:53.140 INFO:tasks.workunit.client.1.smithi193.stdout:7/915: fsync d12/d31/d37/d104/f4a 0 2024-04-25T15:49:53.142 INFO:tasks.workunit.client.1.smithi193.stdout:2/907: write d0/d9/d15/d31/d50/f6b [15207,110018] 0 2024-04-25T15:49:53.146 INFO:tasks.workunit.client.1.smithi193.stdout:5/915: sync 2024-04-25T15:49:53.147 INFO:tasks.workunit.client.1.smithi193.stdout:4/998: mkdir db/d13/dcb/d141 0 2024-04-25T15:49:53.152 INFO:tasks.workunit.client.0.smithi110.stdout:0/972: symlink d1/d47/d132/l15b 0 2024-04-25T15:49:53.156 INFO:tasks.workunit.client.1.smithi193.stdout:0/925: truncate d0/d49/fe 1078020 0 2024-04-25T15:49:53.156 INFO:tasks.workunit.client.0.smithi110.stdout:0/973: creat d1/d6c/d7f/d79/f15c x:0 0 0 2024-04-25T15:49:53.157 INFO:tasks.workunit.client.0.smithi110.stdout:0/974: chown d1/d23/d9e/cb2 30 1 2024-04-25T15:49:53.157 INFO:tasks.workunit.client.0.smithi110.stdout:0/975: stat d1/d23/f24 0 2024-04-25T15:49:53.158 INFO:tasks.workunit.client.1.smithi193.stdout:1/834: dread dc/d59/f8e [0,4194304] 0 2024-04-25T15:49:53.158 INFO:tasks.workunit.client.1.smithi193.stdout:0/926: write d0/d49/d5/d7/dd/d16/d18/d28/f102 [937025,33444] 0 2024-04-25T15:49:53.158 INFO:tasks.workunit.client.0.smithi110.stdout:0/976: chown d1/d23/d30/d33/d5b/d78/ddb/d10b/f140 157236808 1 2024-04-25T15:49:53.161 INFO:tasks.workunit.client.1.smithi193.stdout:9/878: rename df/d2d to df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/d10d 0 2024-04-25T15:49:53.163 INFO:tasks.workunit.client.1.smithi193.stdout:3/934: write d8/d30/d72/db2/fdc [186646,64409] 0 2024-04-25T15:49:53.163 INFO:tasks.workunit.client.1.smithi193.stdout:4/999: readlink db/d13/d1e/d1f/l52 0 2024-04-25T15:49:53.165 INFO:tasks.workunit.client.1.smithi193.stdout:6/892: write dd/d12/d19/d1c/d40/f23 [268576,119241] 0 2024-04-25T15:49:53.168 INFO:tasks.workunit.client.0.smithi110.stdout:2/956: dwrite de/d40/d70/d11c/d5c/d5f/db2/dad/ff1 [0,4194304] 0 2024-04-25T15:49:53.172 INFO:tasks.workunit.client.1.smithi193.stdout:3/935: truncate d8/d13/d14/da0/fb5 851283 0 2024-04-25T15:49:53.173 INFO:tasks.workunit.client.0.smithi110.stdout:0/977: dread d1/f19 [0,4194304] 0 2024-04-25T15:49:53.174 INFO:tasks.workunit.client.1.smithi193.stdout:6/893: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/fd9 [0,4194304] 0 2024-04-25T15:49:53.178 INFO:tasks.workunit.client.0.smithi110.stdout:0/978: fdatasync d1/d47/f8e 0 2024-04-25T15:49:53.182 INFO:tasks.workunit.client.1.smithi193.stdout:5/916: creat d17/f137 x:0 0 0 2024-04-25T15:49:53.182 INFO:tasks.workunit.client.0.smithi110.stdout:0/979: creat d1/d6c/f15d x:0 0 0 2024-04-25T15:49:53.186 INFO:tasks.workunit.client.1.smithi193.stdout:0/927: dwrite d0/d49/d5/d7/dd/d16/fb8 [0,4194304] 0 2024-04-25T15:49:53.186 INFO:tasks.workunit.client.1.smithi193.stdout:9/879: creat df/d2c/d2f/d5c/d7e/f10e x:0 0 0 2024-04-25T15:49:53.189 INFO:tasks.workunit.client.1.smithi193.stdout:7/916: dwrite d12/d31/d37/d62/d6e/da5/dd7/fe5 [0,4194304] 0 2024-04-25T15:49:53.191 INFO:tasks.workunit.client.1.smithi193.stdout:1/835: unlink dc/d59/dc0/deb/d55/d8a/lf3 0 2024-04-25T15:49:53.192 INFO:tasks.workunit.client.0.smithi110.stdout:0/980: dwrite d1/d23/d9e/ff9 [0,4194304] 0 2024-04-25T15:49:53.193 INFO:tasks.workunit.client.1.smithi193.stdout:5/917: creat d17/d18/d9b/f138 x:0 0 0 2024-04-25T15:49:53.196 INFO:tasks.workunit.client.0.smithi110.stdout:2/957: dwrite de/d40/dcd/ddf/f117 [0,4194304] 0 2024-04-25T15:49:53.197 INFO:tasks.workunit.client.1.smithi193.stdout:3/936: dread d8/d30/d72/f7e [0,4194304] 0 2024-04-25T15:49:53.198 INFO:tasks.workunit.client.0.smithi110.stdout:0/981: read d1/d23/d30/d33/d5b/d83/f84 [1927803,106686] 0 2024-04-25T15:49:53.200 INFO:tasks.workunit.client.0.smithi110.stdout:0/982: chown d1/d23/d30/d33/d35/d14a/c12f 126 1 2024-04-25T15:49:53.202 INFO:tasks.workunit.client.0.smithi110.stdout:0/983: chown d1/d23/d9e/ff9 243982565 1 2024-04-25T15:49:53.202 INFO:tasks.workunit.client.1.smithi193.stdout:1/836: unlink dc/d4a/d50/d7c/d82/lc7 0 2024-04-25T15:49:53.204 INFO:tasks.workunit.client.1.smithi193.stdout:1/837: write dc/d59/dc0/deb/f97 [4117875,32620] 0 2024-04-25T15:49:53.207 INFO:tasks.workunit.client.1.smithi193.stdout:6/894: rename dd/d12/d2d/l53 to dd/d12/d19/d1c/l10e 0 2024-04-25T15:49:53.218 INFO:tasks.workunit.client.1.smithi193.stdout:3/937: creat d8/d13/d1f/d69/dcd/dd9/dde/f121 x:0 0 0 2024-04-25T15:49:53.219 INFO:tasks.workunit.client.0.smithi110.stdout:2/958: write de/d40/d6e/f9d [2028014,72950] 0 2024-04-25T15:49:53.222 INFO:tasks.workunit.client.0.smithi110.stdout:2/959: chown de/d40/l37 8 1 2024-04-25T15:49:53.223 INFO:tasks.workunit.client.0.smithi110.stdout:2/960: stat de/d40/d6e/d52/d6d/d99/c10e 0 2024-04-25T15:49:53.224 INFO:tasks.workunit.client.0.smithi110.stdout:2/961: chown de/d40/dde/f122 3 1 2024-04-25T15:49:53.229 INFO:tasks.workunit.client.1.smithi193.stdout:9/880: rename df/d2c/d2f/d4e/d1c/d30/ced to df/d2c/d2f/d5c/c10f 0 2024-04-25T15:49:53.229 INFO:tasks.workunit.client.1.smithi193.stdout:9/881: chown df/d2c/d2f/d4e/d69/cbd 96486085 1 2024-04-25T15:49:53.230 INFO:tasks.workunit.client.0.smithi110.stdout:2/962: mknod de/d40/dcd/ddf/c135 0 2024-04-25T15:49:53.230 INFO:tasks.workunit.client.1.smithi193.stdout:7/917: write d12/d1c/d96/d12e/f140 [954523,80803] 0 2024-04-25T15:49:53.231 INFO:tasks.workunit.client.1.smithi193.stdout:3/938: mknod d8/d13/d1f/d9f/df9/c122 0 2024-04-25T15:49:53.231 INFO:tasks.workunit.client.1.smithi193.stdout:7/918: chown d12/d31/d37/d104/f125 826542 1 2024-04-25T15:49:53.231 INFO:tasks.workunit.client.1.smithi193.stdout:3/939: stat d8/d13/f10f 0 2024-04-25T15:49:53.233 INFO:tasks.workunit.client.0.smithi110.stdout:2/963: creat de/d40/d70/dc6/f136 x:0 0 0 2024-04-25T15:49:53.234 INFO:tasks.workunit.client.1.smithi193.stdout:9/882: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/f40 [0,4194304] 0 2024-04-25T15:49:53.235 INFO:tasks.workunit.client.1.smithi193.stdout:9/883: chown df/d2c/d34/d3e/d4d/l7f 13 1 2024-04-25T15:49:53.235 INFO:tasks.workunit.client.1.smithi193.stdout:9/884: dread - df/d2c/d2f/d5c/fdd zero size 2024-04-25T15:49:53.236 INFO:tasks.workunit.client.1.smithi193.stdout:7/919: mkdir d12/d1c/d63/d70/d68/d8d/d14e 0 2024-04-25T15:49:53.238 INFO:tasks.workunit.client.1.smithi193.stdout:0/928: rename d0/d49/d5/d7/dd/d16/d26/d5e/d99/la5 to d0/d49/d5/d7/l144 0 2024-04-25T15:49:53.241 INFO:tasks.workunit.client.1.smithi193.stdout:7/920: read d12/d1c/d63/d70/f25 [2844202,52795] 0 2024-04-25T15:49:53.243 INFO:tasks.workunit.client.1.smithi193.stdout:1/838: dwrite dc/d59/dc0/deb/d55/fcc [0,4194304] 0 2024-04-25T15:49:53.243 INFO:tasks.workunit.client.1.smithi193.stdout:5/918: rename d17/d18/d30/d38/f120 to d17/d18/d30/d38/d48/f139 0 2024-04-25T15:49:53.244 INFO:tasks.workunit.client.1.smithi193.stdout:7/921: chown d12/d31/d37/d104/d11a/f12b 117 1 2024-04-25T15:49:53.246 INFO:tasks.workunit.client.1.smithi193.stdout:7/922: write d12/d1c/d63/d109/fea [2044254,20995] 0 2024-04-25T15:49:53.246 INFO:tasks.workunit.client.1.smithi193.stdout:3/940: rename d8/d13/d1f/f16 to d8/d13/d1f/d69/dcd/dd9/dde/dfc/f123 0 2024-04-25T15:49:53.247 INFO:tasks.workunit.client.1.smithi193.stdout:3/941: readlink d8/d13/d1f/d69/d77/l46 0 2024-04-25T15:49:53.248 INFO:tasks.workunit.client.1.smithi193.stdout:5/919: fdatasync d17/d18/ff1 0 2024-04-25T15:49:53.252 INFO:tasks.workunit.client.1.smithi193.stdout:3/942: mkdir d8/d13/d1f/d9f/d124 0 2024-04-25T15:49:53.254 INFO:tasks.workunit.client.1.smithi193.stdout:5/920: mknod d17/d18/d30/d34/c13a 0 2024-04-25T15:49:53.257 INFO:tasks.workunit.client.1.smithi193.stdout:1/839: dwrite dc/d59/dc0/deb/d55/d8a/f10c [0,4194304] 0 2024-04-25T15:49:53.259 INFO:tasks.workunit.client.1.smithi193.stdout:5/921: symlink d17/d18/d3e/dd1/ddc/l13b 0 2024-04-25T15:49:53.259 INFO:tasks.workunit.client.0.smithi110.stdout:2/964: truncate de/d40/d4f/d9f/fa3 2858175 0 2024-04-25T15:49:53.260 INFO:tasks.workunit.client.1.smithi193.stdout:5/922: readlink d17/d18/d30/d3b/d12b/dc4/l51 0 2024-04-25T15:49:53.262 INFO:tasks.workunit.client.1.smithi193.stdout:0/929: write d0/d49/d5/d7/dd/d46/f6b [3954268,120029] 0 2024-04-25T15:49:53.262 INFO:tasks.workunit.client.0.smithi110.stdout:2/965: creat de/d40/d70/d11c/d5c/d5f/db2/dad/dbd/dce/f137 x:0 0 0 2024-04-25T15:49:53.263 INFO:tasks.workunit.client.1.smithi193.stdout:5/923: read d17/d18/d3e/f3f [3649178,76466] 0 2024-04-25T15:49:53.264 INFO:tasks.workunit.client.1.smithi193.stdout:1/840: mknod dc/d59/dc0/deb/db0/dff/d104/c118 0 2024-04-25T15:49:53.266 INFO:tasks.workunit.client.1.smithi193.stdout:7/923: dread d12/d1c/d63/d70/f42 [0,4194304] 0 2024-04-25T15:49:53.268 INFO:tasks.workunit.client.1.smithi193.stdout:0/930: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d93/d145 0 2024-04-25T15:49:53.269 INFO:tasks.workunit.client.1.smithi193.stdout:0/931: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/ffe 138677 1 2024-04-25T15:49:53.269 INFO:tasks.workunit.client.1.smithi193.stdout:3/943: write d8/d13/d1f/d69/dcd/dd9/fec [662732,26682] 0 2024-04-25T15:49:53.269 INFO:tasks.workunit.client.1.smithi193.stdout:0/932: stat d0/d49/d5 0 2024-04-25T15:49:53.271 INFO:tasks.workunit.client.0.smithi110.stdout:2/966: creat de/d40/d6e/f138 x:0 0 0 2024-04-25T15:49:53.273 INFO:tasks.workunit.client.1.smithi193.stdout:1/841: truncate dc/d4a/d50/d7c/d7e/f88 955554 0 2024-04-25T15:49:53.273 INFO:tasks.workunit.client.1.smithi193.stdout:1/842: readlink dc/d59/l81 0 2024-04-25T15:49:53.273 INFO:tasks.workunit.client.0.smithi110.stdout:2/967: creat de/d40/dd7/f139 x:0 0 0 2024-04-25T15:49:53.276 INFO:tasks.workunit.client.1.smithi193.stdout:1/843: rename dc/f30 to dc/d4a/d50/d5c/d5d/ddd/f119 0 2024-04-25T15:49:53.277 INFO:tasks.workunit.client.0.smithi110.stdout:2/968: truncate de/d40/d70/d11c/d5c/d5f/db2/fc4 369351 0 2024-04-25T15:49:53.278 INFO:tasks.workunit.client.1.smithi193.stdout:0/933: creat d0/d49/d5/d7/dd/d46/d48/f146 x:0 0 0 2024-04-25T15:49:53.280 INFO:tasks.workunit.client.0.smithi110.stdout:2/969: symlink de/d40/d6e/d52/d6d/d99/l13a 0 2024-04-25T15:49:53.282 INFO:tasks.workunit.client.1.smithi193.stdout:0/934: readlink d0/d49/d5/d7/dd/d16/d26/d5e/d99/deb/l109 0 2024-04-25T15:49:53.283 INFO:tasks.workunit.client.0.smithi110.stdout:2/970: mkdir de/d40/d86/d13b 0 2024-04-25T15:49:53.287 INFO:tasks.workunit.client.1.smithi193.stdout:1/844: getdents dc/d59 0 2024-04-25T15:49:53.287 INFO:tasks.workunit.client.1.smithi193.stdout:5/924: truncate d17/d53/fc1 827826 0 2024-04-25T15:49:53.290 INFO:tasks.workunit.client.1.smithi193.stdout:5/925: chown d17/d18/d75/d85/d5d/dad/f104 60151060 1 2024-04-25T15:49:53.291 INFO:tasks.workunit.client.1.smithi193.stdout:0/935: dwrite d0/d49/d5/d7/dd/d16/d18/d28/d56/f139 [0,4194304] 0 2024-04-25T15:49:53.291 INFO:tasks.workunit.client.1.smithi193.stdout:7/924: write d12/d31/d37/fc5 [2653547,57656] 0 2024-04-25T15:49:53.293 INFO:tasks.workunit.client.1.smithi193.stdout:0/936: chown d0/d49/d5/d7/dd/d16/d120 879 1 2024-04-25T15:49:53.297 INFO:tasks.workunit.client.1.smithi193.stdout:3/944: dread d8/d13/d14/d27/d54/dd7/f11a [4194304,4194304] 0 2024-04-25T15:49:53.299 INFO:tasks.workunit.client.1.smithi193.stdout:7/925: symlink d12/l14f 0 2024-04-25T15:49:53.301 INFO:tasks.workunit.client.1.smithi193.stdout:1/845: write dc/f17 [2624813,64695] 0 2024-04-25T15:49:53.301 INFO:tasks.workunit.client.1.smithi193.stdout:1/846: readlink dc/l32 0 2024-04-25T15:49:53.303 INFO:tasks.workunit.client.0.smithi110.stdout:2/971: dread f8 [0,4194304] 0 2024-04-25T15:49:53.303 INFO:tasks.workunit.client.1.smithi193.stdout:7/926: creat d12/d13/dc4/dfa/f150 x:0 0 0 2024-04-25T15:49:53.306 INFO:tasks.workunit.client.1.smithi193.stdout:0/937: getdents d0/d49/d5/d7/dd/d78 0 2024-04-25T15:49:53.308 INFO:tasks.workunit.client.0.smithi110.stdout:2/972: rename de/d40/d6e/d52 to de/d40/dde/d13c 0 2024-04-25T15:49:53.309 INFO:tasks.workunit.client.1.smithi193.stdout:1/847: dwrite dc/d3c/f79 [0,4194304] 0 2024-04-25T15:49:53.310 INFO:tasks.workunit.client.1.smithi193.stdout:0/938: chown d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/f138 6040000 1 2024-04-25T15:49:53.311 INFO:tasks.workunit.client.1.smithi193.stdout:5/926: dread d17/f2a [0,4194304] 0 2024-04-25T15:49:53.311 INFO:tasks.workunit.client.1.smithi193.stdout:1/848: dread - dc/d59/dc0/deb/fcd zero size 2024-04-25T15:49:53.312 INFO:tasks.workunit.client.1.smithi193.stdout:5/927: chown d17/d18/d3e/c63 15764956 1 2024-04-25T15:49:53.313 INFO:tasks.workunit.client.1.smithi193.stdout:1/849: chown dc/d4a/d4d/d7a/d85/d92/cec 18035 1 2024-04-25T15:49:53.313 INFO:tasks.workunit.client.1.smithi193.stdout:5/928: fsync d17/d18/d30/d34/f11c 0 2024-04-25T15:49:53.313 INFO:tasks.workunit.client.1.smithi193.stdout:6/895: sync 2024-04-25T15:49:53.313 INFO:tasks.workunit.client.1.smithi193.stdout:2/908: sync 2024-04-25T15:49:53.313 INFO:tasks.workunit.client.1.smithi193.stdout:8/908: sync 2024-04-25T15:49:53.314 INFO:tasks.workunit.client.0.smithi110.stdout:2/973: dread de/d40/d6e/feb [0,4194304] 0 2024-04-25T15:49:53.315 INFO:tasks.workunit.client.1.smithi193.stdout:7/927: link d12/d1c/d63/d109/fdc d12/d13/dcc/df4/f151 0 2024-04-25T15:49:53.315 INFO:tasks.workunit.client.1.smithi193.stdout:0/939: symlink d0/l147 0 2024-04-25T15:49:53.317 INFO:tasks.workunit.client.0.smithi110.stdout:2/974: creat de/d40/d70/d11c/d5c/d5f/db2/da9/dac/f13d x:0 0 0 2024-04-25T15:49:53.318 INFO:tasks.workunit.client.1.smithi193.stdout:3/945: dread d8/d30/f42 [0,4194304] 0 2024-04-25T15:49:53.319 INFO:tasks.workunit.client.1.smithi193.stdout:5/929: mknod d17/d18/d75/d85/d5d/dad/c13c 0 2024-04-25T15:49:53.320 INFO:tasks.workunit.client.0.smithi110.stdout:2/975: mkdir de/d40/d70/d11c/d13e 0 2024-04-25T15:49:53.321 INFO:tasks.workunit.client.1.smithi193.stdout:6/896: dread dd/d2f/f5b [0,4194304] 0 2024-04-25T15:49:53.321 INFO:tasks.workunit.client.1.smithi193.stdout:2/909: mknod d0/d44/c147 0 2024-04-25T15:49:53.323 INFO:tasks.workunit.client.1.smithi193.stdout:6/897: stat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0 0 2024-04-25T15:49:53.323 INFO:tasks.workunit.client.0.smithi110.stdout:2/976: symlink de/d40/d70/d11c/d5c/dbe/l13f 0 2024-04-25T15:49:53.324 INFO:tasks.workunit.client.1.smithi193.stdout:8/909: unlink d2/d18/db1/fb3 0 2024-04-25T15:49:53.324 INFO:tasks.workunit.client.0.smithi110.stdout:2/977: mknod de/d40/d4f/d105/c140 0 2024-04-25T15:49:53.325 INFO:tasks.workunit.client.1.smithi193.stdout:6/898: dread - dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0/fe1 zero size 2024-04-25T15:49:53.325 INFO:tasks.workunit.client.1.smithi193.stdout:5/930: symlink d17/d53/l13d 0 2024-04-25T15:49:53.326 INFO:tasks.workunit.client.1.smithi193.stdout:1/850: creat dc/d4a/d50/d5c/f11a x:0 0 0 2024-04-25T15:49:53.327 INFO:tasks.workunit.client.1.smithi193.stdout:0/940: creat d0/d49/d5/d7/dd/d46/d48/d62/d13c/f148 x:0 0 0 2024-04-25T15:49:53.327 INFO:tasks.workunit.client.0.smithi110.stdout:2/978: truncate de/d40/d70/d11c/d5c/d5f/db2/d110/f127 745717 0 2024-04-25T15:49:53.327 INFO:tasks.workunit.client.0.smithi110.stdout:2/979: dread - de/d40/d6e/d50/db7/f107 zero size 2024-04-25T15:49:53.328 INFO:tasks.workunit.client.1.smithi193.stdout:3/946: mknod d8/d13/d1f/d69/c125 0 2024-04-25T15:49:53.328 INFO:tasks.workunit.client.0.smithi110.stdout:2/980: stat de/d40/d6e/d50/l9e 0 2024-04-25T15:49:53.329 INFO:tasks.workunit.client.0.smithi110.stdout:2/981: read de/d40/d4f/d9f/f100 [1166780,116722] 0 2024-04-25T15:49:53.330 INFO:tasks.workunit.client.1.smithi193.stdout:7/928: rmdir d12/d1c/d63/d70/d138 0 2024-04-25T15:49:53.335 INFO:tasks.workunit.client.1.smithi193.stdout:8/910: mknod d2/d3/d27/d69/dbf/d105/c122 0 2024-04-25T15:49:53.337 INFO:tasks.workunit.client.1.smithi193.stdout:5/931: fdatasync d17/d83/fba 0 2024-04-25T15:49:53.339 INFO:tasks.workunit.client.1.smithi193.stdout:6/899: creat dd/d12/d19/d1c/d40/d3c/d3f/d96/f10f x:0 0 0 2024-04-25T15:49:53.339 INFO:tasks.workunit.client.0.smithi110.stdout:2/982: link de/d6f/fb4 de/d40/d4f/d105/f141 0 2024-04-25T15:49:53.340 INFO:tasks.workunit.client.1.smithi193.stdout:7/929: truncate d12/d31/d37/d92/fee 2205849 0 2024-04-25T15:49:53.344 INFO:tasks.workunit.client.1.smithi193.stdout:2/910: link d0/d37/d25/d5f/d3a/d6d/d85/lcb d0/d37/d25/d5f/d3a/d6d/d85/l148 0 2024-04-25T15:49:53.346 INFO:tasks.workunit.client.1.smithi193.stdout:2/911: write d0/d37/d25/d5f/d3a/d6d/d6e/f96 [9329155,6162] 0 2024-04-25T15:49:53.349 INFO:tasks.workunit.client.1.smithi193.stdout:3/947: rename d8/df0/cf4 to d8/d30/d72/c126 0 2024-04-25T15:49:53.350 INFO:tasks.workunit.client.1.smithi193.stdout:3/948: write d8/d30/d45/fe0 [2919135,64636] 0 2024-04-25T15:49:53.350 INFO:tasks.workunit.client.1.smithi193.stdout:9/885: sync 2024-04-25T15:49:53.353 INFO:tasks.workunit.client.1.smithi193.stdout:5/932: symlink d17/d18/d30/d34/l13e 0 2024-04-25T15:49:53.353 INFO:tasks.workunit.client.1.smithi193.stdout:7/930: dread d12/d31/d39/f53 [0,4194304] 0 2024-04-25T15:49:53.354 INFO:tasks.workunit.client.1.smithi193.stdout:5/933: dread - d17/d18/d30/d34/d52/dee/f125 zero size 2024-04-25T15:49:53.359 INFO:tasks.workunit.client.1.smithi193.stdout:0/941: write d0/d49/d5/d7/dd/f6e [1213245,33033] 0 2024-04-25T15:49:53.361 INFO:tasks.workunit.client.1.smithi193.stdout:0/942: dread - d0/d49/d5/d7/dd/d46/d48/d62/d13c/f148 zero size 2024-04-25T15:49:53.363 INFO:tasks.workunit.client.1.smithi193.stdout:2/912: creat d0/d37/d25/d5f/d3a/d6d/d6e/f149 x:0 0 0 2024-04-25T15:49:53.369 INFO:tasks.workunit.client.1.smithi193.stdout:1/851: dwrite dc/dc9/f116 [0,4194304] 0 2024-04-25T15:49:53.380 INFO:tasks.workunit.client.1.smithi193.stdout:0/943: mkdir d0/d49/d5/d7/dd/d16/d18/d28/dcf/d149 0 2024-04-25T15:49:53.383 INFO:tasks.workunit.client.1.smithi193.stdout:2/913: unlink d0/d37/dbb/df8/fca 0 2024-04-25T15:49:53.385 INFO:tasks.workunit.client.1.smithi193.stdout:7/931: dread d12/d1c/d63/d70/d68/d8f/fab [0,4194304] 0 2024-04-25T15:49:53.387 INFO:tasks.workunit.client.1.smithi193.stdout:1/852: truncate dc/f18 4348356 0 2024-04-25T15:49:53.388 INFO:tasks.workunit.client.1.smithi193.stdout:6/900: link dd/d2f/ca9 dd/d12/d2d/c110 0 2024-04-25T15:49:53.389 INFO:tasks.workunit.client.1.smithi193.stdout:7/932: dread d12/d31/d37/d104/f4a [0,4194304] 0 2024-04-25T15:49:53.389 INFO:tasks.workunit.client.1.smithi193.stdout:5/934: mknod d17/d18/d75/d85/d10c/c13f 0 2024-04-25T15:49:53.392 INFO:tasks.workunit.client.1.smithi193.stdout:9/886: rmdir df/d2c/dde 0 2024-04-25T15:49:53.393 INFO:tasks.workunit.client.1.smithi193.stdout:3/949: creat d8/d13/d1f/d69/f127 x:0 0 0 2024-04-25T15:49:53.394 INFO:tasks.workunit.client.1.smithi193.stdout:6/901: symlink dd/d2f/d57/l111 0 2024-04-25T15:49:53.395 INFO:tasks.workunit.client.1.smithi193.stdout:6/902: chown dd/d12/d19/d45/c6b 485906046 1 2024-04-25T15:49:53.396 INFO:tasks.workunit.client.1.smithi193.stdout:7/933: symlink d12/d31/d37/d104/l152 0 2024-04-25T15:49:53.398 INFO:tasks.workunit.client.1.smithi193.stdout:5/935: unlink d17/d18/d30/d3b/d12b/dc4/f47 0 2024-04-25T15:49:53.399 INFO:tasks.workunit.client.1.smithi193.stdout:2/914: mkdir d0/d14a 0 2024-04-25T15:49:53.401 INFO:tasks.workunit.client.1.smithi193.stdout:6/903: dwrite dd/d12/d19/d45/f55 [0,4194304] 0 2024-04-25T15:49:53.401 INFO:tasks.workunit.client.1.smithi193.stdout:1/853: mknod dc/d59/dc0/deb/c11b 0 2024-04-25T15:49:53.402 INFO:tasks.workunit.client.1.smithi193.stdout:6/904: chown dd/db5/l102 201802594 1 2024-04-25T15:49:53.402 INFO:tasks.workunit.client.1.smithi193.stdout:6/905: readlink dd/db5/lfe 0 2024-04-25T15:49:53.403 INFO:tasks.workunit.client.1.smithi193.stdout:7/934: fsync d12/d31/dc3/f10a 0 2024-04-25T15:49:53.405 INFO:tasks.workunit.client.1.smithi193.stdout:7/935: read - d12/d13/dc4/f134 zero size 2024-04-25T15:49:53.407 INFO:tasks.workunit.client.1.smithi193.stdout:9/887: dread df/d2c/d2f/d4e/d1c/f76 [4194304,4194304] 0 2024-04-25T15:49:53.409 INFO:tasks.workunit.client.1.smithi193.stdout:5/936: chown d17/d18/d3e/f65 13 1 2024-04-25T15:49:53.411 INFO:tasks.workunit.client.1.smithi193.stdout:3/950: symlink d8/d30/d72/de1/d120/l128 0 2024-04-25T15:49:53.412 INFO:tasks.workunit.client.1.smithi193.stdout:3/951: write d8/d13/d14/f11f [709839,39537] 0 2024-04-25T15:49:53.418 INFO:tasks.workunit.client.1.smithi193.stdout:6/906: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/l112 0 2024-04-25T15:49:53.419 INFO:tasks.workunit.client.1.smithi193.stdout:6/907: write dd/d12/d19/d45/fd4 [571889,90984] 0 2024-04-25T15:49:53.420 INFO:tasks.workunit.client.1.smithi193.stdout:7/936: rmdir d12/d31/dc3 39 2024-04-25T15:49:53.422 INFO:tasks.workunit.client.1.smithi193.stdout:6/908: truncate dd/d12/d19/d1c/d40/d3c/d3f/d4f/fb9 1038100 0 2024-04-25T15:49:53.425 INFO:tasks.workunit.client.1.smithi193.stdout:9/888: rename df/d2c/d34/d3e/d4d to df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d110 0 2024-04-25T15:49:53.426 INFO:tasks.workunit.client.1.smithi193.stdout:9/889: write f9 [4057191,116219] 0 2024-04-25T15:49:53.427 INFO:tasks.workunit.client.1.smithi193.stdout:9/890: read df/d2c/d2f/d4e/d1c/d30/d3b/da9/f43 [3048453,59776] 0 2024-04-25T15:49:53.429 INFO:tasks.workunit.client.1.smithi193.stdout:2/915: creat d0/d9/d3d/d114/d130/f14b x:0 0 0 2024-04-25T15:49:53.430 INFO:tasks.workunit.client.1.smithi193.stdout:3/952: stat d8/d30/d45/f114 0 2024-04-25T15:49:53.434 INFO:tasks.workunit.client.1.smithi193.stdout:3/953: dread d8/d13/d4b/d8b/de3/fee [0,4194304] 0 2024-04-25T15:49:53.434 INFO:tasks.workunit.client.1.smithi193.stdout:7/937: fdatasync d12/d1c/d96/f11e 0 2024-04-25T15:49:53.435 INFO:tasks.workunit.client.1.smithi193.stdout:7/938: stat d12/d31/d37/d62/d6e/da5/dd7/f113 0 2024-04-25T15:49:53.436 INFO:tasks.workunit.client.1.smithi193.stdout:6/909: mknod dd/d12/d19/d1c/d40/d3c/d3f/c113 0 2024-04-25T15:49:53.437 INFO:tasks.workunit.client.1.smithi193.stdout:5/937: dread d17/d18/d30/d38/fbc [0,4194304] 0 2024-04-25T15:49:53.441 INFO:tasks.workunit.client.0.smithi110.stdout:2/983: truncate de/d40/f35 4085229 0 2024-04-25T15:49:53.444 INFO:tasks.workunit.client.0.smithi110.stdout:2/984: mkdir de/d40/d4f/d9f/d142 0 2024-04-25T15:49:53.450 INFO:tasks.workunit.client.1.smithi193.stdout:9/891: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/f111 x:0 0 0 2024-04-25T15:49:53.453 INFO:tasks.workunit.client.1.smithi193.stdout:3/954: getdents d8/d4d 0 2024-04-25T15:49:53.454 INFO:tasks.workunit.client.1.smithi193.stdout:9/892: symlink df/d2c/d34/l112 0 2024-04-25T15:49:53.455 INFO:tasks.workunit.client.1.smithi193.stdout:3/955: rmdir d8/d13/d1f/d69/dcd/dd9/d11b 39 2024-04-25T15:49:53.455 INFO:tasks.workunit.client.1.smithi193.stdout:9/893: write df/d2c/d2f/d5c/fec [2491649,85208] 0 2024-04-25T15:49:53.456 INFO:tasks.workunit.client.1.smithi193.stdout:9/894: stat df/d2c/d2f/d5c/da3/dac/ddc 0 2024-04-25T15:49:53.460 INFO:tasks.workunit.client.1.smithi193.stdout:1/854: write dc/d4a/d4d/d7a/dd0/de2/df1/fea [1000284,100533] 0 2024-04-25T15:49:53.462 INFO:tasks.workunit.client.1.smithi193.stdout:8/911: sync 2024-04-25T15:49:53.463 INFO:tasks.workunit.client.1.smithi193.stdout:3/956: dwrite d8/d13/d14/d51/f106 [0,4194304] 0 2024-04-25T15:49:53.463 INFO:tasks.workunit.client.1.smithi193.stdout:2/916: write d0/d42/d51/df1/d9f/dc5/fe7 [1196528,113598] 0 2024-04-25T15:49:53.465 INFO:tasks.workunit.client.1.smithi193.stdout:7/939: write d12/d1c/d45/fec [417678,76878] 0 2024-04-25T15:49:53.466 INFO:tasks.workunit.client.0.smithi110.stdout:2/985: write de/d40/dcd/f10a [467987,100940] 0 2024-04-25T15:49:53.467 INFO:tasks.workunit.client.1.smithi193.stdout:9/895: dwrite df/d2c/d2f/f77 [0,4194304] 0 2024-04-25T15:49:53.467 INFO:tasks.workunit.client.1.smithi193.stdout:8/912: chown d2/d3/d20/db8/dd0/l98 92317393 1 2024-04-25T15:49:53.468 INFO:tasks.workunit.client.1.smithi193.stdout:5/938: truncate d17/d18/d30/d38/d48/d4f/deb/f107 1303159 0 2024-04-25T15:49:53.469 INFO:tasks.workunit.client.1.smithi193.stdout:3/957: write d8/d13/d4b/d8b/de3/f119 [572075,11324] 0 2024-04-25T15:49:53.469 INFO:tasks.workunit.client.1.smithi193.stdout:6/910: rename dd/d12/d19/d1c/d40/d3c/c58 to dd/d12/d19/d1c/d40/d3c/d3f/d96/c114 0 2024-04-25T15:49:53.470 INFO:tasks.workunit.client.0.smithi110.stdout:2/986: mknod de/d40/dcd/c143 0 2024-04-25T15:49:53.471 INFO:tasks.workunit.client.1.smithi193.stdout:3/958: dread d8/d13/d4b/f6e [0,4194304] 0 2024-04-25T15:49:53.473 INFO:tasks.workunit.client.1.smithi193.stdout:2/917: mknod d0/d9/d13a/d6a/c14c 0 2024-04-25T15:49:53.475 INFO:tasks.workunit.client.0.smithi110.stdout:2/987: chown de/d40/d6e/l69 1408308955 1 2024-04-25T15:49:53.476 INFO:tasks.workunit.client.1.smithi193.stdout:2/918: dread d0/d37/d25/d5f/d3a/d6d/d6e/fe2 [0,4194304] 0 2024-04-25T15:49:53.477 INFO:tasks.workunit.client.1.smithi193.stdout:7/940: creat d12/d31/d37/d104/d11a/f153 x:0 0 0 2024-04-25T15:49:53.478 INFO:tasks.workunit.client.0.smithi110.stdout:2/988: truncate de/d40/d70/d11c/d5c/d5f/db2/f95 2283948 0 2024-04-25T15:49:53.479 INFO:tasks.workunit.client.1.smithi193.stdout:5/939: dread d17/d18/d30/d38/f69 [0,4194304] 0 2024-04-25T15:49:53.480 INFO:tasks.workunit.client.1.smithi193.stdout:1/855: dwrite dc/f12 [0,4194304] 0 2024-04-25T15:49:53.482 INFO:tasks.workunit.client.1.smithi193.stdout:0/944: sync 2024-04-25T15:49:53.483 INFO:tasks.workunit.client.1.smithi193.stdout:3/959: symlink d8/d13/d14/d23/l129 0 2024-04-25T15:49:53.484 INFO:tasks.workunit.client.1.smithi193.stdout:8/913: truncate d2/d3/f68 2466945 0 2024-04-25T15:49:53.484 INFO:tasks.workunit.client.1.smithi193.stdout:2/919: creat d0/d9/d3d/d114/d130/f14d x:0 0 0 2024-04-25T15:49:53.485 INFO:tasks.workunit.client.0.smithi110.stdout:2/989: link de/d40/d70/f7f de/d40/d70/d11c/d5c/f144 0 2024-04-25T15:49:53.485 INFO:tasks.workunit.client.1.smithi193.stdout:9/896: truncate df/d2c/d2f/d4e/d1c/d30/f7a 402317 0 2024-04-25T15:49:53.486 INFO:tasks.workunit.client.1.smithi193.stdout:1/856: truncate dc/d59/dc0/fc1 1533337 0 2024-04-25T15:49:53.486 INFO:tasks.workunit.client.1.smithi193.stdout:2/920: chown d0/d37/dbb/cf0 2317 1 2024-04-25T15:49:53.486 INFO:tasks.workunit.client.1.smithi193.stdout:9/897: dread - df/fd8 zero size 2024-04-25T15:49:53.487 INFO:tasks.workunit.client.1.smithi193.stdout:5/940: getdents d17/d18/d75/d85/d5d/d12f 0 2024-04-25T15:49:53.488 INFO:tasks.workunit.client.1.smithi193.stdout:1/857: fsync dc/d59/dc0/deb/d55/fe0 0 2024-04-25T15:49:53.489 INFO:tasks.workunit.client.1.smithi193.stdout:0/945: dread - d0/d49/d5/d7/dd/d16/d26/d2e/d143/ff9 zero size 2024-04-25T15:49:53.489 INFO:tasks.workunit.client.1.smithi193.stdout:9/898: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6 372 1 2024-04-25T15:49:53.489 INFO:tasks.workunit.client.1.smithi193.stdout:5/941: chown d17/d18/d75/d85/l4e 4314212 1 2024-04-25T15:49:53.491 INFO:tasks.workunit.client.1.smithi193.stdout:1/858: write dc/d4a/d50/fef [5077487,56008] 0 2024-04-25T15:49:53.491 INFO:tasks.workunit.client.1.smithi193.stdout:5/942: chown d17/f89 195133 1 2024-04-25T15:49:53.491 INFO:tasks.workunit.client.1.smithi193.stdout:2/921: mkdir d0/d9/d3d/d114/d14e 0 2024-04-25T15:49:53.492 INFO:tasks.workunit.client.0.smithi110.stdout:2/990: getdents de/d40/d6e/d50 0 2024-04-25T15:49:53.492 INFO:tasks.workunit.client.0.smithi110.stdout:0/984: sync 2024-04-25T15:49:53.493 INFO:tasks.workunit.client.1.smithi193.stdout:5/943: fsync d17/d18/f11f 0 2024-04-25T15:49:53.493 INFO:tasks.workunit.client.0.smithi110.stdout:0/985: fdatasync d1/d23/f146 0 2024-04-25T15:49:53.493 INFO:tasks.workunit.client.0.smithi110.stdout:0/986: fsync d1/d20/f73 0 2024-04-25T15:49:53.494 INFO:tasks.workunit.client.1.smithi193.stdout:2/922: mkdir d0/d44/d14f 0 2024-04-25T15:49:53.494 INFO:tasks.workunit.client.0.smithi110.stdout:0/987: stat d1/d6c/d116/d77/ced 0 2024-04-25T15:49:53.495 INFO:tasks.workunit.client.1.smithi193.stdout:2/923: chown d0/d37/d25/f77 24 1 2024-04-25T15:49:53.497 INFO:tasks.workunit.client.0.smithi110.stdout:2/991: creat de/d40/dd7/f145 x:0 0 0 2024-04-25T15:49:53.500 INFO:tasks.workunit.client.0.smithi110.stdout:0/988: mknod d1/d23/d30/d33/d5b/d78/ddb/d115/c15e 0 2024-04-25T15:49:53.501 INFO:tasks.workunit.client.1.smithi193.stdout:9/899: creat df/d2c/d2f/d5c/da3/dac/f113 x:0 0 0 2024-04-25T15:49:53.502 INFO:tasks.workunit.client.0.smithi110.stdout:2/992: creat de/d40/dde/d13c/d6d/d7d/dff/f146 x:0 0 0 2024-04-25T15:49:53.504 INFO:tasks.workunit.client.0.smithi110.stdout:0/989: mkdir d1/d47/d113/d15f 0 2024-04-25T15:49:53.504 INFO:tasks.workunit.client.0.smithi110.stdout:2/993: fdatasync de/d40/d70/d11c/d5c/d5f/db2/dad/dbd/dce/f137 0 2024-04-25T15:49:53.506 INFO:tasks.workunit.client.1.smithi193.stdout:5/944: write d17/d18/d3e/dd1/ddc/fdf [12701,96901] 0 2024-04-25T15:49:53.506 INFO:tasks.workunit.client.1.smithi193.stdout:6/911: dwrite dd/d2f/d57/f7d [0,4194304] 0 2024-04-25T15:49:53.507 INFO:tasks.workunit.client.1.smithi193.stdout:2/924: mkdir d0/d37/d25/dda/d150 0 2024-04-25T15:49:53.508 INFO:tasks.workunit.client.1.smithi193.stdout:3/960: link d8/d13/d1f/f78 d8/d13/f12a 0 2024-04-25T15:49:53.509 INFO:tasks.workunit.client.1.smithi193.stdout:7/941: write d12/d1c/f8a [3407968,127375] 0 2024-04-25T15:49:53.509 INFO:tasks.workunit.client.1.smithi193.stdout:7/942: stat d12/l14f 0 2024-04-25T15:49:53.511 INFO:tasks.workunit.client.1.smithi193.stdout:9/900: rename df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/c24 to df/d2c/d2f/d5c/da3/dac/c114 0 2024-04-25T15:49:53.512 INFO:tasks.workunit.client.0.smithi110.stdout:2/994: dwrite de/d40/dd7/f132 [0,4194304] 0 2024-04-25T15:49:53.518 INFO:tasks.workunit.client.1.smithi193.stdout:0/946: creat d0/d49/d5/d7/dd/d16/d26/d2e/d93/d145/f14a x:0 0 0 2024-04-25T15:49:53.518 INFO:tasks.workunit.client.1.smithi193.stdout:8/914: write d2/fb4 [977168,93764] 0 2024-04-25T15:49:53.519 INFO:tasks.workunit.client.0.smithi110.stdout:2/995: truncate de/d40/d70/d11c/d5c/d5f/db2/fa0 4871607 0 2024-04-25T15:49:53.519 INFO:tasks.workunit.client.1.smithi193.stdout:0/947: stat d0/d49/d5/d61 0 2024-04-25T15:49:53.520 INFO:tasks.workunit.client.1.smithi193.stdout:9/901: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/fd4 [0,4194304] 0 2024-04-25T15:49:53.520 INFO:tasks.workunit.client.0.smithi110.stdout:2/996: chown de/d40/dde/d13c/d6d/d99/fef 4 1 2024-04-25T15:49:53.524 INFO:tasks.workunit.client.0.smithi110.stdout:2/997: rmdir de/d6f 39 2024-04-25T15:49:53.527 INFO:tasks.workunit.client.1.smithi193.stdout:3/961: creat d8/d13/d1f/d9f/df9/f12b x:0 0 0 2024-04-25T15:49:53.528 INFO:tasks.workunit.client.1.smithi193.stdout:7/943: creat d12/d1c/dd6/d101/f154 x:0 0 0 2024-04-25T15:49:53.528 INFO:tasks.workunit.client.0.smithi110.stdout:2/998: mknod de/d40/dde/d13c/dda/c147 0 2024-04-25T15:49:53.531 INFO:tasks.workunit.client.0.smithi110.stdout:0/990: dread d1/d23/d30/d33/f88 [0,4194304] 0 2024-04-25T15:49:53.532 INFO:tasks.workunit.client.1.smithi193.stdout:0/948: dwrite d0/d49/d5/d7/dd/d46/d48/fa2 [0,4194304] 0 2024-04-25T15:49:53.532 INFO:tasks.workunit.client.1.smithi193.stdout:9/902: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/f40 [0,4194304] 0 2024-04-25T15:49:53.533 INFO:tasks.workunit.client.0.smithi110.stdout:0/991: write d1/d6c/f15d [888019,16187] 0 2024-04-25T15:49:53.534 INFO:tasks.workunit.client.0.smithi110.stdout:2/999: fsync de/d40/d70/dfa/f133 0 2024-04-25T15:49:53.538 INFO:tasks.workunit.client.1.smithi193.stdout:5/945: fdatasync d17/fef 0 2024-04-25T15:49:53.540 INFO:tasks.workunit.client.1.smithi193.stdout:2/925: mkdir d0/d42/d51/d145/d151 0 2024-04-25T15:49:53.540 INFO:tasks.workunit.client.1.smithi193.stdout:8/915: mkdir d2/d3/d27/d69/dae/d4a/d123 0 2024-04-25T15:49:53.541 INFO:tasks.workunit.client.1.smithi193.stdout:6/912: mknod dd/d12/d19/d1c/d40/dda/d10b/c115 0 2024-04-25T15:49:53.542 INFO:tasks.workunit.client.1.smithi193.stdout:3/962: symlink d8/d13/d4b/d8b/dca/l12c 0 2024-04-25T15:49:53.543 INFO:tasks.workunit.client.1.smithi193.stdout:1/859: link dc/d4a/d50/d5c/d5d/l4c dc/d4a/d50/dab/l11c 0 2024-04-25T15:49:53.544 INFO:tasks.workunit.client.1.smithi193.stdout:8/916: stat d2/d3/d27/d69/dae/d4a/dc1 0 2024-04-25T15:49:53.544 INFO:tasks.workunit.client.1.smithi193.stdout:0/949: read - d0/d49/d5/d7/dd/d46/d48/d62/dd9/fe9 zero size 2024-04-25T15:49:53.545 INFO:tasks.workunit.client.1.smithi193.stdout:8/917: write d2/d18/ddb/feb [407548,55154] 0 2024-04-25T15:49:53.545 INFO:tasks.workunit.client.1.smithi193.stdout:5/946: truncate d17/d18/d30/f35 1759588 0 2024-04-25T15:49:53.546 INFO:tasks.workunit.client.1.smithi193.stdout:1/860: write dc/d59/dc0/deb/d55/d8a/fc8 [691386,29695] 0 2024-04-25T15:49:53.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/913: fsync dd/d12/d19/d1c/f46 0 2024-04-25T15:49:53.549 INFO:tasks.workunit.client.1.smithi193.stdout:7/944: rename d12/d31/d37/d62/d6e/ccf to d12/d31/d37/d62/d6e/da5/c155 0 2024-04-25T15:49:53.552 INFO:tasks.workunit.client.1.smithi193.stdout:2/926: symlink d0/d37/d4d/dbe/l152 0 2024-04-25T15:49:53.554 INFO:tasks.workunit.client.1.smithi193.stdout:5/947: symlink d17/d18/d75/d85/d5d/l140 0 2024-04-25T15:49:53.555 INFO:tasks.workunit.client.1.smithi193.stdout:5/948: stat d17/d18/d30/d3b/d12b/dc4/c49 0 2024-04-25T15:49:53.555 INFO:tasks.workunit.client.1.smithi193.stdout:6/914: readlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dd7/le5 0 2024-04-25T15:49:53.560 INFO:tasks.workunit.client.1.smithi193.stdout:8/918: rmdir d2/d10a 0 2024-04-25T15:49:53.561 INFO:tasks.workunit.client.1.smithi193.stdout:6/915: symlink dd/d12/d19/d1c/d40/dda/l116 0 2024-04-25T15:49:53.563 INFO:tasks.workunit.client.1.smithi193.stdout:2/927: dread d0/d9/d3d/d7d/fab [0,4194304] 0 2024-04-25T15:49:53.567 INFO:tasks.workunit.client.1.smithi193.stdout:0/950: getdents d0/d49/d5/d7/dd/d16/d26/d2e 0 2024-04-25T15:49:53.568 INFO:tasks.workunit.client.1.smithi193.stdout:2/928: read d0/d42/d51/f40 [7678880,106626] 0 2024-04-25T15:49:53.569 INFO:tasks.workunit.client.1.smithi193.stdout:2/929: chown d0/d9/d3d/d114/l12b 5033600 1 2024-04-25T15:49:53.630 INFO:tasks.workunit.client.1.smithi193.stdout:8/919: dwrite d2/d3/d20/d8c/da6/fe6 [4194304,4194304] 0 2024-04-25T15:49:53.653 INFO:tasks.workunit.client.1.smithi193.stdout:8/920: dread d2/d18/f107 [0,4194304] 0 2024-04-25T15:49:53.654 INFO:tasks.workunit.client.1.smithi193.stdout:8/921: readlink d2/d3/d3a/lbe 0 2024-04-25T15:49:53.681 INFO:tasks.workunit.client.1.smithi193.stdout:9/903: write df/f108 [4143994,73208] 0 2024-04-25T15:49:53.684 INFO:tasks.workunit.client.1.smithi193.stdout:9/904: fsync df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d110/d5e/f6c 0 2024-04-25T15:49:53.686 INFO:tasks.workunit.client.1.smithi193.stdout:7/945: write d12/d31/d37/d104/d9d/fed [1014303,127635] 0 2024-04-25T15:49:53.689 INFO:tasks.workunit.client.1.smithi193.stdout:5/949: write d17/d18/d30/d38/d48/fa0 [491267,42617] 0 2024-04-25T15:49:53.690 INFO:tasks.workunit.client.1.smithi193.stdout:9/905: link l7 df/d2c/d2f/dfa/l115 0 2024-04-25T15:49:53.695 INFO:tasks.workunit.client.1.smithi193.stdout:3/963: dwrite d8/d30/d72/fa3 [0,4194304] 0 2024-04-25T15:49:53.697 INFO:tasks.workunit.client.1.smithi193.stdout:6/916: mknod dd/d12/d19/d1c/d40/d3c/d3f/c117 0 2024-04-25T15:49:53.697 INFO:tasks.workunit.client.1.smithi193.stdout:1/861: dwrite dc/d4a/d50/d5c/d5d/f3a [0,4194304] 0 2024-04-25T15:49:53.699 INFO:tasks.workunit.client.1.smithi193.stdout:9/906: rmdir df/d2c/d2f/d5c/da3 39 2024-04-25T15:49:53.700 INFO:tasks.workunit.client.1.smithi193.stdout:1/862: fsync dc/d4a/d50/d5c/d5d/ddd/ffd 0 2024-04-25T15:49:53.702 INFO:tasks.workunit.client.1.smithi193.stdout:1/863: read - dc/d4a/d50/d5c/d5d/f101 zero size 2024-04-25T15:49:53.703 INFO:tasks.workunit.client.1.smithi193.stdout:1/864: dread - dc/d4a/d77/f114 zero size 2024-04-25T15:49:53.705 INFO:tasks.workunit.client.1.smithi193.stdout:6/917: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/c118 0 2024-04-25T15:49:53.706 INFO:tasks.workunit.client.1.smithi193.stdout:6/918: dread - dd/d12/d19/d45/f7e zero size 2024-04-25T15:49:53.707 INFO:tasks.workunit.client.1.smithi193.stdout:9/907: symlink df/d2c/d2f/d4e/d1c/d30/d102/l116 0 2024-04-25T15:49:53.710 INFO:tasks.workunit.client.1.smithi193.stdout:6/919: fsync dd/d12/d19/d1c/d40/d3c/d3f/f85 0 2024-04-25T15:49:53.711 INFO:tasks.workunit.client.1.smithi193.stdout:1/865: mknod dc/c11d 0 2024-04-25T15:49:53.712 INFO:tasks.workunit.client.1.smithi193.stdout:1/866: write dc/d59/dc0/deb/d55/fcc [2854992,66080] 0 2024-04-25T15:49:53.713 INFO:tasks.workunit.client.1.smithi193.stdout:9/908: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d8b/f117 x:0 0 0 2024-04-25T15:49:53.718 INFO:tasks.workunit.client.1.smithi193.stdout:1/867: dread dc/d59/dc0/deb/d55/d8a/fc8 [0,4194304] 0 2024-04-25T15:49:53.719 INFO:tasks.workunit.client.1.smithi193.stdout:6/920: creat dd/d12/d19/d45/f119 x:0 0 0 2024-04-25T15:49:53.720 INFO:tasks.workunit.client.1.smithi193.stdout:9/909: dwrite df/d2c/d34/d3e/f6a [4194304,4194304] 0 2024-04-25T15:49:53.726 INFO:tasks.workunit.client.1.smithi193.stdout:6/921: truncate dd/d2f/f5b 1341551 0 2024-04-25T15:49:53.729 INFO:tasks.workunit.client.1.smithi193.stdout:1/868: symlink dc/d4a/d4d/d80/l11e 0 2024-04-25T15:49:53.733 INFO:tasks.workunit.client.1.smithi193.stdout:6/922: mkdir dd/d12/d19/d45/d11a 0 2024-04-25T15:49:53.735 INFO:tasks.workunit.client.1.smithi193.stdout:1/869: rmdir dc/d59 39 2024-04-25T15:49:53.738 INFO:tasks.workunit.client.1.smithi193.stdout:9/910: truncate df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d110/d5e/f6c 1087431 0 2024-04-25T15:49:53.739 INFO:tasks.workunit.client.1.smithi193.stdout:9/911: fsync df/d2c/d2f/d4e/d1c/fc3 0 2024-04-25T15:49:53.798 INFO:tasks.workunit.client.1.smithi193.stdout:8/922: sync 2024-04-25T15:49:53.815 INFO:tasks.workunit.client.1.smithi193.stdout:9/912: symlink df/d2c/d2f/d4e/d1c/d30/d60/l118 0 2024-04-25T15:49:53.842 INFO:tasks.workunit.client.1.smithi193.stdout:8/923: link d2/d3/d27/d69/dbf/f40 d2/d3/d20/d8c/dde/f124 0 2024-04-25T15:49:53.842 INFO:tasks.workunit.client.1.smithi193.stdout:8/924: chown d2/f6e 80 1 2024-04-25T15:49:53.843 INFO:tasks.workunit.client.1.smithi193.stdout:8/925: write d2/d18/dbc/dd3/f111 [1706208,93399] 0 2024-04-25T15:49:53.846 INFO:tasks.workunit.client.1.smithi193.stdout:8/926: mkdir d2/d3/d27/d69/dae/d29/d125 0 2024-04-25T15:49:53.847 INFO:tasks.workunit.client.1.smithi193.stdout:8/927: chown d2/d3/d3a/f4c 24297839 1 2024-04-25T15:49:53.849 INFO:tasks.workunit.client.1.smithi193.stdout:8/928: mkdir d2/d3/d20/d8c/dde/d94/d126 0 2024-04-25T15:49:53.850 INFO:tasks.workunit.client.1.smithi193.stdout:8/929: write d2/d18/db1/fbd [4050921,88313] 0 2024-04-25T15:49:53.853 INFO:tasks.workunit.client.1.smithi193.stdout:8/930: creat d2/d3/d27/d69/dbf/d106/f127 x:0 0 0 2024-04-25T15:49:53.871 INFO:tasks.workunit.client.1.smithi193.stdout:7/946: mknod d12/d31/d39/c156 0 2024-04-25T15:49:53.872 INFO:tasks.workunit.client.1.smithi193.stdout:8/931: dread d2/d3/d3a/f5e [0,4194304] 0 2024-04-25T15:49:53.892 INFO:tasks.workunit.client.1.smithi193.stdout:0/951: dwrite d0/d49/d5/d7/dd/d16/d18/f7c [0,4194304] 0 2024-04-25T15:49:53.895 INFO:tasks.workunit.client.1.smithi193.stdout:0/952: mknod d0/d49/d5/d7/dd/d16/d26/d5e/d99/c14b 0 2024-04-25T15:49:53.899 INFO:tasks.workunit.client.1.smithi193.stdout:0/953: read d0/d49/d6a/f97 [1932363,102707] 0 2024-04-25T15:49:53.901 INFO:tasks.workunit.client.1.smithi193.stdout:0/954: truncate d0/d49/d5/d7/dd/d46/d48/d62/f8f 4773717 0 2024-04-25T15:49:53.902 INFO:tasks.workunit.client.1.smithi193.stdout:0/955: stat d0/d49/d5/d7/dd/d16 0 2024-04-25T15:49:53.923 INFO:tasks.workunit.client.1.smithi193.stdout:9/913: dwrite df/d2c/d34/d3e/f7d [0,4194304] 0 2024-04-25T15:49:53.923 INFO:tasks.workunit.client.1.smithi193.stdout:1/870: dwrite dc/d59/dc0/fc1 [0,4194304] 0 2024-04-25T15:49:53.924 INFO:tasks.workunit.client.1.smithi193.stdout:9/914: write df/d2c/d2f/d4e/d69/de7/ff8 [201361,96129] 0 2024-04-25T15:49:53.926 INFO:tasks.workunit.client.1.smithi193.stdout:1/871: mknod dc/d4a/d4d/d7a/da6/c11f 0 2024-04-25T15:49:53.927 INFO:tasks.workunit.client.1.smithi193.stdout:1/872: stat dc/d4a/d4d/d7a/d85/d92/cec 0 2024-04-25T15:49:53.928 INFO:tasks.workunit.client.1.smithi193.stdout:3/964: creat d8/d13/d14/d27/f12d x:0 0 0 2024-04-25T15:49:53.929 INFO:tasks.workunit.client.1.smithi193.stdout:9/915: creat df/d2c/d2f/d4e/d69/d6b/de3/d107/f119 x:0 0 0 2024-04-25T15:49:53.930 INFO:tasks.workunit.client.1.smithi193.stdout:9/916: chown df/d2c/d2f/d4e/d1c/d30/d60 10955167 1 2024-04-25T15:49:53.938 INFO:tasks.workunit.client.1.smithi193.stdout:6/923: creat dd/f11b x:0 0 0 2024-04-25T15:49:53.939 INFO:tasks.workunit.client.1.smithi193.stdout:3/965: mkdir d8/d13/d1f/d12e 0 2024-04-25T15:49:53.940 INFO:tasks.workunit.client.1.smithi193.stdout:7/947: write d12/d31/d37/d104/f74 [9130376,19330] 0 2024-04-25T15:49:53.941 INFO:tasks.workunit.client.1.smithi193.stdout:8/932: write d2/d3/d27/d69/dae/d4a/d63/da2/fa4 [1962153,18223] 0 2024-04-25T15:49:53.942 INFO:tasks.workunit.client.1.smithi193.stdout:6/924: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0/f11c x:0 0 0 2024-04-25T15:49:53.943 INFO:tasks.workunit.client.1.smithi193.stdout:8/933: write d2/f11d [784371,20882] 0 2024-04-25T15:49:53.950 INFO:tasks.workunit.client.1.smithi193.stdout:8/934: getdents d2/d3/d27/d69 0 2024-04-25T15:49:53.953 INFO:tasks.workunit.client.1.smithi193.stdout:6/925: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0/f11c [0,4194304] 0 2024-04-25T15:49:53.956 INFO:tasks.workunit.client.1.smithi193.stdout:2/930: rename d0/d9/d13a to d0/d37/d25/dda/d153 0 2024-04-25T15:49:53.960 INFO:tasks.workunit.client.1.smithi193.stdout:2/931: chown d0/d9/d15/df3/l11b 498746 1 2024-04-25T15:49:53.961 INFO:tasks.workunit.client.1.smithi193.stdout:8/935: dwrite d2/d3/d27/fd7 [0,4194304] 0 2024-04-25T15:49:53.962 INFO:tasks.workunit.client.1.smithi193.stdout:6/926: dwrite dd/d12/f5d [4194304,4194304] 0 2024-04-25T15:49:53.964 INFO:tasks.workunit.client.1.smithi193.stdout:5/950: rename d17/d18/d30/d38/d48/la3 to d17/d18/d75/l141 0 2024-04-25T15:49:53.964 INFO:tasks.workunit.client.1.smithi193.stdout:6/927: fdatasync dd/d12/f35 0 2024-04-25T15:49:53.966 INFO:tasks.workunit.client.1.smithi193.stdout:3/966: write d8/d13/d4b/d8b/dca/fcb [540958,123903] 0 2024-04-25T15:49:53.968 INFO:tasks.workunit.client.0.smithi110.stdout:0/992: sync 2024-04-25T15:49:53.970 INFO:tasks.workunit.client.0.smithi110.stdout:0/993: symlink d1/d23/d30/d33/d5b/d83/l160 0 2024-04-25T15:49:53.972 INFO:tasks.workunit.client.0.smithi110.stdout:0/994: mknod d1/d6c/d7f/d79/c161 0 2024-04-25T15:49:53.972 INFO:tasks.workunit.client.1.smithi193.stdout:2/932: creat d0/d37/d4d/dbe/df7/f154 x:0 0 0 2024-04-25T15:49:53.973 INFO:tasks.workunit.client.1.smithi193.stdout:3/967: chown d8/d30/d45 2 1 2024-04-25T15:49:53.973 INFO:tasks.workunit.client.1.smithi193.stdout:1/873: rename dc/d3c to dc/d4a/d50/d5c/d5d/dfb/d120 0 2024-04-25T15:49:53.974 INFO:tasks.workunit.client.1.smithi193.stdout:7/948: dwrite d12/f18 [0,4194304] 0 2024-04-25T15:49:53.975 INFO:tasks.workunit.client.0.smithi110.stdout:0/995: unlink d1/d20/cbe 0 2024-04-25T15:49:53.975 INFO:tasks.workunit.client.1.smithi193.stdout:3/968: readlink d8/d13/d1f/l34 0 2024-04-25T15:49:53.976 INFO:tasks.workunit.client.1.smithi193.stdout:8/936: mkdir d2/d3/d128 0 2024-04-25T15:49:53.976 INFO:tasks.workunit.client.1.smithi193.stdout:6/928: mkdir dd/d12/d19/d45/d11a/d11d 0 2024-04-25T15:49:53.979 INFO:tasks.workunit.client.1.smithi193.stdout:5/951: mknod d17/d18/d30/d38/d71/c142 0 2024-04-25T15:49:53.980 INFO:tasks.workunit.client.1.smithi193.stdout:6/929: read dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa [10468,19132] 0 2024-04-25T15:49:53.981 INFO:tasks.workunit.client.1.smithi193.stdout:2/933: rmdir d0/d9/d15/d31 39 2024-04-25T15:49:53.982 INFO:tasks.workunit.client.1.smithi193.stdout:1/874: readlink dc/d4a/d50/d5c/d5d/l60 0 2024-04-25T15:49:53.982 INFO:tasks.workunit.client.1.smithi193.stdout:8/937: mknod d2/d3/d27/c129 0 2024-04-25T15:49:53.984 INFO:tasks.workunit.client.1.smithi193.stdout:9/917: rename ca to df/d2c/d2f/d4e/d69/df9/c11a 0 2024-04-25T15:49:53.984 INFO:tasks.workunit.client.0.smithi110.stdout:0/996: dread d1/d23/fc2 [0,4194304] 0 2024-04-25T15:49:53.989 INFO:tasks.workunit.client.1.smithi193.stdout:7/949: rename d12/d1c/d63/d70/d68/fb4 to d12/d1c/d63/d70/f157 0 2024-04-25T15:49:53.989 INFO:tasks.workunit.client.0.smithi110.stdout:0/997: truncate d1/f32 4927926 0 2024-04-25T15:49:53.992 INFO:tasks.workunit.client.1.smithi193.stdout:2/934: rename d0/c52 to d0/df/da5/d12d/c155 0 2024-04-25T15:49:53.993 INFO:tasks.workunit.client.0.smithi110.stdout:0/998: fdatasync d1/d23/d30/d33/d5b/d78/ddb/ffc 0 2024-04-25T15:49:53.993 INFO:tasks.workunit.client.1.smithi193.stdout:5/952: dread d17/d18/d30/d38/d48/d4f/deb/f107 [0,4194304] 0 2024-04-25T15:49:53.993 INFO:tasks.workunit.client.1.smithi193.stdout:5/953: dread - d17/f110 zero size 2024-04-25T15:49:53.995 INFO:tasks.workunit.client.1.smithi193.stdout:7/950: rmdir d12/d31/d37/d62/d6e/df2 39 2024-04-25T15:49:53.997 INFO:tasks.workunit.client.1.smithi193.stdout:9/918: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/d10d/fb1 [0,4194304] 0 2024-04-25T15:49:53.998 INFO:tasks.workunit.client.0.smithi110.stdout:0/999: symlink d1/d23/d30/d33/d5b/d78/ddb/de7/l162 0 2024-04-25T15:49:54.002 INFO:tasks.workunit.client.0.smithi110.stderr:+ rm -rf -- ./tmp.2GmEzzzUTX 2024-04-25T15:49:54.005 INFO:tasks.workunit.client.1.smithi193.stdout:2/935: creat d0/d37/f156 x:0 0 0 2024-04-25T15:49:54.006 INFO:tasks.workunit.client.1.smithi193.stdout:9/919: mkdir df/d2c/d2f/d5c/da3/d11b 0 2024-04-25T15:49:54.008 INFO:tasks.workunit.client.1.smithi193.stdout:8/938: write d2/d18/f107 [2625291,38229] 0 2024-04-25T15:49:54.009 INFO:tasks.workunit.client.1.smithi193.stdout:2/936: mkdir d0/d37/d25/d5f/d3a/d12a/d157 0 2024-04-25T15:49:54.011 INFO:tasks.workunit.client.1.smithi193.stdout:9/920: mknod df/d2c/d2f/d5c/da3/dac/c11c 0 2024-04-25T15:49:54.013 INFO:tasks.workunit.client.1.smithi193.stdout:8/939: rmdir d2/d3/d20/d8c 39 2024-04-25T15:49:54.013 INFO:tasks.workunit.client.1.smithi193.stdout:6/930: dwrite dd/d12/d19/d1c/d40/d3c/d3f/d4f/fca [0,4194304] 0 2024-04-25T15:49:54.015 INFO:tasks.workunit.client.1.smithi193.stdout:9/921: rmdir df/d2c/d2f/d4e/d1c/d30/d3b 39 2024-04-25T15:49:54.016 INFO:tasks.workunit.client.1.smithi193.stdout:7/951: write d12/f36 [3141867,101244] 0 2024-04-25T15:49:54.020 INFO:tasks.workunit.client.1.smithi193.stdout:9/922: write df/d2c/d2f/d4e/d1c/fc3 [958124,52685] 0 2024-04-25T15:49:54.021 INFO:tasks.workunit.client.1.smithi193.stdout:8/940: symlink d2/d3/d20/d8c/dde/l12a 0 2024-04-25T15:49:54.023 INFO:tasks.workunit.client.1.smithi193.stdout:5/954: dwrite d17/f89 [0,4194304] 0 2024-04-25T15:49:54.023 INFO:tasks.workunit.client.1.smithi193.stdout:2/937: rename d0/d42/d51/d4f/fac to d0/d42/d51/f158 0 2024-04-25T15:49:54.023 INFO:tasks.workunit.client.1.smithi193.stdout:8/941: fsync d2/d3/f5f 0 2024-04-25T15:49:54.024 INFO:tasks.workunit.client.1.smithi193.stdout:5/955: stat d17/f59 0 2024-04-25T15:49:54.027 INFO:tasks.workunit.client.1.smithi193.stdout:9/923: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/fd4 [3285929,53373] 0 2024-04-25T15:49:54.028 INFO:tasks.workunit.client.1.smithi193.stdout:6/931: getdents dd/d12/d19/d1c/d40/d3c 0 2024-04-25T15:49:54.031 INFO:tasks.workunit.client.1.smithi193.stdout:5/956: fsync d17/d18/d30/d3b/d12b/fd2 0 2024-04-25T15:49:54.032 INFO:tasks.workunit.client.1.smithi193.stdout:8/942: read d2/d3/d27/d69/dae/f5a [883279,96634] 0 2024-04-25T15:49:54.033 INFO:tasks.workunit.client.1.smithi193.stdout:2/938: truncate d0/d9/d3d/d7d/fab 1273622 0 2024-04-25T15:49:54.034 INFO:tasks.workunit.client.1.smithi193.stdout:9/924: mkdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d8b/d11d 0 2024-04-25T15:49:54.035 INFO:tasks.workunit.client.1.smithi193.stdout:2/939: chown d0/d37/dbb/df8/f86 447220 1 2024-04-25T15:49:54.036 INFO:tasks.workunit.client.1.smithi193.stdout:8/943: fsync d2/d3/d3a/f54 0 2024-04-25T15:49:54.040 INFO:tasks.workunit.client.1.smithi193.stdout:7/952: truncate d12/d1c/d45/f6a 3297489 0 2024-04-25T15:49:54.042 INFO:tasks.workunit.client.1.smithi193.stdout:9/925: rename df/fc5 to df/d2c/d2f/d4e/d1c/f11e 0 2024-04-25T15:49:54.045 INFO:tasks.workunit.client.1.smithi193.stdout:8/944: dwrite d2/fb4 [0,4194304] 0 2024-04-25T15:49:54.046 INFO:tasks.workunit.client.1.smithi193.stdout:9/926: fsync df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/ffc 0 2024-04-25T15:49:54.047 INFO:tasks.workunit.client.1.smithi193.stdout:5/957: truncate d17/d18/d30/d38/d48/d4f/f10d 1032843 0 2024-04-25T15:49:54.049 INFO:tasks.workunit.client.1.smithi193.stdout:7/953: creat d12/d1c/d63/d70/d68/d8f/d129/f158 x:0 0 0 2024-04-25T15:49:54.050 INFO:tasks.workunit.client.1.smithi193.stdout:5/958: chown d17/d18/d30/d38/d48/d4f/l10b 32496251 1 2024-04-25T15:49:54.051 INFO:tasks.workunit.client.1.smithi193.stdout:8/945: truncate d2/d18/fc4 136568 0 2024-04-25T15:49:54.051 INFO:tasks.workunit.client.1.smithi193.stdout:6/932: dread dd/d12/d19/d1c/f99 [0,4194304] 0 2024-04-25T15:49:54.051 INFO:tasks.workunit.client.1.smithi193.stdout:2/940: write d0/d9/d15/d10b/f13e [1324820,124070] 0 2024-04-25T15:49:54.052 INFO:tasks.workunit.client.1.smithi193.stdout:9/927: unlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/ce4 0 2024-04-25T15:49:54.053 INFO:tasks.workunit.client.1.smithi193.stdout:8/946: write d2/fb4 [2735058,85305] 0 2024-04-25T15:49:54.053 INFO:tasks.workunit.client.1.smithi193.stdout:7/954: read - d12/d1c/d63/fce zero size 2024-04-25T15:49:54.055 INFO:tasks.workunit.client.1.smithi193.stdout:5/959: fdatasync d17/d18/d75/d85/d5d/fb0 0 2024-04-25T15:49:54.056 INFO:tasks.workunit.client.1.smithi193.stdout:6/933: stat dd/d12/d19/d45/fc7 0 2024-04-25T15:49:54.056 INFO:tasks.workunit.client.1.smithi193.stdout:8/947: chown d2/d3/d20/d8c/dde/d94/c10b 5 1 2024-04-25T15:49:54.056 INFO:tasks.workunit.client.1.smithi193.stdout:5/960: fsync d17/d18/d75/f9c 0 2024-04-25T15:49:54.057 INFO:tasks.workunit.client.1.smithi193.stdout:8/948: stat d2/d3/d20/d8c/d11c 0 2024-04-25T15:49:54.059 INFO:tasks.workunit.client.1.smithi193.stdout:9/928: fsync df/d2c/d2f/d5c/da3/dac/fd1 0 2024-04-25T15:49:54.060 INFO:tasks.workunit.client.1.smithi193.stdout:2/941: chown d0/d9/d15/d31/f141 350716127 1 2024-04-25T15:49:54.061 INFO:tasks.workunit.client.1.smithi193.stdout:2/942: truncate d0/d9/d3d/d114/d130/f14d 220159 0 2024-04-25T15:49:54.064 INFO:tasks.workunit.client.1.smithi193.stdout:7/955: rename d12/d1c/d63/d109/lca to d12/d31/d37/d92/de8/l159 0 2024-04-25T15:49:54.071 INFO:tasks.workunit.client.1.smithi193.stdout:7/956: dwrite d12/d1c/d63/d70/d68/d8d/f136 [0,4194304] 0 2024-04-25T15:49:54.072 INFO:tasks.workunit.client.1.smithi193.stdout:7/957: write d12/d1c/f7e [772371,74989] 0 2024-04-25T15:49:54.073 INFO:tasks.workunit.client.1.smithi193.stdout:9/929: fdatasync df/d2c/d2f/d4e/d1c/d30/d3b/da9/f43 0 2024-04-25T15:49:54.074 INFO:tasks.workunit.client.1.smithi193.stdout:0/956: sync 2024-04-25T15:49:54.075 INFO:tasks.workunit.client.1.smithi193.stdout:7/958: write d12/d31/d37/d104/f74 [5488907,113820] 0 2024-04-25T15:49:54.076 INFO:tasks.workunit.client.1.smithi193.stdout:2/943: dread - d0/d37/fff zero size 2024-04-25T15:49:54.079 INFO:tasks.workunit.client.1.smithi193.stdout:8/949: rename d2/d3/d27/d69/dae/dd8/d113 to d2/d18/db1/d12b 0 2024-04-25T15:49:54.088 INFO:tasks.workunit.client.1.smithi193.stdout:6/934: link dd/d12/d19/d1c/d40/fbe dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/f11e 0 2024-04-25T15:49:54.092 INFO:tasks.workunit.client.1.smithi193.stdout:9/930: mkdir df/d2c/d2f/d4e/d69/d6b/de3/d11f 0 2024-04-25T15:49:54.094 INFO:tasks.workunit.client.1.smithi193.stdout:9/931: write df/d2c/d2f/d4e/d69/d6b/de3/d107/f119 [788636,128420] 0 2024-04-25T15:49:54.094 INFO:tasks.workunit.client.1.smithi193.stdout:0/957: unlink d0/d49/d5/d7/dd/d46/d48/fa2 0 2024-04-25T15:49:54.095 INFO:tasks.workunit.client.1.smithi193.stdout:0/958: readlink d0/d49/d5/d61/dd4/l11a 0 2024-04-25T15:49:54.096 INFO:tasks.workunit.client.1.smithi193.stdout:2/944: creat d0/d42/d51/df1/d9f/dc5/f159 x:0 0 0 2024-04-25T15:49:54.097 INFO:tasks.workunit.client.1.smithi193.stdout:8/950: mkdir d2/d3/d27/d69/dae/d4a/d12c 0 2024-04-25T15:49:54.097 INFO:tasks.workunit.client.1.smithi193.stdout:5/961: link d17/d18/d9b/l121 d17/d18/d30/d119/d126/l143 0 2024-04-25T15:49:54.100 INFO:tasks.workunit.client.1.smithi193.stdout:6/935: symlink dd/d12/d19/d1c/d40/dce/l11f 0 2024-04-25T15:49:54.114 INFO:tasks.workunit.client.1.smithi193.stdout:2/945: mknod d0/d37/d25/d5f/d3a/d6d/d85/c15a 0 2024-04-25T15:49:54.116 INFO:tasks.workunit.client.1.smithi193.stdout:3/969: sync 2024-04-25T15:49:54.120 INFO:tasks.workunit.client.1.smithi193.stdout:6/936: read dd/d2f/f50 [221169,78974] 0 2024-04-25T15:49:54.126 INFO:tasks.workunit.client.1.smithi193.stdout:2/946: dread d0/d37/dbb/df8/f86 [0,4194304] 0 2024-04-25T15:49:54.127 INFO:tasks.workunit.client.1.smithi193.stdout:7/959: dwrite d12/d31/f58 [0,4194304] 0 2024-04-25T15:49:54.131 INFO:tasks.workunit.client.1.smithi193.stdout:3/970: mkdir d8/d13/d14/d23/d108/d12f 0 2024-04-25T15:49:54.133 INFO:tasks.workunit.client.1.smithi193.stdout:0/959: write d0/d49/fe [1175626,113724] 0 2024-04-25T15:49:54.136 INFO:tasks.workunit.client.1.smithi193.stdout:0/960: chown d0/d49/d5/d7/dd/d46/d48/l75 277629 1 2024-04-25T15:49:54.137 INFO:tasks.workunit.client.1.smithi193.stdout:3/971: dwrite d8/d13/d14/fd3 [0,4194304] 0 2024-04-25T15:49:54.138 INFO:tasks.workunit.client.1.smithi193.stdout:0/961: fsync d0/d49/d5/d7/dd/d46/d48/d62/dd9/ff8 0 2024-04-25T15:49:54.140 INFO:tasks.workunit.client.1.smithi193.stdout:8/951: creat d2/d3/d20/d8c/f12d x:0 0 0 2024-04-25T15:49:54.142 INFO:tasks.workunit.client.1.smithi193.stdout:8/952: write d2/d3/d20/f8b [3902834,8375] 0 2024-04-25T15:49:54.143 INFO:tasks.workunit.client.1.smithi193.stdout:2/947: symlink d0/d9/d15/df3/l15b 0 2024-04-25T15:49:54.144 INFO:tasks.workunit.client.1.smithi193.stdout:0/962: write d0/d49/d5/d7/dd/d46/d48/d62/f92 [1954839,53788] 0 2024-04-25T15:49:54.144 INFO:tasks.workunit.client.1.smithi193.stdout:7/960: rmdir d12/d1c/da3 39 2024-04-25T15:49:54.145 INFO:tasks.workunit.client.1.smithi193.stdout:8/953: chown d2/d18/dbc 298 1 2024-04-25T15:49:54.145 INFO:tasks.workunit.client.1.smithi193.stdout:9/932: link df/d2c/d2f/dfa/l115 df/d2c/d95/l120 0 2024-04-25T15:49:54.149 INFO:tasks.workunit.client.1.smithi193.stdout:5/962: getdents d17/d18 0 2024-04-25T15:49:54.150 INFO:tasks.workunit.client.1.smithi193.stdout:6/937: creat dd/d12/d19/d1c/d40/f120 x:0 0 0 2024-04-25T15:49:54.154 INFO:tasks.workunit.client.1.smithi193.stdout:8/954: fdatasync d2/d3/d20/d8c/dde/d94/f90 0 2024-04-25T15:49:54.156 INFO:tasks.workunit.client.1.smithi193.stdout:2/948: mknod d0/c15c 0 2024-04-25T15:49:54.157 INFO:tasks.workunit.client.1.smithi193.stdout:7/961: dread d12/d31/d37/d92/dbf/fe9 [0,4194304] 0 2024-04-25T15:49:54.157 INFO:tasks.workunit.client.1.smithi193.stdout:5/963: creat d17/d18/d30/d38/d48/d4f/deb/f144 x:0 0 0 2024-04-25T15:49:54.160 INFO:tasks.workunit.client.1.smithi193.stdout:2/949: fsync d0/d37/d4d/dbe/df7/f143 0 2024-04-25T15:49:54.160 INFO:tasks.workunit.client.1.smithi193.stdout:6/938: dwrite dd/d12/d19/d45/fee [0,4194304] 0 2024-04-25T15:49:54.162 INFO:tasks.workunit.client.1.smithi193.stdout:9/933: dread df/d2c/d2f/d4e/d1c/d30/f7a [0,4194304] 0 2024-04-25T15:49:54.163 INFO:tasks.workunit.client.1.smithi193.stdout:5/964: readlink d17/d18/d30/d34/d118/l11a 0 2024-04-25T15:49:54.164 INFO:tasks.workunit.client.1.smithi193.stdout:8/955: rename d2/cc5 to d2/d3/d27/d69/dae/d4a/d53/c12e 0 2024-04-25T15:49:54.164 INFO:tasks.workunit.client.1.smithi193.stdout:1/875: sync 2024-04-25T15:49:54.184 INFO:tasks.workunit.client.1.smithi193.stdout:2/950: truncate d0/d42/d51/f94 318917 0 2024-04-25T15:49:54.185 INFO:tasks.workunit.client.1.smithi193.stdout:6/939: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/dc1/l121 0 2024-04-25T15:49:54.186 INFO:tasks.workunit.client.1.smithi193.stdout:8/956: chown d2/d3/d20/d8c/dde/d94/f90 95465089 1 2024-04-25T15:49:54.187 INFO:tasks.workunit.client.1.smithi193.stdout:3/972: dread d8/d13/f10f [0,4194304] 0 2024-04-25T15:49:54.187 INFO:tasks.workunit.client.1.smithi193.stdout:9/934: creat df/d2c/d2f/d5c/da3/dac/f121 x:0 0 0 2024-04-25T15:49:54.188 INFO:tasks.workunit.client.1.smithi193.stdout:5/965: creat d17/d18/d30/d3b/d12b/dc4/da7/d115/f145 x:0 0 0 2024-04-25T15:49:54.190 INFO:tasks.workunit.client.1.smithi193.stdout:1/876: unlink dc/d4a/d4d/d7a/fd7 0 2024-04-25T15:49:54.192 INFO:tasks.workunit.client.1.smithi193.stdout:3/973: dread d8/d13/d14/d27/f96 [0,4194304] 0 2024-04-25T15:49:54.192 INFO:tasks.workunit.client.1.smithi193.stdout:2/951: mkdir d0/d37/d25/d15d 0 2024-04-25T15:49:54.192 INFO:tasks.workunit.client.1.smithi193.stdout:1/877: chown dc/d4a/d50/d7c/d82 79 1 2024-04-25T15:49:54.192 INFO:tasks.workunit.client.1.smithi193.stdout:0/963: dwrite d0/d49/d5/d7/dd/d16/d26/ff6 [0,4194304] 0 2024-04-25T15:49:54.194 INFO:tasks.workunit.client.1.smithi193.stdout:8/957: symlink d2/d3/d27/d69/dae/dd6/l12f 0 2024-04-25T15:49:54.196 INFO:tasks.workunit.client.1.smithi193.stdout:2/952: truncate d0/d37/d25/dda/d153/d6a/f129 90834 0 2024-04-25T15:49:54.196 INFO:tasks.workunit.client.1.smithi193.stdout:9/935: unlink df/d2c/d2f/d4e/d1c/c1d 0 2024-04-25T15:49:54.197 INFO:tasks.workunit.client.1.smithi193.stdout:6/940: mkdir dd/d12/d122 0 2024-04-25T15:49:54.197 INFO:tasks.workunit.client.1.smithi193.stdout:6/941: chown dd/d12/d19/f79 2070199 1 2024-04-25T15:49:54.201 INFO:tasks.workunit.client.1.smithi193.stdout:5/966: truncate d17/d83/fba 79479 0 2024-04-25T15:49:54.207 INFO:tasks.workunit.client.1.smithi193.stdout:1/878: creat dc/d4a/d4d/d7a/dd0/de2/df1/f121 x:0 0 0 2024-04-25T15:49:54.208 INFO:tasks.workunit.client.1.smithi193.stdout:0/964: fdatasync d0/d49/d5/d7/dd/d46/d48/d62/dd9/fe9 0 2024-04-25T15:49:54.208 INFO:tasks.workunit.client.1.smithi193.stdout:8/958: dread d2/f48 [0,4194304] 0 2024-04-25T15:49:54.209 INFO:tasks.workunit.client.1.smithi193.stdout:0/965: chown d0/d49/d5/d7/dd/d16/d26/d2e/l81 24 1 2024-04-25T15:49:54.210 INFO:tasks.workunit.client.1.smithi193.stdout:9/936: creat df/d2c/d2f/d4e/d69/f122 x:0 0 0 2024-04-25T15:49:54.213 INFO:tasks.workunit.client.1.smithi193.stdout:6/942: symlink dd/d12/d19/d1c/d40/d3c/d3f/d4f/l123 0 2024-04-25T15:49:54.217 INFO:tasks.workunit.client.1.smithi193.stdout:1/879: symlink dc/d4a/d4d/d80/l122 0 2024-04-25T15:49:54.220 INFO:tasks.workunit.client.1.smithi193.stdout:6/943: dwrite dd/d12/d19/d45/fd4 [0,4194304] 0 2024-04-25T15:49:54.223 INFO:tasks.workunit.client.1.smithi193.stdout:2/953: rename d0/d9/c49 to d0/d9/d3d/d7d/dc3/c15e 0 2024-04-25T15:49:54.223 INFO:tasks.workunit.client.1.smithi193.stdout:8/959: dread d2/d3/d27/d69/dae/d29/f60 [0,4194304] 0 2024-04-25T15:49:54.223 INFO:tasks.workunit.client.1.smithi193.stdout:9/937: mknod df/d2c/d2f/d4e/d69/c123 0 2024-04-25T15:49:54.229 INFO:tasks.workunit.client.1.smithi193.stdout:3/974: dwrite d8/d13/d1f/f78 [0,4194304] 0 2024-04-25T15:49:54.230 INFO:tasks.workunit.client.1.smithi193.stdout:1/880: symlink dc/d4a/d4d/d7a/d85/d92/l123 0 2024-04-25T15:49:54.231 INFO:tasks.workunit.client.1.smithi193.stdout:1/881: chown dc/dc9/f116 2 1 2024-04-25T15:49:54.234 INFO:tasks.workunit.client.1.smithi193.stdout:6/944: rmdir dd/d12/d19/d45 39 2024-04-25T15:49:54.241 INFO:tasks.workunit.client.1.smithi193.stdout:3/975: mkdir d8/d13/d4b/d8b/de3/d130 0 2024-04-25T15:49:54.245 INFO:tasks.workunit.client.1.smithi193.stdout:7/962: sync 2024-04-25T15:49:54.245 INFO:tasks.workunit.client.1.smithi193.stdout:7/963: stat d12/d13/dc4/dfa/d14a 0 2024-04-25T15:49:54.249 INFO:tasks.workunit.client.1.smithi193.stdout:2/954: unlink d0/d37/d4d/fad 0 2024-04-25T15:49:54.249 INFO:tasks.workunit.client.1.smithi193.stdout:1/882: dread dc/d4a/d77/f7d [0,4194304] 0 2024-04-25T15:49:54.250 INFO:tasks.workunit.client.1.smithi193.stdout:5/967: getdents d17/d18/d30/d119/d126 0 2024-04-25T15:49:54.250 INFO:tasks.workunit.client.1.smithi193.stdout:1/883: read dc/d4a/ff9 [3190955,92806] 0 2024-04-25T15:49:54.252 INFO:tasks.workunit.client.1.smithi193.stdout:9/938: rename df/d2c/d2f/d4e/d69/df9/c11a to df/d2c/d2f/d5c/da3/d11b/c124 0 2024-04-25T15:49:54.255 INFO:tasks.workunit.client.1.smithi193.stdout:0/966: getdents d0/d49/d5/d61 0 2024-04-25T15:49:54.258 INFO:tasks.workunit.client.1.smithi193.stdout:7/964: creat d12/d1c/dd6/d101/f15a x:0 0 0 2024-04-25T15:49:54.259 INFO:tasks.workunit.client.1.smithi193.stdout:7/965: write d12/d31/d39/db0/fb5 [3881447,17808] 0 2024-04-25T15:49:54.260 INFO:tasks.workunit.client.1.smithi193.stdout:8/960: link d2/d18/db1/fbd d2/d3/d20/d8c/f130 0 2024-04-25T15:49:54.262 INFO:tasks.workunit.client.1.smithi193.stdout:8/961: write d2/d3/d27/d69/dae/dd6/fd1 [588665,100002] 0 2024-04-25T15:49:54.265 INFO:tasks.workunit.client.1.smithi193.stdout:9/939: creat df/d2c/d2f/d4e/f125 x:0 0 0 2024-04-25T15:49:54.266 INFO:tasks.workunit.client.1.smithi193.stdout:3/976: fsync d8/d13/d1f/d69/d77/fc9 0 2024-04-25T15:49:54.267 INFO:tasks.workunit.client.1.smithi193.stdout:0/967: symlink d0/d49/d5/d7/dd/d46/d48/d62/d13c/l14c 0 2024-04-25T15:49:54.268 INFO:tasks.workunit.client.1.smithi193.stdout:6/945: mknod dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80/c124 0 2024-04-25T15:49:54.270 INFO:tasks.workunit.client.1.smithi193.stdout:7/966: creat d12/d1c/d63/db7/f15b x:0 0 0 2024-04-25T15:49:54.272 INFO:tasks.workunit.client.1.smithi193.stdout:8/962: mkdir d2/d3/d27/d131 0 2024-04-25T15:49:54.276 INFO:tasks.workunit.client.1.smithi193.stdout:6/946: fsync dd/d12/d19/d1c/f20 0 2024-04-25T15:49:54.276 INFO:tasks.workunit.client.1.smithi193.stdout:7/967: dread d12/d31/d37/d104/d9d/fed [0,4194304] 0 2024-04-25T15:49:54.278 INFO:tasks.workunit.client.1.smithi193.stdout:1/884: dread dc/d59/dc0/deb/f43 [0,4194304] 0 2024-04-25T15:49:54.278 INFO:tasks.workunit.client.1.smithi193.stdout:0/968: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d60/d14d 0 2024-04-25T15:49:54.279 INFO:tasks.workunit.client.1.smithi193.stdout:1/885: dread - dc/d4a/d50/d5c/d5d/f10f zero size 2024-04-25T15:49:54.280 INFO:tasks.workunit.client.1.smithi193.stdout:1/886: write dc/f17 [4725003,46768] 0 2024-04-25T15:49:54.281 INFO:tasks.workunit.client.1.smithi193.stdout:8/963: mknod d2/d3/d27/d69/dbf/c132 0 2024-04-25T15:49:54.281 INFO:tasks.workunit.client.1.smithi193.stdout:1/887: write dc/d59/dc0/deb/f95 [5128753,116423] 0 2024-04-25T15:49:54.283 INFO:tasks.workunit.client.1.smithi193.stdout:3/977: symlink d8/d13/d14/d51/dc6/l131 0 2024-04-25T15:49:54.285 INFO:tasks.workunit.client.1.smithi193.stdout:5/968: write d17/d18/d75/d85/d5d/fb0 [1269305,76009] 0 2024-04-25T15:49:54.287 INFO:tasks.workunit.client.1.smithi193.stdout:9/940: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/d10d/f39 [0,4194304] 0 2024-04-25T15:49:54.287 INFO:tasks.workunit.client.1.smithi193.stdout:6/947: creat dd/d12/d19/d1c/d40/d3c/d3f/d4f/f125 x:0 0 0 2024-04-25T15:49:54.289 INFO:tasks.workunit.client.1.smithi193.stdout:2/955: dwrite d0/d37/d4d/dbe/df7/f112 [0,4194304] 0 2024-04-25T15:49:54.290 INFO:tasks.workunit.client.1.smithi193.stdout:0/969: chown d0/d49/d5/d61/dd4/d131/c137 13 1 2024-04-25T15:49:54.291 INFO:tasks.workunit.client.1.smithi193.stdout:2/956: read d0/d9/d15/f35 [570864,100879] 0 2024-04-25T15:49:54.291 INFO:tasks.workunit.client.1.smithi193.stdout:2/957: dread - d0/d37/fff zero size 2024-04-25T15:49:54.291 INFO:tasks.workunit.client.1.smithi193.stdout:0/970: chown d0/d49/d5/d7/dd/d16/d18/d28/dcf/cd7 339812910 1 2024-04-25T15:49:54.303 INFO:tasks.workunit.client.1.smithi193.stdout:9/941: symlink df/d2c/d95/l126 0 2024-04-25T15:49:54.304 INFO:tasks.workunit.client.1.smithi193.stdout:5/969: read d17/d18/d30/d3b/f60 [1201449,43763] 0 2024-04-25T15:49:54.306 INFO:tasks.workunit.client.1.smithi193.stdout:2/958: symlink d0/d42/d51/df1/d9c/l15f 0 2024-04-25T15:49:54.307 INFO:tasks.workunit.client.1.smithi193.stdout:2/959: write d0/d9/d15/d31/f141 [70389,89996] 0 2024-04-25T15:49:54.307 INFO:tasks.workunit.client.1.smithi193.stdout:0/971: symlink d0/d49/d5/d61/dd4/d131/l14e 0 2024-04-25T15:49:54.308 INFO:tasks.workunit.client.1.smithi193.stdout:0/972: chown d0/d49/d5/d7/dd/d16/d26/df5/d13a 2282785 1 2024-04-25T15:49:54.309 INFO:tasks.workunit.client.1.smithi193.stdout:8/964: symlink d2/d3/d128/l133 0 2024-04-25T15:49:54.310 INFO:tasks.workunit.client.1.smithi193.stdout:1/888: mkdir dc/d59/dc0/deb/d55/d8a/d124 0 2024-04-25T15:49:54.313 INFO:tasks.workunit.client.1.smithi193.stdout:7/968: rename d12/d13/fac to d12/d31/d37/f15c 0 2024-04-25T15:49:54.315 INFO:tasks.workunit.client.1.smithi193.stdout:2/960: truncate d0/fc 3905283 0 2024-04-25T15:49:54.316 INFO:tasks.workunit.client.1.smithi193.stdout:0/973: mkdir d0/d49/d5/d7/dd/d16/d26/d9e/d14f 0 2024-04-25T15:49:54.316 INFO:tasks.workunit.client.1.smithi193.stdout:1/889: dwrite dc/d4a/d77/f114 [0,4194304] 0 2024-04-25T15:49:54.317 INFO:tasks.workunit.client.1.smithi193.stdout:1/890: chown dc/dc9 496936904 1 2024-04-25T15:49:54.318 INFO:tasks.workunit.client.1.smithi193.stdout:3/978: dwrite d8/d13/d1f/d69/d77/fc9 [0,4194304] 0 2024-04-25T15:49:54.321 INFO:tasks.workunit.client.1.smithi193.stdout:9/942: mknod df/d2c/d2f/d5c/da3/dac/ddc/c127 0 2024-04-25T15:49:54.326 INFO:tasks.workunit.client.1.smithi193.stdout:1/891: creat dc/d59/dc0/deb/d55/d110/f125 x:0 0 0 2024-04-25T15:49:54.330 INFO:tasks.workunit.client.1.smithi193.stdout:9/943: symlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/l128 0 2024-04-25T15:49:54.330 INFO:tasks.workunit.client.1.smithi193.stdout:5/970: getdents d17/d18/d75/d85/d5d/dad/dd8 0 2024-04-25T15:49:54.331 INFO:tasks.workunit.client.1.smithi193.stdout:3/979: creat d8/d13/d14/d23/de8/f132 x:0 0 0 2024-04-25T15:49:54.332 INFO:tasks.workunit.client.1.smithi193.stdout:8/965: getdents d2/d3/d27/d69/dbf 0 2024-04-25T15:49:54.336 INFO:tasks.workunit.client.1.smithi193.stdout:5/971: creat d17/d18/d3e/dd1/f146 x:0 0 0 2024-04-25T15:49:54.337 INFO:tasks.workunit.client.1.smithi193.stdout:1/892: getdents dc/d4a/d50/d5c/d63 0 2024-04-25T15:49:54.338 INFO:tasks.workunit.client.1.smithi193.stdout:1/893: write dc/d4a/d77/f114 [2439110,119988] 0 2024-04-25T15:49:54.339 INFO:tasks.workunit.client.1.smithi193.stdout:0/974: dread d0/d49/d5/d7/dd/d16/d18/f22 [0,4194304] 0 2024-04-25T15:49:54.339 INFO:tasks.workunit.client.1.smithi193.stdout:9/944: rename df/d2c/d2f/d4e/d1c/f3d to df/d2c/d2f/d5c/d7e/f129 0 2024-04-25T15:49:54.339 INFO:tasks.workunit.client.1.smithi193.stdout:5/972: mkdir d17/d18/d75/d85/d10c/d147 0 2024-04-25T15:49:54.342 INFO:tasks.workunit.client.1.smithi193.stdout:0/975: readlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d7e/lb7 0 2024-04-25T15:49:54.343 INFO:tasks.workunit.client.1.smithi193.stdout:1/894: rename dc/c1a to dc/d4a/d50/dab/c126 0 2024-04-25T15:49:54.344 INFO:tasks.workunit.client.1.smithi193.stdout:9/945: mkdir df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/d12a 0 2024-04-25T15:49:54.349 INFO:tasks.workunit.client.1.smithi193.stdout:5/973: rename d17/d18/d30/c79 to d17/d18/d75/d85/d5d/dad/dd8/d135/c148 0 2024-04-25T15:49:54.351 INFO:tasks.workunit.client.1.smithi193.stdout:1/895: mknod dc/d4a/d4d/d7a/dd0/c127 0 2024-04-25T15:49:54.351 INFO:tasks.workunit.client.1.smithi193.stdout:9/946: dwrite df/d2c/d2f/d4e/d1c/d30/f103 [0,4194304] 0 2024-04-25T15:49:54.354 INFO:tasks.workunit.client.1.smithi193.stdout:9/947: truncate df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d8b/f117 75914 0 2024-04-25T15:49:54.355 INFO:tasks.workunit.client.1.smithi193.stdout:2/961: write d0/d42/f61 [3213038,19429] 0 2024-04-25T15:49:54.360 INFO:tasks.workunit.client.1.smithi193.stdout:5/974: mknod d17/d18/c149 0 2024-04-25T15:49:54.363 INFO:tasks.workunit.client.1.smithi193.stdout:1/896: mkdir dc/d59/d128 0 2024-04-25T15:49:54.363 INFO:tasks.workunit.client.1.smithi193.stdout:1/897: chown dc/d4a/d50/d5c/d5d/dfb 397111 1 2024-04-25T15:49:54.367 INFO:tasks.workunit.client.1.smithi193.stdout:8/966: write d2/f48 [1087220,84349] 0 2024-04-25T15:49:54.373 INFO:tasks.workunit.client.1.smithi193.stdout:0/976: write d0/d49/d5/d7/dd/d16/d18/d28/d56/f70 [3578473,101022] 0 2024-04-25T15:49:54.373 INFO:tasks.workunit.client.1.smithi193.stdout:0/977: chown d0/d49/d5/d7/dff/cc4 1816552 1 2024-04-25T15:49:54.376 INFO:tasks.workunit.client.1.smithi193.stdout:8/967: mkdir d2/d3/d27/d69/dae/d4a/d63/d134 0 2024-04-25T15:49:54.378 INFO:tasks.workunit.client.1.smithi193.stdout:1/898: mkdir dc/d59/dc0/deb/d55/d8a/d124/d129 0 2024-04-25T15:49:54.380 INFO:tasks.workunit.client.1.smithi193.stdout:0/978: dread d0/d49/d5/d7/dd/d16/d26/d5e/f69 [0,4194304] 0 2024-04-25T15:49:54.401 INFO:tasks.workunit.client.1.smithi193.stdout:6/948: sync 2024-04-25T15:49:54.412 INFO:tasks.workunit.client.1.smithi193.stdout:2/962: getdents d0/d37/d25/d5f/d3a/d6d/d85 0 2024-04-25T15:49:54.413 INFO:tasks.workunit.client.1.smithi193.stdout:5/975: stat d17/d18/d75/d85/l4e 0 2024-04-25T15:49:54.413 INFO:tasks.workunit.client.1.smithi193.stdout:8/968: getdents d2/d3/d27/d108/d11a 0 2024-04-25T15:49:54.414 INFO:tasks.workunit.client.1.smithi193.stdout:1/899: creat dc/d4a/d50/d5c/dfe/f12a x:0 0 0 2024-04-25T15:49:54.414 INFO:tasks.workunit.client.1.smithi193.stdout:9/948: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f8a [0,4194304] 0 2024-04-25T15:49:54.416 INFO:tasks.workunit.client.1.smithi193.stdout:0/979: symlink d0/d49/d5/d7/dd/d16/d26/d2e/d3d/l150 0 2024-04-25T15:49:54.419 INFO:tasks.workunit.client.1.smithi193.stdout:1/900: creat dc/d59/f12b x:0 0 0 2024-04-25T15:49:54.422 INFO:tasks.workunit.client.1.smithi193.stdout:8/969: unlink d2/d3/d20/d8c/dde/l81 0 2024-04-25T15:49:54.424 INFO:tasks.workunit.client.1.smithi193.stdout:6/949: dwrite dd/f11b [0,4194304] 0 2024-04-25T15:49:54.432 INFO:tasks.workunit.client.1.smithi193.stdout:9/949: rmdir df/d2c/d34/d10b 0 2024-04-25T15:49:54.433 INFO:tasks.workunit.client.1.smithi193.stdout:8/970: dwrite d2/da5/ff0 [0,4194304] 0 2024-04-25T15:49:54.444 INFO:tasks.workunit.client.1.smithi193.stdout:2/963: dwrite d0/d42/d51/d4f/fc4 [0,4194304] 0 2024-04-25T15:49:54.445 INFO:tasks.workunit.client.1.smithi193.stdout:9/950: mknod df/d2c/d2f/d5c/da3/d11b/c12b 0 2024-04-25T15:49:54.448 INFO:tasks.workunit.client.1.smithi193.stdout:5/976: dwrite d17/d18/d30/f76 [0,4194304] 0 2024-04-25T15:49:54.448 INFO:tasks.workunit.client.1.smithi193.stdout:1/901: truncate dc/d4a/d50/d7c/fca 673820 0 2024-04-25T15:49:54.448 INFO:tasks.workunit.client.1.smithi193.stdout:6/950: dwrite dd/d12/d19/db2/fc2 [0,4194304] 0 2024-04-25T15:49:54.451 INFO:tasks.workunit.client.1.smithi193.stdout:5/977: write d17/f137 [474469,37532] 0 2024-04-25T15:49:54.456 INFO:tasks.workunit.client.1.smithi193.stdout:9/951: write df/d2c/d2f/d4e/d1c/d30/f103 [5125747,79373] 0 2024-04-25T15:49:54.458 INFO:tasks.workunit.client.1.smithi193.stdout:0/980: dwrite d0/d49/d5/d7/dd/d16/d26/fed [0,4194304] 0 2024-04-25T15:49:54.461 INFO:tasks.workunit.client.1.smithi193.stdout:2/964: rename d0/d37/d25/d5f/d3a/d6d/d6e/db9/fd6 to d0/d37/dbb/df8/f160 0 2024-04-25T15:49:54.464 INFO:tasks.workunit.client.1.smithi193.stdout:1/902: symlink dc/d59/dc0/deb/d111/l12c 0 2024-04-25T15:49:54.466 INFO:tasks.workunit.client.1.smithi193.stdout:2/965: chown d0/d42/d51/d145 340 1 2024-04-25T15:49:54.467 INFO:tasks.workunit.client.1.smithi193.stdout:0/981: creat d0/d49/d5/d7/dd/d16/d18/d28/d56/f151 x:0 0 0 2024-04-25T15:49:54.468 INFO:tasks.workunit.client.1.smithi193.stdout:6/951: dwrite dd/d12/d19/d1c/d40/d3c/fd2 [0,4194304] 0 2024-04-25T15:49:54.471 INFO:tasks.workunit.client.1.smithi193.stdout:6/952: chown dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa 451568 1 2024-04-25T15:49:54.472 INFO:tasks.workunit.client.1.smithi193.stdout:9/952: rename df/d2c/d2f/f9a to df/d2c/d2f/d4e/d69/d6b/de3/f12c 0 2024-04-25T15:49:54.474 INFO:tasks.workunit.client.1.smithi193.stdout:7/969: sync 2024-04-25T15:49:54.477 INFO:tasks.workunit.client.1.smithi193.stdout:7/970: write d12/d31/d37/d92/dd1/ff6 [4472576,91226] 0 2024-04-25T15:49:54.485 INFO:tasks.workunit.client.1.smithi193.stdout:6/953: mknod dd/d12/d19/d1c/d40/dce/de2/df1/c126 0 2024-04-25T15:49:54.487 INFO:tasks.workunit.client.1.smithi193.stdout:9/953: dwrite df/fd8 [0,4194304] 0 2024-04-25T15:49:54.489 INFO:tasks.workunit.client.1.smithi193.stdout:1/903: dread dc/f47 [0,4194304] 0 2024-04-25T15:49:54.491 INFO:tasks.workunit.client.1.smithi193.stdout:7/971: write d12/d31/d37/d92/dbf/f10e [617593,59347] 0 2024-04-25T15:49:54.493 INFO:tasks.workunit.client.1.smithi193.stdout:8/971: dwrite d2/d3/d27/d69/dae/d29/f10e [0,4194304] 0 2024-04-25T15:49:54.496 INFO:tasks.workunit.client.1.smithi193.stdout:0/982: dread d0/d49/d5/d7/dd/f3c [0,4194304] 0 2024-04-25T15:49:54.502 INFO:tasks.workunit.client.1.smithi193.stdout:9/954: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/d10d/f39 [0,4194304] 0 2024-04-25T15:49:54.502 INFO:tasks.workunit.client.1.smithi193.stdout:7/972: symlink d12/d1c/dd6/l15d 0 2024-04-25T15:49:54.503 INFO:tasks.workunit.client.1.smithi193.stdout:5/978: dread d17/d18/f2e [0,4194304] 0 2024-04-25T15:49:54.505 INFO:tasks.workunit.client.1.smithi193.stdout:2/966: link d0/d9/l1f d0/d9/d12e/l161 0 2024-04-25T15:49:54.507 INFO:tasks.workunit.client.1.smithi193.stdout:0/983: unlink d0/d49/d5/d61/dd4/l11a 0 2024-04-25T15:49:54.508 INFO:tasks.workunit.client.1.smithi193.stdout:6/954: dwrite dd/d12/d19/d1c/f71 [0,4194304] 0 2024-04-25T15:49:54.509 INFO:tasks.workunit.client.1.smithi193.stdout:3/980: sync 2024-04-25T15:49:54.509 INFO:tasks.workunit.client.1.smithi193.stdout:1/904: symlink dc/d59/dc0/deb/d55/d8a/d124/d129/l12d 0 2024-04-25T15:49:54.509 INFO:tasks.workunit.client.1.smithi193.stdout:0/984: write d0/d49/d5/d7/dd/d46/d48/d62/d13c/f148 [1021941,23136] 0 2024-04-25T15:49:54.510 INFO:tasks.workunit.client.1.smithi193.stdout:7/973: symlink d12/d1c/dd6/d101/l15e 0 2024-04-25T15:49:54.511 INFO:tasks.workunit.client.1.smithi193.stdout:1/905: chown dc/d4a/d50/fef 439661 1 2024-04-25T15:49:54.513 INFO:tasks.workunit.client.1.smithi193.stdout:5/979: mknod d17/d18/d30/d119/d126/c14a 0 2024-04-25T15:49:54.514 INFO:tasks.workunit.client.1.smithi193.stdout:3/981: creat d8/d13/d14/d27/f133 x:0 0 0 2024-04-25T15:49:54.516 INFO:tasks.workunit.client.1.smithi193.stdout:6/955: mknod dd/d12/d19/d1c/d40/dce/de2/df1/c127 0 2024-04-25T15:49:54.519 INFO:tasks.workunit.client.1.smithi193.stdout:1/906: dwrite dc/d4a/d4d/d7a/dd0/de2/df1/fea [0,4194304] 0 2024-04-25T15:49:54.521 INFO:tasks.workunit.client.1.smithi193.stdout:7/974: fsync d12/d1c/f99 0 2024-04-25T15:49:54.521 INFO:tasks.workunit.client.1.smithi193.stdout:9/955: symlink df/d2c/d2f/d4e/d1c/d30/d3b/da9/l12d 0 2024-04-25T15:49:54.522 INFO:tasks.workunit.client.1.smithi193.stdout:9/956: chown df/d2c/d2f/d4e/d69/d6b/de3/d11f 3 1 2024-04-25T15:49:54.528 INFO:tasks.workunit.client.1.smithi193.stdout:8/972: dwrite d2/d3/d27/d69/dae/dd8/fe2 [0,4194304] 0 2024-04-25T15:49:54.530 INFO:tasks.workunit.client.1.smithi193.stdout:5/980: symlink d17/d18/d30/d38/d48/l14b 0 2024-04-25T15:49:54.534 INFO:tasks.workunit.client.1.smithi193.stdout:6/956: write dd/d12/d19/d45/fd4 [2156034,10357] 0 2024-04-25T15:49:54.536 INFO:tasks.workunit.client.1.smithi193.stdout:1/907: chown dc/d4a/d4d/ca1 525 1 2024-04-25T15:49:54.537 INFO:tasks.workunit.client.1.smithi193.stdout:7/975: mkdir d12/d1c/dd6/d15f 0 2024-04-25T15:49:54.537 INFO:tasks.workunit.client.1.smithi193.stdout:2/967: creat d0/d9/f162 x:0 0 0 2024-04-25T15:49:54.539 INFO:tasks.workunit.client.1.smithi193.stdout:9/957: mknod df/d2c/d2f/d4e/d69/d6b/de3/d107/c12e 0 2024-04-25T15:49:54.542 INFO:tasks.workunit.client.1.smithi193.stdout:0/985: truncate d0/d49/d5/d7/dd/f58 653361 0 2024-04-25T15:49:54.542 INFO:tasks.workunit.client.1.smithi193.stdout:9/958: truncate df/d2c/d2f/d4e/d69/f122 6662 0 2024-04-25T15:49:54.543 INFO:tasks.workunit.client.1.smithi193.stdout:2/968: dread d0/d37/d25/d5f/d3a/d6d/d6e/db9/fee [0,4194304] 0 2024-04-25T15:49:54.544 INFO:tasks.workunit.client.1.smithi193.stdout:8/973: rename d2/d3/d27/d69/dbf/d106 to d2/d3/d27/d69/dae/d4a/d63/d134/d135 0 2024-04-25T15:49:54.545 INFO:tasks.workunit.client.1.smithi193.stdout:9/959: fsync df/d2c/d2f/d4e/d1c/d30/f103 0 2024-04-25T15:49:54.546 INFO:tasks.workunit.client.1.smithi193.stdout:9/960: chown df/d2c/d2f/d4e/d1c/d30/d3b/cb0 0 1 2024-04-25T15:49:54.547 INFO:tasks.workunit.client.1.smithi193.stdout:6/957: write dd/d12/d19/d1c/d40/dda/fe9 [1004579,92038] 0 2024-04-25T15:49:54.549 INFO:tasks.workunit.client.1.smithi193.stdout:7/976: creat d12/d31/d37/d92/f160 x:0 0 0 2024-04-25T15:49:54.552 INFO:tasks.workunit.client.1.smithi193.stdout:0/986: creat d0/d49/d5/d61/dd4/f152 x:0 0 0 2024-04-25T15:49:54.553 INFO:tasks.workunit.client.1.smithi193.stdout:0/987: write d0/d49/d5/d7/dd/d46/d48/d62/f8f [5415396,83285] 0 2024-04-25T15:49:54.554 INFO:tasks.workunit.client.1.smithi193.stdout:8/974: creat d2/d3/d27/d69/dae/d4a/d63/d134/d135/f136 x:0 0 0 2024-04-25T15:49:54.555 INFO:tasks.workunit.client.1.smithi193.stdout:8/975: write d2/f11 [1386129,27453] 0 2024-04-25T15:49:54.559 INFO:tasks.workunit.client.1.smithi193.stdout:6/958: unlink dd/d12/d19/d1c/d40/dce/de2/df1/cfc 0 2024-04-25T15:49:54.561 INFO:tasks.workunit.client.1.smithi193.stdout:7/977: read d12/d31/d37/d92/fee [1568322,65908] 0 2024-04-25T15:49:54.567 INFO:tasks.workunit.client.1.smithi193.stdout:0/988: fdatasync d0/d49/d5/d7/dd/d16/fef 0 2024-04-25T15:49:54.567 INFO:tasks.workunit.client.1.smithi193.stdout:7/978: dwrite d12/d31/d39/d60/f118 [0,4194304] 0 2024-04-25T15:49:54.571 INFO:tasks.workunit.client.1.smithi193.stdout:9/961: creat df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d104/f12f x:0 0 0 2024-04-25T15:49:54.573 INFO:tasks.workunit.client.1.smithi193.stdout:9/962: stat df/d2c/d2f/d4e/d1c/f97 0 2024-04-25T15:49:54.575 INFO:tasks.workunit.client.1.smithi193.stdout:9/963: write df/d2c/d2f/d4e/d1c/fc3 [2035014,61965] 0 2024-04-25T15:49:54.576 INFO:tasks.workunit.client.1.smithi193.stdout:1/908: creat dc/d4a/d50/d5c/f12e x:0 0 0 2024-04-25T15:49:54.576 INFO:tasks.workunit.client.1.smithi193.stdout:7/979: dwrite d12/d31/d37/d104/f149 [0,4194304] 0 2024-04-25T15:49:54.577 INFO:tasks.workunit.client.1.smithi193.stdout:7/980: chown d12/d13/f14 1478692 1 2024-04-25T15:49:54.578 INFO:tasks.workunit.client.1.smithi193.stdout:9/964: write df/d2c/d2f/d4e/d69/de7/ff8 [982503,19565] 0 2024-04-25T15:49:54.578 INFO:tasks.workunit.client.1.smithi193.stdout:7/981: write d12/d31/d37/d92/dd1/ff6 [4657784,64254] 0 2024-04-25T15:49:54.582 INFO:tasks.workunit.client.1.smithi193.stdout:6/959: symlink dd/d12/d19/d1c/d40/d3c/l128 0 2024-04-25T15:49:54.582 INFO:tasks.workunit.client.1.smithi193.stdout:2/969: creat d0/f163 x:0 0 0 2024-04-25T15:49:54.583 INFO:tasks.workunit.client.1.smithi193.stdout:5/981: rename d17/d18/d30/d34/l9f to d17/l14c 0 2024-04-25T15:49:54.583 INFO:tasks.workunit.client.1.smithi193.stdout:3/982: truncate d8/d13/d1f/d69/d77/f50 5986535 0 2024-04-25T15:49:54.586 INFO:tasks.workunit.client.1.smithi193.stdout:0/989: truncate d0/d49/d5/d7/dd/d16/d18/fac 668696 0 2024-04-25T15:49:54.594 INFO:tasks.workunit.client.1.smithi193.stdout:1/909: truncate dc/f47 2839036 0 2024-04-25T15:49:54.595 INFO:tasks.workunit.client.1.smithi193.stdout:1/910: truncate dc/d4a/d50/d5c/d5d/f10f 420189 0 2024-04-25T15:49:54.597 INFO:tasks.workunit.client.1.smithi193.stdout:1/911: truncate dc/d4a/d50/d7c/fd9 664794 0 2024-04-25T15:49:54.597 INFO:tasks.workunit.client.1.smithi193.stdout:1/912: chown dc/d4a/d50/d5c/d5d/dfb/d120/cba 127673 1 2024-04-25T15:49:54.598 INFO:tasks.workunit.client.1.smithi193.stdout:1/913: chown dc/d59/dc0/deb/c52 103391 1 2024-04-25T15:49:54.598 INFO:tasks.workunit.client.1.smithi193.stdout:9/965: dread - df/d2c/d2f/d4e/d69/d6b/de3/f12c zero size 2024-04-25T15:49:54.600 INFO:tasks.workunit.client.1.smithi193.stdout:9/966: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/fbb 482337 1 2024-04-25T15:49:54.601 INFO:tasks.workunit.client.1.smithi193.stdout:7/982: unlink d12/d31/d39/f143 0 2024-04-25T15:49:54.607 INFO:tasks.workunit.client.1.smithi193.stdout:3/983: creat d8/d30/d72/de1/d120/f134 x:0 0 0 2024-04-25T15:49:54.607 INFO:tasks.workunit.client.1.smithi193.stdout:6/960: truncate dd/d12/d19/d1c/fb0 21328 0 2024-04-25T15:49:54.608 INFO:tasks.workunit.client.1.smithi193.stdout:3/984: chown d8/d13/d1f/d69/f127 605 1 2024-04-25T15:49:54.609 INFO:tasks.workunit.client.1.smithi193.stdout:6/961: truncate dd/d12/d19/d1c/d40/f23 5121696 0 2024-04-25T15:49:54.610 INFO:tasks.workunit.client.1.smithi193.stdout:1/914: stat dc/d4a/d50/d5c/d5d/l4c 0 2024-04-25T15:49:54.611 INFO:tasks.workunit.client.1.smithi193.stdout:1/915: write dc/d4a/d50/d5c/f12e [479951,108942] 0 2024-04-25T15:49:54.612 INFO:tasks.workunit.client.1.smithi193.stdout:1/916: dread - dc/d4a/d50/d5c/d5d/ddd/ffd zero size 2024-04-25T15:49:54.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:54 smithi110 ceph-mon[26344]: pgmap v44: 65 pgs: 65 active+clean; 4.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 127 MiB/s rd, 284 MiB/s wr, 638 op/s 2024-04-25T15:49:54.616 INFO:tasks.workunit.client.1.smithi193.stdout:2/970: write d0/df/da5/fc2 [2634549,107098] 0 2024-04-25T15:49:54.618 INFO:tasks.workunit.client.1.smithi193.stdout:0/990: unlink d0/d49/d5/d7/dd/d16/d18/d28/d56/c8a 0 2024-04-25T15:49:54.618 INFO:tasks.workunit.client.1.smithi193.stdout:3/985: dread - d8/d30/d72/db2/fd6 zero size 2024-04-25T15:49:54.619 INFO:tasks.workunit.client.1.smithi193.stdout:6/962: readlink dd/d12/l18 0 2024-04-25T15:49:54.621 INFO:tasks.workunit.client.1.smithi193.stdout:9/967: symlink df/d2c/d2f/dfa/l130 0 2024-04-25T15:49:54.621 INFO:tasks.workunit.client.1.smithi193.stdout:7/983: creat d12/d13/d116/f161 x:0 0 0 2024-04-25T15:49:54.622 INFO:tasks.workunit.client.1.smithi193.stdout:7/984: fdatasync d12/d1c/faf 0 2024-04-25T15:49:54.625 INFO:tasks.workunit.client.1.smithi193.stdout:8/976: rename d2/d3/d20/d8c/dde to d2/d137 0 2024-04-25T15:49:54.627 INFO:tasks.workunit.client.1.smithi193.stdout:6/963: read dd/d2f/f5b [689139,41680] 0 2024-04-25T15:49:54.628 INFO:tasks.workunit.client.1.smithi193.stdout:9/968: rmdir df/d2c/d2f/d4e/d69 39 2024-04-25T15:49:54.630 INFO:tasks.workunit.client.1.smithi193.stdout:1/917: mknod dc/d4a/d4d/d7a/c12f 0 2024-04-25T15:49:54.631 INFO:tasks.workunit.client.1.smithi193.stdout:3/986: dread d8/d30/f42 [4194304,4194304] 0 2024-04-25T15:49:54.632 INFO:tasks.workunit.client.1.smithi193.stdout:1/918: write dc/d4a/d4d/d7a/dd0/de2/df1/f121 [117140,119655] 0 2024-04-25T15:49:54.633 INFO:tasks.workunit.client.1.smithi193.stdout:1/919: dread - dc/d4a/d4d/d7a/d85/fc2 zero size 2024-04-25T15:49:54.637 INFO:tasks.workunit.client.1.smithi193.stdout:0/991: dread d0/d49/d5/d7/dd/d16/f123 [0,4194304] 0 2024-04-25T15:49:54.638 INFO:tasks.workunit.client.1.smithi193.stdout:2/971: rename d0/d42/d51/df1/d9c/fdd to d0/d42/d51/df1/f164 0 2024-04-25T15:49:54.639 INFO:tasks.workunit.client.1.smithi193.stdout:8/977: fsync d2/d3/d27/f7a 0 2024-04-25T15:49:54.639 INFO:tasks.workunit.client.1.smithi193.stdout:0/992: write d0/d49/d5/d7/dd/d16/d26/d9e/f118 [1494956,32622] 0 2024-04-25T15:49:54.643 INFO:tasks.workunit.client.1.smithi193.stdout:9/969: creat df/d2c/d2f/d4e/d1c/d30/d60/f131 x:0 0 0 2024-04-25T15:49:54.645 INFO:tasks.workunit.client.1.smithi193.stdout:3/987: mkdir d8/d30/d45/d135 0 2024-04-25T15:49:54.646 INFO:tasks.workunit.client.1.smithi193.stdout:7/985: mkdir d12/d1c/dd6/d15f/d162 0 2024-04-25T15:49:54.649 INFO:tasks.workunit.client.1.smithi193.stdout:5/982: link d17/c2f d17/d18/d3e/dd1/ddc/c14d 0 2024-04-25T15:49:54.655 INFO:tasks.workunit.client.1.smithi193.stdout:0/993: truncate d0/d49/d5/d7/dd/d16/d26/f8d 2562469 0 2024-04-25T15:49:54.657 INFO:tasks.workunit.client.1.smithi193.stdout:9/970: dread f9 [0,4194304] 0 2024-04-25T15:49:54.657 INFO:tasks.workunit.client.1.smithi193.stdout:3/988: creat d8/d13/d4b/d8b/da2/f136 x:0 0 0 2024-04-25T15:49:54.658 INFO:tasks.workunit.client.1.smithi193.stdout:3/989: chown d8/d30/f36 3898 1 2024-04-25T15:49:54.661 INFO:tasks.workunit.client.1.smithi193.stdout:5/983: mknod d17/d18/d30/d38/d48/d4f/deb/c14e 0 2024-04-25T15:49:54.661 INFO:tasks.workunit.client.1.smithi193.stdout:5/984: chown d17/f110 1837 1 2024-04-25T15:49:54.662 INFO:tasks.workunit.client.1.smithi193.stdout:7/986: dread d12/d1c/d63/d70/d68/d8f/d129/f13c [0,4194304] 0 2024-04-25T15:49:54.662 INFO:tasks.workunit.client.1.smithi193.stdout:2/972: mkdir d0/d37/d25/d15d/d165 0 2024-04-25T15:49:54.663 INFO:tasks.workunit.client.1.smithi193.stdout:7/987: chown d12/d31/d37/d62/db3/c145 896 1 2024-04-25T15:49:54.664 INFO:tasks.workunit.client.1.smithi193.stdout:7/988: write d12/d31/d37/d62/d6e/da5/f108 [2595546,100729] 0 2024-04-25T15:49:54.665 INFO:tasks.workunit.client.1.smithi193.stdout:0/994: mkdir d0/d49/d5/d7/dd/d16/d26/d2e/d93/d145/d153 0 2024-04-25T15:49:54.666 INFO:tasks.workunit.client.1.smithi193.stdout:7/989: dread - d12/d1c/d63/db7/f15b zero size 2024-04-25T15:49:54.668 INFO:tasks.workunit.client.1.smithi193.stdout:3/990: mknod d8/d13/d1f/dbf/c137 0 2024-04-25T15:49:54.669 INFO:tasks.workunit.client.1.smithi193.stdout:1/920: creat dc/d4a/d50/d5c/d5d/f130 x:0 0 0 2024-04-25T15:49:54.670 INFO:tasks.workunit.client.1.smithi193.stdout:5/985: fdatasync d17/d18/d30/d38/d48/f139 0 2024-04-25T15:49:54.671 INFO:tasks.workunit.client.1.smithi193.stdout:2/973: mknod d0/d9/d15/d10b/c166 0 2024-04-25T15:49:54.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/978: rmdir d2/d3/d27/d69/dae/d4a/d123 0 2024-04-25T15:49:54.674 INFO:tasks.workunit.client.1.smithi193.stdout:8/979: chown d2/d3/d27/fd7 218057 1 2024-04-25T15:49:54.675 INFO:tasks.workunit.client.1.smithi193.stdout:6/964: creat dd/d12/d19/d1c/d40/d3c/f129 x:0 0 0 2024-04-25T15:49:54.676 INFO:tasks.workunit.client.1.smithi193.stdout:6/965: chown dd/d12/d19/d1c/d40/f23 1670887 1 2024-04-25T15:49:54.676 INFO:tasks.workunit.client.1.smithi193.stdout:6/966: chown dd/d2f 26328 1 2024-04-25T15:49:54.678 INFO:tasks.workunit.client.1.smithi193.stdout:0/995: creat d0/d49/d5/d7/dd/d16/d18/d28/dcf/f154 x:0 0 0 2024-04-25T15:49:54.680 INFO:tasks.workunit.client.1.smithi193.stdout:7/990: rename d12/d1c/d63/db7/de7/f10f to d12/d1c/dd6/d101/f163 0 2024-04-25T15:49:54.681 INFO:tasks.workunit.client.1.smithi193.stdout:1/921: rmdir dc/d4a/d4d 39 2024-04-25T15:49:54.681 INFO:tasks.workunit.client.1.smithi193.stdout:3/991: truncate d8/d13/d14/f93 1458367 0 2024-04-25T15:49:54.682 INFO:tasks.workunit.client.1.smithi193.stdout:8/980: dread d2/d3/d27/d69/dae/d4a/d63/da2/fa4 [0,4194304] 0 2024-04-25T15:49:54.683 INFO:tasks.workunit.client.1.smithi193.stdout:5/986: chown d17/d18/d30/d38/c103 36592 1 2024-04-25T15:49:54.686 INFO:tasks.workunit.client.1.smithi193.stdout:9/971: link df/d2c/d2f/d4e/d1c/cf3 df/d2c/d2f/d5c/da3/dac/ddc/c132 0 2024-04-25T15:49:54.690 INFO:tasks.workunit.client.1.smithi193.stdout:3/992: dwrite d8/d13/d4b/d8b/da2/f136 [0,4194304] 0 2024-04-25T15:49:54.691 INFO:tasks.workunit.client.1.smithi193.stdout:7/991: rename d12/d31/d37/d62/d6e/da5/fb6 to d12/d1c/d96/d127/f164 0 2024-04-25T15:49:54.698 INFO:tasks.workunit.client.1.smithi193.stdout:2/974: dwrite d0/d37/d25/d5f/d3a/f58 [0,4194304] 0 2024-04-25T15:49:54.699 INFO:tasks.workunit.client.1.smithi193.stdout:8/981: creat d2/d3/d27/d69/dae/d4a/dc1/f138 x:0 0 0 2024-04-25T15:49:54.704 INFO:tasks.workunit.client.1.smithi193.stdout:1/922: dwrite dc/d4a/d50/f53 [4194304,4194304] 0 2024-04-25T15:49:54.705 INFO:tasks.workunit.client.1.smithi193.stdout:1/923: fdatasync dc/d59/dc0/deb/fcd 0 2024-04-25T15:49:54.716 INFO:tasks.workunit.client.1.smithi193.stdout:7/992: creat d12/d31/d37/d62/db3/f165 x:0 0 0 2024-04-25T15:49:54.720 INFO:tasks.workunit.client.1.smithi193.stdout:9/972: dwrite df/d2c/d2f/d4e/d1c/d30/f7a [0,4194304] 0 2024-04-25T15:49:54.720 INFO:tasks.workunit.client.1.smithi193.stdout:2/975: unlink d0/d42/d51/cd0 0 2024-04-25T15:49:54.722 INFO:tasks.workunit.client.1.smithi193.stdout:8/982: mkdir d2/d3/d27/d69/dae/d4a/d63/d139 0 2024-04-25T15:49:54.723 INFO:tasks.workunit.client.1.smithi193.stdout:2/976: write d0/d37/d4d/dbe/df7/f112 [4938918,117385] 0 2024-04-25T15:49:54.726 INFO:tasks.workunit.client.1.smithi193.stdout:1/924: truncate dc/f2d 1441121 0 2024-04-25T15:49:54.730 INFO:tasks.workunit.client.1.smithi193.stdout:9/973: dwrite df/d2c/d2f/d5c/d7e/f10e [0,4194304] 0 2024-04-25T15:49:54.732 INFO:tasks.workunit.client.1.smithi193.stdout:3/993: dwrite d8/d13/d14/f80 [0,4194304] 0 2024-04-25T15:49:54.733 INFO:tasks.workunit.client.1.smithi193.stdout:3/994: fsync d8/d4d/f87 0 2024-04-25T15:49:54.734 INFO:tasks.workunit.client.1.smithi193.stdout:5/987: creat d17/d18/f14f x:0 0 0 2024-04-25T15:49:54.752 INFO:tasks.workunit.client.1.smithi193.stdout:2/977: mkdir d0/d37/d25/d5f/d3a/d6d/d6e/d167 0 2024-04-25T15:49:54.752 INFO:tasks.workunit.client.1.smithi193.stdout:8/983: write d2/d3/d27/d69/f9a [1059312,98179] 0 2024-04-25T15:49:54.754 INFO:tasks.workunit.client.1.smithi193.stdout:2/978: truncate d0/d42/d51/df1/d9f/dc5/f159 209674 0 2024-04-25T15:49:54.762 INFO:tasks.workunit.client.1.smithi193.stdout:0/996: link d0/d49/d5/d7/dd/d16/d26/d2e/ld2 d0/d49/d5/d7/l155 0 2024-04-25T15:49:54.763 INFO:tasks.workunit.client.1.smithi193.stdout:6/967: rename dd/d12/d19/d1c/d40/d3c/d3f/d96/c114 to dd/d12/d19/d1c/d40/dce/de2/df1/c12a 0 2024-04-25T15:49:54.765 INFO:tasks.workunit.client.1.smithi193.stdout:8/984: creat d2/d18/ddb/f13a x:0 0 0 2024-04-25T15:49:54.766 INFO:tasks.workunit.client.1.smithi193.stdout:7/993: creat d12/d13/f166 x:0 0 0 2024-04-25T15:49:54.769 INFO:tasks.workunit.client.1.smithi193.stdout:2/979: mkdir d0/d9/d3d/d7d/d168 0 2024-04-25T15:49:54.772 INFO:tasks.workunit.client.1.smithi193.stdout:5/988: symlink d17/d18/d30/d119/l150 0 2024-04-25T15:49:54.772 INFO:tasks.workunit.client.1.smithi193.stdout:5/989: chown d17/d18/d30/d38/ca4 211923 1 2024-04-25T15:49:54.775 INFO:tasks.workunit.client.1.smithi193.stdout:0/997: unlink d0/d49/d5/d7/dd/d16/d18/f2a 0 2024-04-25T15:49:54.779 INFO:tasks.workunit.client.1.smithi193.stdout:7/994: dread d12/f36 [0,4194304] 0 2024-04-25T15:49:54.779 INFO:tasks.workunit.client.1.smithi193.stdout:9/974: mknod df/d2c/d2f/d4e/d69/d6b/de3/d11f/c133 0 2024-04-25T15:49:54.782 INFO:tasks.workunit.client.1.smithi193.stdout:5/990: dread d17/d83/fba [0,4194304] 0 2024-04-25T15:49:54.787 INFO:tasks.workunit.client.1.smithi193.stdout:7/995: mknod d12/d31/d37/d92/dd1/c167 0 2024-04-25T15:49:54.788 INFO:tasks.workunit.client.1.smithi193.stdout:7/996: read d12/d31/d37/f15c [434365,70342] 0 2024-04-25T15:49:54.789 INFO:tasks.workunit.client.1.smithi193.stdout:7/997: write d12/d31/d37/d92/dd1/fd5 [505478,81664] 0 2024-04-25T15:49:54.790 INFO:tasks.workunit.client.1.smithi193.stdout:7/998: read - d12/d13/f166 zero size 2024-04-25T15:49:54.795 INFO:tasks.workunit.client.1.smithi193.stdout:3/995: rename d8/d30/d72/lb1 to d8/d13/d4b/l138 0 2024-04-25T15:49:54.795 INFO:tasks.workunit.client.1.smithi193.stdout:0/998: write d0/d49/d5/d7/dd/d16/d26/d2e/d3d/d53/fe7 [1029008,17929] 0 2024-04-25T15:49:54.796 INFO:tasks.workunit.client.1.smithi193.stdout:1/925: dread dc/f2d [0,4194304] 0 2024-04-25T15:49:54.797 INFO:tasks.workunit.client.1.smithi193.stdout:3/996: chown d8/d13/d1f/d9f/lda 19800 1 2024-04-25T15:49:54.799 INFO:tasks.workunit.client.1.smithi193.stdout:9/975: fsync df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/f84 0 2024-04-25T15:49:54.800 INFO:tasks.workunit.client.1.smithi193.stdout:7/999: mkdir d12/d1c/d63/d109/d168 0 2024-04-25T15:49:54.803 INFO:tasks.workunit.client.1.smithi193.stdout:8/985: rename d2/da5/ff0 to d2/d18/db1/d12b/f13b 0 2024-04-25T15:49:54.804 INFO:tasks.workunit.client.1.smithi193.stdout:6/968: getdents dd/d12/d19/d1c/d40 0 2024-04-25T15:49:54.806 INFO:tasks.workunit.client.1.smithi193.stdout:5/991: dread d17/d18/d30/d3b/d12b/dc4/f56 [0,4194304] 0 2024-04-25T15:49:54.806 INFO:tasks.workunit.client.1.smithi193.stdout:6/969: chown dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/lc0 220 1 2024-04-25T15:49:54.811 INFO:tasks.workunit.client.1.smithi193.stdout:8/986: read d2/d3/f68 [1083942,45992] 0 2024-04-25T15:49:54.812 INFO:tasks.workunit.client.1.smithi193.stdout:2/980: link d0/d37/d25/l2a d0/d37/d25/l169 0 2024-04-25T15:49:54.814 INFO:tasks.workunit.client.1.smithi193.stdout:3/997: mkdir d8/d13/d1f/dbf/df6/d139 0 2024-04-25T15:49:54.814 INFO:tasks.workunit.client.1.smithi193.stdout:6/970: dwrite dd/d12/d19/d1c/d40/dda/fe9 [0,4194304] 0 2024-04-25T15:49:54.816 INFO:tasks.workunit.client.1.smithi193.stdout:5/992: mknod d17/d18/d30/d38/d48/d4f/deb/c151 0 2024-04-25T15:49:54.824 INFO:tasks.workunit.client.1.smithi193.stdout:9/976: symlink df/d2c/d2f/d5c/d7e/l134 0 2024-04-25T15:49:54.830 INFO:tasks.workunit.client.1.smithi193.stdout:1/926: mknod dc/d4a/d4d/d7a/d85/d92/d105/c131 0 2024-04-25T15:49:54.831 INFO:tasks.workunit.client.1.smithi193.stdout:0/999: dwrite d0/f130 [0,4194304] 0 2024-04-25T15:49:54.832 INFO:tasks.workunit.client.1.smithi193.stdout:2/981: dread d0/d37/d25/d5f/d3a/d6d/d6e/f3e [0,4194304] 0 2024-04-25T15:49:54.840 INFO:tasks.workunit.client.1.smithi193.stdout:5/993: dread d17/d18/d30/d38/d48/d4f/fd6 [0,4194304] 0 2024-04-25T15:49:54.840 INFO:tasks.workunit.client.1.smithi193.stdout:9/977: dread df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d110/fc0 [0,4194304] 0 2024-04-25T15:49:54.841 INFO:tasks.workunit.client.1.smithi193.stdout:2/982: dwrite d0/d9/d15/d31/f141 [0,4194304] 0 2024-04-25T15:49:54.852 INFO:tasks.workunit.client.1.smithi193.stdout:6/971: dread dd/d2f/d57/daf/fcc [0,4194304] 0 2024-04-25T15:49:54.868 INFO:tasks.workunit.client.1.smithi193.stdout:1/927: chown dc/d4a/d4d/d109 221671 1 2024-04-25T15:49:54.871 INFO:tasks.workunit.client.1.smithi193.stdout:9/978: symlink df/d2c/d2f/d5c/da3/l135 0 2024-04-25T15:49:54.872 INFO:tasks.workunit.client.1.smithi193.stdout:9/979: read df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/f8a [1934656,20180] 0 2024-04-25T15:49:54.872 INFO:tasks.workunit.client.1.smithi193.stdout:6/972: creat dd/d12/d19/d1c/d40/dda/d10b/f12b x:0 0 0 2024-04-25T15:49:54.873 INFO:tasks.workunit.client.1.smithi193.stdout:8/987: link d2/d3/d20/lfc d2/d3/d27/d69/dae/d29/d125/l13c 0 2024-04-25T15:49:54.874 INFO:tasks.workunit.client.1.smithi193.stdout:3/998: rename d8/d13/d14/c2e to d8/d13/d1f/d69/c13a 0 2024-04-25T15:49:54.876 INFO:tasks.workunit.client.1.smithi193.stdout:1/928: creat dc/d4a/d50/d5c/d63/d70/f132 x:0 0 0 2024-04-25T15:49:54.877 INFO:tasks.workunit.client.1.smithi193.stdout:2/983: creat d0/d9/d3d/d114/d14e/f16a x:0 0 0 2024-04-25T15:49:54.880 INFO:tasks.workunit.client.1.smithi193.stdout:5/994: mknod d17/d18/d75/d85/d5d/d12f/c152 0 2024-04-25T15:49:54.882 INFO:tasks.workunit.client.1.smithi193.stdout:3/999: dwrite d8/d13/d1f/d69/d77/fc9 [0,4194304] 0 2024-04-25T15:49:54.884 INFO:tasks.workunit.client.1.smithi193.stdout:8/988: mknod d2/d3/d27/d69/dae/d4a/d12c/c13d 0 2024-04-25T15:49:54.886 INFO:tasks.workunit.client.1.smithi193.stdout:5/995: unlink d17/d18/d30/d3b/fed 0 2024-04-25T15:49:54.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/984: link d0/d37/d25/dda/d153/d6a/f129 d0/d9/d3d/d114/d124/f16b 0 2024-04-25T15:49:54.892 INFO:tasks.workunit.client.1.smithi193.stdout:2/985: fsync d0/d9/d15/f113 0 2024-04-25T15:49:54.893 INFO:tasks.workunit.client.1.smithi193.stdout:5/996: mkdir d17/d18/d30/d34/d52/dee/d153 0 2024-04-25T15:49:54.894 INFO:tasks.workunit.client.1.smithi193.stdout:9/980: getdents df/d2c/d2f/d5c 0 2024-04-25T15:49:54.894 INFO:tasks.workunit.client.1.smithi193.stdout:5/997: dread - d17/d18/d30/d38/d48/f139 zero size 2024-04-25T15:49:54.894 INFO:tasks.workunit.client.1.smithi193.stdout:8/989: creat d2/d3/d27/d131/f13e x:0 0 0 2024-04-25T15:49:54.895 INFO:tasks.workunit.client.1.smithi193.stdout:2/986: write d0/d37/d25/d5f/d3a/d6d/d6e/f3e [694285,3392] 0 2024-04-25T15:49:54.896 INFO:tasks.workunit.client.1.smithi193.stdout:9/981: mknod df/d2c/d2f/d4e/d1c/c136 0 2024-04-25T15:49:54.897 INFO:tasks.workunit.client.1.smithi193.stdout:2/987: read - d0/d37/d25/d5f/d3a/d6d/d85/f126 zero size 2024-04-25T15:49:54.898 INFO:tasks.workunit.client.1.smithi193.stdout:9/982: chown df/d2c/d2f/d4e/d1c/f76 21438 1 2024-04-25T15:49:54.900 INFO:tasks.workunit.client.1.smithi193.stdout:2/988: mknod d0/d37/dbb/df8/c16c 0 2024-04-25T15:49:54.900 INFO:tasks.workunit.client.1.smithi193.stdout:2/989: dread - d0/d37/fff zero size 2024-04-25T15:49:54.903 INFO:tasks.workunit.client.1.smithi193.stdout:9/983: rename df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/ffd to df/d2c/d2f/d4e/d69/d6b/de3/d107/f137 0 2024-04-25T15:49:54.904 INFO:tasks.workunit.client.1.smithi193.stdout:8/990: link d2/d3/d3a/fda d2/d3/d27/d69/dae/d4a/d63/da2/f13f 0 2024-04-25T15:49:54.904 INFO:tasks.workunit.client.1.smithi193.stdout:9/984: dread - df/d2c/d2f/d4e/d1c/d30/d60/fa4 zero size 2024-04-25T15:49:54.905 INFO:tasks.workunit.client.1.smithi193.stdout:8/991: stat d2/d3/d27/d69/dae/d4a/l62 0 2024-04-25T15:49:54.905 INFO:tasks.workunit.client.1.smithi193.stdout:9/985: dread - df/d2c/d2f/d5c/fdd zero size 2024-04-25T15:49:54.907 INFO:tasks.workunit.client.1.smithi193.stdout:5/998: link d17/d18/d30/d3b/l7e d17/d18/d30/d38/d71/l154 0 2024-04-25T15:49:54.913 INFO:tasks.workunit.client.1.smithi193.stdout:2/990: getdents d0/df/da5/d12d 0 2024-04-25T15:49:54.915 INFO:tasks.workunit.client.1.smithi193.stdout:5/999: symlink d17/d18/d3e/l155 0 2024-04-25T15:49:54.915 INFO:tasks.workunit.client.1.smithi193.stdout:2/991: rmdir d0/d9/d3d/d114/d124 39 2024-04-25T15:49:54.916 INFO:tasks.workunit.client.1.smithi193.stdout:2/992: stat d0/d37/f43 0 2024-04-25T15:49:54.916 INFO:tasks.workunit.client.1.smithi193.stdout:2/993: chown d0/d37/d25/f77 16622974 1 2024-04-25T15:49:54.917 INFO:tasks.workunit.client.1.smithi193.stdout:2/994: dread - d0/d9/d3d/d114/d14e/f16a zero size 2024-04-25T15:49:54.919 INFO:tasks.workunit.client.1.smithi193.stdout:2/995: dread d0/d42/d51/df1/d9f/dc5/f159 [0,4194304] 0 2024-04-25T15:49:54.921 INFO:tasks.workunit.client.1.smithi193.stdout:2/996: readlink d0/d42/l121 0 2024-04-25T15:49:54.921 INFO:tasks.workunit.client.1.smithi193.stdout:8/992: dread d2/d3/d27/d69/dae/f88 [0,4194304] 0 2024-04-25T15:49:54.922 INFO:tasks.workunit.client.1.smithi193.stdout:2/997: readlink d0/d37/d25/d5f/d3a/d6d/d6e/l13f 0 2024-04-25T15:49:54.925 INFO:tasks.workunit.client.1.smithi193.stdout:8/993: dread d2/d3/d27/d69/dae/f103 [0,4194304] 0 2024-04-25T15:49:54.926 INFO:tasks.workunit.client.1.smithi193.stdout:8/994: readlink d2/d3/d27/d69/dae/l115 0 2024-04-25T15:49:54.926 INFO:tasks.workunit.client.1.smithi193.stdout:8/995: write d2/d3/d27/d69/dae/f10c [315902,23070] 0 2024-04-25T15:49:54.928 INFO:tasks.workunit.client.1.smithi193.stdout:6/973: write dd/d12/d19/d1c/fa4 [3654591,100958] 0 2024-04-25T15:49:54.931 INFO:tasks.workunit.client.1.smithi193.stdout:6/974: readlink dd/d12/l27 0 2024-04-25T15:49:54.932 INFO:tasks.workunit.client.1.smithi193.stdout:8/996: truncate d2/d3/d27/d69/dae/d4a/d63/da2/fa4 2631676 0 2024-04-25T15:49:54.935 INFO:tasks.workunit.client.1.smithi193.stdout:1/929: dwrite dc/d59/f8e [0,4194304] 0 2024-04-25T15:49:54.944 INFO:tasks.workunit.client.1.smithi193.stdout:8/997: link d2/d3/d27/d69/dae/d4a/f4e d2/d137/f140 0 2024-04-25T15:49:54.945 INFO:tasks.workunit.client.1.smithi193.stdout:8/998: readlink d2/da5/lff 0 2024-04-25T15:49:54.947 INFO:tasks.workunit.client.1.smithi193.stdout:6/975: rename dd/d12/d19/d1c/d10d to dd/d12/d19/d12c 0 2024-04-25T15:49:54.953 INFO:tasks.workunit.client.1.smithi193.stdout:9/986: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d8b/f4f [0,4194304] 0 2024-04-25T15:49:54.955 INFO:tasks.workunit.client.1.smithi193.stdout:8/999: mknod d2/d3/d27/d69/dae/c141 0 2024-04-25T15:49:54.956 INFO:tasks.workunit.client.1.smithi193.stdout:6/976: dwrite dd/d12/d19/d1c/d40/dda/fe9 [0,4194304] 0 2024-04-25T15:49:54.957 INFO:tasks.workunit.client.1.smithi193.stdout:6/977: readlink dd/d12/d19/d45/l95 0 2024-04-25T15:49:54.959 INFO:tasks.workunit.client.1.smithi193.stdout:9/987: unlink l0 0 2024-04-25T15:49:54.966 INFO:tasks.workunit.client.1.smithi193.stdout:1/930: rename dc/d4a/d50/d5c/d5d/ddd/f119 to dc/d59/dc0/f133 0 2024-04-25T15:49:54.971 INFO:tasks.workunit.client.1.smithi193.stdout:9/988: read df/d2c/d2f/f49 [244948,60953] 0 2024-04-25T15:49:54.972 INFO:tasks.workunit.client.1.smithi193.stdout:6/978: link dd/f13 dd/db5/f12d 0 2024-04-25T15:49:54.974 INFO:tasks.workunit.client.1.smithi193.stdout:6/979: write dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/dd0/f107 [998688,11909] 0 2024-04-25T15:49:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:54 smithi193 ceph-mon[33747]: pgmap v44: 65 pgs: 65 active+clean; 4.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 127 MiB/s rd, 284 MiB/s wr, 638 op/s 2024-04-25T15:49:54.993 INFO:tasks.workunit.client.1.smithi193.stdout:9/989: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d8b/f52 [4410192,60786] 0 2024-04-25T15:49:54.993 INFO:tasks.workunit.client.1.smithi193.stdout:1/931: dwrite dc/d59/dc0/deb/d55/f73 [0,4194304] 0 2024-04-25T15:49:54.995 INFO:tasks.workunit.client.1.smithi193.stdout:6/980: write dd/d2f/f56 [8155870,111818] 0 2024-04-25T15:49:54.996 INFO:tasks.workunit.client.1.smithi193.stdout:1/932: mknod dc/d4a/d50/d5c/dfe/c134 0 2024-04-25T15:49:55.001 INFO:tasks.workunit.client.1.smithi193.stdout:1/933: creat dc/d4a/d50/d5c/d5d/dfb/f135 x:0 0 0 2024-04-25T15:49:55.004 INFO:tasks.workunit.client.1.smithi193.stdout:6/981: rename dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/lb1 to dd/d12/d19/d45/d11a/l12e 0 2024-04-25T15:49:55.005 INFO:tasks.workunit.client.1.smithi193.stdout:1/934: getdents dc/d4a/d4d/d7a/d85/d92 0 2024-04-25T15:49:55.008 INFO:tasks.workunit.client.1.smithi193.stdout:1/935: truncate dc/d59/dc0/deb/f95 5268249 0 2024-04-25T15:49:55.011 INFO:tasks.workunit.client.1.smithi193.stdout:6/982: dwrite dd/d12/d19/d45/f119 [0,4194304] 0 2024-04-25T15:49:55.011 INFO:tasks.workunit.client.1.smithi193.stdout:1/936: symlink dc/d4a/d4d/d7a/d85/l136 0 2024-04-25T15:49:55.016 INFO:tasks.workunit.client.1.smithi193.stdout:9/990: dwrite df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/db8/dbe/d110/fd3 [0,4194304] 0 2024-04-25T15:49:55.019 INFO:tasks.workunit.client.1.smithi193.stdout:1/937: mknod dc/d4a/d50/d5c/d5d/c137 0 2024-04-25T15:49:55.026 INFO:tasks.workunit.client.1.smithi193.stdout:1/938: mknod dc/d4a/d50/d7c/d82/c138 0 2024-04-25T15:49:55.029 INFO:tasks.workunit.client.1.smithi193.stdout:6/983: link dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/faa dd/d12/d19/d12c/f12f 0 2024-04-25T15:49:55.037 INFO:tasks.workunit.client.1.smithi193.stdout:6/984: mkdir dd/d130 0 2024-04-25T15:49:55.041 INFO:tasks.workunit.client.1.smithi193.stdout:9/991: truncate df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/d10d/f99 2646368 0 2024-04-25T15:49:55.043 INFO:tasks.workunit.client.1.smithi193.stdout:1/939: dwrite dc/f19 [0,4194304] 0 2024-04-25T15:49:55.045 INFO:tasks.workunit.client.1.smithi193.stdout:1/940: stat dc/d59/dc0/fc1 0 2024-04-25T15:49:55.046 INFO:tasks.workunit.client.1.smithi193.stdout:1/941: stat dc/d4a/d50/d5c/d63 0 2024-04-25T15:49:55.047 INFO:tasks.workunit.client.1.smithi193.stdout:1/942: write dc/d4a/d4d/d80/f98 [1065098,129099] 0 2024-04-25T15:49:55.049 INFO:tasks.workunit.client.1.smithi193.stdout:9/992: dread df/d2c/d2f/d4e/d1c/d30/f7a [0,4194304] 0 2024-04-25T15:49:55.051 INFO:tasks.workunit.client.1.smithi193.stdout:9/993: chown df/d2c/d2f/d4e/d1c/d30/d3b/da9/d41/d5d/d10d/fba 12855 1 2024-04-25T15:49:55.057 INFO:tasks.workunit.client.1.smithi193.stdout:6/985: dread dd/d12/d19/d1c/f29 [0,4194304] 0 2024-04-25T15:49:55.058 INFO:tasks.workunit.client.1.smithi193.stdout:1/943: write dc/d4a/d50/d5c/d5d/fa0 [1475682,83825] 0 2024-04-25T15:49:55.059 INFO:tasks.workunit.client.1.smithi193.stdout:1/944: write dc/d59/dc0/deb/d55/f73 [4615079,49638] 0 2024-04-25T15:49:55.060 INFO:tasks.workunit.client.1.smithi193.stdout:9/994: rename df/d2c/d2f/d4e/d1c/f11e to df/d2c/d2f/d5c/d7e/f138 0 2024-04-25T15:49:55.061 INFO:tasks.workunit.client.1.smithi193.stdout:9/995: chown df/d2c/d2f/d4e/l9d 1281179 1 2024-04-25T15:49:55.062 INFO:tasks.workunit.client.1.smithi193.stdout:9/996: stat df/d2c/d2f/d5c/d7e/da2 0 2024-04-25T15:49:55.063 INFO:tasks.workunit.client.1.smithi193.stdout:9/997: write df/f106 [661710,108741] 0 2024-04-25T15:49:55.064 INFO:tasks.workunit.client.1.smithi193.stdout:1/945: rename dc/d4a/d50/d5c/d5d/dfb/d120/f79 to dc/d4a/d50/dab/f139 0 2024-04-25T15:49:55.065 INFO:tasks.workunit.client.1.smithi193.stdout:1/946: write dc/d4a/d50/d5c/d5d/f10f [1279114,65208] 0 2024-04-25T15:49:55.069 INFO:tasks.workunit.client.1.smithi193.stdout:9/998: readlink df/d2c/d95/l120 0 2024-04-25T15:49:55.069 INFO:tasks.workunit.client.1.smithi193.stdout:1/947: write dc/d59/dc0/deb/d55/f73 [695128,19644] 0 2024-04-25T15:49:55.070 INFO:tasks.workunit.client.1.smithi193.stdout:6/986: getdents dd/d12/d19/d1c/d40/d3c/d3f/d4f/d80 0 2024-04-25T15:49:55.076 INFO:tasks.workunit.client.1.smithi193.stdout:9/999: write df/d2c/d2f/d4e/d1c/d30/d3b/da9/daf/da6/fbb [17574,63075] 0 2024-04-25T15:49:55.078 INFO:tasks.workunit.client.1.smithi193.stdout:1/948: rename dc/d4a/d50/d5c/d5d/l5a to dc/d4a/d50/d5c/l13a 0 2024-04-25T15:49:55.081 INFO:tasks.workunit.client.1.smithi193.stdout:6/987: rename dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/fa8 to dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/da5/de8/f131 0 2024-04-25T15:49:55.083 INFO:tasks.workunit.client.1.smithi193.stdout:1/949: dwrite dc/f4b [4194304,4194304] 0 2024-04-25T15:49:55.087 INFO:tasks.workunit.client.1.smithi193.stdout:1/950: mkdir dc/d4a/d50/d5c/dfe/d13b 0 2024-04-25T15:49:55.089 INFO:tasks.workunit.client.1.smithi193.stdout:6/988: dwrite dd/d12/d19/d1c/d40/dda/d10b/f12b [0,4194304] 0 2024-04-25T15:49:55.092 INFO:tasks.workunit.client.1.smithi193.stdout:1/951: creat dc/d4a/d50/d5c/d63/f13c x:0 0 0 2024-04-25T15:49:55.094 INFO:tasks.workunit.client.1.smithi193.stdout:1/952: dread dc/f2d [0,4194304] 0 2024-04-25T15:49:55.094 INFO:tasks.workunit.client.1.smithi193.stdout:6/989: rename dd/d12/d19/d1c/d40/f23 to dd/d12/d19/d45/f132 0 2024-04-25T15:49:55.094 INFO:tasks.workunit.client.1.smithi193.stdout:6/990: chown dd/d2f/d57/l7c 4586033 1 2024-04-25T15:49:55.096 INFO:tasks.workunit.client.1.smithi193.stdout:6/991: write dd/f3d [10872267,120495] 0 2024-04-25T15:49:55.099 INFO:tasks.workunit.client.1.smithi193.stdout:1/953: rmdir dc/d4a/d50/d7c/d82 39 2024-04-25T15:49:55.104 INFO:tasks.workunit.client.1.smithi193.stdout:1/954: mkdir dc/d59/dc0/deb/d55/d8a/d124/d129/d13d 0 2024-04-25T15:49:55.111 INFO:tasks.workunit.client.1.smithi193.stdout:2/998: sync 2024-04-25T15:49:55.111 INFO:tasks.workunit.client.1.smithi193.stdout:1/955: creat dc/d4a/d50/d7c/f13e x:0 0 0 2024-04-25T15:49:55.117 INFO:tasks.workunit.client.1.smithi193.stdout:2/999: symlink d0/d37/d25/dda/d153/l16d 0 2024-04-25T15:49:55.117 INFO:tasks.workunit.client.1.smithi193.stdout:1/956: creat dc/f13f x:0 0 0 2024-04-25T15:49:55.119 INFO:tasks.workunit.client.1.smithi193.stdout:1/957: creat dc/d4a/d4d/d109/f140 x:0 0 0 2024-04-25T15:49:55.120 INFO:tasks.workunit.client.1.smithi193.stdout:1/958: dread - dc/d59/dc0/deb/d55/d110/f125 zero size 2024-04-25T15:49:55.121 INFO:tasks.workunit.client.1.smithi193.stdout:1/959: write dc/d59/dc0/fc1 [4891682,81834] 0 2024-04-25T15:49:55.121 INFO:tasks.workunit.client.1.smithi193.stdout:1/960: readlink dc/d4a/d50/d5c/d5d/l60 0 2024-04-25T15:49:55.123 INFO:tasks.workunit.client.1.smithi193.stdout:6/992: rename dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/da0/cf2 to dd/d12/d19/d1c/d40/d3c/d3f/d4f/d8f/c133 0 2024-04-25T15:49:55.129 INFO:tasks.workunit.client.1.smithi193.stdout:6/993: dwrite dd/d12/d19/db2/fc2 [0,4194304] 0 2024-04-25T15:49:55.136 INFO:tasks.workunit.client.1.smithi193.stdout:6/994: read dd/d12/d19/f7f [2070390,95306] 0 2024-04-25T15:49:55.137 INFO:tasks.workunit.client.1.smithi193.stdout:6/995: chown dd/d2f/d57/l111 6141753 1 2024-04-25T15:49:55.139 INFO:tasks.workunit.client.1.smithi193.stdout:6/996: chown dd/d12/d19/d1c/d40/fcb 28 1 2024-04-25T15:49:55.142 INFO:tasks.workunit.client.1.smithi193.stdout:6/997: mknod dd/d12/d19/d1c/d40/dce/c134 0 2024-04-25T15:49:55.156 INFO:tasks.workunit.client.1.smithi193.stdout:6/998: dread dd/d12/d19/d12c/f12f [0,4194304] 0 2024-04-25T15:49:55.160 INFO:tasks.workunit.client.1.smithi193.stdout:6/999: creat dd/d12/d19/f135 x:0 0 0 2024-04-25T15:49:55.208 INFO:tasks.workunit.client.1.smithi193.stdout:1/961: sync 2024-04-25T15:49:55.211 INFO:tasks.workunit.client.1.smithi193.stdout:1/962: fdatasync dc/f18 0 2024-04-25T15:49:55.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/963: mknod dc/d59/dc0/deb/d55/d110/c141 0 2024-04-25T15:49:55.214 INFO:tasks.workunit.client.1.smithi193.stdout:1/964: fdatasync dc/d59/f12b 0 2024-04-25T15:49:55.218 INFO:tasks.workunit.client.1.smithi193.stdout:1/965: creat dc/d4a/d4d/d7a/d85/d92/d105/f142 x:0 0 0 2024-04-25T15:49:55.222 INFO:tasks.workunit.client.1.smithi193.stdout:1/966: creat dc/d59/dc0/deb/d55/d8a/db7/f143 x:0 0 0 2024-04-25T15:49:55.229 INFO:tasks.workunit.client.1.smithi193.stdout:1/967: mkdir dc/d59/dc0/d144 0 2024-04-25T15:49:55.231 INFO:tasks.workunit.client.1.smithi193.stdout:1/968: symlink dc/d59/dc0/deb/d55/d8a/d124/d129/l145 0 2024-04-25T15:49:55.235 INFO:tasks.workunit.client.1.smithi193.stdout:1/969: symlink dc/d4a/d50/dab/l146 0 2024-04-25T15:49:55.236 INFO:tasks.workunit.client.1.smithi193.stdout:1/970: truncate dc/d4a/d50/d5c/d5d/f10f 1538152 0 2024-04-25T15:49:55.247 INFO:tasks.workunit.client.1.smithi193.stdout:1/971: write dc/d4a/d77/f7d [181269,41649] 0 2024-04-25T15:49:55.249 INFO:tasks.workunit.client.1.smithi193.stdout:1/972: chown dc/d4a/d4d/d7a/dd0/de2/df1/ce4 15249017 1 2024-04-25T15:49:55.250 INFO:tasks.workunit.client.1.smithi193.stdout:1/973: write dc/d4a/d4d/d80/f98 [2181248,106990] 0 2024-04-25T15:49:55.253 INFO:tasks.workunit.client.1.smithi193.stdout:1/974: mknod dc/d59/dc0/deb/d55/c147 0 2024-04-25T15:49:55.254 INFO:tasks.workunit.client.1.smithi193.stdout:1/975: fdatasync dc/d59/dc0/deb/fcd 0 2024-04-25T15:49:55.256 INFO:tasks.workunit.client.1.smithi193.stdout:1/976: mkdir dc/d4a/d50/d5c/d5d/dfb/d148 0 2024-04-25T15:49:55.257 INFO:tasks.workunit.client.1.smithi193.stdout:1/977: fdatasync dc/d4a/d50/d5c/dfe/f12a 0 2024-04-25T15:49:55.262 INFO:tasks.workunit.client.1.smithi193.stdout:1/978: dwrite dc/f4b [0,4194304] 0 2024-04-25T15:49:55.264 INFO:tasks.workunit.client.1.smithi193.stdout:1/979: mknod dc/d4a/d50/c149 0 2024-04-25T15:49:55.273 INFO:tasks.workunit.client.1.smithi193.stdout:1/980: symlink dc/d4a/d50/d5c/d63/d70/l14a 0 2024-04-25T15:49:55.274 INFO:tasks.workunit.client.1.smithi193.stdout:1/981: mkdir dc/d59/dc0/deb/d55/d8a/db7/d14b 0 2024-04-25T15:49:55.285 INFO:tasks.workunit.client.1.smithi193.stdout:1/982: dread dc/d4a/d4d/f5e [0,4194304] 0 2024-04-25T15:49:55.287 INFO:tasks.workunit.client.1.smithi193.stdout:1/983: creat dc/d4a/d4d/d7a/d85/f14c x:0 0 0 2024-04-25T15:49:55.290 INFO:tasks.workunit.client.1.smithi193.stdout:1/984: creat dc/d4a/d4d/d7a/f14d x:0 0 0 2024-04-25T15:49:55.361 INFO:tasks.workunit.client.1.smithi193.stdout:1/985: sync 2024-04-25T15:49:55.371 INFO:tasks.workunit.client.1.smithi193.stdout:1/986: getdents dc/d4a/d4d/d7a/dd0/de2/df1 0 2024-04-25T15:49:55.376 INFO:tasks.workunit.client.1.smithi193.stdout:1/987: symlink dc/d4a/d4d/d7a/dd0/dd2/l14e 0 2024-04-25T15:49:55.381 INFO:tasks.workunit.client.1.smithi193.stdout:1/988: dwrite dc/d4a/d50/d5c/d5d/f3a [0,4194304] 0 2024-04-25T15:49:55.387 INFO:tasks.workunit.client.1.smithi193.stdout:1/989: dread dc/d59/dc0/deb/d55/fe7 [0,4194304] 0 2024-04-25T15:49:55.395 INFO:tasks.workunit.client.1.smithi193.stdout:1/990: link dc/d4a/d50/d7c/d7e/c9d dc/d4a/d50/d5c/c14f 0 2024-04-25T15:49:55.396 INFO:tasks.workunit.client.1.smithi193.stdout:1/991: write dc/d4a/d4d/d7a/dd0/de2/df1/fea [2541736,62106] 0 2024-04-25T15:49:55.400 INFO:tasks.workunit.client.1.smithi193.stdout:1/992: mkdir dc/d4a/d4d/d109/d150 0 2024-04-25T15:49:55.405 INFO:tasks.workunit.client.1.smithi193.stdout:1/993: dwrite dc/f2b [0,4194304] 0 2024-04-25T15:49:55.408 INFO:tasks.workunit.client.1.smithi193.stdout:1/994: unlink dc/d4a/d50/d5c/d63/la8 0 2024-04-25T15:49:55.409 INFO:tasks.workunit.client.1.smithi193.stdout:1/995: fsync dc/f36 0 2024-04-25T15:49:55.417 INFO:tasks.workunit.client.1.smithi193.stdout:1/996: getdents dc/d4a/d4d/d7a/dd0/de2/df1 0 2024-04-25T15:49:55.422 INFO:tasks.workunit.client.1.smithi193.stdout:1/997: symlink dc/d59/dc0/l151 0 2024-04-25T15:49:55.424 INFO:tasks.workunit.client.1.smithi193.stdout:1/998: creat dc/d4a/d4d/d80/f152 x:0 0 0 2024-04-25T15:49:55.426 INFO:tasks.workunit.client.1.smithi193.stdout:1/999: creat dc/f153 x:0 0 0 2024-04-25T15:49:55.429 INFO:tasks.workunit.client.1.smithi193.stderr:+ rm -rf -- ./tmp.Vx2iyfLbHL 2024-04-25T15:49:56.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:56 smithi110 ceph-mon[26344]: pgmap v45: 65 pgs: 65 active+clean; 4.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 87 MiB/s rd, 188 MiB/s wr, 430 op/s 2024-04-25T15:49:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:56 smithi193 ceph-mon[33747]: pgmap v45: 65 pgs: 65 active+clean; 4.1 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 87 MiB/s rd, 188 MiB/s wr, 430 op/s 2024-04-25T15:49:58.347 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:58 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:58.347 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:58 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:58.347 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:58 smithi110 ceph-mon[26344]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-04-25T15:49:58.348 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:58 smithi110 ceph-mon[26344]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-04-25T15:49:58.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:58 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:58.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:58 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:49:58.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:58 smithi193 ceph-mon[33747]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-04-25T15:49:58.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:58 smithi193 ceph-mon[33747]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-04-25T15:49:59.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:49:59 smithi110 ceph-mon[26344]: pgmap v46: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 94 MiB/s rd, 202 MiB/s wr, 486 op/s 2024-04-25T15:49:59.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.369+0000 7faa9f8d8700 1 -- 172.21.15.110:0/342519135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 msgr2=0x7faa980fd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:59.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.369+0000 7faa9f8d8700 1 --2- 172.21.15.110:0/342519135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 0x7faa980fd050 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7faa88004940 tx=0x7faa88009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:59.372 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.370+0000 7faa9f8d8700 1 -- 172.21.15.110:0/342519135 shutdown_connections 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.370+0000 7faa9f8d8700 1 --2- 172.21.15.110:0/342519135 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 0x7faa980fd050 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.370+0000 7faa9f8d8700 1 --2- 172.21.15.110:0/342519135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa980fc290 0x7faa980fc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.370+0000 7faa9f8d8700 1 -- 172.21.15.110:0/342519135 >> 172.21.15.110:0/342519135 conn(0x7faa980f6b80 msgr2=0x7faa980f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.370+0000 7faa9f8d8700 1 -- 172.21.15.110:0/342519135 shutdown_connections 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.370+0000 7faa9f8d8700 1 -- 172.21.15.110:0/342519135 wait complete. 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9f8d8700 1 Processor -- start 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9f8d8700 1 -- start start 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9f8d8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa980fc290 0x7faa98103ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9f8d8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 0x7faa981023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9f8d8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa98102960 con 0x7faa980fcbe0 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9f8d8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa98102ad0 con 0x7faa980fc290 2024-04-25T15:49:59.373 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9ce73700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 0x7faa981023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9ce73700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 0x7faa981023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58682/0 (socket says 172.21.15.110:58682) 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9ce73700 1 -- 172.21.15.110:0/1297152629 learned_addr learned my addr 172.21.15.110:0/1297152629 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9d674700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa980fc290 0x7faa98103ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9ce73700 1 -- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa980fc290 msgr2=0x7faa98103ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9ce73700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa980fc290 0x7faa98103ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.371+0000 7faa9ce73700 1 -- 172.21.15.110:0/1297152629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88005040 con 0x7faa980fcbe0 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.372+0000 7faa9ce73700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 0x7faa981023b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faa88009150 tx=0x7faa88009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.372+0000 7faa8dffb700 1 -- 172.21.15.110:0/1297152629 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8800e040 con 0x7faa980fcbe0 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.372+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa98102d50 con 0x7faa980fcbe0 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.372+0000 7faa8dffb700 1 -- 172.21.15.110:0/1297152629 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa88004290 con 0x7faa980fcbe0 2024-04-25T15:49:59.374 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.372+0000 7faa8dffb700 1 -- 172.21.15.110:0/1297152629 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa8801c630 con 0x7faa980fcbe0 2024-04-25T15:49:59.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.372+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa98115a20 con 0x7faa980fcbe0 2024-04-25T15:49:59.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.373+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9805f5c0 con 0x7faa980fcbe0 2024-04-25T15:49:59.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.374+0000 7faa8dffb700 1 -- 172.21.15.110:0/1297152629 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7faa8800bbf0 con 0x7faa980fcbe0 2024-04-25T15:49:59.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.374+0000 7faa8dffb700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa8406fcb0 0x7faa84072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:59.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.374+0000 7faa8dffb700 1 -- 172.21.15.110:0/1297152629 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7faa880903b0 con 0x7faa980fcbe0 2024-04-25T15:49:59.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.374+0000 7faa9d674700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa8406fcb0 0x7faa84072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:59.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.375+0000 7faa9d674700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa8406fcb0 0x7faa84072160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa98103c70 tx=0x7faa9400b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:59.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.381+0000 7faa8dffb700 1 -- 172.21.15.110:0/1297152629 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7faa88059810 con 0x7faa980fcbe0 2024-04-25T15:49:59.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:49:59 smithi193 ceph-mon[33747]: pgmap v46: 65 pgs: 65 active+clean; 4.0 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 94 MiB/s rd, 202 MiB/s wr, 486 op/s 2024-04-25T15:49:59.713 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.712+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa981013c0 con 0x7faa8406fcb0 2024-04-25T15:49:59.714 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.714+0000 7faa8dffb700 1 -- 172.21.15.110:0/1297152629 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7faa981013c0 con 0x7faa8406fcb0 2024-04-25T15:49:59.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa8406fcb0 msgr2=0x7faa84072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:59.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa8406fcb0 0x7faa84072160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa98103c70 tx=0x7faa9400b040 comp rx=0 tx=0).stop 2024-04-25T15:49:59.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 msgr2=0x7faa981023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:59.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 0x7faa981023b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7faa88009150 tx=0x7faa88009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:59.717 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 shutdown_connections 2024-04-25T15:49:59.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa8406fcb0 0x7faa84072160 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa980fc290 0x7faa98103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 --2- 172.21.15.110:0/1297152629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa980fcbe0 0x7faa981023b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 >> 172.21.15.110:0/1297152629 conn(0x7faa980f6b80 msgr2=0x7faa980f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:59.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 shutdown_connections 2024-04-25T15:49:59.718 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.716+0000 7faa9f8d8700 1 -- 172.21.15.110:0/1297152629 wait complete. 2024-04-25T15:49:59.731 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:49:59.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.865+0000 7f790f408700 1 -- 172.21.15.110:0/2730317704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7908066af0 msgr2=0x7f79080ffa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:59.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.865+0000 7f790f408700 1 --2- 172.21.15.110:0/2730317704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7908066af0 0x7f79080ffa10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f7904004940 tx=0x7f7904009e30 comp rx=0 tx=0).stop 2024-04-25T15:49:59.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.865+0000 7f790f408700 1 -- 172.21.15.110:0/2730317704 shutdown_connections 2024-04-25T15:49:59.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.865+0000 7f790f408700 1 --2- 172.21.15.110:0/2730317704 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79080fff50 0x7f7908102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.865+0000 7f790f408700 1 --2- 172.21.15.110:0/2730317704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7908066af0 0x7f79080ffa10 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.865+0000 7f790f408700 1 -- 172.21.15.110:0/2730317704 >> 172.21.15.110:0/2730317704 conn(0x7f79080f7420 msgr2=0x7f79080f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:49:59.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.866+0000 7f790f408700 1 -- 172.21.15.110:0/2730317704 shutdown_connections 2024-04-25T15:49:59.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.866+0000 7f790f408700 1 -- 172.21.15.110:0/2730317704 wait complete. 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.866+0000 7f790f408700 1 Processor -- start 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.866+0000 7f790f408700 1 -- start start 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790f408700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7908066af0 0x7f790810da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790f408700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79080fff50 0x7f790810dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790f408700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790810e5b0 con 0x7f79080fff50 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790f408700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f790810ee00 con 0x7f7908066af0 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790c9a3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79080fff50 0x7f790810dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790c9a3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79080fff50 0x7f790810dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53276/0 (socket says 172.21.15.110:53276) 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790c9a3700 1 -- 172.21.15.110:0/4111384763 learned_addr learned my addr 172.21.15.110:0/4111384763 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790d1a4700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7908066af0 0x7f790810da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790c9a3700 1 -- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7908066af0 msgr2=0x7f790810da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790c9a3700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7908066af0 0x7f790810da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:49:59.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790c9a3700 1 -- 172.21.15.110:0/4111384763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904005040 con 0x7f79080fff50 2024-04-25T15:49:59.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f790c9a3700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79080fff50 0x7f790810dfb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f78f8002a50 tx=0x7f78f80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:59.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.867+0000 7f78fdffb700 1 -- 172.21.15.110:0/4111384763 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78f8007cc0 con 0x7f79080fff50 2024-04-25T15:49:59.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.868+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f790810f0e0 con 0x7f79080fff50 2024-04-25T15:49:59.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.868+0000 7f78fdffb700 1 -- 172.21.15.110:0/4111384763 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78f800f040 con 0x7f79080fff50 2024-04-25T15:49:59.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.868+0000 7f78fdffb700 1 -- 172.21.15.110:0/4111384763 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78f800da30 con 0x7f79080fff50 2024-04-25T15:49:59.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.868+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f790810f620 con 0x7f79080fff50 2024-04-25T15:49:59.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.869+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f790805f5c0 con 0x7f79080fff50 2024-04-25T15:49:59.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.870+0000 7f78fdffb700 1 -- 172.21.15.110:0/4111384763 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f78f800db90 con 0x7f79080fff50 2024-04-25T15:49:59.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.870+0000 7f78fdffb700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f78f406fcb0 0x7f78f4072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:49:59.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.870+0000 7f78fdffb700 1 -- 172.21.15.110:0/4111384763 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f78f808f5b0 con 0x7f79080fff50 2024-04-25T15:49:59.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.871+0000 7f790d1a4700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f78f406fcb0 0x7f78f4072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:49:59.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.871+0000 7f790d1a4700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f78f406fcb0 0x7f78f4072160 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f790400b970 tx=0x7f7904009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:49:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:49:59.875+0000 7f78fdffb700 1 -- 172.21.15.110:0/4111384763 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f78f8058870 con 0x7f79080fff50 2024-04-25T15:50:00.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.209+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f790810f9a0 con 0x7f78f406fcb0 2024-04-25T15:50:00.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.210+0000 7f78fdffb700 1 -- 172.21.15.110:0/4111384763 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7f790810f9a0 con 0x7f78f406fcb0 2024-04-25T15:50:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f78f406fcb0 msgr2=0x7f78f4072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f78f406fcb0 0x7f78f4072160 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f790400b970 tx=0x7f7904009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79080fff50 msgr2=0x7f790810dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79080fff50 0x7f790810dfb0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f78f8002a50 tx=0x7f78f80046b0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 shutdown_connections 2024-04-25T15:50:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f78f406fcb0 0x7f78f4072160 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7908066af0 0x7f790810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 --2- 172.21.15.110:0/4111384763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79080fff50 0x7f790810dfb0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 >> 172.21.15.110:0/4111384763 conn(0x7f79080f7420 msgr2=0x7f79080f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 shutdown_connections 2024-04-25T15:50:00.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.213+0000 7f790f408700 1 -- 172.21.15.110:0/4111384763 wait complete. 2024-04-25T15:50:00.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.348+0000 7f7c8759e700 1 -- 172.21.15.110:0/1230736295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 msgr2=0x7f7c800fbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:00.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.348+0000 7f7c8759e700 1 --2- 172.21.15.110:0/1230736295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 0x7f7c800fbf00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7c74004940 tx=0x7f7c74009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:00.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:00 smithi110 ceph-mon[26344]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:50:00.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:00 smithi110 ceph-mon[26344]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:50:00.350 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:00 smithi110 ceph-mon[26344]: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.348+0000 7f7c8759e700 1 -- 172.21.15.110:0/1230736295 shutdown_connections 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.349+0000 7f7c8759e700 1 --2- 172.21.15.110:0/1230736295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c800fcda0 0x7f7c800fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.349+0000 7f7c8759e700 1 --2- 172.21.15.110:0/1230736295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 0x7f7c800fbf00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.349+0000 7f7c8759e700 1 -- 172.21.15.110:0/1230736295 >> 172.21.15.110:0/1230736295 conn(0x7f7c800f7250 msgr2=0x7f7c800f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.349+0000 7f7c8759e700 1 -- 172.21.15.110:0/1230736295 shutdown_connections 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.349+0000 7f7c8759e700 1 -- 172.21.15.110:0/1230736295 wait complete. 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.349+0000 7f7c8759e700 1 Processor -- start 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.349+0000 7f7c8759e700 1 -- start start 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8759e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 0x7f7c8010c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8759e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c800fcda0 0x7f7c8010cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8759e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8010d1f0 con 0x7f7c800fbaf0 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8759e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8010da40 con 0x7f7c800fcda0 2024-04-25T15:50:00.352 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 0x7f7c8010c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 0x7f7c8010c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53282/0 (socket says 172.21.15.110:53282) 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8659c700 1 -- 172.21.15.110:0/974486205 learned_addr learned my addr 172.21.15.110:0/974486205 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c85d9b700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c800fcda0 0x7f7c8010cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8659c700 1 -- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c800fcda0 msgr2=0x7f7c8010cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8659c700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c800fcda0 0x7f7c8010cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8659c700 1 -- 172.21.15.110:0/974486205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c74005040 con 0x7f7c800fbaf0 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c85d9b700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c800fcda0 0x7f7c8010cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c8659c700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 0x7f7c8010c6b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7c74013870 tx=0x7f7c74009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.350+0000 7f7c72ffd700 1 -- 172.21.15.110:0/974486205 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c7400e070 con 0x7f7c800fbaf0 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.351+0000 7f7c72ffd700 1 -- 172.21.15.110:0/974486205 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c7401a480 con 0x7f7c800fbaf0 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.351+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c8010dcc0 con 0x7f7c800fbaf0 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.351+0000 7f7c72ffd700 1 -- 172.21.15.110:0/974486205 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c74023640 con 0x7f7c800fbaf0 2024-04-25T15:50:00.353 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.351+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c8010e200 con 0x7f7c800fbaf0 2024-04-25T15:50:00.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.352+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8005f5c0 con 0x7f7c800fbaf0 2024-04-25T15:50:00.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.353+0000 7f7c72ffd700 1 -- 172.21.15.110:0/974486205 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f7c7401a5f0 con 0x7f7c800fbaf0 2024-04-25T15:50:00.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.353+0000 7f7c72ffd700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7c6c06fcb0 0x7f7c6c072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:00.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.353+0000 7f7c72ffd700 1 -- 172.21.15.110:0/974486205 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f7c74093130 con 0x7f7c800fbaf0 2024-04-25T15:50:00.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.353+0000 7f7c85d9b700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7c6c06fcb0 0x7f7c6c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:00.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.354+0000 7f7c85d9b700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7c6c06fcb0 0x7f7c6c072160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009b80 tx=0x7f7c7800b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:00.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.362+0000 7f7c72ffd700 1 -- 172.21.15.110:0/974486205 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f7c7405c470 con 0x7f7c800fbaf0 2024-04-25T15:50:00.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:00 smithi193 ceph-mon[33747]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:50:00.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:00 smithi193 ceph-mon[33747]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:50:00.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:00 smithi193 ceph-mon[33747]: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:50:00.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.689+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c8010e530 con 0x7f7c6c06fcb0 2024-04-25T15:50:00.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.699+0000 7f7c72ffd700 1 -- 172.21.15.110:0/974486205 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f7c8010e530 con 0x7f7c6c06fcb0 2024-04-25T15:50:00.700 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 starting - - - - 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (9m) 77s ago 9m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (8m) 79s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 starting - - - - 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (3m) 77s ago 3m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (3m) 77s ago 3m 86.6M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (3m) 79s ago 3m 12.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (3m) 79s ago 3m 14.4M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:9283 running (10m) 77s ago 10m 363M - 17.2.7-902-gc61472c8 0de407c1b3af c623b709ef43 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (96s) 79s ago 8m 558M - 19.0.0-3291-g994e2f20 d53c96779a29 82852af5489a 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (10m) 77s ago 10m 49.0M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (7m) 79s ago 7m 40.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 starting - - - - 2024-04-25T15:50:00.701 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (8m) 79s ago 8m 15.3M - 1.5.0 0da6a335fe13 13afa4d53352 2024-04-25T15:50:00.702 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (7m) 77s ago 7m 61.7M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:50:00.702 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (6m) 77s ago 6m 66.0M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:50:00.702 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (6m) 77s ago 6m 56.3M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:50:00.702 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (5m) 79s ago 5m 76.0M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:50:00.702 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (5m) 79s ago 5m 64.8M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:50:00.702 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (5m) 79s ago 5m 69.0M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:50:00.702 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (7m) 77s ago 8m 54.9M - 2.43.0 a07b618ecd1d ce7fd55a0368 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7c6c06fcb0 msgr2=0x7f7c6c072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7c6c06fcb0 0x7f7c6c072160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009b80 tx=0x7f7c7800b040 comp rx=0 tx=0).stop 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 msgr2=0x7f7c8010c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 0x7f7c8010c6b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7c74013870 tx=0x7f7c74009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 shutdown_connections 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f7c6c06fcb0 0x7f7c6c072160 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c800fbaf0 0x7f7c8010c6b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 --2- 172.21.15.110:0/974486205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c800fcda0 0x7f7c8010cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 >> 172.21.15.110:0/974486205 conn(0x7f7c800f7250 msgr2=0x7f7c800f8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:00.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 shutdown_connections 2024-04-25T15:50:00.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.703+0000 7f7c8759e700 1 -- 172.21.15.110:0/974486205 wait complete. 2024-04-25T15:50:00.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.835+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/2212818071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 msgr2=0x7f8b14102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:00.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.835+0000 7f8b1a2b3700 1 --2- 172.21.15.110:0/2212818071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 0x7f8b14102300 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f8b08004940 tx=0x7f8b08009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:00.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.835+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/2212818071 shutdown_connections 2024-04-25T15:50:00.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.835+0000 7f8b1a2b3700 1 --2- 172.21.15.110:0/2212818071 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 0x7f8b14102300 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.835+0000 7f8b1a2b3700 1 --2- 172.21.15.110:0/2212818071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b14066ed0 0x7f8b140ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.835+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/2212818071 >> 172.21.15.110:0/2212818071 conn(0x7f8b140f7430 msgr2=0x7f8b140f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.836+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/2212818071 shutdown_connections 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.836+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/2212818071 wait complete. 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.836+0000 7f8b1a2b3700 1 Processor -- start 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.836+0000 7f8b1a2b3700 1 -- start start 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.836+0000 7f8b1a2b3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b14066ed0 0x7f8b140fd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.836+0000 7f8b1a2b3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 0x7f8b140fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.836+0000 7f8b1a2b3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140fd750 con 0x7f8b140ffee0 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.836+0000 7f8b1a2b3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b140fbde0 con 0x7f8b14066ed0 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b137fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 0x7f8b140fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:00.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b13fff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b14066ed0 0x7f8b140fd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:00.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b137fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 0x7f8b140fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53302/0 (socket says 172.21.15.110:53302) 2024-04-25T15:50:00.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b137fe700 1 -- 172.21.15.110:0/397903753 learned_addr learned my addr 172.21.15.110:0/397903753 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b13fff700 1 -- 172.21.15.110:0/397903753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 msgr2=0x7f8b140fb8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b13fff700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 0x7f8b140fb8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b13fff700 1 -- 172.21.15.110:0/397903753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b08005040 con 0x7f8b14066ed0 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b13fff700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b14066ed0 0x7f8b140fd150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b14066c10 tx=0x7f8b0400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b10ff9700 1 -- 172.21.15.110:0/397903753 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b04004e70 con 0x7f8b14066ed0 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b10ff9700 1 -- 172.21.15.110:0/397903753 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b0400f040 con 0x7f8b14066ed0 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b140fc060 con 0x7f8b14066ed0 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b10ff9700 1 -- 172.21.15.110:0/397903753 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b0400c950 con 0x7f8b14066ed0 2024-04-25T15:50:00.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.837+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b140fc550 con 0x7f8b14066ed0 2024-04-25T15:50:00.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.839+0000 7f8b10ff9700 1 -- 172.21.15.110:0/397903753 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f8b0400cbc0 con 0x7f8b14066ed0 2024-04-25T15:50:00.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.839+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b1404c710 con 0x7f8b14066ed0 2024-04-25T15:50:00.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.839+0000 7f8b10ff9700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8afc06fcc0 0x7f8afc072170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:00.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.839+0000 7f8b10ff9700 1 -- 172.21.15.110:0/397903753 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f8b0408f480 con 0x7f8b14066ed0 2024-04-25T15:50:00.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.839+0000 7f8b137fe700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8afc06fcc0 0x7f8afc072170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:00.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.840+0000 7f8b137fe700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8afc06fcc0 0x7f8afc072170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800b0d0 tx=0x7f8b08009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:00.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:00.844+0000 7f8b10ff9700 1 -- 172.21.15.110:0/397903753 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f8b040586a0 con 0x7f8b14066ed0 2024-04-25T15:50:01.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.247+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8b14115a90 con 0x7f8b14066ed0 2024-04-25T15:50:01.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.247+0000 7f8b10ff9700 1 -- 172.21.15.110:0/397903753 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+846 (secure 0 0 0) 0x7f8b04058040 con 0x7f8b14066ed0 2024-04-25T15:50:01.248 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:50:01.248 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 13, 2024-04-25T15:50:01.249 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:50:01.250 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:50:01.250 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:50:01.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8afc06fcc0 msgr2=0x7f8afc072170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:01.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8afc06fcc0 0x7f8afc072170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f8b0800b0d0 tx=0x7f8b08009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:01.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b14066ed0 msgr2=0x7f8b140fd150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:01.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b14066ed0 0x7f8b140fd150 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8b14066c10 tx=0x7f8b0400b040 comp rx=0 tx=0).stop 2024-04-25T15:50:01.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 shutdown_connections 2024-04-25T15:50:01.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8afc06fcc0 0x7f8afc072170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b14066ed0 0x7f8b140fd150 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 --2- 172.21.15.110:0/397903753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b140ffee0 0x7f8b140fb8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 >> 172.21.15.110:0/397903753 conn(0x7f8b140f7430 msgr2=0x7f8b140f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:01.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 shutdown_connections 2024-04-25T15:50:01.252 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.250+0000 7f8b1a2b3700 1 -- 172.21.15.110:0/397903753 wait complete. 2024-04-25T15:50:01.321 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:01 smithi110 ceph-mon[26344]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:01.321 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:01 smithi110 ceph-mon[26344]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:01.321 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:01 smithi110 ceph-mon[26344]: pgmap v47: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 63 MiB/s rd, 142 MiB/s wr, 384 op/s 2024-04-25T15:50:01.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.391+0000 7f1367a7e700 1 -- 172.21.15.110:0/641712239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600ffee0 msgr2=0x7f1360102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:01.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.391+0000 7f1367a7e700 1 --2- 172.21.15.110:0/641712239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600ffee0 0x7f1360102300 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1350004940 tx=0x7f1350009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:01.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.391+0000 7f1367a7e700 1 -- 172.21.15.110:0/641712239 shutdown_connections 2024-04-25T15:50:01.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.391+0000 7f1367a7e700 1 --2- 172.21.15.110:0/641712239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600ffee0 0x7f1360102300 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.392 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.391+0000 7f1367a7e700 1 --2- 172.21.15.110:0/641712239 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13600662c0 0x7f13600ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.391+0000 7f1367a7e700 1 -- 172.21.15.110:0/641712239 >> 172.21.15.110:0/641712239 conn(0x7f13600f7430 msgr2=0x7f13600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:01.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.391+0000 7f1367a7e700 1 -- 172.21.15.110:0/641712239 shutdown_connections 2024-04-25T15:50:01.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.391+0000 7f1367a7e700 1 -- 172.21.15.110:0/641712239 wait complete. 2024-04-25T15:50:01.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1367a7e700 1 Processor -- start 2024-04-25T15:50:01.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1367a7e700 1 -- start start 2024-04-25T15:50:01.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1367a7e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600662c0 0x7f13600fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:01.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1367a7e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13600ffee0 0x7f13600fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:01.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1367a7e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1360111640 con 0x7f13600662c0 2024-04-25T15:50:01.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1367a7e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1360111780 con 0x7f13600ffee0 2024-04-25T15:50:01.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f136581a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600662c0 0x7f13600fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1365019700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13600ffee0 0x7f13600fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f136581a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600662c0 0x7f13600fd380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53320/0 (socket says 172.21.15.110:53320) 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f136581a700 1 -- 172.21.15.110:0/3424979844 learned_addr learned my addr 172.21.15.110:0/3424979844 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1365019700 1 -- 172.21.15.110:0/3424979844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600662c0 msgr2=0x7f13600fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1365019700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600662c0 0x7f13600fd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1365019700 1 -- 172.21.15.110:0/3424979844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1350005040 con 0x7f13600ffee0 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f136581a700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600662c0 0x7f13600fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.392+0000 7f1365019700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13600ffee0 0x7f13600fb8a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1350000c00 tx=0x7f1350009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.393+0000 7f13567fc700 1 -- 172.21.15.110:0/3424979844 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135000e030 con 0x7f13600ffee0 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.393+0000 7f13567fc700 1 -- 172.21.15.110:0/3424979844 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1350013de0 con 0x7f13600ffee0 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.393+0000 7f13567fc700 1 -- 172.21.15.110:0/3424979844 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135001ce50 con 0x7f13600ffee0 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.393+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13600fbe40 con 0x7f13600ffee0 2024-04-25T15:50:01.395 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.393+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13600fc330 con 0x7f13600ffee0 2024-04-25T15:50:01.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.394+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136005f5c0 con 0x7f13600ffee0 2024-04-25T15:50:01.396 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.394+0000 7f13567fc700 1 -- 172.21.15.110:0/3424979844 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7f135001c400 con 0x7f13600ffee0 2024-04-25T15:50:01.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.395+0000 7f13567fc700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f134c06fcb0 0x7f134c072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:01.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.395+0000 7f13567fc700 1 -- 172.21.15.110:0/3424979844 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f1350091010 con 0x7f13600ffee0 2024-04-25T15:50:01.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.395+0000 7f136581a700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f134c06fcb0 0x7f134c072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:01.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.395+0000 7f136581a700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f134c06fcb0 0x7f134c072160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f13600fd110 tx=0x7f135c00b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:01.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.400+0000 7f13567fc700 1 -- 172.21.15.110:0/3424979844 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f135005a470 con 0x7f13600ffee0 2024-04-25T15:50:01.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:01 smithi193 ceph-mon[33747]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:01.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:01 smithi193 ceph-mon[33747]: from='client.14734 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:01.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:01 smithi193 ceph-mon[33747]: pgmap v47: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 63 MiB/s rd, 142 MiB/s wr, 384 op/s 2024-04-25T15:50:01.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.739+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f13600fc660 con 0x7f13600ffee0 2024-04-25T15:50:01.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.740+0000 7f13567fc700 1 -- 172.21.15.110:0/3424979844 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f1350059e10 con 0x7f13600ffee0 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:50:01.741 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:50:01.742 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:50:01.743 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f134c06fcb0 msgr2=0x7f134c072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f134c06fcb0 0x7f134c072160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f13600fd110 tx=0x7f135c00b040 comp rx=0 tx=0).stop 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13600ffee0 msgr2=0x7f13600fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13600ffee0 0x7f13600fb8a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1350000c00 tx=0x7f1350009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 shutdown_connections 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f134c06fcb0 0x7f134c072160 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13600662c0 0x7f13600fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 --2- 172.21.15.110:0/3424979844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13600ffee0 0x7f13600fb8a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 >> 172.21.15.110:0/3424979844 conn(0x7f13600f7430 msgr2=0x7f13600f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:01.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 shutdown_connections 2024-04-25T15:50:01.745 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.743+0000 7f1367a7e700 1 -- 172.21.15.110:0/3424979844 wait complete. 2024-04-25T15:50:01.745 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:50:01.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.891+0000 7faa2e8c2700 1 -- 172.21.15.110:0/4015449986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fcf60 msgr2=0x7faa280fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:01.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.891+0000 7faa2e8c2700 1 --2- 172.21.15.110:0/4015449986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fcf60 0x7faa280fd3d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7faa1c004940 tx=0x7faa1c009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:01.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.892+0000 7faa2e8c2700 1 -- 172.21.15.110:0/4015449986 shutdown_connections 2024-04-25T15:50:01.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.892+0000 7faa2e8c2700 1 --2- 172.21.15.110:0/4015449986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fcf60 0x7faa280fd3d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.892+0000 7faa2e8c2700 1 --2- 172.21.15.110:0/4015449986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa280fbcb0 0x7faa280fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.892+0000 7faa2e8c2700 1 -- 172.21.15.110:0/4015449986 >> 172.21.15.110:0/4015449986 conn(0x7faa280f7430 msgr2=0x7faa280f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:01.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.892+0000 7faa2e8c2700 1 -- 172.21.15.110:0/4015449986 shutdown_connections 2024-04-25T15:50:01.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.892+0000 7faa2e8c2700 1 -- 172.21.15.110:0/4015449986 wait complete. 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa2e8c2700 1 Processor -- start 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa2e8c2700 1 -- start start 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa2e8c2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fbcb0 0x7faa28113f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa2e8c2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa280fcf60 0x7faa281144c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa2e8c2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28114ac0 con 0x7faa280fbcb0 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa2e8c2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa28115310 con 0x7faa280fcf60 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa27fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fbcb0 0x7faa28113f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa27fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fbcb0 0x7faa28113f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53342/0 (socket says 172.21.15.110:53342) 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa27fff700 1 -- 172.21.15.110:0/2496998842 learned_addr learned my addr 172.21.15.110:0/2496998842 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.893+0000 7faa277fe700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa280fcf60 0x7faa281144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa27fff700 1 -- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa280fcf60 msgr2=0x7faa281144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:01.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa27fff700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa280fcf60 0x7faa281144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:01.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa27fff700 1 -- 172.21.15.110:0/2496998842 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa1c005040 con 0x7faa280fbcb0 2024-04-25T15:50:01.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa277fe700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa280fcf60 0x7faa281144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:50:01.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa27fff700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fbcb0 0x7faa28113f80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faa2000ec30 tx=0x7faa2000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:01.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa24ff9700 1 -- 172.21.15.110:0/2496998842 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa20007b30 con 0x7faa280fbcb0 2024-04-25T15:50:01.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa281155f0 con 0x7faa280fbcb0 2024-04-25T15:50:01.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa24ff9700 1 -- 172.21.15.110:0/2496998842 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa2000f040 con 0x7faa280fbcb0 2024-04-25T15:50:01.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa24ff9700 1 -- 172.21.15.110:0/2496998842 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa2000cc90 con 0x7faa280fbcb0 2024-04-25T15:50:01.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.894+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa28115b30 con 0x7faa280fbcb0 2024-04-25T15:50:01.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.895+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa281008f0 con 0x7faa280fbcb0 2024-04-25T15:50:01.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.896+0000 7faa24ff9700 1 -- 172.21.15.110:0/2496998842 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7faa20009070 con 0x7faa280fbcb0 2024-04-25T15:50:01.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.896+0000 7faa24ff9700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa1006fcb0 0x7faa10072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:01.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.896+0000 7faa24ff9700 1 -- 172.21.15.110:0/2496998842 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7faa2008fd50 con 0x7faa280fbcb0 2024-04-25T15:50:01.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.896+0000 7faa277fe700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa1006fcb0 0x7faa10072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:01.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.897+0000 7faa277fe700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa1006fcb0 0x7faa10072160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0137d0 tx=0x7faa1c009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:01.902 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:01.901+0000 7faa24ff9700 1 -- 172.21.15.110:0/2496998842 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7faa20059010 con 0x7faa280fbcb0 2024-04-25T15:50:02.163 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.163+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa28105a30 con 0x7faa1006fcb0 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.164+0000 7faa24ff9700 1 -- 172.21.15.110:0/2496998842 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+318 (secure 0 0 0) 0x7faa28105a30 con 0x7faa1006fcb0 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "1/21 daemons upgraded", 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:50:02.165 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:50:02.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa1006fcb0 msgr2=0x7faa10072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:02.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa1006fcb0 0x7faa10072160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0137d0 tx=0x7faa1c009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:02.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fbcb0 msgr2=0x7faa28113f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:02.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fbcb0 0x7faa28113f80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7faa2000ec30 tx=0x7faa2000b040 comp rx=0 tx=0).stop 2024-04-25T15:50:02.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 shutdown_connections 2024-04-25T15:50:02.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7faa1006fcb0 0x7faa10072160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa280fbcb0 0x7faa28113f80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 --2- 172.21.15.110:0/2496998842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa280fcf60 0x7faa281144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 >> 172.21.15.110:0/2496998842 conn(0x7faa280f7430 msgr2=0x7faa280f8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:02.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 shutdown_connections 2024-04-25T15:50:02.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.166+0000 7faa2e8c2700 1 -- 172.21.15.110:0/2496998842 wait complete. 2024-04-25T15:50:02.244 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:02 smithi110 ceph-mon[26344]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:02.244 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:02 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/397903753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:50:02.244 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:02 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/3424979844' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:50:02.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1064811120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9080fff80 msgr2=0x7fc908102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:02.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 --2- 172.21.15.110:0/1064811120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9080fff80 0x7fc908102360 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc004940 tx=0x7fc8fc009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:02.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1064811120 shutdown_connections 2024-04-25T15:50:02.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 --2- 172.21.15.110:0/1064811120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9080fff80 0x7fc908102360 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 --2- 172.21.15.110:0/1064811120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc908066220 0x7fc9080ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1064811120 >> 172.21.15.110:0/1064811120 conn(0x7fc9080f7430 msgr2=0x7fc9080f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:02.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1064811120 shutdown_connections 2024-04-25T15:50:02.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1064811120 wait complete. 2024-04-25T15:50:02.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.294+0000 7fc90dd2e700 1 Processor -- start 2024-04-25T15:50:02.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc90dd2e700 1 -- start start 2024-04-25T15:50:02.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc90dd2e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc908066220 0x7fc9080fd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc90dd2e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9080fff80 0x7fc9080fb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc90dd2e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9080fbef0 con 0x7fc908066220 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc90dd2e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9080fc060 con 0x7fc9080fff80 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc9077fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc908066220 0x7fc9080fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc9077fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc908066220 0x7fc9080fd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53364/0 (socket says 172.21.15.110:53364) 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc906ffd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9080fff80 0x7fc9080fb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.295+0000 7fc9077fe700 1 -- 172.21.15.110:0/1870076565 learned_addr learned my addr 172.21.15.110:0/1870076565 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc9077fe700 1 -- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9080fff80 msgr2=0x7fc9080fb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc9077fe700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9080fff80 0x7fc9080fb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc9077fe700 1 -- 172.21.15.110:0/1870076565 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc005040 con 0x7fc908066220 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc906ffd700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9080fff80 0x7fc9080fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc9077fe700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc908066220 0x7fc9080fd470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8002a80 tx=0x7fc8f8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc8effff700 1 -- 172.21.15.110:0/1870076565 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f8007cc0 con 0x7fc908066220 2024-04-25T15:50:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc8effff700 1 -- 172.21.15.110:0/1870076565 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8f800f040 con 0x7fc908066220 2024-04-25T15:50:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9080fc340 con 0x7fc908066220 2024-04-25T15:50:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc8effff700 1 -- 172.21.15.110:0/1870076565 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8f800da30 con 0x7fc908066220 2024-04-25T15:50:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.296+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc908115a50 con 0x7fc908066220 2024-04-25T15:50:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.297+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc90805f5c0 con 0x7fc908066220 2024-04-25T15:50:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.298+0000 7fc8effff700 1 -- 172.21.15.110:0/1870076565 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) v1 ==== 92610+0+0 (secure 0 0 0) 0x7fc8f800db90 con 0x7fc908066220 2024-04-25T15:50:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.299+0000 7fc8effff700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fc8f006fcb0 0x7fc8f0072160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:02.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.299+0000 7fc8effff700 1 -- 172.21.15.110:0/1870076565 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7fc8f808f580 con 0x7fc908066220 2024-04-25T15:50:02.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.299+0000 7fc906ffd700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fc8f006fcb0 0x7fc8f0072160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:02.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.300+0000 7fc906ffd700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fc8f006fcb0 0x7fc8f0072160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00b970 tx=0x7fc8fc009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:02.307 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.306+0000 7fc8effff700 1 -- 172.21.15.110:0/1870076565 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7fc8f8058820 con 0x7fc908066220 2024-04-25T15:50:02.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:02 smithi193 ceph-mon[33747]: from='client.14738 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:02.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:02 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/397903753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:50:02.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:02 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3424979844' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:50:02.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.742+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc9081058a0 con 0x7fc908066220 2024-04-25T15:50:02.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.743+0000 7fc8effff700 1 -- 172.21.15.110:0/1870076565 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fc8f80581c0 con 0x7fc908066220 2024-04-25T15:50:02.744 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:50:02.744 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:50:02.744 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fc8f006fcb0 msgr2=0x7fc8f0072160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fc8f006fcb0 0x7fc8f0072160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00b970 tx=0x7fc8fc009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc908066220 msgr2=0x7fc9080fd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc908066220 0x7fc9080fd470 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8f8002a80 tx=0x7fc8f8004600 comp rx=0 tx=0).stop 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 shutdown_connections 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7fc8f006fcb0 0x7fc8f0072160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc908066220 0x7fc9080fd470 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 --2- 172.21.15.110:0/1870076565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9080fff80 0x7fc9080fb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 >> 172.21.15.110:0/1870076565 conn(0x7fc9080f7430 msgr2=0x7fc9080f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:02.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 shutdown_connections 2024-04-25T15:50:02.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:02.746+0000 7fc90dd2e700 1 -- 172.21.15.110:0/1870076565 wait complete. 2024-04-25T15:50:03.283 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:03 smithi110 ceph-mon[26344]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:03.283 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:03 smithi110 ceph-mon[26344]: pgmap v48: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 58 MiB/s rd, 123 MiB/s wr, 338 op/s 2024-04-25T15:50:03.283 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:03 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:50:03.283 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:03 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/1870076565' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:50:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:03 smithi193 ceph-mon[33747]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:03 smithi193 ceph-mon[33747]: pgmap v48: 65 pgs: 65 active+clean; 3.9 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 58 MiB/s rd, 123 MiB/s wr, 338 op/s 2024-04-25T15:50:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:03 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:50:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:03 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/1870076565' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:50:04.638 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:04 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:04.638 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:04 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:04.638 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:04 smithi193 ceph-mon[33747]: Reconfiguring node-exporter.smithi193 (dependencies changed)... 2024-04-25T15:50:04.638 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:04 smithi193 ceph-mon[33747]: Reconfiguring daemon node-exporter.smithi193 on smithi193 2024-04-25T15:50:04.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:04 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:04.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:04 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:04.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:04 smithi110 ceph-mon[26344]: Reconfiguring node-exporter.smithi193 (dependencies changed)... 2024-04-25T15:50:04.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:04 smithi110 ceph-mon[26344]: Reconfiguring daemon node-exporter.smithi193 on smithi193 2024-04-25T15:50:05.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:05 smithi193 ceph-mon[33747]: pgmap v49: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 58 MiB/s rd, 124 MiB/s wr, 424 op/s 2024-04-25T15:50:05.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:05 smithi110 ceph-mon[26344]: pgmap v49: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 58 MiB/s rd, 124 MiB/s wr, 424 op/s 2024-04-25T15:50:06.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: mgrmap e26: smithi193.mgoteh(active, since 93s), standbys: smithi110.lvgbkf 2024-04-25T15:50:06.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110.front.sepia.ceph.com:3000"}]: dispatch 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110.front.sepia.ceph.com:9095"}]: dispatch 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:06 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:50:06.753 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: mgrmap e26: smithi193.mgoteh(active, since 93s), standbys: smithi110.lvgbkf 2024-04-25T15:50:06.753 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110.front.sepia.ceph.com:3000"}]: dispatch 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110.front.sepia.ceph.com:9095"}]: dispatch 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:06.754 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:06 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:50:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:50:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-04-25T15:50:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:50:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110.front.sepia.ceph.com:3000"}]: dispatch 2024-04-25T15:50:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:50:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110.front.sepia.ceph.com:9095"}]: dispatch 2024-04-25T15:50:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: Upgrade: Need to upgrade myself (mgr.smithi193.mgoteh) 2024-04-25T15:50:07.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: Upgrade: Need to upgrade myself (mgr.smithi193.mgoteh) 2024-04-25T15:50:07.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:07 smithi193 ceph-mon[33747]: pgmap v50: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 11 MiB/s rd, 25 MiB/s wr, 198 op/s 2024-04-25T15:50:07.795 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:50:07.795 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110.front.sepia.ceph.com:9093"}]: dispatch 2024-04-25T15:50:07.796 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:50:07.796 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110.front.sepia.ceph.com:3000"}]: dispatch 2024-04-25T15:50:07.796 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:50:07.796 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110.front.sepia.ceph.com:9095"}]: dispatch 2024-04-25T15:50:07.796 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: Upgrade: Need to upgrade myself (mgr.smithi193.mgoteh) 2024-04-25T15:50:07.796 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: Upgrade: Need to upgrade myself (mgr.smithi193.mgoteh) 2024-04-25T15:50:07.796 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:07 smithi110 ceph-mon[26344]: pgmap v50: 65 pgs: 65 active+clean; 3.1 GiB data, 11 GiB used, 526 GiB / 536 GiB avail; 11 MiB/s rd, 25 MiB/s wr, 198 op/s 2024-04-25T15:50:09.007 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:08 smithi110 ceph-mon[26344]: Upgrade: Updating mgr.smithi110.lvgbkf 2024-04-25T15:50:09.007 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:08 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:09.007 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:08 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.lvgbkf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:50:09.007 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:08 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.lvgbkf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:50:09.007 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:08 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:50:09.007 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:08 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:50:09.008 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:08 smithi110 ceph-mon[26344]: Deploying daemon mgr.smithi110.lvgbkf on smithi110 2024-04-25T15:50:09.008 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:08 smithi110 ceph-mon[26344]: pgmap v51: 65 pgs: 65 active+clean; 2.1 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 11 MiB/s rd, 25 MiB/s wr, 247 op/s 2024-04-25T15:50:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:08 smithi193 ceph-mon[33747]: Upgrade: Updating mgr.smithi110.lvgbkf 2024-04-25T15:50:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:08 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:08 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.lvgbkf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:50:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:08 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.lvgbkf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:50:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:08 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:50:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:08 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:50:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:08 smithi193 ceph-mon[33747]: Deploying daemon mgr.smithi110.lvgbkf on smithi110 2024-04-25T15:50:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:08 smithi193 ceph-mon[33747]: pgmap v51: 65 pgs: 65 active+clean; 2.1 GiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 11 MiB/s rd, 25 MiB/s wr, 247 op/s 2024-04-25T15:50:10.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:10 smithi193 ceph-mon[33747]: pgmap v52: 65 pgs: 65 active+clean; 1.9 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 225 op/s 2024-04-25T15:50:10.824 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:10 smithi110 ceph-mon[26344]: pgmap v52: 65 pgs: 65 active+clean; 1.9 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 4.3 MiB/s rd, 12 MiB/s wr, 225 op/s 2024-04-25T15:50:11.212 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T15:50:11.212 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T15:50:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:12 smithi110 ceph-mon[26344]: pgmap v53: 65 pgs: 65 active+clean; 1.9 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-04-25T15:50:12.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:12 smithi193 ceph-mon[33747]: pgmap v53: 65 pgs: 65 active+clean; 1.9 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 168 op/s 2024-04-25T15:50:13.840 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:13 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:13.841 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:13 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:13.841 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:13 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:50:13.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:13 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:13.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:13 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:13.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:13 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:50:14.232 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T15:50:14.233 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-25T15:50:14.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:14 smithi193 ceph-mon[33747]: pgmap v54: 65 pgs: 65 active+clean; 811 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 241 op/s 2024-04-25T15:50:14.996 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:14 smithi110 ceph-mon[26344]: pgmap v54: 65 pgs: 65 active+clean; 811 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 32 KiB/s rd, 1.8 MiB/s wr, 241 op/s 2024-04-25T15:50:17.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:17 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:17.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:17 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:17.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:17 smithi110 ceph-mon[26344]: pgmap v55: 65 pgs: 65 active+clean; 811 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-25T15:50:17.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:17 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:17.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:17 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:17.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:17 smithi193 ceph-mon[33747]: pgmap v55: 65 pgs: 65 active+clean; 811 MiB data, 4.2 GiB used, 532 GiB / 536 GiB avail; 22 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-04-25T15:50:18.659 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:18 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:50:18.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:18 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:50:19.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:19 smithi193 ceph-mon[33747]: pgmap v56: 65 pgs: 65 active+clean; 340 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T15:50:19.748 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:19 smithi110 ceph-mon[26344]: pgmap v56: 65 pgs: 65 active+clean; 340 MiB data, 2.9 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T15:50:20.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:20 smithi193 ceph-mon[33747]: pgmap v57: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-04-25T15:50:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:20 smithi110 ceph-mon[26344]: pgmap v57: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-04-25T15:50:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:21 smithi110 ceph-mon[26344]: Standby manager daemon smithi110.lvgbkf restarted 2024-04-25T15:50:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:21 smithi110 ceph-mon[26344]: Standby manager daemon smithi110.lvgbkf started 2024-04-25T15:50:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:21 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.lvgbkf/crt"}]: dispatch 2024-04-25T15:50:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:21 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:50:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:21 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.lvgbkf/key"}]: dispatch 2024-04-25T15:50:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:21 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:50:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:21 smithi193 ceph-mon[33747]: Standby manager daemon smithi110.lvgbkf restarted 2024-04-25T15:50:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:21 smithi193 ceph-mon[33747]: Standby manager daemon smithi110.lvgbkf started 2024-04-25T15:50:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:21 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.lvgbkf/crt"}]: dispatch 2024-04-25T15:50:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:21 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:50:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:21 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi110.lvgbkf/key"}]: dispatch 2024-04-25T15:50:21.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:21 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:50:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:22 smithi110 ceph-mon[26344]: mgrmap e27: smithi193.mgoteh(active, since 109s), standbys: smithi110.lvgbkf 2024-04-25T15:50:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:22 smithi110 ceph-mon[26344]: pgmap v58: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-25T15:50:22.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:22 smithi193 ceph-mon[33747]: mgrmap e27: smithi193.mgoteh(active, since 109s), standbys: smithi110.lvgbkf 2024-04-25T15:50:22.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:22 smithi193 ceph-mon[33747]: pgmap v58: 65 pgs: 65 active+clean; 282 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-04-25T15:50:24.648 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:24 smithi110 ceph-mon[26344]: pgmap v59: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.8 MiB/s wr, 167 op/s 2024-04-25T15:50:24.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:24 smithi193 ceph-mon[33747]: pgmap v59: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 36 KiB/s rd, 1.8 MiB/s wr, 167 op/s 2024-04-25T15:50:26.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:26 smithi110 ceph-mon[26344]: pgmap v60: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-04-25T15:50:26.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:26 smithi193 ceph-mon[33747]: pgmap v60: 65 pgs: 65 active+clean; 288 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 94 op/s 2024-04-25T15:50:28.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:28 smithi110 ceph-mon[26344]: pgmap v61: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.9 MiB/s wr, 127 op/s 2024-04-25T15:50:28.791 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:28 smithi193 ceph-mon[33747]: pgmap v61: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.9 MiB/s wr, 127 op/s 2024-04-25T15:50:30.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:30 smithi110 ceph-mon[26344]: pgmap v62: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-04-25T15:50:30.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:30 smithi193 ceph-mon[33747]: pgmap v62: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 79 op/s 2024-04-25T15:50:32.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:32 smithi110 ceph-mon[26344]: pgmap v63: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-04-25T15:50:32.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.872+0000 7f47d331d700 1 -- 172.21.15.110:0/1908353641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47cc0fdff0 msgr2=0x7f47cc0fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:32.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.872+0000 7f47d331d700 1 --2- 172.21.15.110:0/1908353641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47cc0fdff0 0x7f47cc0fe460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f47bc004880 tx=0x7f47bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:32.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 -- 172.21.15.110:0/1908353641 shutdown_connections 2024-04-25T15:50:32.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 --2- 172.21.15.110:0/1908353641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47cc0fdff0 0x7f47cc0fe460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:32.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 --2- 172.21.15.110:0/1908353641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47cc0fce80 0x7f47cc0fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:32.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 -- 172.21.15.110:0/1908353641 >> 172.21.15.110:0/1908353641 conn(0x7f47cc0f8540 msgr2=0x7f47cc0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:32.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 -- 172.21.15.110:0/1908353641 shutdown_connections 2024-04-25T15:50:32.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 -- 172.21.15.110:0/1908353641 wait complete. 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 Processor -- start 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 -- start start 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47cc0fce80 0x7f47cc10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47cc0fdff0 0x7f47cc10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc10e560 con 0x7f47cc0fdff0 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.873+0000 7f47d331d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47cc10edb0 con 0x7f47cc0fce80 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d08b8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47cc0fdff0 0x7f47cc10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d08b8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47cc0fdff0 0x7f47cc10df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49078/0 (socket says 172.21.15.110:49078) 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d08b8700 1 -- 172.21.15.110:0/2178116329 learned_addr learned my addr 172.21.15.110:0/2178116329 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d10b9700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47cc0fce80 0x7f47cc10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d08b8700 1 -- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47cc0fce80 msgr2=0x7f47cc10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d08b8700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47cc0fce80 0x7f47cc10da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d08b8700 1 -- 172.21.15.110:0/2178116329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c8005040 con 0x7f47cc0fdff0 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d08b8700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47cc0fdff0 0x7f47cc10df60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f47bc008d00 tx=0x7f47bc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:32.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47c1ffb700 1 -- 172.21.15.110:0/2178116329 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47bc00e030 con 0x7f47cc0fdff0 2024-04-25T15:50:32.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47bc005040 con 0x7f47cc0fdff0 2024-04-25T15:50:32.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47c1ffb700 1 -- 172.21.15.110:0/2178116329 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47bc013e60 con 0x7f47cc0fdff0 2024-04-25T15:50:32.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.874+0000 7f47c1ffb700 1 -- 172.21.15.110:0/2178116329 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47bc0123c0 con 0x7f47cc0fdff0 2024-04-25T15:50:32.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.875+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47cc10f3e0 con 0x7f47cc0fdff0 2024-04-25T15:50:32.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.876+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47cc05f5c0 con 0x7f47cc0fdff0 2024-04-25T15:50:32.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.877+0000 7f47c1ffb700 1 -- 172.21.15.110:0/2178116329 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 98385+0+0 (secure 0 0 0) 0x7f47bc01c400 con 0x7f47cc0fdff0 2024-04-25T15:50:32.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.877+0000 7f47c1ffb700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f47b8076480 0x7f47b8078930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:32.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.877+0000 7f47c1ffb700 1 -- 172.21.15.110:0/2178116329 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f47bc0991b0 con 0x7f47cc0fdff0 2024-04-25T15:50:32.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.877+0000 7f47d10b9700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f47b8076480 0x7f47b8078930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:32.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.878+0000 7f47d10b9700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f47b8076480 0x7f47b8078930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009fd0 tx=0x7f47c800b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:32.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:32.883+0000 7f47c1ffb700 1 -- 172.21.15.110:0/2178116329 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f47bc0627b0 con 0x7f47cc0fdff0 2024-04-25T15:50:32.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:32 smithi193 ceph-mon[33747]: pgmap v63: 65 pgs: 65 active+clean; 296 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 74 op/s 2024-04-25T15:50:33.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.134+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47cc10f700 con 0x7f47b8076480 2024-04-25T15:50:33.136 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.135+0000 7f47c1ffb700 1 -- 172.21.15.110:0/2178116329 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f47cc10f700 con 0x7f47b8076480 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f47b8076480 msgr2=0x7f47b8078930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f47b8076480 0x7f47b8078930 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009fd0 tx=0x7f47c800b040 comp rx=0 tx=0).stop 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47cc0fdff0 msgr2=0x7f47cc10df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47cc0fdff0 0x7f47cc10df60 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f47bc008d00 tx=0x7f47bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 shutdown_connections 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f47b8076480 0x7f47b8078930 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47cc0fce80 0x7f47cc10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 --2- 172.21.15.110:0/2178116329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f47cc0fdff0 0x7f47cc10df60 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 >> 172.21.15.110:0/2178116329 conn(0x7f47cc0f8540 msgr2=0x7f47cc0f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 shutdown_connections 2024-04-25T15:50:33.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.138+0000 7f47d331d700 1 -- 172.21.15.110:0/2178116329 wait complete. 2024-04-25T15:50:33.157 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:50:33.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.298+0000 7f3e1569f700 1 -- 172.21.15.110:0/1663931046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100ffee0 msgr2=0x7f3e10102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:33.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.298+0000 7f3e1569f700 1 --2- 172.21.15.110:0/1663931046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100ffee0 0x7f3e10102300 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3e04004940 tx=0x7f3e04009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:33.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.298+0000 7f3e1569f700 1 -- 172.21.15.110:0/1663931046 shutdown_connections 2024-04-25T15:50:33.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.298+0000 7f3e1569f700 1 --2- 172.21.15.110:0/1663931046 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100ffee0 0x7f3e10102300 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.299+0000 7f3e1569f700 1 --2- 172.21.15.110:0/1663931046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e100662c0 0x7f3e100ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.299+0000 7f3e1569f700 1 -- 172.21.15.110:0/1663931046 >> 172.21.15.110:0/1663931046 conn(0x7f3e100f7430 msgr2=0x7f3e100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:33.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.299+0000 7f3e1569f700 1 -- 172.21.15.110:0/1663931046 shutdown_connections 2024-04-25T15:50:33.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.299+0000 7f3e1569f700 1 -- 172.21.15.110:0/1663931046 wait complete. 2024-04-25T15:50:33.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.299+0000 7f3e1569f700 1 Processor -- start 2024-04-25T15:50:33.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e1569f700 1 -- start start 2024-04-25T15:50:33.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e1569f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100662c0 0x7f3e1010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:33.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e1569f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e100ffee0 0x7f3e1010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:33.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e1569f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1010e4f0 con 0x7f3e100662c0 2024-04-25T15:50:33.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e1569f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e1010ed40 con 0x7f3e100ffee0 2024-04-25T15:50:33.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100662c0 0x7f3e1010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100662c0 0x7f3e1010d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49096/0 (socket says 172.21.15.110:49096) 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0effd700 1 -- 172.21.15.110:0/3341971159 learned_addr learned my addr 172.21.15.110:0/3341971159 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0e7fc700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e100ffee0 0x7f3e1010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0e7fc700 1 -- 172.21.15.110:0/3341971159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100662c0 msgr2=0x7f3e1010d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0e7fc700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100662c0 0x7f3e1010d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0e7fc700 1 -- 172.21.15.110:0/3341971159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04005040 con 0x7f3e100ffee0 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0effd700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100662c0 0x7f3e1010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.300+0000 7f3e0e7fc700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e100ffee0 0x7f3e1010def0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e04000c00 tx=0x7f3e04009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.301+0000 7f3df77fe700 1 -- 172.21.15.110:0/3341971159 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e0400e030 con 0x7f3e100ffee0 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.301+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e1010efc0 con 0x7f3e100ffee0 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.301+0000 7f3df77fe700 1 -- 172.21.15.110:0/3341971159 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e04013de0 con 0x7f3e100ffee0 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.301+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1010f500 con 0x7f3e100ffee0 2024-04-25T15:50:33.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.301+0000 7f3df77fe700 1 -- 172.21.15.110:0/3341971159 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e0401ce50 con 0x7f3e100ffee0 2024-04-25T15:50:33.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.302+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1005f5c0 con 0x7f3e100ffee0 2024-04-25T15:50:33.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.303+0000 7f3df77fe700 1 -- 172.21.15.110:0/3341971159 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 27) v1 ==== 98385+0+0 (secure 0 0 0) 0x7f3e0401c400 con 0x7f3e100ffee0 2024-04-25T15:50:33.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.303+0000 7f3df77fe700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3df8076490 0x7f3df8078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:33.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.303+0000 7f3df77fe700 1 -- 172.21.15.110:0/3341971159 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f3e040991f0 con 0x7f3e100ffee0 2024-04-25T15:50:33.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.303+0000 7f3e0effd700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3df8076490 0x7f3df8078940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:33.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.304+0000 7f3e0effd700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3df8076490 0x7f3df8078940 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009b80 tx=0x7f3e0000b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:33.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.307+0000 7f3df77fe700 1 -- 172.21.15.110:0/3341971159 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f3e040626b0 con 0x7f3e100ffee0 2024-04-25T15:50:33.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.645+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e1010f7e0 con 0x7f3df8076490 2024-04-25T15:50:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.647+0000 7f3df77fe700 1 -- 172.21.15.110:0/3341971159 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f3e1010f7e0 con 0x7f3df8076490 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3df8076490 msgr2=0x7f3df8078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3df8076490 0x7f3df8078940 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3e00009b80 tx=0x7f3e0000b040 comp rx=0 tx=0).stop 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e100ffee0 msgr2=0x7f3e1010def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e100ffee0 0x7f3e1010def0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e04000c00 tx=0x7f3e04009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 shutdown_connections 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f3df8076490 0x7f3df8078940 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e100662c0 0x7f3e1010d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 --2- 172.21.15.110:0/3341971159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e100ffee0 0x7f3e1010def0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 >> 172.21.15.110:0/3341971159 conn(0x7f3e100f7430 msgr2=0x7f3e100f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:33.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 shutdown_connections 2024-04-25T15:50:33.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.649+0000 7f3e1569f700 1 -- 172.21.15.110:0/3341971159 wait complete. 2024-04-25T15:50:33.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.771+0000 7f25fb1a9700 1 -- 172.21.15.110:0/3091259620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fed30 msgr2=0x7f25f40fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:33.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.771+0000 7f25fb1a9700 1 --2- 172.21.15.110:0/3091259620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fed30 0x7f25f40fae70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f25dc004940 tx=0x7f25dc009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:33.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.771+0000 7f25fb1a9700 1 -- 172.21.15.110:0/3091259620 shutdown_connections 2024-04-25T15:50:33.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.771+0000 7f25fb1a9700 1 --2- 172.21.15.110:0/3091259620 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fed30 0x7f25f40fae70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.771+0000 7f25fb1a9700 1 --2- 172.21.15.110:0/3091259620 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f40fe3e0 0x7f25f40fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.771+0000 7f25fb1a9700 1 -- 172.21.15.110:0/3091259620 >> 172.21.15.110:0/3091259620 conn(0x7f25f40f6b80 msgr2=0x7f25f40f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:33.772 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:33 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:50:33.773 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:33 smithi110 ceph-mon[26344]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.771+0000 7f25fb1a9700 1 -- 172.21.15.110:0/3091259620 shutdown_connections 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.771+0000 7f25fb1a9700 1 -- 172.21.15.110:0/3091259620 wait complete. 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25fb1a9700 1 Processor -- start 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25fb1a9700 1 -- start start 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25fb1a9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fe3e0 0x7f25f40fdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25fb1a9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f40fed30 0x7f25f40fc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25fb1a9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f40fc650 con 0x7f25f40fe3e0 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25fb1a9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f40fc7c0 con 0x7f25f40fed30 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25f8f45700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fe3e0 0x7f25f40fdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25f8f45700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fe3e0 0x7f25f40fdbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49110/0 (socket says 172.21.15.110:49110) 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25f8f45700 1 -- 172.21.15.110:0/2295468527 learned_addr learned my addr 172.21.15.110:0/2295468527 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25f3fff700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f40fed30 0x7f25f40fc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:33.774 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25f3fff700 1 -- 172.21.15.110:0/2295468527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fe3e0 msgr2=0x7f25f40fdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25f3fff700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fe3e0 0x7f25f40fdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.772+0000 7f25f3fff700 1 -- 172.21.15.110:0/2295468527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25dc005040 con 0x7f25f40fed30 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.773+0000 7f25f3fff700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f40fed30 0x7f25f40fc0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.773+0000 7f25f17fa700 1 -- 172.21.15.110:0/2295468527 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25dc00e070 con 0x7f25f40fed30 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.773+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f40fca40 con 0x7f25f40fed30 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.773+0000 7f25f17fa700 1 -- 172.21.15.110:0/2295468527 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25dc01c400 con 0x7f25f40fed30 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.773+0000 7f25f17fa700 1 -- 172.21.15.110:0/2295468527 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25dc00b670 con 0x7f25f40fed30 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.773+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f4115990 con 0x7f25f40fed30 2024-04-25T15:50:33.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.774+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f405f5c0 con 0x7f25f40fed30 2024-04-25T15:50:33.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.775+0000 7f25f17fa700 1 -- 172.21.15.110:0/2295468527 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 27) v1 ==== 98385+0+0 (secure 0 0 0) 0x7f25dc013af0 con 0x7f25f40fed30 2024-04-25T15:50:33.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.776+0000 7f25f17fa700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f25e0076490 0x7f25e0078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:33.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.776+0000 7f25f17fa700 1 -- 172.21.15.110:0/2295468527 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f25dc098c10 con 0x7f25f40fed30 2024-04-25T15:50:33.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.776+0000 7f25f8f45700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f25e0076490 0x7f25e0078940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:33.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.776+0000 7f25f8f45700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f25e0076490 0x7f25e0078940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25f40fd960 tx=0x7f25e400b040 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:33.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:33.780+0000 7f25f17fa700 1 -- 172.21.15.110:0/2295468527 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f25dc062110 con 0x7f25f40fed30 2024-04-25T15:50:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:33 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:50:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:33 smithi193 ceph-mon[33747]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:34.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.102+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f25f4102470 con 0x7f25e0076490 2024-04-25T15:50:34.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.112+0000 7f25f17fa700 1 -- 172.21.15.110:0/2295468527 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f25f4102470 con 0x7f25e0076490 2024-04-25T15:50:34.112 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:50:34.112 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 starting - - - - 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (9m) 110s ago 9m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (8m) 18s ago 8m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 starting - - - - 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (4m) 110s ago 4m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (4m) 110s ago 4m 86.6M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (4m) 18s ago 4m 13.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (4m) 18s ago 4m 15.2M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 starting - - - - 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (2m) 18s ago 8m 574M - 19.0.0-3291-g994e2f20 d53c96779a29 82852af5489a 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (10m) 110s ago 11m 49.0M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (8m) 18s ago 8m 42.7M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 starting - - - - 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (28s) 18s ago 8m 5121k - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (7m) 110s ago 7m 61.7M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:50:34.113 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (7m) 110s ago 7m 66.0M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:50:34.114 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (6m) 110s ago 6m 56.3M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:50:34.114 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (6m) 18s ago 6m 200M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:50:34.114 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (6m) 18s ago 6m 176M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:50:34.114 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (5m) 18s ago 5m 166M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:50:34.114 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 starting - - - - 2024-04-25T15:50:34.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.114+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f25e0076490 msgr2=0x7f25e0078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:34.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.114+0000 7f25fb1a9700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f25e0076490 0x7f25e0078940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f25f40fd960 tx=0x7f25e400b040 comp rx=0 tx=0).stop 2024-04-25T15:50:34.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.114+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f40fed30 msgr2=0x7f25f40fc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:34.115 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.114+0000 7f25fb1a9700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f40fed30 0x7f25f40fc0a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f25dc000c00 tx=0x7f25dc009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:34.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.115+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 shutdown_connections 2024-04-25T15:50:34.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.115+0000 7f25fb1a9700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f25e0076490 0x7f25e0078940 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.115+0000 7f25fb1a9700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f40fe3e0 0x7f25f40fdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.115+0000 7f25fb1a9700 1 --2- 172.21.15.110:0/2295468527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f40fed30 0x7f25f40fc0a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.115+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 >> 172.21.15.110:0/2295468527 conn(0x7f25f40f6b80 msgr2=0x7f25f40f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:34.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.115+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 shutdown_connections 2024-04-25T15:50:34.116 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.115+0000 7f25fb1a9700 1 -- 172.21.15.110:0/2295468527 wait complete. 2024-04-25T15:50:34.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.242+0000 7f4b8250e700 1 -- 172.21.15.110:0/3458062281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 msgr2=0x7f4b7c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:34.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.242+0000 7f4b8250e700 1 --2- 172.21.15.110:0/3458062281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 0x7f4b7c0fd3d0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f4b70004940 tx=0x7f4b70009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:34.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.242+0000 7f4b8250e700 1 -- 172.21.15.110:0/3458062281 shutdown_connections 2024-04-25T15:50:34.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.242+0000 7f4b8250e700 1 --2- 172.21.15.110:0/3458062281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 0x7f4b7c0fd3d0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.242+0000 7f4b8250e700 1 --2- 172.21.15.110:0/3458062281 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b7c0fbcb0 0x7f4b7c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.242+0000 7f4b8250e700 1 -- 172.21.15.110:0/3458062281 >> 172.21.15.110:0/3458062281 conn(0x7f4b7c0f7430 msgr2=0x7f4b7c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:34.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.242+0000 7f4b8250e700 1 -- 172.21.15.110:0/3458062281 shutdown_connections 2024-04-25T15:50:34.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b8250e700 1 -- 172.21.15.110:0/3458062281 wait complete. 2024-04-25T15:50:34.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b8250e700 1 Processor -- start 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b8250e700 1 -- start start 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b8250e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b7c0fbcb0 0x7f4b7c110750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b8250e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 0x7f4b7c112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b8250e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c066330 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b8250e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7c066470 con 0x7f4b7c0fbcb0 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b7b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 0x7f4b7c112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b7b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 0x7f4b7c112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49132/0 (socket says 172.21.15.110:49132) 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.243+0000 7f4b7b7fe700 1 -- 172.21.15.110:0/390970783 learned_addr learned my addr 172.21.15.110:0/390970783 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:34.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b7bfff700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b7c0fbcb0 0x7f4b7c110750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b7b7fe700 1 -- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b7c0fbcb0 msgr2=0x7f4b7c110750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b7b7fe700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b7c0fbcb0 0x7f4b7c110750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b7b7fe700 1 -- 172.21.15.110:0/390970783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b70005040 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b7b7fe700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 0x7f4b7c112ca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000b3b0 tx=0x7f4b70009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b78ff9700 1 -- 172.21.15.110:0/390970783 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7000e040 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7c1131e0 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b78ff9700 1 -- 172.21.15.110:0/390970783 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b7000bcc0 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7c1137a0 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.244+0000 7f4b78ff9700 1 -- 172.21.15.110:0/390970783 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b7001c630 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.245+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b7c05f5c0 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.246+0000 7f4b78ff9700 1 -- 172.21.15.110:0/390970783 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 98385+0+0 (secure 0 0 0) 0x7f4b70013980 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.246+0000 7f4b78ff9700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f4b64076490 0x7f4b64078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:34.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.246+0000 7f4b78ff9700 1 -- 172.21.15.110:0/390970783 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f4b70098580 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.246+0000 7f4b7bfff700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f4b64076490 0x7f4b64078940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:34.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.247+0000 7f4b7bfff700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f4b64076490 0x7f4b64078940 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c065fb0 tx=0x7f4b6c009ee0 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:34.251 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.251+0000 7f4b78ff9700 1 -- 172.21.15.110:0/390970783 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f4b70061a60 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.669+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4b7c113ad0 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.669+0000 7f4b78ff9700 1 -- 172.21.15.110:0/390970783 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f4b70061400 con 0x7f4b7c0fcf60 2024-04-25T15:50:34.670 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:50:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:50:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:50:34.670 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:50:34.671 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:50:34.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f4b64076490 msgr2=0x7f4b64078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:34.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f4b64076490 0x7f4b64078940 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c065fb0 tx=0x7f4b6c009ee0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 msgr2=0x7f4b7c112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:34.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 0x7f4b7c112ca0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000b3b0 tx=0x7f4b70009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:34.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 shutdown_connections 2024-04-25T15:50:34.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f4b64076490 0x7f4b64078940 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b7c0fbcb0 0x7f4b7c110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 --2- 172.21.15.110:0/390970783 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4b7c0fcf60 0x7f4b7c112ca0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 >> 172.21.15.110:0/390970783 conn(0x7f4b7c0f7430 msgr2=0x7f4b7c0f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:34.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 shutdown_connections 2024-04-25T15:50:34.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.673+0000 7f4b8250e700 1 -- 172.21.15.110:0/390970783 wait complete. 2024-04-25T15:50:34.815 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:34 smithi110 ceph-mon[26344]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:34.815 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:34 smithi110 ceph-mon[26344]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:34.815 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:34 smithi110 ceph-mon[26344]: pgmap v64: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 116 op/s 2024-04-25T15:50:34.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.817+0000 7ff250fc7700 1 -- 172.21.15.110:0/2368322974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fe3e0 msgr2=0x7ff24c0fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:34.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.817+0000 7ff250fc7700 1 --2- 172.21.15.110:0/2368322974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fe3e0 0x7ff24c0fe7f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff23c004940 tx=0x7ff23c009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:34.818 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.818+0000 7ff250fc7700 1 -- 172.21.15.110:0/2368322974 shutdown_connections 2024-04-25T15:50:34.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.818+0000 7ff250fc7700 1 --2- 172.21.15.110:0/2368322974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24c0fed30 0x7ff24c0fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.818+0000 7ff250fc7700 1 --2- 172.21.15.110:0/2368322974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fe3e0 0x7ff24c0fe7f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.818+0000 7ff250fc7700 1 -- 172.21.15.110:0/2368322974 >> 172.21.15.110:0/2368322974 conn(0x7ff24c0f6b80 msgr2=0x7ff24c0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:34.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.818+0000 7ff250fc7700 1 -- 172.21.15.110:0/2368322974 shutdown_connections 2024-04-25T15:50:34.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.818+0000 7ff250fc7700 1 -- 172.21.15.110:0/2368322974 wait complete. 2024-04-25T15:50:34.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.818+0000 7ff250fc7700 1 Processor -- start 2024-04-25T15:50:34.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.819+0000 7ff250fc7700 1 -- start start 2024-04-25T15:50:34.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.819+0000 7ff250fc7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24c0fe3e0 0x7ff24c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:34.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.819+0000 7ff250fc7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fed30 0x7ff24c10df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:34.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.819+0000 7ff250fc7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c10e560 con 0x7ff24c0fed30 2024-04-25T15:50:34.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.819+0000 7ff250fc7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24c10edb0 con 0x7ff24c0fe3e0 2024-04-25T15:50:34.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.819+0000 7ff249d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fed30 0x7ff24c10df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:34.820 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.819+0000 7ff249d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fed30 0x7ff24c10df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49136/0 (socket says 172.21.15.110:49136) 2024-04-25T15:50:34.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.819+0000 7ff249d9b700 1 -- 172.21.15.110:0/2690141579 learned_addr learned my addr 172.21.15.110:0/2690141579 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:34.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.820+0000 7ff249d9b700 1 -- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24c0fe3e0 msgr2=0x7ff24c10da20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T15:50:34.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.820+0000 7ff249d9b700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24c0fe3e0 0x7ff24c10da20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:34.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.820+0000 7ff249d9b700 1 -- 172.21.15.110:0/2690141579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff23c005040 con 0x7ff24c0fed30 2024-04-25T15:50:34.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.820+0000 7ff249d9b700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fed30 0x7ff24c10df60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff24000c8c0 tx=0x7ff2400044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:34.821 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.821+0000 7ff23affd700 1 -- 172.21.15.110:0/2690141579 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff24000a980 con 0x7ff24c0fed30 2024-04-25T15:50:34.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.821+0000 7ff23affd700 1 -- 172.21.15.110:0/2690141579 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff24000ce60 con 0x7ff24c0fed30 2024-04-25T15:50:34.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.821+0000 7ff23affd700 1 -- 172.21.15.110:0/2690141579 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2400115e0 con 0x7ff24c0fed30 2024-04-25T15:50:34.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.821+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff24c10f010 con 0x7ff24c0fed30 2024-04-25T15:50:34.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.821+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff24c10f5d0 con 0x7ff24c0fed30 2024-04-25T15:50:34.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.823+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24c05f5c0 con 0x7ff24c0fed30 2024-04-25T15:50:34.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.823+0000 7ff23affd700 1 -- 172.21.15.110:0/2690141579 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 98385+0+0 (secure 0 0 0) 0x7ff240012400 con 0x7ff24c0fed30 2024-04-25T15:50:34.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.824+0000 7ff23affd700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff23407a950 0x7ff23407ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:34.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.824+0000 7ff23affd700 1 -- 172.21.15.110:0/2690141579 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7ff2400977c0 con 0x7ff24c0fed30 2024-04-25T15:50:34.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.824+0000 7ff24a59c700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff23407a950 0x7ff23407ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:34.825 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.824+0000 7ff24a59c700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff23407a950 0x7ff23407ce00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00b970 tx=0x7ff23c009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:34.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:34.833+0000 7ff23affd700 1 -- 172.21.15.110:0/2690141579 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7ff240060dc0 con 0x7ff24c0fed30 2024-04-25T15:50:34.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:34 smithi193 ceph-mon[33747]: from='client.24491 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:34.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:34 smithi193 ceph-mon[33747]: from='client.24495 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:34.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:34 smithi193 ceph-mon[33747]: pgmap v64: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 2.1 MiB/s wr, 116 op/s 2024-04-25T15:50:35.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.221+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff24c10f940 con 0x7ff24c0fed30 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.222+0000 7ff23affd700 1 -- 172.21.15.110:0/2690141579 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7ff2400126b0 con 0x7ff24c0fed30 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:50:35.223 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:50:35.224 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:50:35.225 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:50:35.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff23407a950 msgr2=0x7ff23407ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:35.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff23407a950 0x7ff23407ce00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00b970 tx=0x7ff23c009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:35.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fed30 msgr2=0x7ff24c10df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:35.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fed30 0x7ff24c10df60 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff24000c8c0 tx=0x7ff2400044c0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 shutdown_connections 2024-04-25T15:50:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7ff23407a950 0x7ff23407ce00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff24c0fe3e0 0x7ff24c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 --2- 172.21.15.110:0/2690141579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff24c0fed30 0x7ff24c10df60 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 >> 172.21.15.110:0/2690141579 conn(0x7ff24c0f6b80 msgr2=0x7ff24c0f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 shutdown_connections 2024-04-25T15:50:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.225+0000 7ff250fc7700 1 -- 172.21.15.110:0/2690141579 wait complete. 2024-04-25T15:50:35.227 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:50:35.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.363+0000 7f8e65da0700 1 -- 172.21.15.110:0/1190126242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fbcb0 msgr2=0x7f8e600fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:35.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.363+0000 7f8e65da0700 1 --2- 172.21.15.110:0/1190126242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fbcb0 0x7f8e600fc0c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8e48004940 tx=0x7f8e48009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:35.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.363+0000 7f8e65da0700 1 -- 172.21.15.110:0/1190126242 shutdown_connections 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.363+0000 7f8e65da0700 1 --2- 172.21.15.110:0/1190126242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e600fcf60 0x7f8e600fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.363+0000 7f8e65da0700 1 --2- 172.21.15.110:0/1190126242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fbcb0 0x7f8e600fc0c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.363+0000 7f8e65da0700 1 -- 172.21.15.110:0/1190126242 >> 172.21.15.110:0/1190126242 conn(0x7f8e600f7430 msgr2=0x7f8e600f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.363+0000 7f8e65da0700 1 -- 172.21.15.110:0/1190126242 shutdown_connections 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.363+0000 7f8e65da0700 1 -- 172.21.15.110:0/1190126242 wait complete. 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.364+0000 7f8e65da0700 1 Processor -- start 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.364+0000 7f8e65da0700 1 -- start start 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.364+0000 7f8e65da0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e600fbcb0 0x7f8e6010c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:35.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.364+0000 7f8e65da0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fcf60 0x7f8e6010cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:35.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.364+0000 7f8e65da0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6010d390 con 0x7f8e600fcf60 2024-04-25T15:50:35.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.364+0000 7f8e65da0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6010d500 con 0x7f8e600fbcb0 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.364+0000 7f8e5effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fcf60 0x7f8e6010cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.364+0000 7f8e5f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e600fbcb0 0x7f8e6010c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e5f7fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e600fbcb0 0x7f8e6010c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:47248/0 (socket says 172.21.15.110:47248) 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e5f7fe700 1 -- 172.21.15.110:0/2207534683 learned_addr learned my addr 172.21.15.110:0/2207534683 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e5effd700 1 -- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e600fbcb0 msgr2=0x7f8e6010c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e5effd700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e600fbcb0 0x7f8e6010c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e5effd700 1 -- 172.21.15.110:0/2207534683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e48005040 con 0x7f8e600fcf60 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e5effd700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fcf60 0x7f8e6010cd90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000ed70 tx=0x7f8e500044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e57fff700 1 -- 172.21.15.110:0/2207534683 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5000cd10 con 0x7f8e600fcf60 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e57fff700 1 -- 172.21.15.110:0/2207534683 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8e5000ee70 con 0x7f8e600fcf60 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e60066250 con 0x7f8e600fcf60 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.365+0000 7f8e57fff700 1 -- 172.21.15.110:0/2207534683 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e50010870 con 0x7f8e600fcf60 2024-04-25T15:50:35.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.366+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e60066810 con 0x7f8e600fcf60 2024-04-25T15:50:35.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.367+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6004c710 con 0x7f8e600fcf60 2024-04-25T15:50:35.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.368+0000 7f8e57fff700 1 -- 172.21.15.110:0/2207534683 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 27) v1 ==== 98385+0+0 (secure 0 0 0) 0x7f8e50010a60 con 0x7f8e600fcf60 2024-04-25T15:50:35.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.368+0000 7f8e57fff700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8e4c076490 0x7f8e4c078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:35.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.369+0000 7f8e57fff700 1 -- 172.21.15.110:0/2207534683 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f8e5009a0c0 con 0x7f8e600fcf60 2024-04-25T15:50:35.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.369+0000 7f8e5f7fe700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8e4c076490 0x7f8e4c078940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:35.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.369+0000 7f8e5f7fe700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8e4c076490 0x7f8e4c078940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8e48004f90 tx=0x7f8e48009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:35.378 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.377+0000 7f8e57fff700 1 -- 172.21.15.110:0/2207534683 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f8e500635c0 con 0x7f8e600fcf60 2024-04-25T15:50:35.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.686+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 --> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e600663e0 con 0x7f8e4c076490 2024-04-25T15:50:35.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.687+0000 7f8e57fff700 1 -- 172.21.15.110:0/2207534683 <== mgr.24379 v2:172.21.15.193:6828/2194935512 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+328 (secure 0 0 0) 0x7f8e600663e0 con 0x7f8e4c076490 2024-04-25T15:50:35.689 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:50:35.689 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:50:35.689 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:50:35.690 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:50:35.690 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-04-25T15:50:35.690 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "", 2024-04-25T15:50:35.690 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mgr daemons", 2024-04-25T15:50:35.690 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:50:35.690 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:50:35.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8e4c076490 msgr2=0x7f8e4c078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:35.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8e4c076490 0x7f8e4c078940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8e48004f90 tx=0x7f8e48009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:35.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fcf60 msgr2=0x7f8e6010cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:35.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fcf60 0x7f8e6010cd90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8e5000ed70 tx=0x7f8e500044c0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 shutdown_connections 2024-04-25T15:50:35.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f8e4c076490 0x7f8e4c078940 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e600fbcb0 0x7f8e6010c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 --2- 172.21.15.110:0/2207534683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e600fcf60 0x7f8e6010cd90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 >> 172.21.15.110:0/2207534683 conn(0x7f8e600f7430 msgr2=0x7f8e600f8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:35.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 shutdown_connections 2024-04-25T15:50:35.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.690+0000 7f8e65da0700 1 -- 172.21.15.110:0/2207534683 wait complete. 2024-04-25T15:50:35.837 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:35 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/390970783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:50:35.837 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:35 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2690141579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:50:35.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.834+0000 7f397310c700 1 -- 172.21.15.110:0/90809782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396c0fcf60 msgr2=0x7f396c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:35.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.835+0000 7f397310c700 1 --2- 172.21.15.110:0/90809782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396c0fcf60 0x7f396c0fd3d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3958004940 tx=0x7f3958009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:35.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.835+0000 7f397310c700 1 -- 172.21.15.110:0/90809782 shutdown_connections 2024-04-25T15:50:35.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.835+0000 7f397310c700 1 --2- 172.21.15.110:0/90809782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396c0fcf60 0x7f396c0fd3d0 secure :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3958004940 tx=0x7f3958009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:35.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.835+0000 7f397310c700 1 --2- 172.21.15.110:0/90809782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f396c0fbcb0 0x7f396c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.835+0000 7f397310c700 1 -- 172.21.15.110:0/90809782 >> 172.21.15.110:0/90809782 conn(0x7f396c0f7430 msgr2=0x7f396c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:35.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.836+0000 7f397310c700 1 -- 172.21.15.110:0/90809782 shutdown_connections 2024-04-25T15:50:35.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.836+0000 7f397310c700 1 -- 172.21.15.110:0/90809782 wait complete. 2024-04-25T15:50:35.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.836+0000 7f397310c700 1 Processor -- start 2024-04-25T15:50:35.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.836+0000 7f397310c700 1 -- start start 2024-04-25T15:50:35.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f397310c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f396c0fbcb0 0x7f396c10fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:35.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f397310c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396c110320 0x7f396c111040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:35.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f397310c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c1107e0 con 0x7f396c110320 2024-04-25T15:50:35.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f397310c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f396c110950 con 0x7f396c0fbcb0 2024-04-25T15:50:35.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f3970ea8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f396c0fbcb0 0x7f396c10fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:35.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f3970ea8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f396c0fbcb0 0x7f396c10fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:47272/0 (socket says 172.21.15.110:47272) 2024-04-25T15:50:35.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f3970ea8700 1 -- 172.21.15.110:0/65535849 learned_addr learned my addr 172.21.15.110:0/65535849 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:50:35.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f3970ea8700 1 -- 172.21.15.110:0/65535849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396c110320 msgr2=0x7f396c111040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:50:35.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f3970ea8700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396c110320 0x7f396c111040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:35.840 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.837+0000 7f3970ea8700 1 -- 172.21.15.110:0/65535849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3958005040 con 0x7f396c0fbcb0 2024-04-25T15:50:35.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.838+0000 7f3970ea8700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f396c0fbcb0 0x7f396c10fde0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f396000eb40 tx=0x7f396000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:35.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.838+0000 7f39697fa700 1 -- 172.21.15.110:0/65535849 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3960007cc0 con 0x7f396c0fbcb0 2024-04-25T15:50:35.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.838+0000 7f39697fa700 1 -- 172.21.15.110:0/65535849 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f396000f040 con 0x7f396c0fbcb0 2024-04-25T15:50:35.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.838+0000 7f39697fa700 1 -- 172.21.15.110:0/65535849 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3960009620 con 0x7f396c0fbcb0 2024-04-25T15:50:35.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.838+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f396c1116c0 con 0x7f396c0fbcb0 2024-04-25T15:50:35.841 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.838+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396c111c00 con 0x7f396c0fbcb0 2024-04-25T15:50:35.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.840+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f396c1018f0 con 0x7f396c0fbcb0 2024-04-25T15:50:35.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.841+0000 7f39697fa700 1 -- 172.21.15.110:0/65535849 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 27) v1 ==== 98385+0+0 (secure 0 0 0) 0x7f3960009780 con 0x7f396c0fbcb0 2024-04-25T15:50:35.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.841+0000 7f39697fa700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f395c076490 0x7f395c078940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:50:35.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.841+0000 7f39697fa700 1 -- 172.21.15.110:0/65535849 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5722+0+0 (secure 0 0 0) 0x7f3960097370 con 0x7f396c0fbcb0 2024-04-25T15:50:35.843 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.841+0000 7f396bfff700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f395c076490 0x7f395c078940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:50:35.844 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.842+0000 7f396bfff700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f395c076490 0x7f395c078940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3958013870 tx=0x7f3958009e30 comp rx=0 tx=0).ready entity=mgr.24379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:50:35.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:35.846+0000 7f39697fa700 1 -- 172.21.15.110:0/65535849 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f3960060980 con 0x7f396c0fbcb0 2024-04-25T15:50:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:35 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/390970783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:50:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:35 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2690141579' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:50:36.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.298+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f396c104fb0 con 0x7f396c0fbcb0 2024-04-25T15:50:36.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.299+0000 7f39697fa700 1 -- 172.21.15.110:0/65535849 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f3960060320 con 0x7f396c0fbcb0 2024-04-25T15:50:36.301 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:50:36.301 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:50:36.301 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:50:36.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f395c076490 msgr2=0x7f395c078940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:36.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f395c076490 0x7f395c078940 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3958013870 tx=0x7f3958009e30 comp rx=0 tx=0).stop 2024-04-25T15:50:36.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f396c0fbcb0 msgr2=0x7f396c10fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:50:36.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f396c0fbcb0 0x7f396c10fde0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f396000eb40 tx=0x7f396000b040 comp rx=0 tx=0).stop 2024-04-25T15:50:36.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 shutdown_connections 2024-04-25T15:50:36.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:6828/2194935512,v1:172.21.15.193:6829/2194935512] conn(0x7f395c076490 0x7f395c078940 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:36.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f396c0fbcb0 0x7f396c10fde0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:36.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 --2- 172.21.15.110:0/65535849 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f396c110320 0x7f396c111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:50:36.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 >> 172.21.15.110:0/65535849 conn(0x7f396c0f7430 msgr2=0x7f396c065cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:50:36.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 shutdown_connections 2024-04-25T15:50:36.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:50:36.301+0000 7f397310c700 1 -- 172.21.15.110:0/65535849 wait complete. 2024-04-25T15:50:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:36 smithi110 ceph-mon[26344]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:36 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/65535849' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:50:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:36 smithi110 ceph-mon[26344]: pgmap v65: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-04-25T15:50:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:36 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:36 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:36.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:36 smithi193 ceph-mon[33747]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:50:36.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:36 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/65535849' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:50:36.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:36 smithi193 ceph-mon[33747]: pgmap v65: 65 pgs: 65 active+clean; 303 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.4 MiB/s wr, 78 op/s 2024-04-25T15:50:36.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:36 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:36.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:36 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:39.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:38 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:39.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:38 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:39.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:38 smithi110 ceph-mon[26344]: pgmap v66: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 123 op/s 2024-04-25T15:50:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:38 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:38 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:38 smithi193 ceph-mon[33747]: pgmap v66: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 2.2 MiB/s wr, 123 op/s 2024-04-25T15:50:40.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:40 smithi110 ceph-mon[26344]: pgmap v67: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 89 op/s 2024-04-25T15:50:40.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:40 smithi193 ceph-mon[33747]: pgmap v67: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 89 op/s 2024-04-25T15:50:42.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:42 smithi110 ceph-mon[26344]: pgmap v68: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 86 op/s 2024-04-25T15:50:42.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:42 smithi193 ceph-mon[33747]: pgmap v68: 65 pgs: 65 active+clean; 305 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 30 KiB/s rd, 1.5 MiB/s wr, 86 op/s 2024-04-25T15:50:44.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:44 smithi110 ceph-mon[26344]: pgmap v69: 65 pgs: 65 active+clean; 301 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 2.3 MiB/s wr, 138 op/s 2024-04-25T15:50:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:44 smithi193 ceph-mon[33747]: pgmap v69: 65 pgs: 65 active+clean; 301 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 47 KiB/s rd, 2.3 MiB/s wr, 138 op/s 2024-04-25T15:50:47.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr fail", "who": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr fail", "who": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: Activating manager daemon smithi110.lvgbkf 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: osdmap e48: 6 total, 6 up, 6 in 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd='[{"prefix": "mgr fail", "who": "smithi193.mgoteh"}]': finished 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: mgrmap e28: smithi110.lvgbkf(active, starting, since 0.0113375s) 2024-04-25T15:50:47.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:50:47.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:50:47.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: Manager daemon smithi110.lvgbkf is now available 2024-04-25T15:50:47.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:50:47.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:50:47.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:50:47.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:50:47.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:50:47.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:47 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 172.21.15.193:0/2512234961' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr fail", "who": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "mgr fail", "who": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: Activating manager daemon smithi110.lvgbkf 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: osdmap e48: 6 total, 6 up, 6 in 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.24379 ' entity='mgr.smithi193.mgoteh' cmd='[{"prefix": "mgr fail", "who": "smithi193.mgoteh"}]': finished 2024-04-25T15:50:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: mgrmap e28: smithi110.lvgbkf(active, starting, since 0.0113375s) 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:50:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:50:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:50:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: Manager daemon smithi110.lvgbkf is now available 2024-04-25T15:50:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:50:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:50:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:50:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:50:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:50:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:47 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:50:48.724 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:48 smithi193 ceph-mon[33747]: [25/Apr/2024:15:50:47] ENGINE Bus STARTING 2024-04-25T15:50:48.724 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:48 smithi193 ceph-mon[33747]: [25/Apr/2024:15:50:47] ENGINE Serving on https://172.21.15.110:7150 2024-04-25T15:50:48.724 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:48 smithi193 ceph-mon[33747]: mgrmap e29: smithi110.lvgbkf(active, since 1.01283s) 2024-04-25T15:50:48.724 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:48 smithi193 ceph-mon[33747]: pgmap v3: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:50:48.724 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:48 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:48.724 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:48 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:48.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:48 smithi110 ceph-mon[26344]: [25/Apr/2024:15:50:47] ENGINE Bus STARTING 2024-04-25T15:50:48.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:48 smithi110 ceph-mon[26344]: [25/Apr/2024:15:50:47] ENGINE Serving on https://172.21.15.110:7150 2024-04-25T15:50:48.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:48 smithi110 ceph-mon[26344]: mgrmap e29: smithi110.lvgbkf(active, since 1.01283s) 2024-04-25T15:50:48.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:48 smithi110 ceph-mon[26344]: pgmap v3: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:50:48.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:48 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:48.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:48 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:49 smithi110 ceph-mon[26344]: [25/Apr/2024:15:50:47] ENGINE Serving on http://172.21.15.110:8765 2024-04-25T15:50:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:49 smithi110 ceph-mon[26344]: [25/Apr/2024:15:50:47] ENGINE Bus STARTED 2024-04-25T15:50:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:49 smithi110 ceph-mon[26344]: pgmap v4: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:50:49.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:49 smithi193 ceph-mon[33747]: [25/Apr/2024:15:50:47] ENGINE Serving on http://172.21.15.110:8765 2024-04-25T15:50:49.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:49 smithi193 ceph-mon[33747]: [25/Apr/2024:15:50:47] ENGINE Bus STARTED 2024-04-25T15:50:49.962 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:49 smithi193 ceph-mon[33747]: pgmap v4: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:50:50.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:50 smithi193 ceph-mon[33747]: mgrmap e30: smithi110.lvgbkf(active, since 3s) 2024-04-25T15:50:50.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:50 smithi110 ceph-mon[26344]: mgrmap e30: smithi110.lvgbkf(active, since 3s) 2024-04-25T15:50:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:51 smithi193 ceph-mon[33747]: pgmap v5: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:50:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:51 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:51 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:51 smithi110 ceph-mon[26344]: pgmap v5: 65 pgs: 65 active+clean; 299 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:50:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:51 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:51 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:53.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:53 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:53.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:53 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:53.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:53 smithi193 ceph-mon[33747]: pgmap v6: 65 pgs: 65 active+clean; 300 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 983 KiB/s wr, 68 op/s 2024-04-25T15:50:53.612 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:53 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:53.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:53 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:53.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:53 smithi110 ceph-mon[26344]: pgmap v6: 65 pgs: 65 active+clean; 300 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 35 KiB/s rd, 983 KiB/s wr, 68 op/s 2024-04-25T15:50:54.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:54 smithi110 ceph-mon[26344]: pgmap v7: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-04-25T15:50:54.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:54 smithi110 ceph-mon[26344]: Standby manager daemon smithi193.mgoteh started 2024-04-25T15:50:54.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:54 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/1166588351' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:50:54.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:54 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/1166588351' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:50:54.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:54 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/1166588351' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:50:54.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:54 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/1166588351' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:50:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:54 smithi193 ceph-mon[33747]: pgmap v7: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 54 KiB/s rd, 1.4 MiB/s wr, 84 op/s 2024-04-25T15:50:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:54 smithi193 ceph-mon[33747]: Standby manager daemon smithi193.mgoteh started 2024-04-25T15:50:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:54 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/1166588351' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:50:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:54 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/1166588351' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:50:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:54 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/1166588351' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:50:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:54 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/1166588351' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:50:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:56 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mgoteh", "id": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:50:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:56 smithi110 ceph-mon[26344]: mgrmap e31: smithi110.lvgbkf(active, since 9s), standbys: smithi193.mgoteh 2024-04-25T15:50:56.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:56 smithi110 ceph-mon[26344]: pgmap v8: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 66 op/s 2024-04-25T15:50:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:56 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mgoteh", "id": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:50:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:56 smithi193 ceph-mon[33747]: mgrmap e31: smithi110.lvgbkf(active, since 9s), standbys: smithi193.mgoteh 2024-04-25T15:50:56.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:56 smithi193 ceph-mon[33747]: pgmap v8: 65 pgs: 65 active+clean; 303 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 42 KiB/s rd, 1.1 MiB/s wr, 66 op/s 2024-04-25T15:50:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:58 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:58 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:50:58 smithi110 ceph-mon[26344]: pgmap v9: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-04-25T15:50:58.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:58 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:58.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:58 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:50:58.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:50:58 smithi193 ceph-mon[33747]: pgmap v9: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 49 KiB/s rd, 1.8 MiB/s wr, 253 op/s 2024-04-25T15:51:00.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:00 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:00.327 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:00 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:00.362 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:00 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:00.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:00 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:01 smithi110 ceph-mon[26344]: pgmap v10: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-04-25T15:51:01.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:01 smithi193 ceph-mon[33747]: pgmap v10: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 45 KiB/s rd, 1.6 MiB/s wr, 232 op/s 2024-04-25T15:51:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:02 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:51:02.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:02 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:51:03.352 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:03 smithi110 ceph-mon[26344]: pgmap v11: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 2.3 MiB/s wr, 393 op/s 2024-04-25T15:51:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:03 smithi193 ceph-mon[33747]: pgmap v11: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 58 KiB/s rd, 2.3 MiB/s wr, 393 op/s 2024-04-25T15:51:04.850 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:04 smithi110 ceph-mon[26344]: pgmap v12: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.9 MiB/s wr, 364 op/s 2024-04-25T15:51:04.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:04 smithi193 ceph-mon[33747]: pgmap v12: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 43 KiB/s rd, 1.9 MiB/s wr, 364 op/s 2024-04-25T15:51:06.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 -- 172.21.15.110:0/2420591191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7704100280 msgr2=0x7f77041006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 --2- 172.21.15.110:0/2420591191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7704100280 0x7f77041006f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f76f4004940 tx=0x7f76f4009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 -- 172.21.15.110:0/2420591191 shutdown_connections 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 --2- 172.21.15.110:0/2420591191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7704100280 0x7f77041006f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 --2- 172.21.15.110:0/2420591191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77040fefd0 0x7f77040ff3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 -- 172.21.15.110:0/2420591191 >> 172.21.15.110:0/2420591191 conn(0x7f77040faab0 msgr2=0x7f77040fce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 -- 172.21.15.110:0/2420591191 shutdown_connections 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 -- 172.21.15.110:0/2420591191 wait complete. 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 Processor -- start 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.446+0000 7f770b890700 1 -- start start 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770b890700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77040fefd0 0x7f770410fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770b890700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7704100280 0x7f77041100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770b890700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77041106e0 con 0x7f77040fefd0 2024-04-25T15:51:06.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770b890700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7704110f30 con 0x7f7704100280 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770962c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77040fefd0 0x7f770410fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770962c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77040fefd0 0x7f770410fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59286/0 (socket says 172.21.15.110:59286) 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770962c700 1 -- 172.21.15.110:0/428508255 learned_addr learned my addr 172.21.15.110:0/428508255 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f7708e2b700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7704100280 0x7f77041100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770962c700 1 -- 172.21.15.110:0/428508255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7704100280 msgr2=0x7f77041100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770962c700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7704100280 0x7f77041100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770962c700 1 -- 172.21.15.110:0/428508255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f4005040 con 0x7f77040fefd0 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.447+0000 7f770962c700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77040fefd0 0x7f770410fba0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f770000eb40 tx=0x7f770000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.448+0000 7f76f9ffb700 1 -- 172.21.15.110:0/428508255 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7700007cc0 con 0x7f77040fefd0 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.448+0000 7f76f9ffb700 1 -- 172.21.15.110:0/428508255 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f770000f040 con 0x7f77040fefd0 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.448+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77041111b0 con 0x7f77040fefd0 2024-04-25T15:51:06.450 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.448+0000 7f76f9ffb700 1 -- 172.21.15.110:0/428508255 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f770000ca10 con 0x7f77040fefd0 2024-04-25T15:51:06.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.448+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77041116f0 con 0x7f77040fefd0 2024-04-25T15:51:06.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.449+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f770405f5c0 con 0x7f77040fefd0 2024-04-25T15:51:06.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.450+0000 7f76f9ffb700 1 -- 172.21.15.110:0/428508255 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 31) v1 ==== 98487+0+0 (secure 0 0 0) 0x7f770000cb70 con 0x7f77040fefd0 2024-04-25T15:51:06.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.451+0000 7f76f9ffb700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f76f00765a0 0x7f76f0078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:06.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.451+0000 7f7708e2b700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f76f00765a0 0x7f76f0078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:06.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.451+0000 7f76f9ffb700 1 -- 172.21.15.110:0/428508255 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f77000976e0 con 0x7f77040fefd0 2024-04-25T15:51:06.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.452+0000 7f7708e2b700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f76f00765a0 0x7f76f0078a50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009b30 tx=0x7f76f4009e30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:06.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.456+0000 7f76f9ffb700 1 -- 172.21.15.110:0/428508255 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f77000609d0 con 0x7f77040fefd0 2024-04-25T15:51:06.780 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:06 smithi110 ceph-mon[26344]: pgmap v13: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-04-25T15:51:06.832 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.830+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 --> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7704111a20 con 0x7f76f00765a0 2024-04-25T15:51:06.833 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.832+0000 7f76f9ffb700 1 -- 172.21.15.110:0/428508255 <== mgr.14758 v2:172.21.15.110:6800/1601336217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f7704111a20 con 0x7f76f00765a0 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f76f00765a0 msgr2=0x7f76f0078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f76f00765a0 0x7f76f0078a50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f76f4009b30 tx=0x7f76f4009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77040fefd0 msgr2=0x7f770410fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77040fefd0 0x7f770410fba0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f770000eb40 tx=0x7f770000b040 comp rx=0 tx=0).stop 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 shutdown_connections 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f76f00765a0 0x7f76f0078a50 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77040fefd0 0x7f770410fba0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 --2- 172.21.15.110:0/428508255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7704100280 0x7f77041100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 >> 172.21.15.110:0/428508255 conn(0x7f77040faab0 msgr2=0x7f77040fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 shutdown_connections 2024-04-25T15:51:06.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.836+0000 7f770b890700 1 -- 172.21.15.110:0/428508255 wait complete. 2024-04-25T15:51:06.849 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:51:06.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.967+0000 7f0dcb271700 1 -- 172.21.15.110:0/3117455840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 msgr2=0x7f0dc40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:06.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.967+0000 7f0dcb271700 1 --2- 172.21.15.110:0/3117455840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 0x7f0dc40fd3d0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0db4004940 tx=0x7f0db4009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:06.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.967+0000 7f0dcb271700 1 -- 172.21.15.110:0/3117455840 shutdown_connections 2024-04-25T15:51:06.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.967+0000 7f0dcb271700 1 --2- 172.21.15.110:0/3117455840 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 0x7f0dc40fd3d0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.967+0000 7f0dcb271700 1 --2- 172.21.15.110:0/3117455840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc40fbcb0 0x7f0dc40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.967+0000 7f0dcb271700 1 -- 172.21.15.110:0/3117455840 >> 172.21.15.110:0/3117455840 conn(0x7f0dc40f7430 msgr2=0x7f0dc40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.967+0000 7f0dcb271700 1 -- 172.21.15.110:0/3117455840 shutdown_connections 2024-04-25T15:51:06.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.967+0000 7f0dcb271700 1 -- 172.21.15.110:0/3117455840 wait complete. 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.968+0000 7f0dcb271700 1 Processor -- start 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.968+0000 7f0dcb271700 1 -- start start 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dcb271700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc40fbcb0 0x7f0dc410d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dcb271700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 0x7f0dc410de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dcb271700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc410e490 con 0x7f0dc40fcf60 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dcb271700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dc410ece0 con 0x7f0dc40fbcb0 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dc880c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 0x7f0dc410de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dc880c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 0x7f0dc410de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59298/0 (socket says 172.21.15.110:59298) 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dc880c700 1 -- 172.21.15.110:0/167306415 learned_addr learned my addr 172.21.15.110:0/167306415 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dc900d700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc40fbcb0 0x7f0dc410d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dc880c700 1 -- 172.21.15.110:0/167306415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc40fbcb0 msgr2=0x7f0dc410d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:06.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dc880c700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc40fbcb0 0x7f0dc410d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:06.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dc880c700 1 -- 172.21.15.110:0/167306415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db4005040 con 0x7f0dc40fcf60 2024-04-25T15:51:06.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.969+0000 7f0dc880c700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 0x7f0dc410de90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0db400b0d0 tx=0x7f0db4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:06.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.970+0000 7f0db9ffb700 1 -- 172.21.15.110:0/167306415 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db400e030 con 0x7f0dc40fcf60 2024-04-25T15:51:06.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.970+0000 7f0db9ffb700 1 -- 172.21.15.110:0/167306415 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0db401c400 con 0x7f0dc40fcf60 2024-04-25T15:51:06.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.970+0000 7f0db9ffb700 1 -- 172.21.15.110:0/167306415 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0db400b630 con 0x7f0dc40fcf60 2024-04-25T15:51:06.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.970+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dc410ef60 con 0x7f0dc40fcf60 2024-04-25T15:51:06.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.970+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dc410f3e0 con 0x7f0dc40fcf60 2024-04-25T15:51:06.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.971+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dc405f5c0 con 0x7f0dc40fcf60 2024-04-25T15:51:06.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.971+0000 7f0db9ffb700 1 -- 172.21.15.110:0/167306415 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 31) v1 ==== 98487+0+0 (secure 0 0 0) 0x7f0db401c570 con 0x7f0dc40fcf60 2024-04-25T15:51:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.972+0000 7f0db9ffb700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f0db00765a0 0x7f0db0078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.972+0000 7f0db9ffb700 1 -- 172.21.15.110:0/167306415 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f0db40994d0 con 0x7f0dc40fcf60 2024-04-25T15:51:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.972+0000 7f0dc900d700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f0db00765a0 0x7f0db0078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:06.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.973+0000 7f0dc900d700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f0db00765a0 0x7f0db0078a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009b80 tx=0x7f0dc000b040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:06.978 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:06.977+0000 7f0db9ffb700 1 -- 172.21.15.110:0/167306415 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f0db4062710 con 0x7f0dc40fcf60 2024-04-25T15:51:06.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:06 smithi193 ceph-mon[33747]: pgmap v13: 65 pgs: 65 active+clean; 309 MiB data, 2.7 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.4 MiB/s wr, 343 op/s 2024-04-25T15:51:07.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.292+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 --> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0dc410f750 con 0x7f0db00765a0 2024-04-25T15:51:07.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.294+0000 7f0db9ffb700 1 -- 172.21.15.110:0/167306415 <== mgr.14758 v2:172.21.15.110:6800/1601336217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f0dc410f750 con 0x7f0db00765a0 2024-04-25T15:51:07.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f0db00765a0 msgr2=0x7f0db0078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:07.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f0db00765a0 0x7f0db0078a50 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0009b80 tx=0x7f0dc000b040 comp rx=0 tx=0).stop 2024-04-25T15:51:07.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 msgr2=0x7f0dc410de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:07.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 0x7f0dc410de90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0db400b0d0 tx=0x7f0db4009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:07.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 shutdown_connections 2024-04-25T15:51:07.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f0db00765a0 0x7f0db0078a50 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dc40fbcb0 0x7f0dc410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 --2- 172.21.15.110:0/167306415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0dc40fcf60 0x7f0dc410de90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 >> 172.21.15.110:0/167306415 conn(0x7f0dc40f7430 msgr2=0x7f0dc40f8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:07.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 shutdown_connections 2024-04-25T15:51:07.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.296+0000 7f0dcb271700 1 -- 172.21.15.110:0/167306415 wait complete. 2024-04-25T15:51:07.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.436+0000 7f49671d2700 1 -- 172.21.15.110:0/2480312007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 msgr2=0x7f49600fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:07.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.436+0000 7f49671d2700 1 --2- 172.21.15.110:0/2480312007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 0x7f49600fd270 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4954004940 tx=0x7f4954009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:07.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.436+0000 7f49671d2700 1 -- 172.21.15.110:0/2480312007 shutdown_connections 2024-04-25T15:51:07.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.436+0000 7f49671d2700 1 --2- 172.21.15.110:0/2480312007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49600fdfd0 0x7f49600fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.436+0000 7f49671d2700 1 --2- 172.21.15.110:0/2480312007 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 0x7f49600fd270 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.436+0000 7f49671d2700 1 -- 172.21.15.110:0/2480312007 >> 172.21.15.110:0/2480312007 conn(0x7f49600f8540 msgr2=0x7f49600fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:07.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.436+0000 7f49671d2700 1 -- 172.21.15.110:0/2480312007 shutdown_connections 2024-04-25T15:51:07.438 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.436+0000 7f49671d2700 1 -- 172.21.15.110:0/2480312007 wait complete. 2024-04-25T15:51:07.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f49671d2700 1 Processor -- start 2024-04-25T15:51:07.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f49671d2700 1 -- start start 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f49671d2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 0x7f496010d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f49671d2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49600fdfd0 0x7f496010ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f49671d2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496010e4d0 con 0x7f49600fce60 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f49671d2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496010ed20 con 0x7f49600fdfd0 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f4964f6e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 0x7f496010d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f4964f6e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 0x7f496010d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59324/0 (socket says 172.21.15.110:59324) 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f4964f6e700 1 -- 172.21.15.110:0/1368787380 learned_addr learned my addr 172.21.15.110:0/1368787380 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.437+0000 7f495ffff700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49600fdfd0 0x7f496010ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f4964f6e700 1 -- 172.21.15.110:0/1368787380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49600fdfd0 msgr2=0x7f496010ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f4964f6e700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49600fdfd0 0x7f496010ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f4964f6e700 1 -- 172.21.15.110:0/1368787380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4954005040 con 0x7f49600fce60 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f4964f6e700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 0x7f496010d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f495400b0d0 tx=0x7f4954009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f495d7fa700 1 -- 172.21.15.110:0/1368787380 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495400e030 con 0x7f49600fce60 2024-04-25T15:51:07.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f495d7fa700 1 -- 172.21.15.110:0/1368787380 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f495401c400 con 0x7f49600fce60 2024-04-25T15:51:07.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f496010efa0 con 0x7f49600fce60 2024-04-25T15:51:07.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f495d7fa700 1 -- 172.21.15.110:0/1368787380 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f495400b630 con 0x7f49600fce60 2024-04-25T15:51:07.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.438+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f496010f4e0 con 0x7f49600fce60 2024-04-25T15:51:07.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.439+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f496005f5c0 con 0x7f49600fce60 2024-04-25T15:51:07.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.440+0000 7f495d7fa700 1 -- 172.21.15.110:0/1368787380 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 31) v1 ==== 98487+0+0 (secure 0 0 0) 0x7f495401c570 con 0x7f49600fce60 2024-04-25T15:51:07.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.440+0000 7f495d7fa700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f49500765a0 0x7f4950078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:07.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.441+0000 7f495d7fa700 1 -- 172.21.15.110:0/1368787380 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f49540994e0 con 0x7f49600fce60 2024-04-25T15:51:07.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.441+0000 7f495ffff700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f49500765a0 0x7f4950078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:07.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.441+0000 7f495ffff700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f49500765a0 0x7f4950078a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f494c009b80 tx=0x7f494c00b040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:07.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.446+0000 7f495d7fa700 1 -- 172.21.15.110:0/1368787380 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f4954062720 con 0x7f49600fce60 2024-04-25T15:51:07.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.763+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 --> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f496010f860 con 0x7f49500765a0 2024-04-25T15:51:07.765 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:07 smithi110 ceph-mon[26344]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:07.765 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:07 smithi110 ceph-mon[26344]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:07.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.783+0000 7f495d7fa700 1 -- 172.21.15.110:0/1368787380 <== mgr.14758 v2:172.21.15.110:6800/1601336217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f496010f860 con 0x7f49500765a0 2024-04-25T15:51:07.784 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:51:07.784 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (86s) 10s ago 10m 14.8M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:51:07.784 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (10m) 10s ago 10m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (9m) 16s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (71s) 10s ago 9m 79.6M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (4m) 10s ago 4m 17.6M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (4m) 10s ago 4m 238M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (4m) 16s ago 4m 13.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (4m) 16s ago 4m 15.5M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (55s) 10s ago 11m 547M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283 running (2m) 16s ago 9m 147M - 19.0.0-3291-g994e2f20 d53c96779a29 82852af5489a 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (11m) 10s ago 11m 52.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (9m) 16s ago 9m 46.0M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (2m) 10s ago 10m 8468k - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (62s) 16s ago 9m 15.8M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (8m) 10s ago 8m 179M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (7m) 10s ago 7m 192M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (7m) 10s ago 7m 158M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:51:07.785 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (7m) 16s ago 7m 215M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:51:07.786 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (6m) 16s ago 6m 185M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:51:07.786 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (6m) 16s ago 6m 170M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:51:07.786 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (64s) 10s ago 9m 47.2M - 2.43.0 a07b618ecd1d 596d4bb0a96e 2024-04-25T15:51:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f49500765a0 msgr2=0x7f4950078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f49500765a0 0x7f4950078a50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f494c009b80 tx=0x7f494c00b040 comp rx=0 tx=0).stop 2024-04-25T15:51:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 msgr2=0x7f496010d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 0x7f496010d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f495400b0d0 tx=0x7f4954009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 shutdown_connections 2024-04-25T15:51:07.790 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f49500765a0 0x7f4950078a50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49600fce60 0x7f496010d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 --2- 172.21.15.110:0/1368787380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49600fdfd0 0x7f496010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.789+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 >> 172.21.15.110:0/1368787380 conn(0x7f49600f8540 msgr2=0x7f49600f9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:07.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.791+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 shutdown_connections 2024-04-25T15:51:07.792 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.791+0000 7f49671d2700 1 -- 172.21.15.110:0/1368787380 wait complete. 2024-04-25T15:51:07.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.932+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2574503317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fb420 msgr2=0x7f9ccc0fb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.932+0000 7f9cd36f8700 1 --2- 172.21.15.110:0/2574503317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fb420 0x7f9ccc0fb830 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8004940 tx=0x7f9cc8009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.932+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2574503317 shutdown_connections 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.932+0000 7f9cd36f8700 1 --2- 172.21.15.110:0/2574503317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ccc0fc6d0 0x7f9ccc0fcb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.932+0000 7f9cd36f8700 1 --2- 172.21.15.110:0/2574503317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fb420 0x7f9ccc0fb830 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.932+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2574503317 >> 172.21.15.110:0/2574503317 conn(0x7f9ccc0f6b80 msgr2=0x7f9ccc0f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.932+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2574503317 shutdown_connections 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.932+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2574503317 wait complete. 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd36f8700 1 Processor -- start 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd36f8700 1 -- start start 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd36f8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ccc0fb420 0x7f9ccc0ff540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd36f8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fc6d0 0x7f9ccc0fda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd36f8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0fdfc0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd36f8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc0fe130 con 0x7f9ccc0fb420 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd0c93700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fc6d0 0x7f9ccc0fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd1494700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ccc0fb420 0x7f9ccc0ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd1494700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ccc0fb420 0x7f9ccc0ff540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.110:44030/0 (socket says 172.21.15.110:44030) 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd1494700 1 -- 172.21.15.110:0/2243539543 learned_addr learned my addr 172.21.15.110:0/2243539543 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.933+0000 7f9cd0c93700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fc6d0 0x7f9ccc0fda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59350/0 (socket says 172.21.15.110:59350) 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.934+0000 7f9cd0c93700 1 -- 172.21.15.110:0/2243539543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ccc0fb420 msgr2=0x7f9ccc0ff540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.934+0000 7f9cd0c93700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ccc0fb420 0x7f9ccc0ff540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.934+0000 7f9cd0c93700 1 -- 172.21.15.110:0/2243539543 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc8005040 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.934+0000 7f9cd0c93700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fc6d0 0x7f9ccc0fda10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00a6c0 tx=0x7f9cbc00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.934+0000 7f9cc1ffb700 1 -- 172.21.15.110:0/2243539543 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cbc007ab0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.934+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc0fe3b0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.934+0000 7f9cc1ffb700 1 -- 172.21.15.110:0/2243539543 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cbc010040 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.934+0000 7f9cc1ffb700 1 -- 172.21.15.110:0/2243539543 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cbc00e920 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.935+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc1159b0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.936+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc05f5c0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.936+0000 7f9cc1ffb700 1 -- 172.21.15.110:0/2243539543 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 31) v1 ==== 98487+0+0 (secure 0 0 0) 0x7f9cbc0090d0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.937+0000 7f9cc1ffb700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f9cb80765a0 0x7f9cb8078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.937+0000 7f9cc1ffb700 1 -- 172.21.15.110:0/2243539543 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f9cbc0971d0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.937+0000 7f9cd1494700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f9cb80765a0 0x7f9cb8078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:07.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.938+0000 7f9cd1494700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f9cb80765a0 0x7f9cb8078a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800b0d0 tx=0x7f9cc8009e30 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:07.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:07.944+0000 7f9cc1ffb700 1 -- 172.21.15.110:0/2243539543 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f9cbc0604c0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:07.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:07 smithi193 ceph-mon[33747]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:07.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:07 smithi193 ceph-mon[33747]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:08.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.389+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9ccc04bed0 con 0x7f9ccc0fc6d0 2024-04-25T15:51:08.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.389+0000 7f9cc1ffb700 1 -- 172.21.15.110:0/2243539543 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+739 (secure 0 0 0) 0x7f9cbc05fe60 con 0x7f9ccc0fc6d0 2024-04-25T15:51:08.390 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:51:08.390 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:51:08.390 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 12, 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:51:08.391 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:51:08.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f9cb80765a0 msgr2=0x7f9cb8078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:08.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f9cb80765a0 0x7f9cb8078a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800b0d0 tx=0x7f9cc8009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:08.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fc6d0 msgr2=0x7f9ccc0fda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:08.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fc6d0 0x7f9ccc0fda10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00a6c0 tx=0x7f9cbc00c040 comp rx=0 tx=0).stop 2024-04-25T15:51:08.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 shutdown_connections 2024-04-25T15:51:08.393 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f9cb80765a0 0x7f9cb8078a50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9ccc0fb420 0x7f9ccc0ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 --2- 172.21.15.110:0/2243539543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ccc0fc6d0 0x7f9ccc0fda10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 >> 172.21.15.110:0/2243539543 conn(0x7f9ccc0f6b80 msgr2=0x7f9ccc0f8160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:08.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 shutdown_connections 2024-04-25T15:51:08.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.392+0000 7f9cd36f8700 1 -- 172.21.15.110:0/2243539543 wait complete. 2024-04-25T15:51:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 -- 172.21.15.110:0/2723603674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 msgr2=0x7f94840fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:08.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 --2- 172.21.15.110:0/2723603674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 0x7f94840fc0c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9474004940 tx=0x7f9474009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 -- 172.21.15.110:0/2723603674 shutdown_connections 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 --2- 172.21.15.110:0/2723603674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94840fcf60 0x7f94840fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 --2- 172.21.15.110:0/2723603674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 0x7f94840fc0c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 -- 172.21.15.110:0/2723603674 >> 172.21.15.110:0/2723603674 conn(0x7f94840f7430 msgr2=0x7f94840f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 -- 172.21.15.110:0/2723603674 shutdown_connections 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 -- 172.21.15.110:0/2723603674 wait complete. 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.534+0000 7f948a5a0700 1 Processor -- start 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f948a5a0700 1 -- start start 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f948a5a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 0x7f9484103df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f948a5a0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94840fcf60 0x7f94841022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f948a5a0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9484102870 con 0x7f94840fbcb0 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f948a5a0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94841029e0 con 0x7f94840fcf60 2024-04-25T15:51:08.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f9483fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 0x7f9484103df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f9483fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 0x7f9484103df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59366/0 (socket says 172.21.15.110:59366) 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f9483fff700 1 -- 172.21.15.110:0/2245992454 learned_addr learned my addr 172.21.15.110:0/2245992454 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f94837fe700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94840fcf60 0x7f94841022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f9483fff700 1 -- 172.21.15.110:0/2245992454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94840fcf60 msgr2=0x7f94841022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f9483fff700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94840fcf60 0x7f94841022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.535+0000 7f9483fff700 1 -- 172.21.15.110:0/2245992454 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474005040 con 0x7f94840fbcb0 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.536+0000 7f9483fff700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 0x7f9484103df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9474013870 tx=0x7f9474009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.536+0000 7f9480ff9700 1 -- 172.21.15.110:0/2245992454 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9474021070 con 0x7f94840fbcb0 2024-04-25T15:51:08.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.536+0000 7f9480ff9700 1 -- 172.21.15.110:0/2245992454 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9474009430 con 0x7f94840fbcb0 2024-04-25T15:51:08.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.536+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9484102c60 con 0x7f94840fbcb0 2024-04-25T15:51:08.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.536+0000 7f9480ff9700 1 -- 172.21.15.110:0/2245992454 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f947400e070 con 0x7f94840fbcb0 2024-04-25T15:51:08.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.536+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94841159b0 con 0x7f94840fbcb0 2024-04-25T15:51:08.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.537+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f948405f5c0 con 0x7f94840fbcb0 2024-04-25T15:51:08.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.538+0000 7f9480ff9700 1 -- 172.21.15.110:0/2245992454 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 31) v1 ==== 98487+0+0 (secure 0 0 0) 0x7f947401c750 con 0x7f94840fbcb0 2024-04-25T15:51:08.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.539+0000 7f9480ff9700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f946c076530 0x7f946c0789e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:08.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.539+0000 7f9480ff9700 1 -- 172.21.15.110:0/2245992454 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f9474098d20 con 0x7f94840fbcb0 2024-04-25T15:51:08.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.539+0000 7f94837fe700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f946c076530 0x7f946c0789e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:08.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.539+0000 7f94837fe700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f946c076530 0x7f946c0789e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9478004fd0 tx=0x7f947800b040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:08.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.545+0000 7f9480ff9700 1 -- 172.21.15.110:0/2245992454 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f9474061f60 con 0x7f94840fbcb0 2024-04-25T15:51:08.697 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:08.697 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:08.697 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:08.697 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:51:08.697 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:51:08.697 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:51:08.697 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:08.697 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:51:08.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.955+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f94841058a0 con 0x7f94840fbcb0 2024-04-25T15:51:08.956 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:51:08.956 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:51:08.956 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:51:08.956 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2243539543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:51:08.956 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:08 smithi110 ceph-mon[26344]: pgmap v14: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.1 MiB/s wr, 425 op/s 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.955+0000 7f9480ff9700 1 -- 172.21.15.110:0/2245992454 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1808 (secure 0 0 0) 0x7f9474061900 con 0x7f94840fbcb0 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:51:08.957 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:51:08.958 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:08.959 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:08.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f946c076530 msgr2=0x7f946c0789e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:08.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f946c076530 0x7f946c0789e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f9478004fd0 tx=0x7f947800b040 comp rx=0 tx=0).stop 2024-04-25T15:51:08.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 msgr2=0x7f9484103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:08.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 0x7f9484103df0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9474013870 tx=0x7f9474009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:08.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 shutdown_connections 2024-04-25T15:51:08.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f946c076530 0x7f946c0789e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f94840fbcb0 0x7f9484103df0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 --2- 172.21.15.110:0/2245992454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94840fcf60 0x7f94841022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:08.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 >> 172.21.15.110:0/2245992454 conn(0x7f94840f7430 msgr2=0x7f94840f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:08.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 shutdown_connections 2024-04-25T15:51:08.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:08.958+0000 7f948a5a0700 1 -- 172.21.15.110:0/2245992454 wait complete. 2024-04-25T15:51:08.961 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:51:08.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:51:08.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:51:08.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2243539543' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:51:08.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:08 smithi193 ceph-mon[33747]: pgmap v14: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 41 KiB/s rd, 2.1 MiB/s wr, 425 op/s 2024-04-25T15:51:09.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.099+0000 7f77b02c7700 1 -- 172.21.15.110:0/4122553105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fbcb0 msgr2=0x7f77a80fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:09.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.099+0000 7f77b02c7700 1 --2- 172.21.15.110:0/4122553105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fbcb0 0x7f77a80fc0c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009ce0 tx=0x7f77a0004d10 comp rx=0 tx=0).stop 2024-04-25T15:51:09.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.099+0000 7f77b02c7700 1 -- 172.21.15.110:0/4122553105 shutdown_connections 2024-04-25T15:51:09.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.099+0000 7f77b02c7700 1 --2- 172.21.15.110:0/4122553105 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a80fcf60 0x7f77a80fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.099+0000 7f77b02c7700 1 --2- 172.21.15.110:0/4122553105 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fbcb0 0x7f77a80fc0c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.099+0000 7f77b02c7700 1 -- 172.21.15.110:0/4122553105 >> 172.21.15.110:0/4122553105 conn(0x7f77a80f7410 msgr2=0x7f77a80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:09.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.099+0000 7f77b02c7700 1 -- 172.21.15.110:0/4122553105 shutdown_connections 2024-04-25T15:51:09.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.099+0000 7f77b02c7700 1 -- 172.21.15.110:0/4122553105 wait complete. 2024-04-25T15:51:09.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77b02c7700 1 Processor -- start 2024-04-25T15:51:09.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77b02c7700 1 -- start start 2024-04-25T15:51:09.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77b02c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a80fbcb0 0x7f77a8114090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:09.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77b02c7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fcf60 0x7f77a81145d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:09.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77b02c7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8114bd0 con 0x7f77a80fcf60 2024-04-25T15:51:09.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77b02c7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a8115420 con 0x7f77a80fbcb0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77ad862700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fcf60 0x7f77a81145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77ad862700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fcf60 0x7f77a81145d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:59384/0 (socket says 172.21.15.110:59384) 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77ad862700 1 -- 172.21.15.110:0/1890692689 learned_addr learned my addr 172.21.15.110:0/1890692689 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77ae063700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a80fbcb0 0x7f77a8114090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77ae063700 1 -- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fcf60 msgr2=0x7f77a81145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77ae063700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fcf60 0x7f77a81145d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.100+0000 7f77ae063700 1 -- 172.21.15.110:0/1890692689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a0009940 con 0x7f77a80fbcb0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.101+0000 7f77ae063700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a80fbcb0 0x7f77a8114090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009cb0 tx=0x7f77a0004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.101+0000 7f779affd700 1 -- 172.21.15.110:0/1890692689 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a000e040 con 0x7f77a80fbcb0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.101+0000 7f779affd700 1 -- 172.21.15.110:0/1890692689 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77a001cb20 con 0x7f77a80fbcb0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.101+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a81156a0 con 0x7f77a80fbcb0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.101+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a8115be0 con 0x7f77a80fbcb0 2024-04-25T15:51:09.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.101+0000 7f779affd700 1 -- 172.21.15.110:0/1890692689 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a0012b10 con 0x7f77a80fbcb0 2024-04-25T15:51:09.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.102+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f778c0052e0 con 0x7f77a80fbcb0 2024-04-25T15:51:09.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.103+0000 7f779affd700 1 -- 172.21.15.110:0/1890692689 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 31) v1 ==== 98487+0+0 (secure 0 0 0) 0x7f77a001c650 con 0x7f77a80fbcb0 2024-04-25T15:51:09.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.104+0000 7f779affd700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f77940765a0 0x7f7794078a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:09.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.104+0000 7f779affd700 1 -- 172.21.15.110:0/1890692689 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7f77a0099c60 con 0x7f77a80fbcb0 2024-04-25T15:51:09.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.104+0000 7f77ad862700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f77940765a0 0x7f7794078a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:09.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.104+0000 7f77ad862700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f77940765a0 0x7f7794078a50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f779c008fd0 tx=0x7f779c00b040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:09.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.108+0000 7f779affd700 1 -- 172.21.15.110:0/1890692689 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7f77a0062ea0 con 0x7f77a80fbcb0 2024-04-25T15:51:09.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.425+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 --> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f778c000bf0 con 0x7f77940765a0 2024-04-25T15:51:09.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.426+0000 7f779affd700 1 -- 172.21.15.110:0/1890692689 <== mgr.14758 v2:172.21.15.110:6800/1601336217 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f778c000bf0 con 0x7f77940765a0 2024-04-25T15:51:09.427 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:51:09.427 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:51:09.427 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:51:09.427 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:51:09.428 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:51:09.428 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:51:09.428 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:51:09.428 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "2/21 daemons upgraded", 2024-04-25T15:51:09.428 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-04-25T15:51:09.428 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:51:09.428 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f77940765a0 msgr2=0x7f7794078a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f77940765a0 0x7f7794078a50 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f779c008fd0 tx=0x7f779c00b040 comp rx=0 tx=0).stop 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a80fbcb0 msgr2=0x7f77a8114090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a80fbcb0 0x7f77a8114090 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009cb0 tx=0x7f77a0004d10 comp rx=0 tx=0).stop 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 shutdown_connections 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7f77940765a0 0x7f7794078a50 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77a80fbcb0 0x7f77a8114090 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 --2- 172.21.15.110:0/1890692689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77a80fcf60 0x7f77a81145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 >> 172.21.15.110:0/1890692689 conn(0x7f77a80f7410 msgr2=0x7f77a80f8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:09.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 shutdown_connections 2024-04-25T15:51:09.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.430+0000 7f77b02c7700 1 -- 172.21.15.110:0/1890692689 wait complete. 2024-04-25T15:51:09.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.563+0000 7faa52a9c700 1 -- 172.21.15.110:0/4059937942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 msgr2=0x7faa4c102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:09.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.563+0000 7faa52a9c700 1 --2- 172.21.15.110:0/4059937942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 0x7faa4c102300 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7faa38004940 tx=0x7faa38009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:09.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.563+0000 7faa52a9c700 1 -- 172.21.15.110:0/4059937942 shutdown_connections 2024-04-25T15:51:09.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.563+0000 7faa52a9c700 1 --2- 172.21.15.110:0/4059937942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 0x7faa4c102300 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.563+0000 7faa52a9c700 1 --2- 172.21.15.110:0/4059937942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4c0662c0 0x7faa4c0ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.563+0000 7faa52a9c700 1 -- 172.21.15.110:0/4059937942 >> 172.21.15.110:0/4059937942 conn(0x7faa4c0f7430 msgr2=0x7faa4c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:09.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.563+0000 7faa52a9c700 1 -- 172.21.15.110:0/4059937942 shutdown_connections 2024-04-25T15:51:09.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.563+0000 7faa52a9c700 1 -- 172.21.15.110:0/4059937942 wait complete. 2024-04-25T15:51:09.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa52a9c700 1 Processor -- start 2024-04-25T15:51:09.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa52a9c700 1 -- start start 2024-04-25T15:51:09.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa52a9c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4c0662c0 0x7faa4c0fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:09.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa52a9c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 0x7faa4c0fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:09.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa52a9c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c111640 con 0x7faa4c0ffee0 2024-04-25T15:51:09.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa52a9c700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c111780 con 0x7faa4c0662c0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa4bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 0x7faa4c0fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa4bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 0x7faa4c0fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38672/0 (socket says 172.21.15.110:38672) 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa4bfff700 1 -- 172.21.15.110:0/317488645 learned_addr learned my addr 172.21.15.110:0/317488645 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.564+0000 7faa50838700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4c0662c0 0x7faa4c0fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.565+0000 7faa4bfff700 1 -- 172.21.15.110:0/317488645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4c0662c0 msgr2=0x7faa4c0fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.565+0000 7faa4bfff700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4c0662c0 0x7faa4c0fd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.565+0000 7faa4bfff700 1 -- 172.21.15.110:0/317488645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa38005040 con 0x7faa4c0ffee0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.565+0000 7faa4bfff700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 0x7faa4c0fb8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faa38009fd0 tx=0x7faa38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.565+0000 7faa497fa700 1 -- 172.21.15.110:0/317488645 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa3800e070 con 0x7faa4c0ffee0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.565+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c0fbe40 con 0x7faa4c0ffee0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.565+0000 7faa497fa700 1 -- 172.21.15.110:0/317488645 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa3801c400 con 0x7faa4c0ffee0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.566+0000 7faa497fa700 1 -- 172.21.15.110:0/317488645 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa38026640 con 0x7faa4c0ffee0 2024-04-25T15:51:09.567 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.566+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c0fc3a0 con 0x7faa4c0ffee0 2024-04-25T15:51:09.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.567+0000 7faa497fa700 1 -- 172.21.15.110:0/317488645 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 31) v1 ==== 98487+0+0 (secure 0 0 0) 0x7faa3801ca90 con 0x7faa4c0ffee0 2024-04-25T15:51:09.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.567+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c05f5c0 con 0x7faa4c0ffee0 2024-04-25T15:51:09.568 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.567+0000 7faa497fa700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7faa3c076640 0x7faa3c078af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:09.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.568+0000 7faa497fa700 1 -- 172.21.15.110:0/317488645 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7faa38099230 con 0x7faa4c0ffee0 2024-04-25T15:51:09.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.568+0000 7faa50838700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7faa3c076640 0x7faa3c078af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:09.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.568+0000 7faa50838700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7faa3c076640 0x7faa3c078af0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0fd110 tx=0x7faa4000b040 comp rx=0 tx=0).ready entity=mgr.14758 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:09.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:09.574+0000 7faa497fa700 1 -- 172.21.15.110:0/317488645 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186652 (secure 0 0 0) 0x7faa380623d0 con 0x7faa4c0ffee0 2024-04-25T15:51:10.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.046+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7faa4c0fc610 con 0x7faa4c0ffee0 2024-04-25T15:51:10.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.046+0000 7faa497fa700 1 -- 172.21.15.110:0/317488645 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7faa38061d70 con 0x7faa4c0ffee0 2024-04-25T15:51:10.047 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:51:10.047 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:51:10.047 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:51:10.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7faa3c076640 msgr2=0x7faa3c078af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7faa3c076640 0x7faa3c078af0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7faa4c0fd110 tx=0x7faa4000b040 comp rx=0 tx=0).stop 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 msgr2=0x7faa4c0fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 0x7faa4c0fb8a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7faa38009fd0 tx=0x7faa38009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 shutdown_connections 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7faa3c076640 0x7faa3c078af0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faa4c0662c0 0x7faa4c0fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 --2- 172.21.15.110:0/317488645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faa4c0ffee0 0x7faa4c0fb8a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.049+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 >> 172.21.15.110:0/317488645 conn(0x7faa4c0f7430 msgr2=0x7faa4c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.050+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 shutdown_connections 2024-04-25T15:51:10.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:10.050+0000 7faa52a9c700 1 -- 172.21.15.110:0/317488645 wait complete. 2024-04-25T15:51:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:51:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:51:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2245992454' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:51:10.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-04-25T15:51:10.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-04-25T15:51:10.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:09 smithi193 ceph-mon[33747]: from='client.24555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/2245992454' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: Reconfiguring prometheus.smithi110 (dependencies changed)... 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: Reconfiguring daemon prometheus.smithi110 on smithi110 2024-04-25T15:51:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:09 smithi110 ceph-mon[26344]: from='client.24555 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:11.160 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:10 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/317488645' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:51:11.161 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:10 smithi193 ceph-mon[33747]: pgmap v15: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 242 op/s 2024-04-25T15:51:11.320 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:10 smithi110 ceph-mon[26344]: from='client.? 172.21.15.110:0/317488645' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:51:11.320 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:10 smithi110 ceph-mon[26344]: pgmap v15: 65 pgs: 65 active+clean; 305 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 27 KiB/s rd, 1.3 MiB/s wr, 242 op/s 2024-04-25T15:51:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:12 smithi110 ceph-mon[26344]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-04-25T15:51:12.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:12 smithi193 ceph-mon[33747]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 40 KiB/s rd, 1.9 MiB/s wr, 303 op/s 2024-04-25T15:51:14.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:14 smithi110 ceph-mon[26344]: pgmap v17: 65 pgs: 65 active+clean; 293 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-04-25T15:51:14.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:14 smithi193 ceph-mon[33747]: pgmap v17: 65 pgs: 65 active+clean; 293 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-04-25T15:51:16.817 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:16 smithi110 ceph-mon[26344]: pgmap v18: 65 pgs: 65 active+clean; 293 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-04-25T15:51:16.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:16 smithi193 ceph-mon[33747]: pgmap v18: 65 pgs: 65 active+clean; 293 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 26 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-04-25T15:51:17.836 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:17 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:51:17.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:17 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:51:18.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:18 smithi110 ceph-mon[26344]: pgmap v19: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.8 MiB/s wr, 236 op/s 2024-04-25T15:51:18.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:18 smithi193 ceph-mon[33747]: pgmap v19: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.8 MiB/s wr, 236 op/s 2024-04-25T15:51:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:19 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:19 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:19 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:51:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:19 smithi110 ceph-mon[26344]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:51:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:19 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110:9095"}]: dispatch 2024-04-25T15:51:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:19 smithi110 ceph-mon[26344]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110:9095"}]: dispatch 2024-04-25T15:51:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:19 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:19.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:19 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:19 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:19 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:19 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:51:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:19 smithi193 ceph-mon[33747]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:51:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:19 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110:9095"}]: dispatch 2024-04-25T15:51:19.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:19 smithi193 ceph-mon[33747]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi110:9095"}]: dispatch 2024-04-25T15:51:19.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:19 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:19.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:19 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:20 smithi110 ceph-mon[26344]: Upgrade: Updating mgr.smithi193.mgoteh 2024-04-25T15:51:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:20 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:51:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:20 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:51:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:20 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:51:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:20 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:20 smithi110 ceph-mon[26344]: Deploying daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:51:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:20 smithi110 ceph-mon[26344]: pgmap v20: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-04-25T15:51:20.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:20 smithi193 ceph-mon[33747]: Upgrade: Updating mgr.smithi193.mgoteh 2024-04-25T15:51:20.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:20 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:51:20.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:20 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:51:20.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:20 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:51:20.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:20 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:20.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:20 smithi193 ceph-mon[33747]: Deploying daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:51:20.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:20 smithi193 ceph-mon[33747]: pgmap v20: 65 pgs: 65 active+clean; 283 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 23 KiB/s rd, 1.1 MiB/s wr, 155 op/s 2024-04-25T15:51:22.354 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-25T15:51:22.354 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T15:51:22.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:22 smithi110 ceph-mon[26344]: pgmap v21: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-04-25T15:51:22.691 DEBUG:teuthology.parallel:result is None 2024-04-25T15:51:22.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:22 smithi193 ceph-mon[33747]: pgmap v21: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 37 KiB/s rd, 1.8 MiB/s wr, 260 op/s 2024-04-25T15:51:25.544 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:25 smithi193 ceph-mon[33747]: pgmap v22: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-04-25T15:51:25.544 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:25 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:25.544 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:25 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:25.545 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:25 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:25.546 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-25T15:51:25.546 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-25T15:51:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:25 smithi110 ceph-mon[26344]: pgmap v22: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-04-25T15:51:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:25 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:25 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:25 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:25.948 DEBUG:teuthology.parallel:result is None 2024-04-25T15:51:25.949 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T15:51:25.983 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T15:51:25.983 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T15:51:26.019 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-25T15:51:26.019 DEBUG:teuthology.parallel:result is None 2024-04-25T15:51:27.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:27 smithi110 ceph-mon[26344]: pgmap v23: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-04-25T15:51:27.893 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:27 smithi193 ceph-mon[33747]: pgmap v23: 65 pgs: 65 active+clean; 279 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 24 KiB/s rd, 1.2 MiB/s wr, 199 op/s 2024-04-25T15:51:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:28 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:28 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:28 smithi110 ceph-mon[26344]: pgmap v24: 65 pgs: 65 active+clean; 272 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-25T15:51:29.206 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:28 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:29.207 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:28 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:29.207 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:28 smithi193 ceph-mon[33747]: pgmap v24: 65 pgs: 65 active+clean; 272 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 39 KiB/s rd, 1.6 MiB/s wr, 280 op/s 2024-04-25T15:51:30.178 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:29 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:30.178 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:29 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:30.236 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:29 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:30.237 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:29 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:31.051 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:30 smithi110 ceph-mon[26344]: pgmap v25: 65 pgs: 65 active+clean; 272 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1008 KiB/s wr, 186 op/s 2024-04-25T15:51:31.098 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:30 smithi193 ceph-mon[33747]: pgmap v25: 65 pgs: 65 active+clean; 272 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 28 KiB/s rd, 1008 KiB/s wr, 186 op/s 2024-04-25T15:51:32.630 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:32.630 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:32.631 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:51:32.631 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: Standby manager daemon smithi193.mgoteh restarted 2024-04-25T15:51:32.631 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: Standby manager daemon smithi193.mgoteh started 2024-04-25T15:51:32.631 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/1281934472' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:51:32.631 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/1281934472' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:51:32.631 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/1281934472' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:51:32.631 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/1281934472' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:51:32.631 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:32 smithi193 ceph-mon[33747]: pgmap v26: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.0 MiB/s wr, 192 op/s 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: Standby manager daemon smithi193.mgoteh restarted 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: Standby manager daemon smithi193.mgoteh started 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/1281934472' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/1281934472' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/1281934472' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: from='mgr.? 172.21.15.193:0/1281934472' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:51:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:32 smithi110 ceph-mon[26344]: pgmap v26: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 29 KiB/s rd, 1.0 MiB/s wr, 192 op/s 2024-04-25T15:51:33.738 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: mgrmap e32: smithi110.lvgbkf(active, since 46s), standbys: smithi193.mgoteh 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: Upgrade: Setting container_image for all mgr 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.lvgbkf"}]: dispatch 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.lvgbkf"}]: dispatch 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.lvgbkf"}]': finished 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mgoteh"}]: dispatch 2024-04-25T15:51:33.739 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mgoteh"}]: dispatch 2024-04-25T15:51:33.740 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mgoteh"}]': finished 2024-04-25T15:51:33.740 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:33 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-25T15:51:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: mgrmap e32: smithi110.lvgbkf(active, since 46s), standbys: smithi193.mgoteh 2024-04-25T15:51:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: Upgrade: Setting container_image for all mgr 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.lvgbkf"}]: dispatch 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.lvgbkf"}]: dispatch 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi110.lvgbkf"}]': finished 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mgoteh"}]: dispatch 2024-04-25T15:51:33.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mgoteh"}]: dispatch 2024-04-25T15:51:33.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi193.mgoteh"}]': finished 2024-04-25T15:51:33.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:33 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-25T15:51:34.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:34 smithi110 ceph-mon[26344]: pgmap v27: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 391 KiB/s wr, 87 op/s 2024-04-25T15:51:34.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:34 smithi193 ceph-mon[33747]: pgmap v27: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 391 KiB/s wr, 87 op/s 2024-04-25T15:51:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:35 smithi110 ceph-mon[26344]: Upgrade: Updating mon.smithi110 2024-04-25T15:51:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:35 smithi110 ceph-mon[26344]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:35 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:51:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:35 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:51:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:35 smithi110 ceph-mon[26344]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:35 smithi110 ceph-mon[26344]: Deploying daemon mon.smithi110 on smithi110 2024-04-25T15:51:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:35 smithi193 ceph-mon[33747]: Upgrade: Updating mon.smithi110 2024-04-25T15:51:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:35 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:35 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:51:35.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:35 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:51:35.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:35 smithi193 ceph-mon[33747]: from='mgr.14758 172.21.15.110:0/2774016552' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:35.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:35 smithi193 ceph-mon[33747]: Deploying daemon mon.smithi110 on smithi110 2024-04-25T15:51:36.588 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:36 smithi110 systemd[1]: Stopping Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:51:36.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:36 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[26340]: 2024-04-25T15:51:36.585+0000 7f893ad0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi110 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:51:36.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:36 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[26340]: 2024-04-25T15:51:36.585+0000 7f893ad0d700 -1 mon.smithi110@0(leader) e2 *** Got Signal Terminated *** 2024-04-25T15:51:36.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:36 smithi110 podman[90829]: 2024-04-25 15:51:36.679218596 +0000 UTC m=+0.211398451 container died e6fee4ab4b51e376c1e7197c8a7a3f024b4a10a24982a393a9636d06f66c58a5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=quincy-c61472c, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T15:51:37.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 podman[90829]: 2024-04-25 15:51:37.162546061 +0000 UTC m=+0.694725963 container cleanup e6fee4ab4b51e376c1e7197c8a7a3f024b4a10a24982a393a9636d06f66c58a5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, RELEASE=quincy-c61472c, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:51:37.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 bash[90829]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110 2024-04-25T15:51:37.778 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 podman[90843]: 2024-04-25 15:51:37.417094946 +0000 UTC m=+0.738968361 container remove e6fee4ab4b51e376c1e7197c8a7a3f024b4a10a24982a393a9636d06f66c58a5 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T15:51:37.778 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110.service: Deactivated successfully. 2024-04-25T15:51:37.778 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 systemd[1]: Stopped Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:51:37.778 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110.service: Consumed 11.120s CPU time. 2024-04-25T15:51:38.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 systemd[1]: Starting Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:51:38.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 podman[90972]: 2024-04-25 15:51:37.896661028 +0000 UTC m=+0.019159990 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:51:38.102 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:37 smithi110 podman[90972]: 2024-04-25 15:51:37.998259835 +0000 UTC m=+0.120758896 container create 59d163e103ed26d0770be306cb476e40b2f071b33a09091abd5d92aa58797603 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T15:51:38.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 podman[90972]: 2024-04-25 15:51:38.234924435 +0000 UTC m=+0.357423383 container init 59d163e103ed26d0770be306cb476e40b2f071b33a09091abd5d92aa58797603 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T15:51:38.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 podman[90972]: 2024-04-25 15:51:38.23810706 +0000 UTC m=+0.360606012 container start 59d163e103ed26d0770be306cb476e40b2f071b33a09091abd5d92aa58797603 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424) 2024-04-25T15:51:38.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T15:51:38.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: pidfile_write: ignore empty --pid-file 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: load: jerasure load: lrc 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: RocksDB version: 7.9.2 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Git sha 0 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: DB SUMMARY 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: DB Session ID: F0CTD43R209N4IV54YG4 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: CURRENT file: CURRENT 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: MANIFEST file: MANIFEST-000015 size: 972 Bytes 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi110/store.db dir, Total Num: 1, files: 000030.sst 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi110/store.db: 000028.log size: 2638782 ; 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.error_if_exists: 0 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.create_if_missing: 0 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.paranoid_checks: 1 2024-04-25T15:51:38.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T15:51:38.365 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T15:51:38.365 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T15:51:38.365 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.env: 0x55abe23fd500 2024-04-25T15:51:38.365 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T15:51:38.365 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.info_log: 0x55abe42a7e80 2024-04-25T15:51:38.365 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T15:51:38.366 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.statistics: (nil) 2024-04-25T15:51:38.366 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.use_fsync: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_log_file_size: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.allow_fallocate: 1 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.use_direct_reads: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.db_log_dir: 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.wal_dir: 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T15:51:38.367 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.write_buffer_manager: 0x55abe42788c0 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T15:51:38.368 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.unordered_write: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.row_cache: None 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.wal_filter: None 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.two_write_queues: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.wal_compression: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.atomic_flush: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.log_readahead_size: 0 2024-04-25T15:51:38.369 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_background_jobs: 2 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_background_compactions: -1 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_subcompactions: 1 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T15:51:38.370 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_open_files: -1 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_background_flushes: -1 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Compression algorithms supported: 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: kZSTD supported: 0 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: kXpressCompression supported: 0 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: kZlibCompression supported: 1 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: kSnappyCompression supported: 1 2024-04-25T15:51:38.371 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: kLZ4Compression supported: 1 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: kBZip2Compression supported: 0 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi110/store.db/MANIFEST-000015 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.merge_operator: 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_filter: None 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55abe43ae060) 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-04-25T15:51:38.372 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_type: 0 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_block_index_type: 0 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_shortening: 1 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: checksum: 4 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: no_block_cache: 0 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache: 0x55abe42ab1f0 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_options: 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: capacity : 536870912 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: num_shard_bits : 4 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: strict_capacity_limit : 0 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_cache_compressed: (nil) 2024-04-25T15:51:38.373 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: persistent_cache: (nil) 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_size: 4096 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_size_deviation: 10 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_restart_interval: 16 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: index_block_restart_interval: 1 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: metadata_block_size: 4096 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: partition_filters: 0 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: use_delta_encoding: 1 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: filter_policy: bloomfilter 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: whole_key_filtering: 1 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: verify_compression: 0 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: format_version: 5 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: enable_index_compression: 1 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: block_align: 0 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_auto_readahead_size: 262144 2024-04-25T15:51:38.374 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: prepopulate_block_cache: 0 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: initial_auto_readahead_size: 8192 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression: NoCompression 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.num_levels: 7 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T15:51:38.375 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T15:51:38.376 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T15:51:38.377 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T15:51:38.378 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.inplace_update_support: 0 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.bloom_locality: 0 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.max_successive_merges: 0 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T15:51:38.379 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.ttl: 2592000 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.enable_blob_files: false 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.min_blob_size: 0 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T15:51:38.380 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 30.sst 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi110/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 32, last_sequence is 10294, log_number is 28,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 28 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 716ef407-baec-4d5b-ab88-8f1408e30c08 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714060298277914, "job": 1, "event": "recovery_started", "wal_files": [28]} 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #28 mode 2 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714060298286705, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 33, "file_size": 2423154, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 10295, "largest_seqno": 11495, "table_properties": {"data_size": 2416945, "index_size": 3820, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 1349, "raw_key_size": 12419, "raw_average_key_size": 23, "raw_value_size": 2405090, "raw_average_value_size": 4478, "num_data_blocks": 175, "num_entries": 537, "num_filter_entries": 537, "num_deletions": 2, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714060298, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "716ef407-baec-4d5b-ab88-8f1408e30c08", "db_session_id": "F0CTD43R209N4IV54YG4", "orig_file_number": 33, "seqno_to_time_mapping": "N/A"}} 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714060298286820, "job": 1, "event": "recovery_finished"} 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/version_set.cc:5047] Creating manifest 35 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi110/store.db/000028.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55abe428fc00 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: DB pointer 0x55abe9082000 2024-04-25T15:51:38.381 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** DB Stats ** 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: L0 1/0 2.31 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 552.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: L6 1/0 5.21 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-25T15:51:38.382 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Sum 2/0 7.53 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 552.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 552.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** Compaction Stats [default] ** 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 552.3 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Flush(GB): cumulative 0.002, interval 0.002 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Cumulative compaction: 0.00 GB write, 206.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T15:51:38.383 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Interval compaction: 0.00 GB write, 206.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Block cache BinnedLRUCache@0x55abe42ab1f0#2 capacity: 512.00 MB usage: 5.49 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,1.41 KB,0.000269711%) IndexBlock(1,4.08 KB,0.000777841%) Misc(1,0.00 KB,0%) 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: starting mon.smithi110 rank 0 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi110 fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: mon.smithi110@-1(???) e2 preinit fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: mon.smithi110@-1(???).mds e13 new map 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: mon.smithi110@-1(???).mds e13 print_map 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: e13 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: legacy client fscid: 1 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Filesystem 'cephfs' (1) 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: fs_name cephfs 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: epoch 13 2024-04-25T15:51:38.384 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: tableserver 0 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: root 0 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: session_timeout 60 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: session_autoclose 300 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_file_size 1099511627776 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_xattr_size 65536 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: required_client_features {} 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: last_failure 0 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: last_failure_osd_epoch 0 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: max_mds 1 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: in 0 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: up {0=14468} 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: failed 2024-04-25T15:51:38.385 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: damaged 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: stopped 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: data_pools [3] 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: metadata_pool 2 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: inline_data enabled 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: balancer 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: bal_rank_mask -1 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: standby_count_wanted 1 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: qdb_cluster leader: 0 members: 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: [mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: Standby daemons: 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: [mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: [mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout: [mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:38.386 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: mon.smithi110@-1(???).osd e48 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-25T15:51:38.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: mon.smithi110@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T15:51:38.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: mon.smithi110@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T15:51:38.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: mon.smithi110@-1(???).osd e48 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T15:51:38.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 ceph-mon[90991]: mon.smithi110@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2024-04-25T15:51:38.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 bash[90972]: 59d163e103ed26d0770be306cb476e40b2f071b33a09091abd5d92aa58797603 2024-04-25T15:51:38.387 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:38 smithi110 systemd[1]: Started Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:51:40.112 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: pgmap v29: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 391 KiB/s wr, 87 op/s 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: mon.smithi110 calling monitor election 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: mon.smithi110 is new leader, mons smithi110,smithi193 in quorum (ranks 0,1) 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: monmap epoch 2 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: last_changed 2024-04-25T15:42:03.287721+0000 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: created 2024-04-25T15:39:28.650237+0000 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: min_mon_release 17 (quincy) 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: election_strategy: 1 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.smithi110 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: osdmap e48: 6 total, 6 up, 6 in 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: mgrmap e32: smithi110.lvgbkf(active, since 53s), standbys: smithi193.mgoteh 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:51:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:51:40.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:51:40.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: from='mgr.14758 ' entity='' 2024-04-25T15:51:40.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:39 smithi110 ceph-mon[90991]: mgrmap e33: smithi110.lvgbkf(active, since 53s), standbys: smithi193.mgoteh 2024-04-25T15:51:40.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: pgmap v29: 65 pgs: 65 active+clean; 263 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 391 KiB/s wr, 87 op/s 2024-04-25T15:51:40.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: mon.smithi110 calling monitor election 2024-04-25T15:51:40.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: mon.smithi110 is new leader, mons smithi110,smithi193 in quorum (ranks 0,1) 2024-04-25T15:51:40.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: monmap epoch 2 2024-04-25T15:51:40.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:51:40.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: last_changed 2024-04-25T15:42:03.287721+0000 2024-04-25T15:51:40.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: created 2024-04-25T15:39:28.650237+0000 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: min_mon_release 17 (quincy) 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: election_strategy: 1 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.smithi110 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: osdmap e48: 6 total, 6 up, 6 in 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: mgrmap e32: smithi110.lvgbkf(active, since 53s), standbys: smithi193.mgoteh 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: from='mgr.14758 ' entity='' 2024-04-25T15:51:40.117 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:39 smithi193 ceph-mon[33747]: mgrmap e33: smithi110.lvgbkf(active, since 53s), standbys: smithi193.mgoteh 2024-04-25T15:51:40.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 -- 172.21.15.110:0/2245127456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 msgr2=0x7fb8200fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:40.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 --2- 172.21.15.110:0/2245127456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 0x7fb8200fc0c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb810004940 tx=0x7fb810009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 -- 172.21.15.110:0/2245127456 shutdown_connections 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 --2- 172.21.15.110:0/2245127456 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8200fcf60 0x7fb8200fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 --2- 172.21.15.110:0/2245127456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 0x7fb8200fc0c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 -- 172.21.15.110:0/2245127456 >> 172.21.15.110:0/2245127456 conn(0x7fb8200f7430 msgr2=0x7fb8200f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 -- 172.21.15.110:0/2245127456 shutdown_connections 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 -- 172.21.15.110:0/2245127456 wait complete. 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 Processor -- start 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 -- start start 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 0x7fb82010d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8200fcf60 0x7fb82010deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82010e4b0 con 0x7fb8200fbcb0 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.163+0000 7fb824e10700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82010ed00 con 0x7fb8200fcf60 2024-04-25T15:51:40.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb81e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 0x7fb82010d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb81e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 0x7fb82010d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:40854/0 (socket says 172.21.15.110:40854) 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb81e59c700 1 -- 172.21.15.110:0/2633165751 learned_addr learned my addr 172.21.15.110:0/2633165751 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8200fcf60 0x7fb82010deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb81e59c700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8200fcf60 msgr2=0x7fb82010deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb81e59c700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8200fcf60 0x7fb82010deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb81e59c700 1 -- 172.21.15.110:0/2633165751 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb810005040 con 0x7fb8200fbcb0 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb81e59c700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 0x7fb82010d970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb810013870 tx=0x7fb810009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb810021070 con 0x7fb8200fbcb0 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82010ef80 con 0x7fb8200fbcb0 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb810009430 con 0x7fb8200fbcb0 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb81000e070 con 0x7fb8200fbcb0 2024-04-25T15:51:40.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.164+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82010f4c0 con 0x7fb8200fbcb0 2024-04-25T15:51:40.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.165+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb820101af0 con 0x7fb8200fbcb0 2024-04-25T15:51:40.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.166+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 33) v1 ==== 98765+0+0 (secure 0 0 0) 0x7fb81001c750 con 0x7fb8200fbcb0 2024-04-25T15:51:40.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.167+0000 7fb80effd700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:40.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.167+0000 7fb81dd9b700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 msgr2=0x7fb8080790e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1601336217 2024-04-25T15:51:40.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.167+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5980+0+0 (secure 0 0 0) 0x7fb8100995d0 con 0x7fb8200fbcb0 2024-04-25T15:51:40.168 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.167+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T15:51:40.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.172+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb810062110 con 0x7fb8200fbcb0 2024-04-25T15:51:40.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.368+0000 7fb81dd9b700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 msgr2=0x7fb8080790e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1601336217 2024-04-25T15:51:40.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.368+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T15:51:40.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.399+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 --> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb820104910 con 0x7fb808076c30 2024-04-25T15:51:40.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.769+0000 7fb81dd9b700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 msgr2=0x7fb8080790e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1601336217 2024-04-25T15:51:40.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:40.769+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T15:51:41.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:41.570+0000 7fb81dd9b700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 msgr2=0x7fb8080790e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1601336217 2024-04-25T15:51:41.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:41.570+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T15:51:43.171 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:43.171+0000 7fb81dd9b700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 msgr2=0x7fb8080790e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1601336217 2024-04-25T15:51:43.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:43.171+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T15:51:46.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:46.374+0000 7fb81dd9b700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 msgr2=0x7fb8080790e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.110:6800/1601336217 2024-04-25T15:51:46.375 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:46.374+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-25T15:51:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:46.661+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 34) v1 ==== 98286+0+0 (secure 0 0 0) 0x7fb8100979d0 con 0x7fb8200fbcb0 2024-04-25T15:51:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:46.662+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 msgr2=0x7fb8080790e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T15:51:46.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:46.662+0000 7fb80effd700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:46.775 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: Active manager daemon smithi110.lvgbkf restarted 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: Activating manager daemon smithi110.lvgbkf 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: osdmap e49: 6 total, 6 up, 6 in 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: mgrmap e34: smithi110.lvgbkf(active, starting, since 0.0064004s), standbys: smithi193.mgoteh 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mgoteh", "id": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:51:46.776 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:51:46.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:51:46.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:51:46.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:51:46.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:51:46.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:51:46.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:51:46.777 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:46 smithi193 ceph-mon[33747]: Manager daemon smithi110.lvgbkf is now available 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: Active manager daemon smithi110.lvgbkf restarted 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: Activating manager daemon smithi110.lvgbkf 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: osdmap e49: 6 total, 6 up, 6 in 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: mgrmap e34: smithi110.lvgbkf(active, starting, since 0.0064004s), standbys: smithi193.mgoteh 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:51:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi110.lvgbkf", "id": "smithi110.lvgbkf"}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr metadata", "who": "smithi193.mgoteh", "id": "smithi193.mgoteh"}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T15:51:47.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:46 smithi110 ceph-mon[90991]: Manager daemon smithi110.lvgbkf is now available 2024-04-25T15:51:47.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.664+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mgrmap(e 35) v1 ==== 98413+0+0 (secure 0 0 0) 0x7fb81001c400 con 0x7fb8200fbcb0 2024-04-25T15:51:47.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.664+0000 7fb80effd700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb80803abe0 0x7fb80803afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:47.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.664+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb820104910 con 0x7fb80803abe0 2024-04-25T15:51:47.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.665+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb80803abe0 0x7fb80803afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:47.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.665+0000 7fb81dd9b700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb80803abe0 0x7fb80803afb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb814009b80 tx=0x7fb81400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:47.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.668+0000 7fb80effd700 1 -- 172.21.15.110:0/2633165751 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fb820104910 con 0x7fb80803abe0 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb80803abe0 msgr2=0x7fb80803afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb80803abe0 0x7fb80803afb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb814009b80 tx=0x7fb81400b040 comp rx=0 tx=0).stop 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 msgr2=0x7fb82010d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 0x7fb82010d970 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb810013870 tx=0x7fb810009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 shutdown_connections 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1601336217,v1:172.21.15.110:6801/1601336217] conn(0x7fb808076c30 0x7fb8080790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb80803abe0 0x7fb80803afb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb8200fbcb0 0x7fb82010d970 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 --2- 172.21.15.110:0/2633165751 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8200fcf60 0x7fb82010deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:47.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 >> 172.21.15.110:0/2633165751 conn(0x7fb8200f7430 msgr2=0x7fb820065b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:47.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 shutdown_connections 2024-04-25T15:51:47.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.672+0000 7fb824e10700 1 -- 172.21.15.110:0/2633165751 wait complete. 2024-04-25T15:51:47.684 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:51:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 -- 172.21.15.110:0/303234009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 msgr2=0x7fafb00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 --2- 172.21.15.110:0/303234009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 0x7fafb00fd3d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fafa0004940 tx=0x7fafa0009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 -- 172.21.15.110:0/303234009 shutdown_connections 2024-04-25T15:51:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 --2- 172.21.15.110:0/303234009 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 0x7fafb00fd3d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 --2- 172.21.15.110:0/303234009 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb00fbcb0 0x7fafb00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 -- 172.21.15.110:0/303234009 >> 172.21.15.110:0/303234009 conn(0x7fafb00f7430 msgr2=0x7fafb00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 -- 172.21.15.110:0/303234009 shutdown_connections 2024-04-25T15:51:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 -- 172.21.15.110:0/303234009 wait complete. 2024-04-25T15:51:47.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 Processor -- start 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.778+0000 7fafb7e72700 1 -- start start 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb7e72700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb00fbcb0 0x7fafb010c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb7e72700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 0x7fafb010ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb7e72700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb010d440 con 0x7fafb00fcf60 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb7e72700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb010dc90 con 0x7fafb00fbcb0 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb540d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 0x7fafb010ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb540d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 0x7fafb010ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41000/0 (socket says 172.21.15.110:41000) 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb540d700 1 -- 172.21.15.110:0/3253187297 learned_addr learned my addr 172.21.15.110:0/3253187297 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb5c0e700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb00fbcb0 0x7fafb010c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb540d700 1 -- 172.21.15.110:0/3253187297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb00fbcb0 msgr2=0x7fafb010c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb540d700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb00fbcb0 0x7fafb010c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb540d700 1 -- 172.21.15.110:0/3253187297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa0005040 con 0x7fafb00fcf60 2024-04-25T15:51:47.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb540d700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 0x7fafb010ce40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fafa000b3b0 tx=0x7fafa0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafa67fc700 1 -- 172.21.15.110:0/3253187297 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa000e040 con 0x7fafb00fcf60 2024-04-25T15:51:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb010df10 con 0x7fafb00fcf60 2024-04-25T15:51:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.779+0000 7fafa67fc700 1 -- 172.21.15.110:0/3253187297 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fafa000bbf0 con 0x7fafb00fcf60 2024-04-25T15:51:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.780+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb010e450 con 0x7fafb00fcf60 2024-04-25T15:51:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.780+0000 7fafa67fc700 1 -- 172.21.15.110:0/3253187297 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafa001c630 con 0x7fafb00fcf60 2024-04-25T15:51:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.781+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb005f5c0 con 0x7fafb00fcf60 2024-04-25T15:51:47.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.781+0000 7fafa67fc700 1 -- 172.21.15.110:0/3253187297 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 35) v1 ==== 98413+0+0 (secure 0 0 0) 0x7fafa000bd60 con 0x7fafb00fcf60 2024-04-25T15:51:47.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.782+0000 7fafa67fc700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf9c078a90 0x7faf9c07af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:47.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.782+0000 7fafa67fc700 1 -- 172.21.15.110:0/3253187297 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fafa0098a80 con 0x7fafb00fcf60 2024-04-25T15:51:47.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.782+0000 7fafb5c0e700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf9c078a90 0x7faf9c07af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:47.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.782+0000 7fafb5c0e700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf9c078a90 0x7faf9c07af40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fafac004f40 tx=0x7fafac00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:47.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:47.787+0000 7fafa67fc700 1 -- 172.21.15.110:0/3253187297 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fafa005d9d0 con 0x7fafb00fcf60 2024-04-25T15:51:47.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:47.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:51:47.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: Standby manager daemon smithi193.mgoteh restarted 2024-04-25T15:51:47.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: Standby manager daemon smithi193.mgoteh started 2024-04-25T15:51:47.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:51:47.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/3497299534' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:51:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/3497299534' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:51:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/3497299534' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:51:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.? 172.21.15.193:0/3497299534' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:51:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:51:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:47 smithi193 ceph-mon[33747]: mgrmap e35: smithi110.lvgbkf(active, since 1.01081s), standbys: smithi193.mgoteh 2024-04-25T15:51:48.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.016+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafb010e890 con 0x7faf9c078a90 2024-04-25T15:51:48.017 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: Standby manager daemon smithi193.mgoteh restarted 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: Standby manager daemon smithi193.mgoteh started 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/mirror_snapshot_schedule"}]: dispatch 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.? 172.21.15.193:0/3497299534' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/crt"}]: dispatch 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.? 172.21.15.193:0/3497299534' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.? 172.21.15.193:0/3497299534' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi193.mgoteh/key"}]: dispatch 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.? 172.21.15.193:0/3497299534' entity='mgr.smithi193.mgoteh' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi110.lvgbkf/trash_purge_schedule"}]: dispatch 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:48.018 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:47 smithi110 ceph-mon[90991]: mgrmap e35: smithi110.lvgbkf(active, since 1.01081s), standbys: smithi193.mgoteh 2024-04-25T15:51:48.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.018+0000 7fafa67fc700 1 -- 172.21.15.110:0/3253187297 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7fafb010e890 con 0x7faf9c078a90 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf9c078a90 msgr2=0x7faf9c07af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf9c078a90 0x7faf9c07af40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fafac004f40 tx=0x7fafac00b040 comp rx=0 tx=0).stop 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 msgr2=0x7fafb010ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 0x7fafb010ce40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fafa000b3b0 tx=0x7fafa0009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 shutdown_connections 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fafb00fbcb0 0x7fafb010c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf9c078a90 0x7faf9c07af40 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 --2- 172.21.15.110:0/3253187297 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fafb00fcf60 0x7fafb010ce40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 >> 172.21.15.110:0/3253187297 conn(0x7fafb00f7430 msgr2=0x7fafb00f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 shutdown_connections 2024-04-25T15:51:48.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.020+0000 7fafb7e72700 1 -- 172.21.15.110:0/3253187297 wait complete. 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.121+0000 7f7930643700 1 -- 172.21.15.110:0/2039238048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fce60 msgr2=0x7f79280fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.121+0000 7f7930643700 1 --2- 172.21.15.110:0/2039238048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fce60 0x7f79280fd270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7924004940 tx=0x7f7924009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.121+0000 7f7930643700 1 -- 172.21.15.110:0/2039238048 shutdown_connections 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.121+0000 7f7930643700 1 --2- 172.21.15.110:0/2039238048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79280fdfd0 0x7f79280fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.121+0000 7f7930643700 1 --2- 172.21.15.110:0/2039238048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fce60 0x7f79280fd270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.121+0000 7f7930643700 1 -- 172.21.15.110:0/2039238048 >> 172.21.15.110:0/2039238048 conn(0x7f79280f8540 msgr2=0x7f79280fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f7930643700 1 -- 172.21.15.110:0/2039238048 shutdown_connections 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f7930643700 1 -- 172.21.15.110:0/2039238048 wait complete. 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f7930643700 1 Processor -- start 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f7930643700 1 -- start start 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f7930643700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79280fce60 0x7f792810d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f7930643700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fdfd0 0x7f792810de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f7930643700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792810e410 con 0x7f79280fdfd0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f7930643700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792810e580 con 0x7f79280fce60 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f792dbde700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fdfd0 0x7f792810de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f792dbde700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fdfd0 0x7f792810de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41018/0 (socket says 172.21.15.110:41018) 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f792dbde700 1 -- 172.21.15.110:0/1531018667 learned_addr learned my addr 172.21.15.110:0/1531018667 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.122+0000 7f792e3df700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79280fce60 0x7f792810d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f792dbde700 1 -- 172.21.15.110:0/1531018667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79280fce60 msgr2=0x7f792810d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f792dbde700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79280fce60 0x7f792810d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f792dbde700 1 -- 172.21.15.110:0/1531018667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7924005040 con 0x7f79280fdfd0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f792dbde700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fdfd0 0x7f792810de10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f791800eb70 tx=0x7f791800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f791effd700 1 -- 172.21.15.110:0/1531018667 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7918007cc0 con 0x7f79280fdfd0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7928112f80 con 0x7f79280fdfd0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f791effd700 1 -- 172.21.15.110:0/1531018667 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f791800f040 con 0x7f79280fdfd0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f791effd700 1 -- 172.21.15.110:0/1531018667 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f791800ca90 con 0x7f79280fdfd0 2024-04-25T15:51:48.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.123+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7928113490 con 0x7f79280fdfd0 2024-04-25T15:51:48.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.124+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792804c710 con 0x7f79280fdfd0 2024-04-25T15:51:48.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.125+0000 7f791effd700 1 -- 172.21.15.110:0/1531018667 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 35) v1 ==== 98413+0+0 (secure 0 0 0) 0x7f791800cbf0 con 0x7f79280fdfd0 2024-04-25T15:51:48.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.125+0000 7f791effd700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7914076880 0x7f7914078d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.126 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.125+0000 7f791effd700 1 -- 172.21.15.110:0/1531018667 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f79180979e0 con 0x7f79280fdfd0 2024-04-25T15:51:48.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.125+0000 7f792e3df700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7914076880 0x7f7914078d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.127 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.126+0000 7f792e3df700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7914076880 0x7f7914078d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7924009b30 tx=0x7f7924009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:48.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.130+0000 7f791effd700 1 -- 172.21.15.110:0/1531018667 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f791805c8b0 con 0x7f79280fdfd0 2024-04-25T15:51:48.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.343+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f79281022b0 con 0x7f7914076880 2024-04-25T15:51:48.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.354+0000 7f791effd700 1 -- 172.21.15.110:0/1531018667 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f79281022b0 con 0x7f7914076880 2024-04-25T15:51:48.355 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:51:48.355 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (2m) 16s ago 10m 14.8M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (10m) 16s ago 10m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (9m) 20s ago 9m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (111s) 16s ago 10m 80.0M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (5m) 16s ago 5m 17.9M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (5m) 16s ago 5m 169M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (5m) 20s ago 5m 13.6M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (5m) 20s ago 5m 15.8M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (96s) 16s ago 12m 562M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (24s) 20s ago 9m 130M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (12m) 16s ago 12m 53.4M 2048M 17.2.7-902-gc61472c8 0de407c1b3af e6fee4ab4b51 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (9m) 20s ago 9m 46.2M 2048M 17.2.7-902-gc61472c8 0de407c1b3af 2f882196f427 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (2m) 16s ago 10m 13.9M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (103s) 20s ago 9m 15.4M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (9m) 16s ago 8m 204M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (8m) 16s ago 8m 216M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:51:48.356 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (8m) 16s ago 8m 176M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:51:48.357 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (7m) 20s ago 7m 239M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:51:48.357 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (7m) 20s ago 7m 206M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:51:48.357 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (6m) 20s ago 6m 186M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:51:48.357 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (30s) 16s ago 10m 47.1M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:51:48.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7914076880 msgr2=0x7f7914078d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7914076880 0x7f7914078d30 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7924009b30 tx=0x7f7924009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:48.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fdfd0 msgr2=0x7f792810de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fdfd0 0x7f792810de10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f791800eb70 tx=0x7f791800b040 comp rx=0 tx=0).stop 2024-04-25T15:51:48.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 shutdown_connections 2024-04-25T15:51:48.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79280fce60 0x7f792810d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7914076880 0x7f7914078d30 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 --2- 172.21.15.110:0/1531018667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79280fdfd0 0x7f792810de10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 >> 172.21.15.110:0/1531018667 conn(0x7f79280f8540 msgr2=0x7f79280f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:48.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 shutdown_connections 2024-04-25T15:51:48.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.357+0000 7f7930643700 1 -- 172.21.15.110:0/1531018667 wait complete. 2024-04-25T15:51:48.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 -- 172.21.15.110:0/1591461530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 msgr2=0x7f10480fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 --2- 172.21.15.110:0/1591461530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 0x7f10480fe440 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f103c004940 tx=0x7f103c009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:48.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 -- 172.21.15.110:0/1591461530 shutdown_connections 2024-04-25T15:51:48.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 --2- 172.21.15.110:0/1591461530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 0x7f10480fe440 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 --2- 172.21.15.110:0/1591461530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10480fce60 0x7f10480fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 -- 172.21.15.110:0/1591461530 >> 172.21.15.110:0/1591461530 conn(0x7f10480f8520 msgr2=0x7f10480fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:48.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 -- 172.21.15.110:0/1591461530 shutdown_connections 2024-04-25T15:51:48.466 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 -- 172.21.15.110:0/1591461530 wait complete. 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 Processor -- start 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.465+0000 7f104d761700 1 -- start start 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f104d761700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10480fce60 0x7f104810fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f104d761700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 0x7f1048110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f104d761700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1048110650 con 0x7f10480fdfd0 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f104d761700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10481107c0 con 0x7f10480fce60 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f10467fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 0x7f1048110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f10467fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 0x7f1048110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41044/0 (socket says 172.21.15.110:41044) 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f10467fc700 1 -- 172.21.15.110:0/3394306904 learned_addr learned my addr 172.21.15.110:0/3394306904 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f1046ffd700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10480fce60 0x7f104810fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f10467fc700 1 -- 172.21.15.110:0/3394306904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10480fce60 msgr2=0x7f104810fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f10467fc700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10480fce60 0x7f104810fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f10467fc700 1 -- 172.21.15.110:0/3394306904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c005040 con 0x7f10480fdfd0 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f10467fc700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 0x7f1048110050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f103c00b3b0 tx=0x7f103c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:48.468 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f102f7fe700 1 -- 172.21.15.110:0/3394306904 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c00e040 con 0x7f10480fdfd0 2024-04-25T15:51:48.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10481151c0 con 0x7f10480fdfd0 2024-04-25T15:51:48.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.466+0000 7f102f7fe700 1 -- 172.21.15.110:0/3394306904 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f103c00bbf0 con 0x7f10480fdfd0 2024-04-25T15:51:48.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.467+0000 7f102f7fe700 1 -- 172.21.15.110:0/3394306904 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f103c01c630 con 0x7f10480fdfd0 2024-04-25T15:51:48.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.467+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1048115780 con 0x7f10480fdfd0 2024-04-25T15:51:48.469 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.468+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1048101960 con 0x7f10480fdfd0 2024-04-25T15:51:48.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.468+0000 7f102f7fe700 1 -- 172.21.15.110:0/3394306904 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 35) v1 ==== 98413+0+0 (secure 0 0 0) 0x7f103c00bd60 con 0x7f10480fdfd0 2024-04-25T15:51:48.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.468+0000 7f102f7fe700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1030076880 0x7f1030078d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.468+0000 7f1046ffd700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1030076880 0x7f1030078d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.468+0000 7f102f7fe700 1 -- 172.21.15.110:0/3394306904 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f103c098a30 con 0x7f10480fdfd0 2024-04-25T15:51:48.470 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.469+0000 7f1046ffd700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1030076880 0x7f1030078d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1038004f40 tx=0x7f103800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:48.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.473+0000 7f102f7fe700 1 -- 172.21.15.110:0/3394306904 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f103c09ae60 con 0x7f10480fdfd0 2024-04-25T15:51:48.722 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.721+0000 7f102f7fe700 1 -- 172.21.15.110:0/3394306904 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mgrmap(e 36) v1 ==== 98709+0+0 (secure 0 0 0) 0x7f103c013980 con 0x7f10480fdfd0 2024-04-25T15:51:48.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.777+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f104802b2a0 con 0x7f10480fdfd0 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.778+0000 7f102f7fe700 1 -- 172.21.15.110:0/3394306904 <== mon.0 v2:172.21.15.110:3300/0 8 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+843 (secure 0 0 0) 0x7f103c0957d0 con 0x7f10480fdfd0 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:51:48.779 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:51:48.780 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:51:48.780 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:51:48.780 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:51:48.780 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:51:48.780 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 11, 2024-04-25T15:51:48.780 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 3 2024-04-25T15:51:48.780 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:51:48.780 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:51:48.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1030076880 msgr2=0x7f1030078d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1030076880 0x7f1030078d30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1038004f40 tx=0x7f103800b040 comp rx=0 tx=0).stop 2024-04-25T15:51:48.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 msgr2=0x7f1048110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 0x7f1048110050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f103c00b3b0 tx=0x7f103c009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:48.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 shutdown_connections 2024-04-25T15:51:48.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10480fce60 0x7f104810fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1030076880 0x7f1030078d30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 --2- 172.21.15.110:0/3394306904 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f10480fdfd0 0x7f1048110050 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 >> 172.21.15.110:0/3394306904 conn(0x7f10480f8520 msgr2=0x7f10480f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:48.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 shutdown_connections 2024-04-25T15:51:48.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.780+0000 7f104d761700 1 -- 172.21.15.110:0/3394306904 wait complete. 2024-04-25T15:51:48.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.883+0000 7f24ff3b6700 1 -- 172.21.15.110:0/1284531324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80ffee0 msgr2=0x7f24f8102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.883+0000 7f24ff3b6700 1 --2- 172.21.15.110:0/1284531324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80ffee0 0x7f24f8102300 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f24e8004940 tx=0x7f24e8009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 -- 172.21.15.110:0/1284531324 shutdown_connections 2024-04-25T15:51:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 --2- 172.21.15.110:0/1284531324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80ffee0 0x7f24f8102300 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 --2- 172.21.15.110:0/1284531324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24f80662c0 0x7f24f80ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 -- 172.21.15.110:0/1284531324 >> 172.21.15.110:0/1284531324 conn(0x7f24f80f7430 msgr2=0x7f24f80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 -- 172.21.15.110:0/1284531324 shutdown_connections 2024-04-25T15:51:48.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 -- 172.21.15.110:0/1284531324 wait complete. 2024-04-25T15:51:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 Processor -- start 2024-04-25T15:51:48.887 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 -- start start 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80662c0 0x7f24f810dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24f80ffee0 0x7f24f810dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f810e680 con 0x7f24f80662c0 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.884+0000 7f24ff3b6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f810ee20 con 0x7f24f80ffee0 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24fd152700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80662c0 0x7f24f810dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24fd152700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80662c0 0x7f24f810dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41098/0 (socket says 172.21.15.110:41098) 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24fd152700 1 -- 172.21.15.110:0/2448528050 learned_addr learned my addr 172.21.15.110:0/2448528050 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24fc951700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24f80ffee0 0x7f24f810dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24fd152700 1 -- 172.21.15.110:0/2448528050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24f80ffee0 msgr2=0x7f24f810dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24fd152700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24f80ffee0 0x7f24f810dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24fd152700 1 -- 172.21.15.110:0/2448528050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e8005040 con 0x7f24f80662c0 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24fd152700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80662c0 0x7f24f810dab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b790 tx=0x7f24f4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24edffb700 1 -- 172.21.15.110:0/2448528050 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24f400f6f0 con 0x7f24f80662c0 2024-04-25T15:51:48.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24edffb700 1 -- 172.21.15.110:0/2448528050 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24f400fd20 con 0x7f24f80662c0 2024-04-25T15:51:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f810f100 con 0x7f24f80662c0 2024-04-25T15:51:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24edffb700 1 -- 172.21.15.110:0/2448528050 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24f40103a0 con 0x7f24f80662c0 2024-04-25T15:51:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.885+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f810f6c0 con 0x7f24f80662c0 2024-04-25T15:51:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.886+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f805f5c0 con 0x7f24f80662c0 2024-04-25T15:51:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.887+0000 7f24edffb700 1 -- 172.21.15.110:0/2448528050 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 36) v1 ==== 98709+0+0 (secure 0 0 0) 0x7f24f4011400 con 0x7f24f80662c0 2024-04-25T15:51:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.887+0000 7f24edffb700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f24e4076c10 0x7f24e40790c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:48.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.887+0000 7f24fc951700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f24e4076c10 0x7f24e40790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:48.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.887+0000 7f24edffb700 1 -- 172.21.15.110:0/2448528050 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f24f40980f0 con 0x7f24f80662c0 2024-04-25T15:51:48.890 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.888+0000 7f24fc951700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f24e4076c10 0x7f24e40790c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f24e800b970 tx=0x7f24e8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:48.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:48.892+0000 7f24edffb700 1 -- 172.21.15.110:0/2448528050 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f24f4060cc0 con 0x7f24f80662c0 2024-04-25T15:51:48.908 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:48 smithi110 ceph-mon[90991]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:48.909 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:48 smithi110 ceph-mon[90991]: pgmap v3: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:51:48.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:48 smithi193 ceph-mon[33747]: from='client.34108 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:48.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:48 smithi193 ceph-mon[33747]: pgmap v3: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:51:49.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.141+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f24f804bed0 con 0x7f24f80662c0 2024-04-25T15:51:49.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.141+0000 7f24edffb700 1 -- 172.21.15.110:0/2448528050 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f24f4060660 con 0x7f24f80662c0 2024-04-25T15:51:49.142 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:51:49.142 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:51:49.142 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:51:49.142 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:51:49.142 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:51:49.142 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:51:49.143 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:51:49.144 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:49.145 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:51:49.145 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:51:49.145 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:51:49.145 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:51:49.145 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:49.145 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:49.145 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:51:49.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f24e4076c10 msgr2=0x7f24e40790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f24e4076c10 0x7f24e40790c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f24e800b970 tx=0x7f24e8009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80662c0 msgr2=0x7f24f810dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80662c0 0x7f24f810dab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b790 tx=0x7f24f4009220 comp rx=0 tx=0).stop 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 shutdown_connections 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f24e4076c10 0x7f24e40790c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f24f80662c0 0x7f24f810dab0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 --2- 172.21.15.110:0/2448528050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f24f80ffee0 0x7f24f810dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.144+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 >> 172.21.15.110:0/2448528050 conn(0x7f24f80f7430 msgr2=0x7f24f80f9810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.145+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 shutdown_connections 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.145+0000 7f24ff3b6700 1 -- 172.21.15.110:0/2448528050 wait complete. 2024-04-25T15:51:49.146 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:51:49.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.252+0000 7f6223f0a700 1 -- 172.21.15.110:0/701511924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 msgr2=0x7f621c102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.252+0000 7f6223f0a700 1 --2- 172.21.15.110:0/701511924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 0x7f621c102290 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f620c004940 tx=0x7f620c009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:49.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.252+0000 7f6223f0a700 1 -- 172.21.15.110:0/701511924 shutdown_connections 2024-04-25T15:51:49.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.252+0000 7f6223f0a700 1 --2- 172.21.15.110:0/701511924 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 0x7f621c102290 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.252+0000 7f6223f0a700 1 --2- 172.21.15.110:0/701511924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c066e70 0x7f621c0ff930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.253 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.252+0000 7f6223f0a700 1 -- 172.21.15.110:0/701511924 >> 172.21.15.110:0/701511924 conn(0x7f621c0f73a0 msgr2=0x7f621c0f97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:49.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.252+0000 7f6223f0a700 1 -- 172.21.15.110:0/701511924 shutdown_connections 2024-04-25T15:51:49.254 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.252+0000 7f6223f0a700 1 -- 172.21.15.110:0/701511924 wait complete. 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f6223f0a700 1 Processor -- start 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f6223f0a700 1 -- start start 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f6223f0a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c066e70 0x7f621c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f6223f0a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 0x7f621c10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f6223f0a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c10e520 con 0x7f621c0ffe70 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f6223f0a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621c10ecd0 con 0x7f621c066e70 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f62214a5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 0x7f621c10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f62214a5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 0x7f621c10df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41116/0 (socket says 172.21.15.110:41116) 2024-04-25T15:51:49.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f6221ca6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c066e70 0x7f621c10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f62214a5700 1 -- 172.21.15.110:0/1212409548 learned_addr learned my addr 172.21.15.110:0/1212409548 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f62214a5700 1 -- 172.21.15.110:0/1212409548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c066e70 msgr2=0x7f621c10d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.253+0000 7f62214a5700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c066e70 0x7f621c10d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.254+0000 7f62214a5700 1 -- 172.21.15.110:0/1212409548 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f620c005040 con 0x7f621c0ffe70 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.254+0000 7f6221ca6700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c066e70 0x7f621c10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.254+0000 7f62214a5700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 0x7f621c10df20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f620c009150 tx=0x7f620c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.254+0000 7f62127fc700 1 -- 172.21.15.110:0/1212409548 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f620c00e040 con 0x7f621c0ffe70 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.254+0000 7f62127fc700 1 -- 172.21.15.110:0/1212409548 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f620c004290 con 0x7f621c0ffe70 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.254+0000 7f62127fc700 1 -- 172.21.15.110:0/1212409548 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f620c01c630 con 0x7f621c0ffe70 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.254+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621c10ef50 con 0x7f621c0ffe70 2024-04-25T15:51:49.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.254+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f621c10f490 con 0x7f621c0ffe70 2024-04-25T15:51:49.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.255+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f621c0fb040 con 0x7f621c0ffe70 2024-04-25T15:51:49.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.256+0000 7f62127fc700 1 -- 172.21.15.110:0/1212409548 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 36) v1 ==== 98709+0+0 (secure 0 0 0) 0x7f620c00bbf0 con 0x7f621c0ffe70 2024-04-25T15:51:49.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.256+0000 7f62127fc700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6208076c20 0x7f62080790d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:49.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.256+0000 7f62127fc700 1 -- 172.21.15.110:0/1212409548 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f620c098fe0 con 0x7f621c0ffe70 2024-04-25T15:51:49.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.256+0000 7f6221ca6700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6208076c20 0x7f62080790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:49.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.256+0000 7f6221ca6700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6208076c20 0x7f62080790d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6218009fd0 tx=0x7f621800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:49.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.261+0000 7f62127fc700 1 -- 172.21.15.110:0/1212409548 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f620c061aa0 con 0x7f621c0ffe70 2024-04-25T15:51:49.486 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.485+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f621c10f7f0 con 0x7f6208076c20 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.486+0000 7f62127fc700 1 -- 172.21.15.110:0/1212409548 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+337 (secure 0 0 0) 0x7f621c10f7f0 con 0x7f6208076c20 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "2/21 daemons upgraded", 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:51:49.487 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6208076c20 msgr2=0x7f62080790d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6208076c20 0x7f62080790d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6218009fd0 tx=0x7f621800b040 comp rx=0 tx=0).stop 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 msgr2=0x7f621c10df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 0x7f621c10df20 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f620c009150 tx=0x7f620c009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 shutdown_connections 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f621c066e70 0x7f621c10d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6208076c20 0x7f62080790d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 --2- 172.21.15.110:0/1212409548 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f621c0ffe70 0x7f621c10df20 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 >> 172.21.15.110:0/1212409548 conn(0x7f621c0f73a0 msgr2=0x7f621c0f77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:49.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 shutdown_connections 2024-04-25T15:51:49.490 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.488+0000 7f6223f0a700 1 -- 172.21.15.110:0/1212409548 wait complete. 2024-04-25T15:51:49.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.591+0000 7fb380b32700 1 -- 172.21.15.110:0/1472304739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fcf70 msgr2=0x7fb37c0fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.591+0000 7fb380b32700 1 --2- 172.21.15.110:0/1472304739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fcf70 0x7fb37c0fd3e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb364004940 tx=0x7fb364009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.591+0000 7fb380b32700 1 -- 172.21.15.110:0/1472304739 shutdown_connections 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.591+0000 7fb380b32700 1 --2- 172.21.15.110:0/1472304739 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fcf70 0x7fb37c0fd3e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.591+0000 7fb380b32700 1 --2- 172.21.15.110:0/1472304739 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb37c0fbcc0 0x7fb37c0fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.591+0000 7fb380b32700 1 -- 172.21.15.110:0/1472304739 >> 172.21.15.110:0/1472304739 conn(0x7fb37c0f7440 msgr2=0x7fb37c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.591+0000 7fb380b32700 1 -- 172.21.15.110:0/1472304739 shutdown_connections 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.591+0000 7fb380b32700 1 -- 172.21.15.110:0/1472304739 wait complete. 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb380b32700 1 Processor -- start 2024-04-25T15:51:49.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb380b32700 1 -- start start 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb380b32700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fbcc0 0x7fb37c10c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb380b32700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb37c0fcf70 0x7fb37c10cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb380b32700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c10d380 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb380b32700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c10d4f0 con 0x7fb37c0fcf70 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb37a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fbcc0 0x7fb37c10c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb37a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fbcc0 0x7fb37c10c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53718/0 (socket says 172.21.15.110:53718) 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb37a59c700 1 -- 172.21.15.110:0/317263586 learned_addr learned my addr 172.21.15.110:0/317263586 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.592+0000 7fb379d9b700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb37c0fcf70 0x7fb37c10cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb37a59c700 1 -- 172.21.15.110:0/317263586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb37c0fcf70 msgr2=0x7fb37c10cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb37a59c700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb37c0fcf70 0x7fb37c10cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb37a59c700 1 -- 172.21.15.110:0/317263586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb364005040 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb37a59c700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fbcc0 0x7fb37c10c840 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb37400cac0 tx=0x7fb374007ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb372ffd700 1 -- 172.21.15.110:0/317263586 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb37400a980 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb372ffd700 1 -- 172.21.15.110:0/317263586 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb374012440 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb372ffd700 1 -- 172.21.15.110:0/317263586 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3740115e0 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37c111f50 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.593+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37c112480 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.594+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb37c04c710 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.595+0000 7fb372ffd700 1 -- 172.21.15.110:0/317263586 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 36) v1 ==== 98709+0+0 (secure 0 0 0) 0x7fb3740125b0 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.595+0000 7fb372ffd700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb368076c20 0x7fb3680790d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:51:49.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.595+0000 7fb379d9b700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb368076c20 0x7fb3680790d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:51:49.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.595+0000 7fb372ffd700 1 -- 172.21.15.110:0/317263586 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb374098a70 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.596+0000 7fb379d9b700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb368076c20 0x7fb3680790d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb36400b3b0 tx=0x7fb364009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:51:49.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.600+0000 7fb372ffd700 1 -- 172.21.15.110:0/317263586 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb3740615c0 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: [25/Apr/2024:15:51:47] ENGINE Bus STARTING 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: [25/Apr/2024:15:51:48] ENGINE Serving on https://172.21.15.110:7150 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: [25/Apr/2024:15:51:48] ENGINE Serving on http://172.21.15.110:8765 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: [25/Apr/2024:15:51:48] ENGINE Bus STARTED 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: pgmap v4: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: mgrmap e36: smithi110.lvgbkf(active, since 2s), standbys: smithi193.mgoteh 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3394306904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:51:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:49 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2448528050' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:51:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.884+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb37c1126d0 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.885+0000 7fb372ffd700 1 -- 172.21.15.110:0/317263586 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb374012860 con 0x7fb37c0fbcc0 2024-04-25T15:51:49.886 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:51:49.886 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:51:49.886 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb368076c20 msgr2=0x7fb3680790d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb368076c20 0x7fb3680790d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb36400b3b0 tx=0x7fb364009e30 comp rx=0 tx=0).stop 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fbcc0 msgr2=0x7fb37c10c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fbcc0 0x7fb37c10c840 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fb37400cac0 tx=0x7fb374007ee0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 shutdown_connections 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb368076c20 0x7fb3680790d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb37c0fbcc0 0x7fb37c10c840 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 --2- 172.21.15.110:0/317263586 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb37c0fcf70 0x7fb37c10cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 >> 172.21.15.110:0/317263586 conn(0x7fb37c0f7440 msgr2=0x7fb37c0f8b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:51:49.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 shutdown_connections 2024-04-25T15:51:49.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:51:49.887+0000 7fb380b32700 1 -- 172.21.15.110:0/317263586 wait complete. 2024-04-25T15:51:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: [25/Apr/2024:15:51:47] ENGINE Bus STARTING 2024-04-25T15:51:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: [25/Apr/2024:15:51:48] ENGINE Serving on https://172.21.15.110:7150 2024-04-25T15:51:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: [25/Apr/2024:15:51:48] ENGINE Serving on http://172.21.15.110:8765 2024-04-25T15:51:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: [25/Apr/2024:15:51:48] ENGINE Bus STARTED 2024-04-25T15:51:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: pgmap v4: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:51:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: mgrmap e36: smithi110.lvgbkf(active, since 2s), standbys: smithi193.mgoteh 2024-04-25T15:51:49.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/3394306904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:51:49.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:49 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/2448528050' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:51:50.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:50 smithi193 ceph-mon[33747]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:50.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:50 smithi193 ceph-mon[33747]: from='client.? 172.21.15.110:0/317263586' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:51:50.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:50 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:50.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:50 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:51.092 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:50 smithi110 ceph-mon[90991]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:51:51.107 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:50 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/317263586' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:51:51.107 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:50 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:51.107 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:50 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:51.915 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:51 smithi110 ceph-mon[90991]: pgmap v5: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:51:51.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:51 smithi110 ceph-mon[90991]: mgrmap e37: smithi110.lvgbkf(active, since 4s), standbys: smithi193.mgoteh 2024-04-25T15:51:51.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:51 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:51.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:51 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:51.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:51 smithi193 ceph-mon[33747]: pgmap v5: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:51:51.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:51 smithi193 ceph-mon[33747]: mgrmap e37: smithi110.lvgbkf(active, since 4s), standbys: smithi193.mgoteh 2024-04-25T15:51:51.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:51 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:51.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:51 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:53 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:53 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:53 smithi193 ceph-mon[33747]: pgmap v6: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:51:53.862 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:53 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:53.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:53 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:53.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:53 smithi110 ceph-mon[90991]: pgmap v6: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:51:54.700 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:54 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:54.700 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:54 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:54.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:54.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:56.032 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:55 smithi110 ceph-mon[90991]: pgmap v7: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T15:51:56.032 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:56.032 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:56.032 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:56.032 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:51:56.032 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:55 smithi110 ceph-mon[90991]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:51:56.032 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:55 smithi110 ceph-mon[90991]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:51:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:55 smithi193 ceph-mon[33747]: pgmap v7: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 25 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-25T15:51:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:55 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:55 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:55 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:55 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:51:56.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:55 smithi193 ceph-mon[33747]: Updating smithi110:/etc/ceph/ceph.conf 2024-04-25T15:51:56.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:55 smithi193 ceph-mon[33747]: Updating smithi193:/etc/ceph/ceph.conf 2024-04-25T15:51:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:51:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:51:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:51:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:51:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:51:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:51:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:57.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:51:57.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: Updating smithi193:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: Updating smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: Updating smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:51:57.116 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:56 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-25T15:51:58.059 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:57 smithi193 ceph-mon[33747]: pgmap v8: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T15:51:58.059 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:57 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:58.059 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:57 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:51:58.059 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:57 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:51:58.059 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:57 smithi193 ceph-mon[33747]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:57 smithi110 ceph-mon[90991]: pgmap v8: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-25T15:51:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:51:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:51:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:51:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:51:59.056 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:58 smithi193 ceph-mon[33747]: Upgrade: Updating mon.smithi193 2024-04-25T15:51:59.056 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:58 smithi193 ceph-mon[33747]: Deploying daemon mon.smithi193 on smithi193 2024-04-25T15:51:59.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:58 smithi110 ceph-mon[90991]: Upgrade: Updating mon.smithi193 2024-04-25T15:51:59.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:51:58 smithi110 ceph-mon[90991]: Deploying daemon mon.smithi193 on smithi193 2024-04-25T15:51:59.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:59 smithi193 systemd[1]: Stopping Ceph mon.smithi193 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:51:59.978 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:59 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193[33733]: 2024-04-25T15:51:59.518+0000 7f8bbf9a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi193 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:51:59.979 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:59 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193[33733]: 2024-04-25T15:51:59.518+0000 7f8bbf9a7700 -1 mon.smithi193@1(peon) e2 *** Got Signal Terminated *** 2024-04-25T15:51:59.979 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:51:59 smithi193 podman[79909]: 2024-04-25 15:51:59.713287776 +0000 UTC m=+0.326998089 container died 2f882196f42727bb651809c73a17477fab996cd962802bc1750d0a180230193f (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=quincy-c61472c, io.buildah.version=1.33.5) 2024-04-25T15:52:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 podman[79909]: 2024-04-25 15:52:00.077548072 +0000 UTC m=+0.691258381 container cleanup 2f882196f42727bb651809c73a17477fab996cd962802bc1750d0a180230193f (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T15:52:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 bash[79909]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193 2024-04-25T15:52:00.646 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 podman[79923]: 2024-04-25 15:52:00.337406363 +0000 UTC m=+0.616093189 container remove 2f882196f42727bb651809c73a17477fab996cd962802bc1750d0a180230193f (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T15:52:00.646 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi193.service: Deactivated successfully. 2024-04-25T15:52:00.646 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 systemd[1]: Stopped Ceph mon.smithi193 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:52:00.646 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi193.service: Consumed 6.063s CPU time. 2024-04-25T15:52:00.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 systemd[1]: Starting Ceph mon.smithi193 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:52:00.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 podman[80052]: 2024-04-25 15:52:00.763993796 +0000 UTC m=+0.018195552 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:52:00.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:00 smithi193 podman[80052]: 2024-04-25 15:52:00.880175575 +0000 UTC m=+0.134377316 container create 82fe48722b1a7d2d91f9d08d07189afa0f9a8005f9912488334fb9c15d000956 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 podman[80052]: 2024-04-25 15:52:01.099342475 +0000 UTC m=+0.353544216 container init 82fe48722b1a7d2d91f9d08d07189afa0f9a8005f9912488334fb9c15d000956 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 podman[80052]: 2024-04-25 15:52:01.102336401 +0000 UTC m=+0.356538149 container start 82fe48722b1a7d2d91f9d08d07189afa0f9a8005f9912488334fb9c15d000956 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev), process ceph-mon, pid 2 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: pidfile_write: ignore empty --pid-file 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: load: jerasure load: lrc 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: RocksDB version: 7.9.2 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Git sha 0 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Compile date 2024-04-25 02:57:34 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: DB SUMMARY 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: DB Session ID: R90K7N27HH5MPX3VA9H3 2024-04-25T15:52:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: CURRENT file: CURRENT 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: MANIFEST file: MANIFEST-000009 size: 1084 Bytes 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi193/store.db dir, Total Num: 1, files: 000027.sst 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi193/store.db: 000025.log size: 314790 ; 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.error_if_exists: 0 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.create_if_missing: 0 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.paranoid_checks: 1 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.env: 0x56443597e500 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.info_log: 0x5644379e5e80 2024-04-25T15:52:01.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.statistics: (nil) 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.use_fsync: 0 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_log_file_size: 0 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.allow_fallocate: 1 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T15:52:01.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.use_direct_reads: 0 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.db_log_dir: 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.wal_dir: 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.write_buffer_manager: 0x5644379b68c0 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T15:52:01.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.unordered_write: 0 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.row_cache: None 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.wal_filter: None 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.two_write_queues: 0 2024-04-25T15:52:01.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.wal_compression: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.atomic_flush: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.log_readahead_size: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_background_jobs: 2 2024-04-25T15:52:01.496 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_background_compactions: -1 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_subcompactions: 1 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_open_files: -1 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T15:52:01.497 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_background_flushes: -1 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Compression algorithms supported: 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: kZSTD supported: 0 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: kXpressCompression supported: 0 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: kZlibCompression supported: 1 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: kSnappyCompression supported: 1 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: kLZ4Compression supported: 1 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: kBZip2Compression supported: 0 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000009 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T15:52:01.498 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.merge_operator: 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_filter: None 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564437aec060) 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks: 1 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: pin_top_level_index_and_filter: 1 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_type: 0 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_index_type: 0 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_shortening: 1 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T15:52:01.499 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: checksum: 4 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: no_block_cache: 0 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache: 0x5644379e91f0 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_name: BinnedLRUCache 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_options: 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: capacity : 536870912 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_shard_bits : 4 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: strict_capacity_limit : 0 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: high_pri_pool_ratio: 0.000 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_cache_compressed: (nil) 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: persistent_cache: (nil) 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size: 4096 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_size_deviation: 10 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_restart_interval: 16 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: index_block_restart_interval: 1 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: metadata_block_size: 4096 2024-04-25T15:52:01.500 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: partition_filters: 0 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: use_delta_encoding: 1 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: filter_policy: bloomfilter 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: whole_key_filtering: 1 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: verify_compression: 0 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: read_amp_bytes_per_bit: 0 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: format_version: 5 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_index_compression: 1 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: block_align: 0 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_auto_readahead_size: 262144 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: prepopulate_block_cache: 0 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: initial_auto_readahead_size: 8192 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression: NoCompression 2024-04-25T15:52:01.501 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.num_levels: 7 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T15:52:01.502 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T15:52:01.503 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T15:52:01.504 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T15:52:01.505 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.inplace_update_support: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.bloom_locality: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.max_successive_merges: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.ttl: 2592000 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T15:52:01.506 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.enable_blob_files: false 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.min_blob_size: 0 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [table/block_based/block_based_table_reader.cc:721] At least one SST file opened without unique ID to verify: 27.sst 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi193/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 29, last_sequence is 12321, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 39b53a5f-e672-4996-9c42-bfd0a2cde4d3 2024-04-25T15:52:01.507 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714060321143231, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714060321144739, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 198163, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12326, "largest_seqno": 12468, "table_properties": {"data_size": 196404, "index_size": 529, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 197, "raw_key_size": 1929, "raw_average_key_size": 26, "raw_value_size": 194593, "raw_average_value_size": 2629, "num_data_blocks": 22, "num_entries": 74, "num_filter_entries": 74, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714060321, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "39b53a5f-e672-4996-9c42-bfd0a2cde4d3", "db_session_id": "R90K7N27HH5MPX3VA9H3", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714060321144822, "job": 1, "event": "recovery_finished"} 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi193/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5644379cdc00 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: DB pointer 0x56443c7c0000 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** DB Stats ** 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T15:52:01.508 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L0 1/0 193.52 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 223.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: L6 1/0 7.94 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Sum 2/0 8.13 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 223.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 223.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** Compaction Stats [default] ** 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T15:52:01.509 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 223.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Cumulative compaction: 0.00 GB write, 52.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Interval compaction: 0.00 GB write, 52.18 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Block cache BinnedLRUCache@0x5644379e91f0#2 capacity: 512.00 MB usage: 0.88 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-04-25T15:52:01.510 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.24 KB,4.61936e-05%) IndexBlock(1,0.64 KB,0.00012219%) Misc(1,0.00 KB,0%) 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: starting mon.smithi193 rank 1 at public addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] at bind addrs [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi193 fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: mon.smithi193@-1(???) e2 preinit fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: mon.smithi193@-1(???).mds e13 new map 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: mon.smithi193@-1(???).mds e13 print_map 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: e13 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: legacy client fscid: 1 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Filesystem 'cephfs' (1) 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: fs_name cephfs 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: epoch 13 2024-04-25T15:52:01.511 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: tableserver 0 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: root 0 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: session_timeout 60 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: session_autoclose 300 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_file_size 1099511627776 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_xattr_size 65536 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: required_client_features {} 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: last_failure 0 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: last_failure_osd_epoch 0 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: max_mds 1 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: in 0 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: up {0=14468} 2024-04-25T15:52:01.512 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: failed 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: damaged 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: stopped 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: data_pools [3] 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: metadata_pool 2 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: inline_data enabled 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: balancer 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: bal_rank_mask -1 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: standby_count_wanted 1 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: qdb_cluster leader: 0 members: 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: Standby daemons: 2024-04-25T15:52:01.513 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout: [mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: mon.smithi193@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: mon.smithi193@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: mon.smithi193@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: mon.smithi193@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 ceph-mon[80071]: mon.smithi193@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 bash[80052]: 82fe48722b1a7d2d91f9d08d07189afa0f9a8005f9912488334fb9c15d000956 2024-04-25T15:52:01.514 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:01 smithi193 systemd[1]: Started Ceph mon.smithi193 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:52:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:02 smithi110 ceph-mon[90991]: pgmap v9: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-25T15:52:02.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:02 smithi193 ceph-mon[80071]: pgmap v9: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 16 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-25T15:52:03.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: pgmap v11: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T15:52:03.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: osdmap e49: 6 total, 6 up, 6 in 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: mgrmap e37: smithi110.lvgbkf(active, since 16s), standbys: smithi193.mgoteh 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: mon.smithi110 calling monitor election 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: mon.smithi193 calling monitor election 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: mon.smithi110 is new leader, mons smithi110,smithi193 in quorum (ranks 0,1) 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: monmap epoch 3 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: last_changed 2024-04-25T15:52:02.716234+0000 2024-04-25T15:52:03.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: created 2024-04-25T15:39:28.650237+0000 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: min_mon_release 19 (squid) 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: election_strategy: 1 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.smithi110 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: osdmap e49: 6 total, 6 up, 6 in 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: mgrmap e37: smithi110.lvgbkf(active, since 16s), standbys: smithi193.mgoteh 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:03.865 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:52:03.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: pgmap v11: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T15:52:03.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: osdmap e49: 6 total, 6 up, 6 in 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: mgrmap e37: smithi110.lvgbkf(active, since 16s), standbys: smithi193.mgoteh 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: mon.smithi110 calling monitor election 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: mon.smithi193 calling monitor election 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi110"}]: dispatch 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mon metadata", "id": "smithi193"}]: dispatch 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: mon.smithi110 is new leader, mons smithi110,smithi193 in quorum (ranks 0,1) 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: monmap epoch 3 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: fsid e79cc6fc-0319-11ef-bc93-c7b262605968 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: last_changed 2024-04-25T15:52:02.716234+0000 2024-04-25T15:52:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: created 2024-04-25T15:39:28.650237+0000 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: min_mon_release 19 (squid) 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: election_strategy: 1 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: 0: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.smithi110 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: 1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.smithi193 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: osdmap e49: 6 total, 6 up, 6 in 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: mgrmap e37: smithi110.lvgbkf(active, since 16s), standbys: smithi193.mgoteh 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:03.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:52:05.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:05 smithi193 ceph-mon[80071]: pgmap v12: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T15:52:05.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:05.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:05 smithi110 ceph-mon[90991]: pgmap v12: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 15 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T15:52:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:07.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:07 smithi193 ceph-mon[80071]: pgmap v13: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:07.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:07.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:07 smithi110 ceph-mon[90991]: pgmap v13: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:09 smithi110 ceph-mon[90991]: pgmap v14: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:10.165 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:09 smithi193 ceph-mon[80071]: pgmap v14: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:11.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:11.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:11.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:11.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:52:11.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:11.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: Reconfiguring mon.smithi110 (monmap changed)... 2024-04-25T15:52:11.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:52:11.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:52:11.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:11.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: Reconfiguring daemon mon.smithi110 on smithi110 2024-04-25T15:52:11.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:11 smithi193 ceph-mon[80071]: pgmap v15: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:11.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:11.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:11.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:11.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:52:11.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:11.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: Reconfiguring mon.smithi110 (monmap changed)... 2024-04-25T15:52:11.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:52:11.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:52:11.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:11.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: Reconfiguring daemon mon.smithi110 on smithi110 2024-04-25T15:52:11.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:11 smithi110 ceph-mon[90991]: pgmap v15: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:13 smithi110 ceph-mon[90991]: Reconfiguring mgr.smithi110.lvgbkf (monmap changed)... 2024-04-25T15:52:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.lvgbkf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:52:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:52:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:13 smithi110 ceph-mon[90991]: Reconfiguring daemon mgr.smithi110.lvgbkf on smithi110 2024-04-25T15:52:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:13 smithi110 ceph-mon[90991]: pgmap v16: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:13 smithi193 ceph-mon[80071]: Reconfiguring mgr.smithi110.lvgbkf (monmap changed)... 2024-04-25T15:52:13.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi110.lvgbkf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:52:13.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:52:13.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:13.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:13 smithi193 ceph-mon[80071]: Reconfiguring daemon mgr.smithi110.lvgbkf on smithi110 2024-04-25T15:52:13.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:13 smithi193 ceph-mon[80071]: pgmap v16: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:14 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:14 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:14 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:52:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:14 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:14 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:14 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:14 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:52:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:14 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:15 smithi110 ceph-mon[90991]: Reconfiguring crash.smithi110 (monmap changed)... 2024-04-25T15:52:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:15 smithi110 ceph-mon[90991]: Reconfiguring daemon crash.smithi110 on smithi110 2024-04-25T15:52:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:15 smithi110 ceph-mon[90991]: pgmap v17: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:15 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:15 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:15 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T15:52:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:15 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:15 smithi193 ceph-mon[80071]: Reconfiguring crash.smithi110 (monmap changed)... 2024-04-25T15:52:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:15 smithi193 ceph-mon[80071]: Reconfiguring daemon crash.smithi110 on smithi110 2024-04-25T15:52:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:15 smithi193 ceph-mon[80071]: pgmap v17: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:15 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:15 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:15 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T15:52:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:15 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:16 smithi110 ceph-mon[90991]: Reconfiguring osd.0 (monmap changed)... 2024-04-25T15:52:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:16 smithi110 ceph-mon[90991]: Reconfiguring daemon osd.0 on smithi110 2024-04-25T15:52:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:52:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:16 smithi193 ceph-mon[80071]: Reconfiguring osd.0 (monmap changed)... 2024-04-25T15:52:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:16 smithi193 ceph-mon[80071]: Reconfiguring daemon osd.0 on smithi110 2024-04-25T15:52:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:52:18.094 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:17 smithi110 ceph-mon[90991]: pgmap v18: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:18.094 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:18.094 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:18.094 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T15:52:18.094 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:17 smithi193 ceph-mon[80071]: pgmap v18: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T15:52:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:18.931 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:18 smithi110 ceph-mon[90991]: Reconfiguring osd.1 (monmap changed)... 2024-04-25T15:52:18.931 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:18 smithi110 ceph-mon[90991]: Reconfiguring daemon osd.1 on smithi110 2024-04-25T15:52:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:18 smithi193 ceph-mon[80071]: Reconfiguring osd.1 (monmap changed)... 2024-04-25T15:52:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:18 smithi193 ceph-mon[80071]: Reconfiguring daemon osd.1 on smithi110 2024-04-25T15:52:19.926 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:19 smithi110 ceph-mon[90991]: pgmap v19: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:19.926 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:19 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:19.926 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:19 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:19.927 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:19 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T15:52:19.927 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:19 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 -- 172.21.15.110:0/3771971022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc066f30 msgr2=0x7f11fc0ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 --2- 172.21.15.110:0/3771971022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc066f30 0x7f11fc0ff9b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f11f0004940 tx=0x7f11f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 -- 172.21.15.110:0/3771971022 shutdown_connections 2024-04-25T15:52:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 --2- 172.21.15.110:0/3771971022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11fc0ffef0 0x7f11fc102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 --2- 172.21.15.110:0/3771971022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc066f30 0x7f11fc0ff9b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 -- 172.21.15.110:0/3771971022 >> 172.21.15.110:0/3771971022 conn(0x7f11fc0f7440 msgr2=0x7f11fc0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 -- 172.21.15.110:0/3771971022 shutdown_connections 2024-04-25T15:52:19.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 -- 172.21.15.110:0/3771971022 wait complete. 2024-04-25T15:52:19.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 Processor -- start 2024-04-25T15:52:19.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 -- start start 2024-04-25T15:52:19.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11fc066f30 0x7f11fc0fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:19.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc0ffef0 0x7f11fc0fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:19.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0fd800 con 0x7f11fc0ffef0 2024-04-25T15:52:19.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.992+0000 7f1202b67700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0fbdf0 con 0x7f11fc066f30 2024-04-25T15:52:19.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc0ffef0 0x7f11fc0fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11fbfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc0ffef0 0x7f11fc0fb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53054/0 (socket says 172.21.15.110:53054) 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11fbfff700 1 -- 172.21.15.110:0/117164421 learned_addr learned my addr 172.21.15.110:0/117164421 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f1200903700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11fc066f30 0x7f11fc0fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11fbfff700 1 -- 172.21.15.110:0/117164421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11fc066f30 msgr2=0x7f11fc0fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11fbfff700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11fc066f30 0x7f11fc0fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11fbfff700 1 -- 172.21.15.110:0/117164421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0005040 con 0x7f11fc0ffef0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11fbfff700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc0ffef0 0x7f11fc0fb8b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f11e8002a80 tx=0x7f11e80046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11f97fa700 1 -- 172.21.15.110:0/117164421 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11e8007cc0 con 0x7f11fc0ffef0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11f97fa700 1 -- 172.21.15.110:0/117164421 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11e800f040 con 0x7f11fc0ffef0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc0fc0a0 con 0x7f11fc0ffef0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f11f97fa700 1 -- 172.21.15.110:0/117164421 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11e800dac0 con 0x7f11fc0ffef0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.993+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc0fc660 con 0x7f11fc0ffef0 2024-04-25T15:52:19.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.994+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc04c710 con 0x7f11fc0ffef0 2024-04-25T15:52:19.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.995+0000 7f11f97fa700 1 -- 172.21.15.110:0/117164421 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7f11e800dc20 con 0x7f11fc0ffef0 2024-04-25T15:52:19.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.995+0000 7f11f97fa700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f11ec076ca0 0x7f11ec079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:19.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.995+0000 7f11f97fa700 1 -- 172.21.15.110:0/117164421 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f11e80981e0 con 0x7f11fc0ffef0 2024-04-25T15:52:19.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.995+0000 7f1200903700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f11ec076ca0 0x7f11ec079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:19.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:19.996+0000 7f1200903700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f11ec076ca0 0x7f11ec079150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f11f0004890 tx=0x7f11f0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:20.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.001+0000 7f11f97fa700 1 -- 172.21.15.110:0/117164421 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f11e8060b40 con 0x7f11fc0ffef0 2024-04-25T15:52:20.225 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.224+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11fc0f8f30 con 0x7f11ec076ca0 2024-04-25T15:52:20.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.225+0000 7f11f97fa700 1 -- 172.21.15.110:0/117164421 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f11fc0f8f30 con 0x7f11ec076ca0 2024-04-25T15:52:20.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f11ec076ca0 msgr2=0x7f11ec079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f11ec076ca0 0x7f11ec079150 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f11f0004890 tx=0x7f11f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc0ffef0 msgr2=0x7f11fc0fb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc0ffef0 0x7f11fc0fb8b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f11e8002a80 tx=0x7f11e80046b0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 shutdown_connections 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11fc066f30 0x7f11fc0fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f11ec076ca0 0x7f11ec079150 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 --2- 172.21.15.110:0/117164421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f11fc0ffef0 0x7f11fc0fb8b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 >> 172.21.15.110:0/117164421 conn(0x7f11fc0f7440 msgr2=0x7f11fc0f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.227+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 shutdown_connections 2024-04-25T15:52:20.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.228+0000 7f1202b67700 1 -- 172.21.15.110:0/117164421 wait complete. 2024-04-25T15:52:20.238 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:52:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:19 smithi193 ceph-mon[80071]: pgmap v19: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:19 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:19 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:19 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T15:52:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:19 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:20.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.329+0000 7f63b6c5e700 1 -- 172.21.15.110:0/3265722238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 msgr2=0x7f63b00fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.329+0000 7f63b6c5e700 1 --2- 172.21.15.110:0/3265722238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 0x7f63b00fe460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f639c004940 tx=0x7f639c009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:20.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.329+0000 7f63b6c5e700 1 -- 172.21.15.110:0/3265722238 shutdown_connections 2024-04-25T15:52:20.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.329+0000 7f63b6c5e700 1 --2- 172.21.15.110:0/3265722238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 0x7f63b00fe460 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.329+0000 7f63b6c5e700 1 --2- 172.21.15.110:0/3265722238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b00fce80 0x7f63b00fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 -- 172.21.15.110:0/3265722238 >> 172.21.15.110:0/3265722238 conn(0x7f63b00f8520 msgr2=0x7f63b00fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:20.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 -- 172.21.15.110:0/3265722238 shutdown_connections 2024-04-25T15:52:20.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 -- 172.21.15.110:0/3265722238 wait complete. 2024-04-25T15:52:20.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 Processor -- start 2024-04-25T15:52:20.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 -- start start 2024-04-25T15:52:20.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b00fce80 0x7f63b0071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 0x7f63b00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b0070860 con 0x7f63b00fdff0 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63b6c5e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b00709d0 con 0x7f63b00fce80 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.330+0000 7f63affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 0x7f63b00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 0x7f63b00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53072/0 (socket says 172.21.15.110:53072) 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63affff700 1 -- 172.21.15.110:0/1142067120 learned_addr learned my addr 172.21.15.110:0/1142067120 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63b49fa700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b00fce80 0x7f63b0071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63affff700 1 -- 172.21.15.110:0/1142067120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b00fce80 msgr2=0x7f63b0071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63affff700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b00fce80 0x7f63b0071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63affff700 1 -- 172.21.15.110:0/1142067120 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639c005040 con 0x7f63b00fdff0 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63affff700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 0x7f63b00702b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f639c009150 tx=0x7f639c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63ad7fa700 1 -- 172.21.15.110:0/1142067120 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639c00e040 con 0x7f63b00fdff0 2024-04-25T15:52:20.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63ad7fa700 1 -- 172.21.15.110:0/1142067120 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f639c00bcc0 con 0x7f63b00fdff0 2024-04-25T15:52:20.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b0070c50 con 0x7f63b00fdff0 2024-04-25T15:52:20.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63ad7fa700 1 -- 172.21.15.110:0/1142067120 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f639c01c630 con 0x7f63b00fdff0 2024-04-25T15:52:20.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.331+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b0115a30 con 0x7f63b00fdff0 2024-04-25T15:52:20.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.332+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b005f5c0 con 0x7f63b00fdff0 2024-04-25T15:52:20.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.333+0000 7f63ad7fa700 1 -- 172.21.15.110:0/1142067120 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7f639c013420 con 0x7f63b00fdff0 2024-04-25T15:52:20.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.333+0000 7f63ad7fa700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f63a0076ca0 0x7f63a0079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:20.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.333+0000 7f63ad7fa700 1 -- 172.21.15.110:0/1142067120 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f639c0991f0 con 0x7f63b00fdff0 2024-04-25T15:52:20.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.333+0000 7f63b49fa700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f63a0076ca0 0x7f63a0079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:20.335 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.334+0000 7f63b49fa700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f63a0076ca0 0x7f63a0079150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f63a4007fd0 tx=0x7f63a4009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:20.340 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.339+0000 7f63ad7fa700 1 -- 172.21.15.110:0/1142067120 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f639c061bd0 con 0x7f63b00fdff0 2024-04-25T15:52:20.565 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.564+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63b0102330 con 0x7f63a0076ca0 2024-04-25T15:52:20.566 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.566+0000 7f63ad7fa700 1 -- 172.21.15.110:0/1142067120 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f63b0102330 con 0x7f63a0076ca0 2024-04-25T15:52:20.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f63a0076ca0 msgr2=0x7f63a0079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f63a0076ca0 0x7f63a0079150 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f63a4007fd0 tx=0x7f63a4009040 comp rx=0 tx=0).stop 2024-04-25T15:52:20.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 msgr2=0x7f63b00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 0x7f63b00702b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f639c009150 tx=0x7f639c009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:20.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 shutdown_connections 2024-04-25T15:52:20.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63b00fce80 0x7f63b0071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f63a0076ca0 0x7f63a0079150 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 --2- 172.21.15.110:0/1142067120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f63b00fdff0 0x7f63b00702b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 >> 172.21.15.110:0/1142067120 conn(0x7f63b00f8520 msgr2=0x7f63b00f9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:20.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 shutdown_connections 2024-04-25T15:52:20.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.568+0000 7f63b6c5e700 1 -- 172.21.15.110:0/1142067120 wait complete. 2024-04-25T15:52:20.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 -- 172.21.15.110:0/3934035240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fed30 msgr2=0x7fcb000fae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 --2- 172.21.15.110:0/3934035240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fed30 0x7fcb000fae70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4004940 tx=0x7fcaf4009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:20.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 -- 172.21.15.110:0/3934035240 shutdown_connections 2024-04-25T15:52:20.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 --2- 172.21.15.110:0/3934035240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fed30 0x7fcb000fae70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 --2- 172.21.15.110:0/3934035240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb000fe3e0 0x7fcb000fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 -- 172.21.15.110:0/3934035240 >> 172.21.15.110:0/3934035240 conn(0x7fcb000f6b80 msgr2=0x7fcb000f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:20.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 -- 172.21.15.110:0/3934035240 shutdown_connections 2024-04-25T15:52:20.676 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 -- 172.21.15.110:0/3934035240 wait complete. 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.675+0000 7fcb049bc700 1 Processor -- start 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcb049bc700 1 -- start start 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcb049bc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fe3e0 0x7fcb0010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcb049bc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb000fed30 0x7fcb0010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcb049bc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0010e5c0 con 0x7fcb000fe3e0 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcb049bc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0010edb0 con 0x7fcb000fed30 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcafe59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fe3e0 0x7fcb0010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcafe59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fe3e0 0x7fcb0010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53090/0 (socket says 172.21.15.110:53090) 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcafe59c700 1 -- 172.21.15.110:0/1796328221 learned_addr learned my addr 172.21.15.110:0/1796328221 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcafdd9b700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb000fed30 0x7fcb0010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:20.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcafe59c700 1 -- 172.21.15.110:0/1796328221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb000fed30 msgr2=0x7fcb0010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcafe59c700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb000fed30 0x7fcb0010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcafe59c700 1 -- 172.21.15.110:0/1796328221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf4005040 con 0x7fcb000fe3e0 2024-04-25T15:52:20.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.676+0000 7fcafe59c700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fe3e0 0x7fcb0010d9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000b790 tx=0x7fcaf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:20.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.677+0000 7fcaeeffd700 1 -- 172.21.15.110:0/1796328221 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf000f770 con 0x7fcb000fe3e0 2024-04-25T15:52:20.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.677+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0010f090 con 0x7fcb000fe3e0 2024-04-25T15:52:20.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.677+0000 7fcaeeffd700 1 -- 172.21.15.110:0/1796328221 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcaf000fda0 con 0x7fcb000fe3e0 2024-04-25T15:52:20.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.677+0000 7fcaeeffd700 1 -- 172.21.15.110:0/1796328221 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf000d310 con 0x7fcb000fe3e0 2024-04-25T15:52:20.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.677+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0010f650 con 0x7fcb000fe3e0 2024-04-25T15:52:20.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.678+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb0005f5c0 con 0x7fcb000fe3e0 2024-04-25T15:52:20.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.678+0000 7fcaeeffd700 1 -- 172.21.15.110:0/1796328221 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7fcaf0010420 con 0x7fcb000fe3e0 2024-04-25T15:52:20.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.679+0000 7fcaeeffd700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcae8076ca0 0x7fcae8079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:20.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.679+0000 7fcaeeffd700 1 -- 172.21.15.110:0/1796328221 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fcaf0098530 con 0x7fcb000fe3e0 2024-04-25T15:52:20.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.679+0000 7fcafdd9b700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcae8076ca0 0x7fcae8079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:20.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.679+0000 7fcafdd9b700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcae8076ca0 0x7fcae8079150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400b3b0 tx=0x7fcaf4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:20.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.684+0000 7fcaeeffd700 1 -- 172.21.15.110:0/1796328221 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcaf0060f80 con 0x7fcb000fe3e0 2024-04-25T15:52:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:20 smithi110 ceph-mon[90991]: Reconfiguring osd.2 (monmap changed)... 2024-04-25T15:52:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:20 smithi110 ceph-mon[90991]: Reconfiguring daemon osd.2 on smithi110 2024-04-25T15:52:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:20.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.mvvbtv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:52:20.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:20.911 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.911+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb0010f930 con 0x7fcae8076ca0 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.921+0000 7fcaeeffd700 1 -- 172.21.15.110:0/1796328221 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcb0010f930 con 0x7fcae8076ca0 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (2m) 28s ago 11m 14.9M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (11m) 28s ago 11m 6656k - 17.2.7-902-gc61472c8 0de407c1b3af 176b433d2923 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (10m) 15s ago 10m 6665k - 17.2.7-902-gc61472c8 0de407c1b3af 22a16c0eeb6f 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (2m) 28s ago 11m 77.8M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (5m) 28s ago 5m 18.8M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (5m) 28s ago 5m 153M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (5m) 15s ago 5m 14.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (5m) 15s ago 5m 16.5M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (2m) 28s ago 12m 541M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:52:20.922 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (56s) 15s ago 10m 517M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (43s) 28s ago 12m 95.2M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (20s) 15s ago 10m 85.9M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (3m) 28s ago 11m 15.5M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (2m) 15s ago 10m 15.7M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (9m) 28s ago 9m 204M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (9m) 28s ago 9m 217M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (8m) 28s ago 8m 177M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (8m) 15s ago 8m 238M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (7m) 15s ago 7m 206M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (7m) 15s ago 7m 188M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:52:20.923 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (63s) 28s ago 10m 51.8M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:52:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcae8076ca0 msgr2=0x7fcae8079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcae8076ca0 0x7fcae8079150 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcaf400b3b0 tx=0x7fcaf4009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fe3e0 msgr2=0x7fcb0010d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fe3e0 0x7fcb0010d9f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000b790 tx=0x7fcaf0009250 comp rx=0 tx=0).stop 2024-04-25T15:52:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 shutdown_connections 2024-04-25T15:52:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcae8076ca0 0x7fcae8079150 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcb000fe3e0 0x7fcb0010d9f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 --2- 172.21.15.110:0/1796328221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb000fed30 0x7fcb0010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 >> 172.21.15.110:0/1796328221 conn(0x7fcb000f6b80 msgr2=0x7fcb000f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 shutdown_connections 2024-04-25T15:52:20.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:20.923+0000 7fcb049bc700 1 -- 172.21.15.110:0/1796328221 wait complete. 2024-04-25T15:52:21.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 -- 172.21.15.110:0/2085896339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fbcb0 msgr2=0x7f05200fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 --2- 172.21.15.110:0/2085896339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fbcb0 0x7f05200fc0c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0514004940 tx=0x7f0514009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:21.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 -- 172.21.15.110:0/2085896339 shutdown_connections 2024-04-25T15:52:21.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 --2- 172.21.15.110:0/2085896339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05200fcf60 0x7f05200fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 --2- 172.21.15.110:0/2085896339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fbcb0 0x7f05200fc0c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 -- 172.21.15.110:0/2085896339 >> 172.21.15.110:0/2085896339 conn(0x7f05200f7430 msgr2=0x7f05200f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:21.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 -- 172.21.15.110:0/2085896339 shutdown_connections 2024-04-25T15:52:21.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 -- 172.21.15.110:0/2085896339 wait complete. 2024-04-25T15:52:21.029 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.027+0000 7f052601b700 1 Processor -- start 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f052601b700 1 -- start start 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f052601b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05200fbcb0 0x7f052010c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f052601b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fcf60 0x7f052010ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f052601b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052010d460 con 0x7f05200fcf60 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f052601b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f052010dcb0 con 0x7f05200fbcb0 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fcf60 0x7f052010ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fcf60 0x7f052010ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53108/0 (socket says 172.21.15.110:53108) 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051effd700 1 -- 172.21.15.110:0/985449970 learned_addr learned my addr 172.21.15.110:0/985449970 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:21.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051f7fe700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05200fbcb0 0x7f052010c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051effd700 1 -- 172.21.15.110:0/985449970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05200fbcb0 msgr2=0x7f052010c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051effd700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05200fbcb0 0x7f052010c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051effd700 1 -- 172.21.15.110:0/985449970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0514005040 con 0x7f05200fcf60 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051f7fe700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05200fbcb0 0x7f052010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.028+0000 7f051effd700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fcf60 0x7f052010ce60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f051000c810 tx=0x7f0510004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.029+0000 7f0507fff700 1 -- 172.21.15.110:0/985449970 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0510012810 con 0x7f05200fcf60 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.029+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f052010df90 con 0x7f05200fcf60 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.029+0000 7f0507fff700 1 -- 172.21.15.110:0/985449970 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0510012e40 con 0x7f05200fcf60 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.029+0000 7f0507fff700 1 -- 172.21.15.110:0/985449970 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f051000a370 con 0x7f05200fcf60 2024-04-25T15:52:21.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.029+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f052010e4d0 con 0x7f05200fcf60 2024-04-25T15:52:21.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.030+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052005f5c0 con 0x7f05200fcf60 2024-04-25T15:52:21.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.030+0000 7f0507fff700 1 -- 172.21.15.110:0/985449970 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7f051000a550 con 0x7f05200fcf60 2024-04-25T15:52:21.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.031+0000 7f0507fff700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0508076ca0 0x7f0508079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.031+0000 7f0507fff700 1 -- 172.21.15.110:0/985449970 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f05100988c0 con 0x7f05200fcf60 2024-04-25T15:52:21.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.031+0000 7f051f7fe700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0508076ca0 0x7f0508079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.031+0000 7f051f7fe700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0508076ca0 0x7f0508079150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0514000c00 tx=0x7f0514009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:21.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.036+0000 7f0507fff700 1 -- 172.21.15.110:0/985449970 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f05100613a0 con 0x7f05200fcf60 2024-04-25T15:52:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:20 smithi193 ceph-mon[80071]: Reconfiguring osd.2 (monmap changed)... 2024-04-25T15:52:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:20 smithi193 ceph-mon[80071]: Reconfiguring daemon osd.2 on smithi110 2024-04-25T15:52:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.mvvbtv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:52:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:21.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.333+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f05201047c0 con 0x7f05200fcf60 2024-04-25T15:52:21.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.333+0000 7f0507fff700 1 -- 172.21.15.110:0/985449970 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f0510060d40 con 0x7f05200fcf60 2024-04-25T15:52:21.334 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:52:21.334 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 4 2024-04-25T15:52:21.335 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:52:21.336 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0508076ca0 msgr2=0x7f0508079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0508076ca0 0x7f0508079150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0514000c00 tx=0x7f0514009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fcf60 msgr2=0x7f052010ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fcf60 0x7f052010ce60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f051000c810 tx=0x7f0510004470 comp rx=0 tx=0).stop 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 shutdown_connections 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05200fbcb0 0x7f052010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0508076ca0 0x7f0508079150 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 --2- 172.21.15.110:0/985449970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f05200fcf60 0x7f052010ce60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 >> 172.21.15.110:0/985449970 conn(0x7f05200f7430 msgr2=0x7f05200f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 shutdown_connections 2024-04-25T15:52:21.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.336+0000 7f052601b700 1 -- 172.21.15.110:0/985449970 wait complete. 2024-04-25T15:52:21.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.438+0000 7fc4237e0700 1 -- 172.21.15.110:0/26791716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c0fff40 msgr2=0x7fc41c102320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.438+0000 7fc4237e0700 1 --2- 172.21.15.110:0/26791716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c0fff40 0x7fc41c102320 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc418004940 tx=0x7fc418009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:21.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.438+0000 7fc4237e0700 1 -- 172.21.15.110:0/26791716 shutdown_connections 2024-04-25T15:52:21.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.438+0000 7fc4237e0700 1 --2- 172.21.15.110:0/26791716 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c0fff40 0x7fc41c102320 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.438+0000 7fc4237e0700 1 --2- 172.21.15.110:0/26791716 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc41c066260 0x7fc41c0ffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.438+0000 7fc4237e0700 1 -- 172.21.15.110:0/26791716 >> 172.21.15.110:0/26791716 conn(0x7fc41c0f7430 msgr2=0x7fc41c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:21.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.438+0000 7fc4237e0700 1 -- 172.21.15.110:0/26791716 shutdown_connections 2024-04-25T15:52:21.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.438+0000 7fc4237e0700 1 -- 172.21.15.110:0/26791716 wait complete. 2024-04-25T15:52:21.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc4237e0700 1 Processor -- start 2024-04-25T15:52:21.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc4237e0700 1 -- start start 2024-04-25T15:52:21.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc4237e0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c066260 0x7fc41c10d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc4237e0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc41c0fff40 0x7fc41c10df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc4237e0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c10e520 con 0x7fc41c066260 2024-04-25T15:52:21.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc4237e0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41c10ed70 con 0x7fc41c0fff40 2024-04-25T15:52:21.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc42157c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c066260 0x7fc41c10d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc42157c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c066260 0x7fc41c10d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53120/0 (socket says 172.21.15.110:53120) 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc42157c700 1 -- 172.21.15.110:0/410553869 learned_addr learned my addr 172.21.15.110:0/410553869 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc420d7b700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc41c0fff40 0x7fc41c10df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc42157c700 1 -- 172.21.15.110:0/410553869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc41c0fff40 msgr2=0x7fc41c10df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc42157c700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc41c0fff40 0x7fc41c10df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc42157c700 1 -- 172.21.15.110:0/410553869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc418005040 con 0x7fc41c066260 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.439+0000 7fc42157c700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c066260 0x7fc41c10d9e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c8b0 tx=0x7fc40c004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.440+0000 7fc411ffb700 1 -- 172.21.15.110:0/410553869 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40c0106f0 con 0x7fc41c066260 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.440+0000 7fc411ffb700 1 -- 172.21.15.110:0/410553869 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc40c010d20 con 0x7fc41c066260 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.440+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41c10f050 con 0x7fc41c066260 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.440+0000 7fc411ffb700 1 -- 172.21.15.110:0/410553869 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc40c012e90 con 0x7fc41c066260 2024-04-25T15:52:21.442 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.440+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41c10f590 con 0x7fc41c066260 2024-04-25T15:52:21.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.441+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41c05f5c0 con 0x7fc41c066260 2024-04-25T15:52:21.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.441+0000 7fc411ffb700 1 -- 172.21.15.110:0/410553869 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7fc40c010850 con 0x7fc41c066260 2024-04-25T15:52:21.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.442+0000 7fc411ffb700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc408076ca0 0x7fc408079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.442+0000 7fc411ffb700 1 -- 172.21.15.110:0/410553869 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc40c0983a0 con 0x7fc41c066260 2024-04-25T15:52:21.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.442+0000 7fc420d7b700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc408076ca0 0x7fc408079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.442+0000 7fc420d7b700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc408076ca0 0x7fc408079150 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc41800b970 tx=0x7fc418009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:21.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.447+0000 7fc411ffb700 1 -- 172.21.15.110:0/410553869 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc40c060e10 con 0x7fc41c066260 2024-04-25T15:52:21.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.704+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc41c105890 con 0x7fc41c066260 2024-04-25T15:52:21.705 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.704+0000 7fc411ffb700 1 -- 172.21.15.110:0/410553869 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7fc40c0607b0 con 0x7fc41c066260 2024-04-25T15:52:21.705 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:52:21.706 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:52:21.707 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:21.708 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc408076ca0 msgr2=0x7fc408079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc408076ca0 0x7fc408079150 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fc41800b970 tx=0x7fc418009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c066260 msgr2=0x7fc41c10d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c066260 0x7fc41c10d9e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c8b0 tx=0x7fc40c004470 comp rx=0 tx=0).stop 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 shutdown_connections 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc408076ca0 0x7fc408079150 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc41c066260 0x7fc41c10d9e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 --2- 172.21.15.110:0/410553869 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc41c0fff40 0x7fc41c10df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 >> 172.21.15.110:0/410553869 conn(0x7fc41c0f7430 msgr2=0x7fc41c0f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 shutdown_connections 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.707+0000 7fc4237e0700 1 -- 172.21.15.110:0/410553869 wait complete. 2024-04-25T15:52:21.709 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:52:21.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.810+0000 7efdf2825700 1 -- 172.21.15.110:0/2523807544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 msgr2=0x7efdec0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.810+0000 7efdf2825700 1 --2- 172.21.15.110:0/2523807544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 0x7efdec0fc0d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efddc004940 tx=0x7efddc009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:21.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.810+0000 7efdf2825700 1 -- 172.21.15.110:0/2523807544 shutdown_connections 2024-04-25T15:52:21.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.810+0000 7efdf2825700 1 --2- 172.21.15.110:0/2523807544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdec0fcf70 0x7efdec0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.810+0000 7efdf2825700 1 --2- 172.21.15.110:0/2523807544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 0x7efdec0fc0d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.810+0000 7efdf2825700 1 -- 172.21.15.110:0/2523807544 >> 172.21.15.110:0/2523807544 conn(0x7efdec0f7440 msgr2=0x7efdec0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:21.811 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.810+0000 7efdf2825700 1 -- 172.21.15.110:0/2523807544 shutdown_connections 2024-04-25T15:52:21.812 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.810+0000 7efdf2825700 1 -- 172.21.15.110:0/2523807544 wait complete. 2024-04-25T15:52:21.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdf2825700 1 Processor -- start 2024-04-25T15:52:21.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdf2825700 1 -- start start 2024-04-25T15:52:21.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdf2825700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 0x7efdec066cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdf2825700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdec0fcf70 0x7efdec112730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdf2825700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec112c70 con 0x7efdec0fbcc0 2024-04-25T15:52:21.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdf2825700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdec112de0 con 0x7efdec0fcf70 2024-04-25T15:52:21.813 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdebfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 0x7efdec066cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdebfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 0x7efdec066cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53138/0 (socket says 172.21.15.110:53138) 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdebfff700 1 -- 172.21.15.110:0/2380046824 learned_addr learned my addr 172.21.15.110:0/2380046824 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.811+0000 7efdeb7fe700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdec0fcf70 0x7efdec112730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efdebfff700 1 -- 172.21.15.110:0/2380046824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdec0fcf70 msgr2=0x7efdec112730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efdebfff700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdec0fcf70 0x7efdec112730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efdebfff700 1 -- 172.21.15.110:0/2380046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efddc005040 con 0x7efdec0fbcc0 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efdebfff700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 0x7efdec066cb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efddc00b3b0 tx=0x7efddc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efde8ff9700 1 -- 172.21.15.110:0/2380046824 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efddc00e040 con 0x7efdec0fbcc0 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdec113060 con 0x7efdec0fbcc0 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efde8ff9700 1 -- 172.21.15.110:0/2380046824 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efddc00bcc0 con 0x7efdec0fbcc0 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efde8ff9700 1 -- 172.21.15.110:0/2380046824 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efddc01c630 con 0x7efdec0fbcc0 2024-04-25T15:52:21.814 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.812+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdec113620 con 0x7efdec0fbcc0 2024-04-25T15:52:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.813+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdec04c710 con 0x7efdec0fbcc0 2024-04-25T15:52:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.814+0000 7efde8ff9700 1 -- 172.21.15.110:0/2380046824 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7efddc013980 con 0x7efdec0fbcc0 2024-04-25T15:52:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.814+0000 7efde8ff9700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efdd4076ca0 0x7efdd4079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:21.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.814+0000 7efde8ff9700 1 -- 172.21.15.110:0/2380046824 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7efddc099310 con 0x7efdec0fbcc0 2024-04-25T15:52:21.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.814+0000 7efdeb7fe700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efdd4076ca0 0x7efdd4079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:21.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.814+0000 7efdeb7fe700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efdd4076ca0 0x7efdd4079150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efde0006fd0 tx=0x7efde0008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:21.819 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:21.819+0000 7efde8ff9700 1 -- 172.21.15.110:0/2380046824 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efddc061bd0 con 0x7efdec0fbcc0 2024-04-25T15:52:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:21 smithi110 ceph-mon[90991]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:21 smithi110 ceph-mon[90991]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:21 smithi110 ceph-mon[90991]: pgmap v20: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:21 smithi110 ceph-mon[90991]: Reconfiguring mds.cephfs.smithi110.mvvbtv (monmap changed)... 2024-04-25T15:52:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:21 smithi110 ceph-mon[90991]: Reconfiguring daemon mds.cephfs.smithi110.mvvbtv on smithi110 2024-04-25T15:52:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:21 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/985449970' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:21.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:21 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/410553869' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:52:22.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.044+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efdec101210 con 0x7efdd4076ca0 2024-04-25T15:52:22.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.045+0000 7efde8ff9700 1 -- 172.21.15.110:0/2380046824 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7efdec101210 con 0x7efdd4076ca0 2024-04-25T15:52:22.046 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:52:22.046 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "4/21 daemons upgraded", 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mon daemons", 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:52:22.047 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:52:22.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efdd4076ca0 msgr2=0x7efdd4079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:22.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efdd4076ca0 0x7efdd4079150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efde0006fd0 tx=0x7efde0008040 comp rx=0 tx=0).stop 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 msgr2=0x7efdec066cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 0x7efdec066cb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7efddc00b3b0 tx=0x7efddc009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 shutdown_connections 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efdd4076ca0 0x7efdd4079150 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdec0fbcc0 0x7efdec066cb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 --2- 172.21.15.110:0/2380046824 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdec0fcf70 0x7efdec112730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.047+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 >> 172.21.15.110:0/2380046824 conn(0x7efdec0f7440 msgr2=0x7efdec0f8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.048+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 shutdown_connections 2024-04-25T15:52:22.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.048+0000 7efdf2825700 1 -- 172.21.15.110:0/2380046824 wait complete. 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.149+0000 7fef356f1700 1 -- 172.21.15.110:0/2539555698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 msgr2=0x7fef300faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.149+0000 7fef356f1700 1 --2- 172.21.15.110:0/2539555698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 0x7fef300faea0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fef24004940 tx=0x7fef24009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.149+0000 7fef356f1700 1 -- 172.21.15.110:0/2539555698 shutdown_connections 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.149+0000 7fef356f1700 1 --2- 172.21.15.110:0/2539555698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 0x7fef300faea0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.149+0000 7fef356f1700 1 --2- 172.21.15.110:0/2539555698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef300fc140 0x7fef300fc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.149+0000 7fef356f1700 1 -- 172.21.15.110:0/2539555698 >> 172.21.15.110:0/2539555698 conn(0x7fef300f6bd0 msgr2=0x7fef300f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.149+0000 7fef356f1700 1 -- 172.21.15.110:0/2539555698 shutdown_connections 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.149+0000 7fef356f1700 1 -- 172.21.15.110:0/2539555698 wait complete. 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef356f1700 1 Processor -- start 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef356f1700 1 -- start start 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef356f1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef300fc140 0x7fef30113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:22.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef356f1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 0x7fef30114430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef356f1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef30114a30 con 0x7fef300fca90 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef356f1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef30114ba0 con 0x7fef300fc140 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef2e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 0x7fef30114430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef2e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 0x7fef30114430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:53160/0 (socket says 172.21.15.110:53160) 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.150+0000 7fef2e7fc700 1 -- 172.21.15.110:0/1902435519 learned_addr learned my addr 172.21.15.110:0/1902435519 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef2e7fc700 1 -- 172.21.15.110:0/1902435519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef300fc140 msgr2=0x7fef30113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef2effd700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef300fc140 0x7fef30113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef2e7fc700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef300fc140 0x7fef30113ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef2e7fc700 1 -- 172.21.15.110:0/1902435519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef24005040 con 0x7fef300fca90 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef2e7fc700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 0x7fef30114430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef2400b3b0 tx=0x7fef24009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef177fe700 1 -- 172.21.15.110:0/1902435519 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef2400e040 con 0x7fef300fca90 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef177fe700 1 -- 172.21.15.110:0/1902435519 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef2400bcc0 con 0x7fef300fca90 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef301195a0 con 0x7fef300fca90 2024-04-25T15:52:22.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef177fe700 1 -- 172.21.15.110:0/1902435519 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef2401c630 con 0x7fef300fca90 2024-04-25T15:52:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.151+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef30119b60 con 0x7fef300fca90 2024-04-25T15:52:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.152+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef3004c710 con 0x7fef300fca90 2024-04-25T15:52:22.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.153+0000 7fef177fe700 1 -- 172.21.15.110:0/1902435519 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7fef24013420 con 0x7fef300fca90 2024-04-25T15:52:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.153+0000 7fef177fe700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef18076ca0 0x7fef18079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.153+0000 7fef177fe700 1 -- 172.21.15.110:0/1902435519 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fef240991a0 con 0x7fef300fca90 2024-04-25T15:52:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.153+0000 7fef2effd700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef18076ca0 0x7fef18079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:22.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.154+0000 7fef2effd700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef18076ca0 0x7fef18079150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fef20004d60 tx=0x7fef2000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:22.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.158+0000 7fef177fe700 1 -- 172.21.15.110:0/1902435519 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fef24061b80 con 0x7fef300fca90 2024-04-25T15:52:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:21 smithi193 ceph-mon[80071]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:21 smithi193 ceph-mon[80071]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:21 smithi193 ceph-mon[80071]: pgmap v20: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:21 smithi193 ceph-mon[80071]: Reconfiguring mds.cephfs.smithi110.mvvbtv (monmap changed)... 2024-04-25T15:52:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:21 smithi193 ceph-mon[80071]: Reconfiguring daemon mds.cephfs.smithi110.mvvbtv on smithi110 2024-04-25T15:52:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:21 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/985449970' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:21 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/410553869' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:52:22.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.452+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fef30119fe0 con 0x7fef300fca90 2024-04-25T15:52:22.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.453+0000 7fef177fe700 1 -- 172.21.15.110:0/1902435519 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fef24061520 con 0x7fef300fca90 2024-04-25T15:52:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:22.454 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:22.454 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef18076ca0 msgr2=0x7fef18079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef18076ca0 0x7fef18079150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fef20004d60 tx=0x7fef2000b040 comp rx=0 tx=0).stop 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 msgr2=0x7fef30114430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 0x7fef30114430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fef2400b3b0 tx=0x7fef24009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 shutdown_connections 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef300fc140 0x7fef30113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef18076ca0 0x7fef18079150 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 --2- 172.21.15.110:0/1902435519 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef300fca90 0x7fef30114430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 >> 172.21.15.110:0/1902435519 conn(0x7fef300f6bd0 msgr2=0x7fef300f82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:22.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 shutdown_connections 2024-04-25T15:52:22.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:22.455+0000 7fef356f1700 1 -- 172.21.15.110:0/1902435519 wait complete. 2024-04-25T15:52:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:22 smithi110 ceph-mon[90991]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:22 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:22 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:22 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1902435519' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:52:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:22 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.kdhjqx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:52:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:22 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:22 smithi193 ceph-mon[80071]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:22 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:22 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:22 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1902435519' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:52:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:22 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.kdhjqx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:52:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:22 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:24.006 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:23 smithi110 ceph-mon[90991]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:24.006 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:23 smithi110 ceph-mon[90991]: Reconfiguring mds.cephfs.smithi110.kdhjqx (monmap changed)... 2024-04-25T15:52:24.006 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:23 smithi110 ceph-mon[90991]: Reconfiguring daemon mds.cephfs.smithi110.kdhjqx on smithi110 2024-04-25T15:52:24.006 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:23 smithi110 ceph-mon[90991]: pgmap v21: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:23 smithi193 ceph-mon[80071]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:23 smithi193 ceph-mon[80071]: Reconfiguring mds.cephfs.smithi110.kdhjqx (monmap changed)... 2024-04-25T15:52:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:23 smithi193 ceph-mon[80071]: Reconfiguring daemon mds.cephfs.smithi110.kdhjqx on smithi110 2024-04-25T15:52:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:23 smithi193 ceph-mon[80071]: pgmap v21: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:25 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:25 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:25 smithi193 ceph-mon[80071]: Reconfiguring crash.smithi193 (monmap changed)... 2024-04-25T15:52:25.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:25 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:52:25.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:25 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:25.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:25 smithi193 ceph-mon[80071]: Reconfiguring daemon crash.smithi193 on smithi193 2024-04-25T15:52:25.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:25 smithi193 ceph-mon[80071]: pgmap v22: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:25 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:25 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:25 smithi110 ceph-mon[90991]: Reconfiguring crash.smithi193 (monmap changed)... 2024-04-25T15:52:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:25 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:52:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:25 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:25 smithi110 ceph-mon[90991]: Reconfiguring daemon crash.smithi193 on smithi193 2024-04-25T15:52:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:25 smithi110 ceph-mon[90991]: pgmap v22: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:26.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:26 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:26.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:26 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:26.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:26 smithi110 ceph-mon[90991]: Reconfiguring mgr.smithi193.mgoteh (monmap changed)... 2024-04-25T15:52:26.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:26 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:52:26.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:26 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:52:26.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:26 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:26.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:26 smithi110 ceph-mon[90991]: Reconfiguring daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:52:26.885 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:26 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:26.885 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:26 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:26.885 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:26 smithi193 ceph-mon[80071]: Reconfiguring mgr.smithi193.mgoteh (monmap changed)... 2024-04-25T15:52:26.885 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:26 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi193.mgoteh", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T15:52:26.886 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:26 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T15:52:26.886 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:26 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:26.886 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:26 smithi193 ceph-mon[80071]: Reconfiguring daemon mgr.smithi193.mgoteh on smithi193 2024-04-25T15:52:27.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:27 smithi193 ceph-mon[80071]: pgmap v23: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:27.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:27 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:27.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:27 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:27.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:27 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:52:27.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:27 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:52:27.938 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:27 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:27 smithi110 ceph-mon[90991]: pgmap v23: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:27 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:27 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:27 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T15:52:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:27 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T15:52:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:27 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:28.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:28 smithi193 ceph-mon[80071]: Reconfiguring mon.smithi193 (monmap changed)... 2024-04-25T15:52:28.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:28 smithi193 ceph-mon[80071]: Reconfiguring daemon mon.smithi193 on smithi193 2024-04-25T15:52:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:28 smithi110 ceph-mon[90991]: Reconfiguring mon.smithi193 (monmap changed)... 2024-04-25T15:52:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:28 smithi110 ceph-mon[90991]: Reconfiguring daemon mon.smithi193 on smithi193 2024-04-25T15:52:29.731 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:29 smithi193 ceph-mon[80071]: pgmap v24: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:29.731 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:29.731 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:29.731 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T15:52:29.731 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:29 smithi110 ceph-mon[90991]: pgmap v24: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T15:52:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:30.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:30 smithi193 ceph-mon[80071]: Reconfiguring osd.3 (monmap changed)... 2024-04-25T15:52:30.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:30 smithi193 ceph-mon[80071]: Reconfiguring daemon osd.3 on smithi193 2024-04-25T15:52:30.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:30 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:30.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:30 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:30.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:30 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T15:52:30.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:30 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:31.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:30 smithi110 ceph-mon[90991]: Reconfiguring osd.3 (monmap changed)... 2024-04-25T15:52:31.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:30 smithi110 ceph-mon[90991]: Reconfiguring daemon osd.3 on smithi193 2024-04-25T15:52:31.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:30 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:31.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:30 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:31.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:30 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T15:52:31.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:30 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:31.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:31 smithi193 ceph-mon[80071]: Reconfiguring osd.4 (monmap changed)... 2024-04-25T15:52:31.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:31 smithi193 ceph-mon[80071]: Reconfiguring daemon osd.4 on smithi193 2024-04-25T15:52:31.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:31 smithi193 ceph-mon[80071]: pgmap v25: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:31 smithi110 ceph-mon[90991]: Reconfiguring osd.4 (monmap changed)... 2024-04-25T15:52:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:31 smithi110 ceph-mon[90991]: Reconfiguring daemon osd.4 on smithi193 2024-04-25T15:52:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:31 smithi110 ceph-mon[90991]: pgmap v25: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:52:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T15:52:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:32.918 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:52:32.918 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:32.918 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:32.919 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T15:52:32.919 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:33.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:33 smithi110 ceph-mon[90991]: Reconfiguring osd.5 (monmap changed)... 2024-04-25T15:52:33.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:33 smithi110 ceph-mon[90991]: Reconfiguring daemon osd.5 on smithi193 2024-04-25T15:52:33.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:33 smithi110 ceph-mon[90991]: pgmap v26: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:33 smithi193 ceph-mon[80071]: Reconfiguring osd.5 (monmap changed)... 2024-04-25T15:52:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:33 smithi193 ceph-mon[80071]: Reconfiguring daemon osd.5 on smithi193 2024-04-25T15:52:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:33 smithi193 ceph-mon[80071]: pgmap v26: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:34.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:34 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:34.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:34 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:34.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:34 smithi193 ceph-mon[80071]: Reconfiguring mds.cephfs.smithi193.ewxwfw (monmap changed)... 2024-04-25T15:52:34.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:34 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ewxwfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:52:34.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:34 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:34.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:34 smithi193 ceph-mon[80071]: Reconfiguring daemon mds.cephfs.smithi193.ewxwfw on smithi193 2024-04-25T15:52:35.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:34 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:35.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:34 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:35.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:34 smithi110 ceph-mon[90991]: Reconfiguring mds.cephfs.smithi193.ewxwfw (monmap changed)... 2024-04-25T15:52:35.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:34 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ewxwfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:52:35.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:34 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:35.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:34 smithi110 ceph-mon[90991]: Reconfiguring daemon mds.cephfs.smithi193.ewxwfw on smithi193 2024-04-25T15:52:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:35 smithi193 ceph-mon[80071]: pgmap v27: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:35.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:35.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:35.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.jtyzhl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:52:35.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:35 smithi110 ceph-mon[90991]: pgmap v27: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.jtyzhl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:52:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:36.836 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:36 smithi193 ceph-mon[80071]: Reconfiguring mds.cephfs.smithi193.jtyzhl (monmap changed)... 2024-04-25T15:52:36.836 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:36 smithi193 ceph-mon[80071]: Reconfiguring daemon mds.cephfs.smithi193.jtyzhl on smithi193 2024-04-25T15:52:36.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:36 smithi110 ceph-mon[90991]: Reconfiguring mds.cephfs.smithi193.jtyzhl (monmap changed)... 2024-04-25T15:52:36.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:36 smithi110 ceph-mon[90991]: Reconfiguring daemon mds.cephfs.smithi193.jtyzhl on smithi193 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: pgmap v28: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi110"}]: dispatch 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi110"}]': finished 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]: dispatch 2024-04-25T15:52:37.802 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]': finished 2024-04-25T15:52:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: pgmap v28: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:38.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:52:38.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:38.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:38.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:38.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi110"}]: dispatch 2024-04-25T15:52:38.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi110"}]': finished 2024-04-25T15:52:38.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]: dispatch 2024-04-25T15:52:38.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi193"}]': finished 2024-04-25T15:52:39.104 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:38 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all mon 2024-04-25T15:52:39.104 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:39.105 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:52:39.105 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:38 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all mon 2024-04-25T15:52:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi110", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:52:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:40.021 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:39 smithi110 ceph-mon[90991]: Upgrade: Updating crash.smithi110 (1/2) 2024-04-25T15:52:40.022 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:39 smithi110 ceph-mon[90991]: Deploying daemon crash.smithi110 on smithi110 2024-04-25T15:52:40.022 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:39 smithi110 ceph-mon[90991]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:39 smithi193 ceph-mon[80071]: Upgrade: Updating crash.smithi110 (1/2) 2024-04-25T15:52:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:39 smithi193 ceph-mon[80071]: Deploying daemon crash.smithi110 on smithi110 2024-04-25T15:52:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:39 smithi193 ceph-mon[80071]: pgmap v29: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:41.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:41 smithi193 ceph-mon[80071]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:42.019 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:41 smithi110 ceph-mon[90991]: pgmap v30: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:43.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:43 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:43.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:43 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:43.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:43 smithi193 ceph-mon[80071]: pgmap v31: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:43.503 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:43 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:43.503 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:43 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:43.503 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:43 smithi110 ceph-mon[90991]: pgmap v31: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:44 smithi193 ceph-mon[80071]: Upgrade: Updating crash.smithi193 (2/2) 2024-04-25T15:52:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:44 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:44 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:52:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:44 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:44.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:44 smithi193 ceph-mon[80071]: Deploying daemon crash.smithi193 on smithi193 2024-04-25T15:52:44.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:44 smithi193 ceph-mon[80071]: pgmap v32: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:45.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:44 smithi110 ceph-mon[90991]: Upgrade: Updating crash.smithi193 (2/2) 2024-04-25T15:52:45.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:44 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:45.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:44 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi193", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-25T15:52:45.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:44 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:45.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:44 smithi110 ceph-mon[90991]: Deploying daemon crash.smithi193 on smithi193 2024-04-25T15:52:45.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:44 smithi110 ceph-mon[90991]: pgmap v32: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:47.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:47 smithi193 ceph-mon[80071]: pgmap v33: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:52:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:47.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:52:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:47 smithi110 ceph-mon[90991]: pgmap v33: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:52:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:52:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:49 smithi193 ceph-mon[80071]: pgmap v34: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:50.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:49 smithi110 ceph-mon[90991]: pgmap v34: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:51 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:51 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:51 smithi193 ceph-mon[80071]: pgmap v35: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:51 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:51 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:51 smithi110 ceph-mon[90991]: pgmap v35: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 -- 172.21.15.110:0/4169662203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 msgr2=0x7f6c380fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 --2- 172.21.15.110:0/4169662203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 0x7f6c380fc150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009ce0 tx=0x7f6c30004d10 comp rx=0 tx=0).stop 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 -- 172.21.15.110:0/4169662203 shutdown_connections 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 --2- 172.21.15.110:0/4169662203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fcff0 0x7f6c380fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 --2- 172.21.15.110:0/4169662203 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 0x7f6c380fc150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 -- 172.21.15.110:0/4169662203 >> 172.21.15.110:0/4169662203 conn(0x7f6c380f74c0 msgr2=0x7f6c380f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 -- 172.21.15.110:0/4169662203 shutdown_connections 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 -- 172.21.15.110:0/4169662203 wait complete. 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 Processor -- start 2024-04-25T15:52:52.569 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.566+0000 7f6c40149700 1 -- start start 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c40149700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 0x7f6c38112b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c40149700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fcff0 0x7f6c38115070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c40149700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c38066ab0 con 0x7f6c380fbd40 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c40149700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c38066bf0 con 0x7f6c380fcff0 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c3dee5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 0x7f6c38112b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c3dee5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 0x7f6c38112b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44210/0 (socket says 172.21.15.110:44210) 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c3dee5700 1 -- 172.21.15.110:0/686863533 learned_addr learned my addr 172.21.15.110:0/686863533 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c3d6e4700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fcff0 0x7f6c38115070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c3dee5700 1 -- 172.21.15.110:0/686863533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fcff0 msgr2=0x7f6c38115070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c3dee5700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fcff0 0x7f6c38115070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c3dee5700 1 -- 172.21.15.110:0/686863533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009940 con 0x7f6c380fbd40 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c3dee5700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 0x7f6c38112b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000bfd0 tx=0x7f6c30004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:52.570 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c2e7fc700 1 -- 172.21.15.110:0/686863533 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c3000e040 con 0x7f6c380fbd40 2024-04-25T15:52:52.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c381155b0 con 0x7f6c380fbd40 2024-04-25T15:52:52.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c2e7fc700 1 -- 172.21.15.110:0/686863533 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c3001cd00 con 0x7f6c380fbd40 2024-04-25T15:52:52.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.567+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c38115980 con 0x7f6c380fbd40 2024-04-25T15:52:52.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.568+0000 7f6c2e7fc700 1 -- 172.21.15.110:0/686863533 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c30012cb0 con 0x7f6c380fbd40 2024-04-25T15:52:52.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.569+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3805f5c0 con 0x7f6c380fbd40 2024-04-25T15:52:52.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.569+0000 7f6c2e7fc700 1 -- 172.21.15.110:0/686863533 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7f6c30012410 con 0x7f6c380fbd40 2024-04-25T15:52:52.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.570+0000 7f6c2e7fc700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6c24076a90 0x7f6c24078f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:52.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.570+0000 7f6c2e7fc700 1 -- 172.21.15.110:0/686863533 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6c3009a7c0 con 0x7f6c380fbd40 2024-04-25T15:52:52.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.570+0000 7f6c3d6e4700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6c24076a90 0x7f6c24078f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:52.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.570+0000 7f6c3d6e4700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6c24076a90 0x7f6c24078f40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6c28004600 tx=0x7f6c2800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:52.576 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.575+0000 7f6c2e7fc700 1 -- 172.21.15.110:0/686863533 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6c300631a0 con 0x7f6c380fbd40 2024-04-25T15:52:52.690 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:52 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:52.691 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:52 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:52.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.821+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c38101300 con 0x7f6c24076a90 2024-04-25T15:52:52.822 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:52 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:52.822 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:52 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:52.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.823+0000 7f6c2e7fc700 1 -- 172.21.15.110:0/686863533 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7f6c38101300 con 0x7f6c24076a90 2024-04-25T15:52:52.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6c24076a90 msgr2=0x7f6c24078f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:52.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6c24076a90 0x7f6c24078f40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6c28004600 tx=0x7f6c2800b040 comp rx=0 tx=0).stop 2024-04-25T15:52:52.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 msgr2=0x7f6c38112b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:52.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 0x7f6c38112b20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000bfd0 tx=0x7f6c30004d10 comp rx=0 tx=0).stop 2024-04-25T15:52:52.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 shutdown_connections 2024-04-25T15:52:52.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6c24076a90 0x7f6c24078f40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c380fbd40 0x7f6c38112b20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 --2- 172.21.15.110:0/686863533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c380fcff0 0x7f6c38115070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 >> 172.21.15.110:0/686863533 conn(0x7f6c380f74c0 msgr2=0x7f6c380f8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:52.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 shutdown_connections 2024-04-25T15:52:52.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.825+0000 7f6c40149700 1 -- 172.21.15.110:0/686863533 wait complete. 2024-04-25T15:52:52.836 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:52:52.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.931+0000 7fb611782700 1 -- 172.21.15.110:0/3345128282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 msgr2=0x7fb60c0fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:52.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.931+0000 7fb611782700 1 --2- 172.21.15.110:0/3345128282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 0x7fb60c0fe820 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc004940 tx=0x7fb5fc009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:52.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.931+0000 7fb611782700 1 -- 172.21.15.110:0/3345128282 shutdown_connections 2024-04-25T15:52:52.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.931+0000 7fb611782700 1 --2- 172.21.15.110:0/3345128282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb60c0fed60 0x7fb60c0faea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.931+0000 7fb611782700 1 --2- 172.21.15.110:0/3345128282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 0x7fb60c0fe820 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.932 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.931+0000 7fb611782700 1 -- 172.21.15.110:0/3345128282 >> 172.21.15.110:0/3345128282 conn(0x7fb60c0f6bd0 msgr2=0x7fb60c0f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:52.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.931+0000 7fb611782700 1 -- 172.21.15.110:0/3345128282 shutdown_connections 2024-04-25T15:52:52.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.931+0000 7fb611782700 1 -- 172.21.15.110:0/3345128282 wait complete. 2024-04-25T15:52:52.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb611782700 1 Processor -- start 2024-04-25T15:52:52.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb611782700 1 -- start start 2024-04-25T15:52:52.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb611782700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 0x7fb60c10d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:52.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb611782700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb60c0fed60 0x7fb60c10de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:52.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb611782700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c10e470 con 0x7fb60c0fe410 2024-04-25T15:52:52.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb611782700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c10e5e0 con 0x7fb60c0fed60 2024-04-25T15:52:52.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb60affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 0x7fb60c10d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb60affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 0x7fb60c10d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44236/0 (socket says 172.21.15.110:44236) 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb60affd700 1 -- 172.21.15.110:0/4279225006 learned_addr learned my addr 172.21.15.110:0/4279225006 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb60a7fc700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb60c0fed60 0x7fb60c10de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb60affd700 1 -- 172.21.15.110:0/4279225006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb60c0fed60 msgr2=0x7fb60c10de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.932+0000 7fb60affd700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb60c0fed60 0x7fb60c10de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.933+0000 7fb60affd700 1 -- 172.21.15.110:0/4279225006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc005040 con 0x7fb60c0fe410 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.933+0000 7fb60affd700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 0x7fb60c10d930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00b970 tx=0x7fb5fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.933+0000 7fb5f37fe700 1 -- 172.21.15.110:0/4279225006 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5fc00e040 con 0x7fb60c0fe410 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.933+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c112fe0 con 0x7fb60c0fe410 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.933+0000 7fb5f37fe700 1 -- 172.21.15.110:0/4279225006 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb5fc00bbf0 con 0x7fb60c0fe410 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.933+0000 7fb5f37fe700 1 -- 172.21.15.110:0/4279225006 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5fc01c630 con 0x7fb60c0fe410 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.933+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c113510 con 0x7fb60c0fe410 2024-04-25T15:52:52.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.934+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c04c710 con 0x7fb60c0fe410 2024-04-25T15:52:52.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.935+0000 7fb5f37fe700 1 -- 172.21.15.110:0/4279225006 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7fb5fc00bd60 con 0x7fb60c0fe410 2024-04-25T15:52:52.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.936+0000 7fb5f37fe700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb5f4076ca0 0x7fb5f4079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:52.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.936+0000 7fb5f37fe700 1 -- 172.21.15.110:0/4279225006 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb5fc0997b0 con 0x7fb60c0fe410 2024-04-25T15:52:52.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.936+0000 7fb60a7fc700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb5f4076ca0 0x7fb5f4079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:52.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.936+0000 7fb60a7fc700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb5f4076ca0 0x7fb5f4079150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb600004f60 tx=0x7fb60000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:52.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:52.941+0000 7fb5f37fe700 1 -- 172.21.15.110:0/4279225006 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb5fc062230 con 0x7fb60c0fe410 2024-04-25T15:52:53.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.167+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb60c1024e0 con 0x7fb5f4076ca0 2024-04-25T15:52:53.169 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.169+0000 7fb5f37fe700 1 -- 172.21.15.110:0/4279225006 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fb60c1024e0 con 0x7fb5f4076ca0 2024-04-25T15:52:53.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb5f4076ca0 msgr2=0x7fb5f4079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb5f4076ca0 0x7fb5f4079150 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb600004f60 tx=0x7fb60000b040 comp rx=0 tx=0).stop 2024-04-25T15:52:53.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 msgr2=0x7fb60c10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.172 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 0x7fb60c10d930 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00b970 tx=0x7fb5fc009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:53.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 shutdown_connections 2024-04-25T15:52:53.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb5f4076ca0 0x7fb5f4079150 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb60c0fe410 0x7fb60c10d930 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 --2- 172.21.15.110:0/4279225006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb60c0fed60 0x7fb60c10de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.171+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 >> 172.21.15.110:0/4279225006 conn(0x7fb60c0f6bd0 msgr2=0x7fb60c0f82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:53.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.172+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 shutdown_connections 2024-04-25T15:52:53.173 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.172+0000 7fb611782700 1 -- 172.21.15.110:0/4279225006 wait complete. 2024-04-25T15:52:53.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.277+0000 7f66b6366700 1 -- 172.21.15.110:0/3104275496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b00fff10 msgr2=0x7f66b0102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.277+0000 7f66b6366700 1 --2- 172.21.15.110:0/3104275496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b00fff10 0x7f66b0102330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f66a4004940 tx=0x7f66a4009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:53.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.277+0000 7f66b6366700 1 -- 172.21.15.110:0/3104275496 shutdown_connections 2024-04-25T15:52:53.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.277+0000 7f66b6366700 1 --2- 172.21.15.110:0/3104275496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b00fff10 0x7f66b0102330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.277+0000 7f66b6366700 1 --2- 172.21.15.110:0/3104275496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66b0066b50 0x7f66b00ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.278 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.277+0000 7f66b6366700 1 -- 172.21.15.110:0/3104275496 >> 172.21.15.110:0/3104275496 conn(0x7f66b00f7440 msgr2=0x7f66b00f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:53.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.277+0000 7f66b6366700 1 -- 172.21.15.110:0/3104275496 shutdown_connections 2024-04-25T15:52:53.279 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.277+0000 7f66b6366700 1 -- 172.21.15.110:0/3104275496 wait complete. 2024-04-25T15:52:53.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66b6366700 1 Processor -- start 2024-04-25T15:52:53.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66b6366700 1 -- start start 2024-04-25T15:52:53.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66b6366700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0066b50 0x7f66b010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:53.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66b6366700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66b00fff10 0x7f66b010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:53.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66b6366700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b010e460 con 0x7f66b0066b50 2024-04-25T15:52:53.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66b6366700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b010ecb0 con 0x7f66b00fff10 2024-04-25T15:52:53.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0066b50 0x7f66b010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66affff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0066b50 0x7f66b010d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44264/0 (socket says 172.21.15.110:44264) 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.278+0000 7f66affff700 1 -- 172.21.15.110:0/1677557019 learned_addr learned my addr 172.21.15.110:0/1677557019 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66affff700 1 -- 172.21.15.110:0/1677557019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66b00fff10 msgr2=0x7f66b010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66affff700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66b00fff10 0x7f66b010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66affff700 1 -- 172.21.15.110:0/1677557019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a4005040 con 0x7f66b0066b50 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66affff700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0066b50 0x7f66b010d920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66a000eb40 tx=0x7f66a000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66acff9700 1 -- 172.21.15.110:0/1677557019 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66a0007cc0 con 0x7f66b0066b50 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b010ef30 con 0x7f66b0066b50 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66acff9700 1 -- 172.21.15.110:0/1677557019 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f66a000f040 con 0x7f66b0066b50 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66acff9700 1 -- 172.21.15.110:0/1677557019 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66a00095e0 con 0x7f66b0066b50 2024-04-25T15:52:53.281 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.279+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b010f470 con 0x7f66b0066b50 2024-04-25T15:52:53.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.280+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b005f5c0 con 0x7f66b0066b50 2024-04-25T15:52:53.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.281+0000 7f66acff9700 1 -- 172.21.15.110:0/1677557019 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7f66a0009740 con 0x7f66b0066b50 2024-04-25T15:52:53.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.281+0000 7f66acff9700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6698076ca0 0x7f6698079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:53.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.281+0000 7f66acff9700 1 -- 172.21.15.110:0/1677557019 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f66a0098230 con 0x7f66b0066b50 2024-04-25T15:52:53.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.282+0000 7f66af7fe700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6698076ca0 0x7f6698079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:53.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.282+0000 7f66af7fe700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6698076ca0 0x7f6698079150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009b30 tx=0x7f66a4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:53.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.289+0000 7f66acff9700 1 -- 172.21.15.110:0/1677557019 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f66a0060cc0 con 0x7f66b0066b50 2024-04-25T15:52:53.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.504+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f66b010f800 con 0x7f6698076ca0 2024-04-25T15:52:53.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.524+0000 7f66acff9700 1 -- 172.21.15.110:0/1677557019 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f66b010f800 con 0x7f6698076ca0 2024-04-25T15:52:53.525 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:52:53.525 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (3m) 0s ago 11m 26.3M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:52:53.525 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (11s) 0s ago 12m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:52:53.525 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (6s) 3s ago 11m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:52:53.525 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (2m) 0s ago 11m 77.8M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (6m) 0s ago 6m 19.4M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (6m) 0s ago 6m 150M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (6m) 3s ago 6m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (6m) 3s ago 6m 16.7M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (2m) 0s ago 13m 565M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (89s) 3s ago 10m 517M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (75s) 0s ago 13m 110M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (52s) 3s ago 10m 98.5M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (3m) 0s ago 11m 15.9M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (2m) 3s ago 10m 15.7M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (10m) 0s ago 10m 205M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 89f44aa76091 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (9m) 0s ago 9m 218M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (9m) 0s ago 9m 178M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (8m) 3s ago 8m 239M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:52:53.526 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (8m) 3s ago 8m 206M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:52:53.527 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (7m) 3s ago 7m 189M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:52:53.527 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (96s) 0s ago 11m 54.2M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6698076ca0 msgr2=0x7f6698079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6698076ca0 0x7f6698079150 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009b30 tx=0x7f66a4009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0066b50 msgr2=0x7f66b010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0066b50 0x7f66b010d920 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f66a000eb40 tx=0x7f66a000b040 comp rx=0 tx=0).stop 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 shutdown_connections 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6698076ca0 0x7f6698079150 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f66b0066b50 0x7f66b010d920 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 --2- 172.21.15.110:0/1677557019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f66b00fff10 0x7f66b010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 >> 172.21.15.110:0/1677557019 conn(0x7f66b00f7440 msgr2=0x7f66b00f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 shutdown_connections 2024-04-25T15:52:53.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.526+0000 7f66b6366700 1 -- 172.21.15.110:0/1677557019 wait complete. 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 -- 172.21.15.110:0/3646353841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 msgr2=0x7f93680fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 --2- 172.21.15.110:0/3646353841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 0x7f93680fe500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9358004940 tx=0x7f9358009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 -- 172.21.15.110:0/3646353841 shutdown_connections 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 --2- 172.21.15.110:0/3646353841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 0x7f93680fe500 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 --2- 172.21.15.110:0/3646353841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93680fce90 0x7f93680fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 -- 172.21.15.110:0/3646353841 >> 172.21.15.110:0/3646353841 conn(0x7f93680f85c0 msgr2=0x7f93680fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 -- 172.21.15.110:0/3646353841 shutdown_connections 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 -- 172.21.15.110:0/3646353841 wait complete. 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.629+0000 7f936e856700 1 Processor -- start 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f936e856700 1 -- start start 2024-04-25T15:52:53.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f936e856700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93680fce90 0x7f936810d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f936e856700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 0x7f936810def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f936e856700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936810e4f0 con 0x7f93680fe090 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f936e856700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936810ed40 con 0x7f93680fce90 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f93677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 0x7f936810def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f93677fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 0x7f936810def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44286/0 (socket says 172.21.15.110:44286) 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f93677fe700 1 -- 172.21.15.110:0/1142079326 learned_addr learned my addr 172.21.15.110:0/1142079326 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f9367fff700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93680fce90 0x7f936810d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f93677fe700 1 -- 172.21.15.110:0/1142079326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93680fce90 msgr2=0x7f936810d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f93677fe700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93680fce90 0x7f936810d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.630+0000 7f93677fe700 1 -- 172.21.15.110:0/1142079326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358005040 con 0x7f93680fe090 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.631+0000 7f93677fe700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 0x7f936810def0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9358013870 tx=0x7f9358009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:53.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.631+0000 7f9364ff9700 1 -- 172.21.15.110:0/1142079326 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9358021070 con 0x7f93680fe090 2024-04-25T15:52:53.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.631+0000 7f9364ff9700 1 -- 172.21.15.110:0/1142079326 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9358009430 con 0x7f93680fe090 2024-04-25T15:52:53.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.631+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936810efc0 con 0x7f93680fe090 2024-04-25T15:52:53.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.631+0000 7f9364ff9700 1 -- 172.21.15.110:0/1142079326 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935800e070 con 0x7f93680fe090 2024-04-25T15:52:53.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.631+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f936810f500 con 0x7f93680fe090 2024-04-25T15:52:53.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.632+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f936805f5c0 con 0x7f93680fe090 2024-04-25T15:52:53.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.633+0000 7f9364ff9700 1 -- 172.21.15.110:0/1142079326 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7f935801c750 con 0x7f93680fe090 2024-04-25T15:52:53.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.633+0000 7f9364ff9700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9350076c30 0x7f93500790e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:53.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.633+0000 7f9364ff9700 1 -- 172.21.15.110:0/1142079326 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9358099770 con 0x7f93680fe090 2024-04-25T15:52:53.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.633+0000 7f9367fff700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9350076c30 0x7f93500790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:53.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.633+0000 7f9367fff700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9350076c30 0x7f93500790e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f935c009b80 tx=0x7f935c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:53.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.638+0000 7f9364ff9700 1 -- 172.21.15.110:0/1142079326 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f93580620f0 con 0x7f93680fe090 2024-04-25T15:52:53.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:53 smithi110 ceph-mon[90991]: pgmap v36: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:53.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:53 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:53.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:53 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:53.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.937+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f936804bed0 con 0x7f93680fe090 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.938+0000 7f9364ff9700 1 -- 172.21.15.110:0/1142079326 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+736 (secure 0 0 0) 0x7f9358061a90 con 0x7f93680fe090 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6 2024-04-25T15:52:53.939 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:52:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:52:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:52:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:52:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:52:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 10, 2024-04-25T15:52:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 4 2024-04-25T15:52:53.940 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:52:53.940 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:52:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9350076c30 msgr2=0x7f93500790e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9350076c30 0x7f93500790e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f935c009b80 tx=0x7f935c00b040 comp rx=0 tx=0).stop 2024-04-25T15:52:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 msgr2=0x7f936810def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 0x7f936810def0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9358013870 tx=0x7f9358009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 shutdown_connections 2024-04-25T15:52:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f93680fce90 0x7f936810d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9350076c30 0x7f93500790e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 --2- 172.21.15.110:0/1142079326 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93680fe090 0x7f936810def0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 >> 172.21.15.110:0/1142079326 conn(0x7f93680f85c0 msgr2=0x7f93680f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 shutdown_connections 2024-04-25T15:52:53.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:53.940+0000 7f936e856700 1 -- 172.21.15.110:0/1142079326 wait complete. 2024-04-25T15:52:54.031 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:53 smithi193 ceph-mon[80071]: pgmap v36: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:54.031 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:53 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:54.032 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:53 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:54.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 -- 172.21.15.110:0/3618853213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 msgr2=0x7fc9540fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 --2- 172.21.15.110:0/3618853213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 0x7fc9540fc0c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc944004940 tx=0x7fc944009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:54.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 -- 172.21.15.110:0/3618853213 shutdown_connections 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 --2- 172.21.15.110:0/3618853213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9540fcf60 0x7fc9540fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 --2- 172.21.15.110:0/3618853213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 0x7fc9540fc0c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 -- 172.21.15.110:0/3618853213 >> 172.21.15.110:0/3618853213 conn(0x7fc9540f7430 msgr2=0x7fc9540f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 -- 172.21.15.110:0/3618853213 shutdown_connections 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 -- 172.21.15.110:0/3618853213 wait complete. 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 Processor -- start 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 -- start start 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 0x7fc954110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9540fcf60 0x7fc954112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc954066330 con 0x7fc9540fbcb0 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.046+0000 7fc958f5d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc954066470 con 0x7fc9540fcf60 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc95259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 0x7fc954110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc951d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9540fcf60 0x7fc954112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc95259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 0x7fc954110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44316/0 (socket says 172.21.15.110:44316) 2024-04-25T15:52:54.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc95259c700 1 -- 172.21.15.110:0/3117640043 learned_addr learned my addr 172.21.15.110:0/3117640043 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc95259c700 1 -- 172.21.15.110:0/3117640043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9540fcf60 msgr2=0x7fc954112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc95259c700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9540fcf60 0x7fc954112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc95259c700 1 -- 172.21.15.110:0/3117640043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc944005040 con 0x7fc9540fbcb0 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc95259c700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 0x7fc954110720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc94400b3b0 tx=0x7fc944009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc942ffd700 1 -- 172.21.15.110:0/3117640043 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc94400e040 con 0x7fc9540fbcb0 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc942ffd700 1 -- 172.21.15.110:0/3117640043 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc94400bbf0 con 0x7fc9540fbcb0 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9541131b0 con 0x7fc9540fbcb0 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc942ffd700 1 -- 172.21.15.110:0/3117640043 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc94401c630 con 0x7fc9540fbcb0 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.047+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc954113770 con 0x7fc9540fbcb0 2024-04-25T15:52:54.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.049+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc95405f5c0 con 0x7fc9540fbcb0 2024-04-25T15:52:54.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.049+0000 7fc942ffd700 1 -- 172.21.15.110:0/3117640043 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7fc944026440 con 0x7fc9540fbcb0 2024-04-25T15:52:54.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.050+0000 7fc942ffd700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc93c076ca0 0x7fc93c079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.050+0000 7fc942ffd700 1 -- 172.21.15.110:0/3117640043 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc9440990d0 con 0x7fc9540fbcb0 2024-04-25T15:52:54.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.050+0000 7fc951d9b700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc93c076ca0 0x7fc93c079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.050+0000 7fc951d9b700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc93c076ca0 0x7fc93c079150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc948004f40 tx=0x7fc94800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:54.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.055+0000 7fc942ffd700 1 -- 172.21.15.110:0/3117640043 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc944061ab0 con 0x7fc9540fbcb0 2024-04-25T15:52:54.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.307+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc954113a50 con 0x7fc9540fbcb0 2024-04-25T15:52:54.308 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.307+0000 7fc942ffd700 1 -- 172.21.15.110:0/3117640043 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7fc944061450 con 0x7fc9540fbcb0 2024-04-25T15:52:54.308 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:52:54.309 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:52:54.310 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:54.311 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:54.312 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:52:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc93c076ca0 msgr2=0x7fc93c079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc93c076ca0 0x7fc93c079150 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc948004f40 tx=0x7fc94800b040 comp rx=0 tx=0).stop 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 msgr2=0x7fc954110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 0x7fc954110720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc94400b3b0 tx=0x7fc944009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 shutdown_connections 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc93c076ca0 0x7fc93c079150 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc9540fbcb0 0x7fc954110720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 --2- 172.21.15.110:0/3117640043 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9540fcf60 0x7fc954112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 >> 172.21.15.110:0/3117640043 conn(0x7fc9540f7430 msgr2=0x7fc9540f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 shutdown_connections 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.311+0000 7fc958f5d700 1 -- 172.21.15.110:0/3117640043 wait complete. 2024-04-25T15:52:54.313 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:52:54.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.425+0000 7fa19f389700 1 -- 172.21.15.110:0/3971921485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fe3e0 msgr2=0x7fa1980fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.425+0000 7fa19f389700 1 --2- 172.21.15.110:0/3971921485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fe3e0 0x7fa1980fe7f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa194004940 tx=0x7fa194009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:54.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.425+0000 7fa19f389700 1 -- 172.21.15.110:0/3971921485 shutdown_connections 2024-04-25T15:52:54.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.425+0000 7fa19f389700 1 --2- 172.21.15.110:0/3971921485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1980fed30 0x7fa1980fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 --2- 172.21.15.110:0/3971921485 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fe3e0 0x7fa1980fe7f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 -- 172.21.15.110:0/3971921485 >> 172.21.15.110:0/3971921485 conn(0x7fa1980f6b80 msgr2=0x7fa1980f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:54.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 -- 172.21.15.110:0/3971921485 shutdown_connections 2024-04-25T15:52:54.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 -- 172.21.15.110:0/3971921485 wait complete. 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 Processor -- start 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 -- start start 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1980fe3e0 0x7fa19810d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fed30 0x7fa19810df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19810e510 con 0x7fa1980fed30 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.426+0000 7fa19f389700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19810ed60 con 0x7fa1980fe3e0 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19c924700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fed30 0x7fa19810df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19d125700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1980fe3e0 0x7fa19810d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.429 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19c924700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fed30 0x7fa19810df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44332/0 (socket says 172.21.15.110:44332) 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19c924700 1 -- 172.21.15.110:0/4070746805 learned_addr learned my addr 172.21.15.110:0/4070746805 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19c924700 1 -- 172.21.15.110:0/4070746805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1980fe3e0 msgr2=0x7fa19810d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19c924700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1980fe3e0 0x7fa19810d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19c924700 1 -- 172.21.15.110:0/4070746805 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194005040 con 0x7fa1980fed30 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19c924700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fed30 0x7fa19810df10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa190001d80 tx=0x7fa190009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa18dffb700 1 -- 172.21.15.110:0/4070746805 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa190007d10 con 0x7fa1980fed30 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19810f040 con 0x7fa1980fed30 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa18dffb700 1 -- 172.21.15.110:0/4070746805 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa190001e80 con 0x7fa1980fed30 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.427+0000 7fa18dffb700 1 -- 172.21.15.110:0/4070746805 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa190014640 con 0x7fa1980fed30 2024-04-25T15:52:54.430 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.428+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19810f580 con 0x7fa1980fed30 2024-04-25T15:52:54.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.429+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19805f5c0 con 0x7fa1980fed30 2024-04-25T15:52:54.431 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.429+0000 7fa18dffb700 1 -- 172.21.15.110:0/4070746805 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7fa19000e960 con 0x7fa1980fed30 2024-04-25T15:52:54.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.430+0000 7fa18dffb700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa184076ca0 0x7fa184079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.430+0000 7fa18dffb700 1 -- 172.21.15.110:0/4070746805 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fa1900981d0 con 0x7fa1980fed30 2024-04-25T15:52:54.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.430+0000 7fa19d125700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa184076ca0 0x7fa184079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.431+0000 7fa19d125700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa184076ca0 0x7fa184079150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa19400b970 tx=0x7fa194009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:54.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.435+0000 7fa18dffb700 1 -- 172.21.15.110:0/4070746805 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa190060cf0 con 0x7fa1980fed30 2024-04-25T15:52:54.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.670+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa19810f910 con 0x7fa184076ca0 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.671+0000 7fa18dffb700 1 -- 172.21.15.110:0/4070746805 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+402 (secure 0 0 0) 0x7fa19810f910 con 0x7fa184076ca0 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:52:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "6/21 daemons upgraded", 2024-04-25T15:52:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading crash daemons", 2024-04-25T15:52:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:52:54.673 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:52:54.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa184076ca0 msgr2=0x7fa184079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa184076ca0 0x7fa184079150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa19400b970 tx=0x7fa194009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:54.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fed30 msgr2=0x7fa19810df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fed30 0x7fa19810df10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa190001d80 tx=0x7fa190009ec0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 shutdown_connections 2024-04-25T15:52:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1980fe3e0 0x7fa19810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa184076ca0 0x7fa184079150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 --2- 172.21.15.110:0/4070746805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1980fed30 0x7fa19810df10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 >> 172.21.15.110:0/4070746805 conn(0x7fa1980f6b80 msgr2=0x7fa1980f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 shutdown_connections 2024-04-25T15:52:54.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.673+0000 7fa19f389700 1 -- 172.21.15.110:0/4070746805 wait complete. 2024-04-25T15:52:54.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.777+0000 7f722d832700 1 -- 172.21.15.110:0/2622479123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 msgr2=0x7f72280fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.777+0000 7f722d832700 1 --2- 172.21.15.110:0/2622479123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 0x7f72280fd3e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f721c004940 tx=0x7f721c009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:54.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.777+0000 7f722d832700 1 -- 172.21.15.110:0/2622479123 shutdown_connections 2024-04-25T15:52:54.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.777+0000 7f722d832700 1 --2- 172.21.15.110:0/2622479123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 0x7f72280fd3e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.777+0000 7f722d832700 1 --2- 172.21.15.110:0/2622479123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280fbcc0 0x7f72280fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.777+0000 7f722d832700 1 -- 172.21.15.110:0/2622479123 >> 172.21.15.110:0/2622479123 conn(0x7f72280f7440 msgr2=0x7f72280f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:54.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.777+0000 7f722d832700 1 -- 172.21.15.110:0/2622479123 shutdown_connections 2024-04-25T15:52:54.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.777+0000 7f722d832700 1 -- 172.21.15.110:0/2622479123 wait complete. 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.778+0000 7f722d832700 1 Processor -- start 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.778+0000 7f722d832700 1 -- start start 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.778+0000 7f722d832700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280fbcc0 0x7f722810da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.778+0000 7f722d832700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 0x7f722810df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.778+0000 7f722d832700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722810e550 con 0x7f72280fcf70 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.778+0000 7f722d832700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722810e6c0 con 0x7f72280fbcc0 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.778+0000 7f72267fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 0x7f722810df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f72267fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 0x7f722810df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44344/0 (socket says 172.21.15.110:44344) 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f72267fc700 1 -- 172.21.15.110:0/848096964 learned_addr learned my addr 172.21.15.110:0/848096964 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f7226ffd700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280fbcc0 0x7f722810da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f72267fc700 1 -- 172.21.15.110:0/848096964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280fbcc0 msgr2=0x7f722810da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f72267fc700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280fbcc0 0x7f722810da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:54.781 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f72267fc700 1 -- 172.21.15.110:0/848096964 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c005040 con 0x7f72280fcf70 2024-04-25T15:52:54.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f7226ffd700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280fbcc0 0x7f722810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:52:54.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f72267fc700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 0x7f722810df50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f721c00b970 tx=0x7f721c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:54.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f720f7fe700 1 -- 172.21.15.110:0/848096964 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721c00e040 con 0x7f72280fcf70 2024-04-25T15:52:54.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7228113020 con 0x7f72280fcf70 2024-04-25T15:52:54.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f720f7fe700 1 -- 172.21.15.110:0/848096964 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f721c00bcc0 con 0x7f72280fcf70 2024-04-25T15:52:54.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f720f7fe700 1 -- 172.21.15.110:0/848096964 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f721c013660 con 0x7f72280fcf70 2024-04-25T15:52:54.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.779+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72281135e0 con 0x7f72280fcf70 2024-04-25T15:52:54.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.780+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7228101b00 con 0x7f72280fcf70 2024-04-25T15:52:54.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.781+0000 7f720f7fe700 1 -- 172.21.15.110:0/848096964 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 37) v1 ==== 98765+0+0 (secure 0 0 0) 0x7f721c01ca00 con 0x7f72280fcf70 2024-04-25T15:52:54.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.781+0000 7f720f7fe700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7210076ca0 0x7f7210079150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:52:54.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.781+0000 7f720f7fe700 1 -- 172.21.15.110:0/848096964 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f721c099260 con 0x7f72280fcf70 2024-04-25T15:52:54.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.781+0000 7f7226ffd700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7210076ca0 0x7f7210079150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:52:54.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.782+0000 7f7226ffd700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7210076ca0 0x7f7210079150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f72180041d0 tx=0x7f7218009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:52:54.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:54.786+0000 7f720f7fe700 1 -- 172.21.15.110:0/848096964 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f721c061cf0 con 0x7f72280fcf70 2024-04-25T15:52:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1142079326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:54.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3117640043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:52:54.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:54.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:54 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:55.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.072+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f72281047b0 con 0x7f72280fcf70 2024-04-25T15:52:55.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.073+0000 7f720f7fe700 1 -- 172.21.15.110:0/848096964 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f721c061690 con 0x7f72280fcf70 2024-04-25T15:52:55.074 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:55.074 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:52:55.074 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:52:55.074 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='client.34188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:55.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='client.34192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:55.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='client.34196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:55.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:55.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:55.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1142079326' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:55.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3117640043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:52:55.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:55.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7210076ca0 msgr2=0x7f7210079150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7210076ca0 0x7f7210079150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f72180041d0 tx=0x7f7218009040 comp rx=0 tx=0).stop 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 msgr2=0x7f722810df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 0x7f722810df50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f721c00b970 tx=0x7f721c009e30 comp rx=0 tx=0).stop 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 shutdown_connections 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72280fbcc0 0x7f722810da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7210076ca0 0x7f7210079150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 --2- 172.21.15.110:0/848096964 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72280fcf70 0x7f722810df50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 >> 172.21.15.110:0/848096964 conn(0x7f72280f7440 msgr2=0x7f7228066860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 shutdown_connections 2024-04-25T15:52:55.077 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:52:55.075+0000 7f722d832700 1 -- 172.21.15.110:0/848096964 wait complete. 2024-04-25T15:52:55.850 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:55 smithi110 ceph-mon[90991]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:55.850 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:55 smithi110 ceph-mon[90991]: pgmap v37: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:55.850 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:55 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/848096964' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:52:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:55 smithi193 ceph-mon[80071]: from='client.34208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:52:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:55 smithi193 ceph-mon[80071]: pgmap v37: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:56.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:55 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/848096964' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:52:58.042 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: pgmap v38: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:58.042 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:58.042 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:58.042 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:58.042 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:52:58.042 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:58.042 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:52:58.042 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:58.043 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:58.043 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:58.043 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:58.043 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi110"}]: dispatch 2024-04-25T15:52:58.043 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi110"}]': finished 2024-04-25T15:52:58.043 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]: dispatch 2024-04-25T15:52:58.043 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]': finished 2024-04-25T15:52:58.043 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-25T15:52:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: pgmap v38: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:52:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:52:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:52:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:52:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:58.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:58.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:52:58.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:52:58.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi110"}]: dispatch 2024-04-25T15:52:58.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi110"}]': finished 2024-04-25T15:52:58.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]: dispatch 2024-04-25T15:52:58.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi193"}]': finished 2024-04-25T15:52:58.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-25T15:52:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:58 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all crash 2024-04-25T15:52:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:58 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-25T15:52:58.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:58 smithi110 ceph-mon[90991]: Upgrade: osd.0 is safe to restart 2024-04-25T15:52:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:58 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all crash 2024-04-25T15:52:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:58 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-25T15:52:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:58 smithi193 ceph-mon[80071]: Upgrade: osd.0 is safe to restart 2024-04-25T15:53:00.052 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:59 smithi110 ceph-mon[90991]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:00.052 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:00.052 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T15:53:00.052 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:52:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:53:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:59 smithi193 ceph-mon[80071]: pgmap v39: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:00.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:00.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T15:53:00.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:52:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:53:00.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:00 smithi110 ceph-mon[90991]: Upgrade: Updating osd.0 2024-04-25T15:53:00.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:00 smithi110 ceph-mon[90991]: Deploying daemon osd.0 on smithi110 2024-04-25T15:53:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:00 smithi193 ceph-mon[80071]: Upgrade: Updating osd.0 2024-04-25T15:53:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:00 smithi193 ceph-mon[80071]: Deploying daemon osd.0 on smithi110 2024-04-25T15:53:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:01 smithi110 ceph-mon[90991]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:01.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:53:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:01 smithi193 ceph-mon[80071]: pgmap v40: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:53:02.863 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:02 smithi110 systemd[1]: Stopping Ceph osd.0 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:53:03.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:02 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[45308]: 2024-04-25T15:53:02.932+0000 7f48ca530700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:53:03.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:02 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[45308]: 2024-04-25T15:53:02.932+0000 7f48ca530700 -1 osd.0 49 *** Got signal Terminated *** 2024-04-25T15:53:03.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:02 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[45308]: 2024-04-25T15:53:02.932+0000 7f48ca530700 -1 osd.0 49 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:53:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:03 smithi110 ceph-mon[90991]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:03 smithi110 ceph-mon[90991]: osd.0 marked itself down and dead 2024-04-25T15:53:04.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:03 smithi193 ceph-mon[80071]: pgmap v41: 65 pgs: 65 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:04.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:03 smithi193 ceph-mon[80071]: osd.0 marked itself down and dead 2024-04-25T15:53:04.427 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:04 smithi110 podman[99249]: 2024-04-25 15:53:04.136861181 +0000 UTC m=+1.322774605 container died 89f44aa76091f90c83f72cfa13ef7b7816ed4bfbd7825462f9807f68b3cf1a2d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, RELEASE=quincy-c61472c, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:53:05.026 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:04 smithi110 ceph-mon[90991]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:53:05.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:04 smithi110 ceph-mon[90991]: osdmap e50: 6 total, 5 up, 6 in 2024-04-25T15:53:05.027 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:04 smithi110 ceph-mon[90991]: osdmap e51: 6 total, 5 up, 6 in 2024-04-25T15:53:05.027 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:04 smithi110 podman[99249]: 2024-04-25 15:53:04.708393392 +0000 UTC m=+1.894306820 container cleanup 89f44aa76091f90c83f72cfa13ef7b7816ed4bfbd7825462f9807f68b3cf1a2d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T15:53:05.027 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:04 smithi110 bash[99249]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0 2024-04-25T15:53:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:04 smithi193 ceph-mon[80071]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:53:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:04 smithi193 ceph-mon[80071]: osdmap e50: 6 total, 5 up, 6 in 2024-04-25T15:53:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:04 smithi193 ceph-mon[80071]: osdmap e51: 6 total, 5 up, 6 in 2024-04-25T15:53:05.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:05 smithi110 podman[99266]: 2024-04-25 15:53:05.026441906 +0000 UTC m=+0.890109510 container remove 89f44aa76091f90c83f72cfa13ef7b7816ed4bfbd7825462f9807f68b3cf1a2d (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, ceph=True, GIT_BRANCH=HEAD, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T15:53:05.655 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:05 smithi110 podman[99359]: 2024-04-25 15:53:05.293602446 +0000 UTC m=+0.018557918 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:53:05.655 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:05 smithi110 podman[99359]: 2024-04-25 15:53:05.394246383 +0000 UTC m=+0.119201844 container create 10785ce2b3085f99b3d01bb51e479d10597ae805a59a3668b06dd64dc8493e7c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T15:53:05.959 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:05 smithi110 ceph-mon[90991]: pgmap v43: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:05.959 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:05 smithi110 podman[99359]: 2024-04-25 15:53:05.655395876 +0000 UTC m=+0.380351336 container init 10785ce2b3085f99b3d01bb51e479d10597ae805a59a3668b06dd64dc8493e7c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T15:53:05.959 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:05 smithi110 podman[99359]: 2024-04-25 15:53:05.658773747 +0000 UTC m=+0.383729217 container start 10785ce2b3085f99b3d01bb51e479d10597ae805a59a3668b06dd64dc8493e7c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-25T15:53:05.959 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:05 smithi110 podman[99359]: 2024-04-25 15:53:05.763809222 +0000 UTC m=+0.488764697 container attach 10785ce2b3085f99b3d01bb51e479d10597ae805a59a3668b06dd64dc8493e7c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:53:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:05 smithi193 ceph-mon[80071]: pgmap v43: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:06.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:05 smithi110 podman[99359]: 2024-04-25 15:53:05.95969139 +0000 UTC m=+0.684646844 container died 10785ce2b3085f99b3d01bb51e479d10597ae805a59a3668b06dd64dc8493e7c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T15:53:07.113 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:06 smithi110 podman[99386]: 2024-04-25 15:53:06.684188707 +0000 UTC m=+0.715892210 container remove 10785ce2b3085f99b3d01bb51e479d10597ae805a59a3668b06dd64dc8493e7c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:53:07.113 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:06 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-25T15:53:07.113 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:06 smithi110 systemd[1]: Stopped Ceph osd.0 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:53:07.113 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:06 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.0.service: Consumed 50.250s CPU time. 2024-04-25T15:53:07.113 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:06 smithi110 systemd[1]: Starting Ceph osd.0 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:53:07.421 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:07 smithi110 podman[99492]: 2024-04-25 15:53:07.044778478 +0000 UTC m=+0.018200345 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:53:07.422 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:07 smithi110 podman[99492]: 2024-04-25 15:53:07.177164504 +0000 UTC m=+0.150586357 container create f35fa4016b01de1a06359f9c1c4cc68bd621d1da77ff476557f9adf454dffca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-25T15:53:07.422 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:07 smithi110 podman[99492]: 2024-04-25 15:53:07.421576917 +0000 UTC m=+0.394998767 container init f35fa4016b01de1a06359f9c1c4cc68bd621d1da77ff476557f9adf454dffca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T15:53:07.784 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:07 smithi110 podman[99492]: 2024-04-25 15:53:07.424765861 +0000 UTC m=+0.398187720 container start f35fa4016b01de1a06359f9c1c4cc68bd621d1da77ff476557f9adf454dffca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T15:53:07.784 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:07 smithi110 podman[99492]: 2024-04-25 15:53:07.521803869 +0000 UTC m=+0.495225720 container attach f35fa4016b01de1a06359f9c1c4cc68bd621d1da77ff476557f9adf454dffca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T15:53:08.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:07 smithi110 ceph-mon[90991]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 bash[99492]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T15:53:08.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-25T15:53:08.045 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-25T15:53:08.045 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-25T15:53:08.045 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-25T15:53:08.045 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T15:53:08.045 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate[99507]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-25T15:53:08.093 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:07 smithi193 ceph-mon[80071]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:08.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 podman[99492]: 2024-04-25 15:53:08.043790584 +0000 UTC m=+1.017212442 container died f35fa4016b01de1a06359f9c1c4cc68bd621d1da77ff476557f9adf454dffca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T15:53:08.982 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-mon[90991]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:08.983 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:08 smithi110 ceph-mon[90991]: Health check failed: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:08.983 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 podman[99647]: 2024-04-25 15:53:08.650743994 +0000 UTC m=+0.601300145 container remove f35fa4016b01de1a06359f9c1c4cc68bd621d1da77ff476557f9adf454dffca2 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T15:53:08.983 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 podman[99707]: 2024-04-25 15:53:08.771043806 +0000 UTC m=+0.018248992 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:53:08.983 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:08 smithi110 podman[99707]: 2024-04-25 15:53:08.885330731 +0000 UTC m=+0.132535903 container create 8e46029b07105c8d92c335fe8c871aa0a0fa0787e7db4e915c75472df614880b (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-25T15:53:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:08 smithi193 ceph-mon[80071]: pgmap v46: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:08 smithi193 ceph-mon[80071]: Health check failed: Degraded data redundancy: 50/267 objects degraded (18.727%), 19 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:09.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:09 smithi110 podman[99707]: 2024-04-25 15:53:09.080203707 +0000 UTC m=+0.327408883 container init 8e46029b07105c8d92c335fe8c871aa0a0fa0787e7db4e915c75472df614880b (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:53:09.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:09 smithi110 podman[99707]: 2024-04-25 15:53:09.083314337 +0000 UTC m=+0.330519520 container start 8e46029b07105c8d92c335fe8c871aa0a0fa0787e7db4e915c75472df614880b (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T15:53:09.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:09 smithi110 ceph-osd[99727]: -- 172.21.15.110:0/30112392 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2f4bc5680 con 0x55b2f4c6cd00 2024-04-25T15:53:09.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:09 smithi110 bash[99707]: 8e46029b07105c8d92c335fe8c871aa0a0fa0787e7db4e915c75472df614880b 2024-04-25T15:53:09.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:09 smithi110 systemd[1]: Started Ceph osd.0 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:53:10.044 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:09 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[99723]: 2024-04-25T15:53:09.634+0000 7fd119f737c0 -1 Falling back to public interface 2024-04-25T15:53:10.299 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:10 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:10.299 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:10 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:10.299 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:10 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:53:10.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:10 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:10.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:10 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:10.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:10 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:53:11.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:11 smithi193 ceph-mon[80071]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:11.605 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:11 smithi110 ceph-mon[90991]: pgmap v47: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:13.852 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:13 smithi110 ceph-mon[90991]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:14.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:13 smithi193 ceph-mon[80071]: pgmap v48: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:15.113 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:14 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[99723]: 2024-04-25T15:53:14.670+0000 7fd119f737c0 -1 osd.0 49 log_to_monitors true 2024-04-25T15:53:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:15 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:15 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:15 smithi193 ceph-mon[80071]: from='osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T15:53:15.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:15 smithi193 ceph-mon[80071]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:15 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:15 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:15 smithi110 ceph-mon[90991]: from='osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T15:53:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:15 smithi110 ceph-mon[90991]: pgmap v49: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.5 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:16.456 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 15:53:16 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[99723]: 2024-04-25T15:53:16.143+0000 7fd107385700 -1 osd.0 49 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:53:16.723 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:16 smithi110 ceph-mon[90991]: from='osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T15:53:16.723 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:16 smithi110 ceph-mon[90991]: osdmap e52: 6 total, 5 up, 6 in 2024-04-25T15:53:16.723 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:16 smithi110 ceph-mon[90991]: from='osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:53:16.723 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:16.723 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:16.723 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:16 smithi110 ceph-mon[90991]: from='osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504]' entity='osd.0' 2024-04-25T15:53:16.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:16 smithi193 ceph-mon[80071]: from='osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T15:53:16.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:16 smithi193 ceph-mon[80071]: osdmap e52: 6 total, 5 up, 6 in 2024-04-25T15:53:16.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:16 smithi193 ceph-mon[80071]: from='osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:53:16.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:16.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:16.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:16 smithi193 ceph-mon[80071]: from='osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504]' entity='osd.0' 2024-04-25T15:53:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:17 smithi110 ceph-mon[90991]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:53:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:17 smithi110 ceph-mon[90991]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:53:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:17 smithi110 ceph-mon[90991]: osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504] boot 2024-04-25T15:53:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:17 smithi110 ceph-mon[90991]: osdmap e53: 6 total, 6 up, 6 in 2024-04-25T15:53:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:53:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:17 smithi193 ceph-mon[80071]: pgmap v51: 65 pgs: 19 active+undersized+degraded, 15 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 50/267 objects degraded (18.727%) 2024-04-25T15:53:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:53:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:17 smithi193 ceph-mon[80071]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:53:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:17 smithi193 ceph-mon[80071]: osd.0 [v2:172.21.15.110:6802/1940504504,v1:172.21.15.110:6803/1940504504] boot 2024-04-25T15:53:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:17 smithi193 ceph-mon[80071]: osdmap e53: 6 total, 6 up, 6 in 2024-04-25T15:53:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T15:53:19.185 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:19 smithi110 ceph-mon[90991]: osdmap e54: 6 total, 6 up, 6 in 2024-04-25T15:53:19.185 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:19 smithi110 ceph-mon[90991]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 7/267 objects degraded (2.622%) 2024-04-25T15:53:19.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:19 smithi193 ceph-mon[80071]: osdmap e54: 6 total, 6 up, 6 in 2024-04-25T15:53:19.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:19 smithi193 ceph-mon[80071]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s; 7/267 objects degraded (2.622%) 2024-04-25T15:53:20.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:20.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: mgrmap e38: smithi110.lvgbkf(active, since 92s), standbys: smithi193.mgoteh 2024-04-25T15:53:20.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:20.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:20.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:53:20.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:53:20.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:20.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:53:20.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:20.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:20.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:20.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T15:53:20.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T15:53:20.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:20 smithi193 ceph-mon[80071]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: mgrmap e38: smithi110.lvgbkf(active, since 92s), standbys: smithi193.mgoteh 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:53:20.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:20.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:20.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:20.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T15:53:20.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T15:53:20.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:20 smithi110 ceph-mon[90991]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-25T15:53:21.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:21 smithi193 ceph-mon[80071]: pgmap v55: 65 pgs: 25 peering, 40 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:53:21.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:21 smithi110 ceph-mon[90991]: pgmap v55: 65 pgs: 25 peering, 40 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-25T15:53:22.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:22 smithi193 ceph-mon[80071]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded) 2024-04-25T15:53:22.611 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:22 smithi110 ceph-mon[90991]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7/267 objects degraded (2.622%), 3 pgs degraded) 2024-04-25T15:53:23.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:23 smithi193 ceph-mon[80071]: pgmap v56: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 424 B/s wr, 0 op/s 2024-04-25T15:53:23.506 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:23 smithi110 ceph-mon[90991]: pgmap v56: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 424 B/s wr, 0 op/s 2024-04-25T15:53:25.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.191+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/1139464660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fdfd0 msgr2=0x7fb5a00fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.191+0000 7fb5a6bdf700 1 --2- 172.21.15.110:0/1139464660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fdfd0 0x7fb5a00fe440 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb590004940 tx=0x7fb590009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.191+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/1139464660 shutdown_connections 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.191+0000 7fb5a6bdf700 1 --2- 172.21.15.110:0/1139464660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fdfd0 0x7fb5a00fe440 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.191+0000 7fb5a6bdf700 1 --2- 172.21.15.110:0/1139464660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5a00fce60 0x7fb5a00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.191+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/1139464660 >> 172.21.15.110:0/1139464660 conn(0x7fb5a00f8540 msgr2=0x7fb5a00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.191+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/1139464660 shutdown_connections 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.191+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/1139464660 wait complete. 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a6bdf700 1 Processor -- start 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a6bdf700 1 -- start start 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a6bdf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fce60 0x7fb5a010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a6bdf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5a00fdfd0 0x7fb5a010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a6bdf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a010e450 con 0x7fb5a00fce60 2024-04-25T15:53:25.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a6bdf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5a010e5c0 con 0x7fb5a00fdfd0 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a497b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fce60 0x7fb5a010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a497b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fce60 0x7fb5a010d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50312/0 (socket says 172.21.15.110:50312) 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.192+0000 7fb5a497b700 1 -- 172.21.15.110:0/2084518315 learned_addr learned my addr 172.21.15.110:0/2084518315 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb597fff700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5a00fdfd0 0x7fb5a010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5a497b700 1 -- 172.21.15.110:0/2084518315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5a00fdfd0 msgr2=0x7fb5a010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5a497b700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5a00fdfd0 0x7fb5a010de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5a497b700 1 -- 172.21.15.110:0/2084518315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb590005040 con 0x7fb5a00fce60 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5a497b700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fce60 0x7fb5a010d910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb59c002a50 tx=0x7fb59c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5957fa700 1 -- 172.21.15.110:0/2084518315 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb59c007cc0 con 0x7fb5a00fce60 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5a0112fc0 con 0x7fb5a00fce60 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5957fa700 1 -- 172.21.15.110:0/2084518315 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb59c00f040 con 0x7fb5a00fce60 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5957fa700 1 -- 172.21.15.110:0/2084518315 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb59c009640 con 0x7fb5a00fce60 2024-04-25T15:53:25.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.193+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5a0113550 con 0x7fb5a00fce60 2024-04-25T15:53:25.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.194+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5a004c710 con 0x7fb5a00fce60 2024-04-25T15:53:25.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.195+0000 7fb5957fa700 1 -- 172.21.15.110:0/2084518315 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fb59c0097a0 con 0x7fb5a00fce60 2024-04-25T15:53:25.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.195+0000 7fb5957fa700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb588076b90 0x7fb588079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.195+0000 7fb5957fa700 1 -- 172.21.15.110:0/2084518315 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb59c098070 con 0x7fb5a00fce60 2024-04-25T15:53:25.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.195+0000 7fb597fff700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb588076b90 0x7fb588079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.196+0000 7fb597fff700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb588076b90 0x7fb588079040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb59000b3b0 tx=0x7fb590009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:25.201 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.200+0000 7fb5957fa700 1 -- 172.21.15.110:0/2084518315 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb59c060bf0 con 0x7fb5a00fce60 2024-04-25T15:53:25.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.417+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5a0113740 con 0x7fb588076b90 2024-04-25T15:53:25.419 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.418+0000 7fb5957fa700 1 -- 172.21.15.110:0/2084518315 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb5a0113740 con 0x7fb588076b90 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.420+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb588076b90 msgr2=0x7fb588079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.420+0000 7fb5a6bdf700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb588076b90 0x7fb588079040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb59000b3b0 tx=0x7fb590009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.420+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fce60 msgr2=0x7fb5a010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.420+0000 7fb5a6bdf700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fce60 0x7fb5a010d910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb59c002a50 tx=0x7fb59c0046b0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.421+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 shutdown_connections 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.421+0000 7fb5a6bdf700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb588076b90 0x7fb588079040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.421+0000 7fb5a6bdf700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5a00fce60 0x7fb5a010d910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.421+0000 7fb5a6bdf700 1 --2- 172.21.15.110:0/2084518315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5a00fdfd0 0x7fb5a010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.421+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 >> 172.21.15.110:0/2084518315 conn(0x7fb5a00f8540 msgr2=0x7fb5a00f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.421+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 shutdown_connections 2024-04-25T15:53:25.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.421+0000 7fb5a6bdf700 1 -- 172.21.15.110:0/2084518315 wait complete. 2024-04-25T15:53:25.432 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:53:25.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.524+0000 7fae61ca1700 1 -- 172.21.15.110:0/3156015133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fbcc0 msgr2=0x7fae5c0fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.524+0000 7fae61ca1700 1 --2- 172.21.15.110:0/3156015133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fbcc0 0x7fae5c0fc0d0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fae4c004940 tx=0x7fae4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:25.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.524+0000 7fae61ca1700 1 -- 172.21.15.110:0/3156015133 shutdown_connections 2024-04-25T15:53:25.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.524+0000 7fae61ca1700 1 --2- 172.21.15.110:0/3156015133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae5c0fcf70 0x7fae5c0fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.524+0000 7fae61ca1700 1 --2- 172.21.15.110:0/3156015133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fbcc0 0x7fae5c0fc0d0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.524+0000 7fae61ca1700 1 -- 172.21.15.110:0/3156015133 >> 172.21.15.110:0/3156015133 conn(0x7fae5c0f7420 msgr2=0x7fae5c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:25.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.525+0000 7fae61ca1700 1 -- 172.21.15.110:0/3156015133 shutdown_connections 2024-04-25T15:53:25.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.525+0000 7fae61ca1700 1 -- 172.21.15.110:0/3156015133 wait complete. 2024-04-25T15:53:25.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.525+0000 7fae61ca1700 1 Processor -- start 2024-04-25T15:53:25.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae61ca1700 1 -- start start 2024-04-25T15:53:25.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae61ca1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae5c0fbcc0 0x7fae5c10d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae61ca1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fcf70 0x7fae5c10dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.528 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae61ca1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c10e4c0 con 0x7fae5c0fcf70 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae61ca1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5c10e630 con 0x7fae5c0fbcc0 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae5affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fcf70 0x7fae5c10dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae5affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fcf70 0x7fae5c10dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50320/0 (socket says 172.21.15.110:50320) 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae5affd700 1 -- 172.21.15.110:0/3379871578 learned_addr learned my addr 172.21.15.110:0/3379871578 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae5b7fe700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae5c0fbcc0 0x7fae5c10d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae5affd700 1 -- 172.21.15.110:0/3379871578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae5c0fbcc0 msgr2=0x7fae5c10d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae5affd700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae5c0fbcc0 0x7fae5c10d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.526+0000 7fae5affd700 1 -- 172.21.15.110:0/3379871578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4c005040 con 0x7fae5c0fcf70 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.527+0000 7fae5affd700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fcf70 0x7fae5c10dec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae5000c8c0 tx=0x7fae500044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.527+0000 7fae43fff700 1 -- 172.21.15.110:0/3379871578 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae5000a980 con 0x7fae5c0fcf70 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.527+0000 7fae43fff700 1 -- 172.21.15.110:0/3379871578 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae5000ce60 con 0x7fae5c0fcf70 2024-04-25T15:53:25.529 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.527+0000 7fae43fff700 1 -- 172.21.15.110:0/3379871578 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae500115e0 con 0x7fae5c0fcf70 2024-04-25T15:53:25.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.527+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae5c113040 con 0x7fae5c0fcf70 2024-04-25T15:53:25.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.527+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae5c1135d0 con 0x7fae5c0fcf70 2024-04-25T15:53:25.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.528+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae5c101b00 con 0x7fae5c0fcf70 2024-04-25T15:53:25.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.528+0000 7fae43fff700 1 -- 172.21.15.110:0/3379871578 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fae50012400 con 0x7fae5c0fcf70 2024-04-25T15:53:25.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.529+0000 7fae43fff700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fae44076b90 0x7fae44079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.529+0000 7fae43fff700 1 -- 172.21.15.110:0/3379871578 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fae50098300 con 0x7fae5c0fcf70 2024-04-25T15:53:25.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.529+0000 7fae5b7fe700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fae44076b90 0x7fae44079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.529+0000 7fae5b7fe700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fae44076b90 0x7fae44079040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00b3b0 tx=0x7fae4c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:25.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.534+0000 7fae43fff700 1 -- 172.21.15.110:0/3379871578 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fae50060fa0 con 0x7fae5c0fcf70 2024-04-25T15:53:25.760 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.759+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae5c02b360 con 0x7fae44076b90 2024-04-25T15:53:25.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.760+0000 7fae43fff700 1 -- 172.21.15.110:0/3379871578 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fae5c02b360 con 0x7fae44076b90 2024-04-25T15:53:25.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fae44076b90 msgr2=0x7fae44079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fae44076b90 0x7fae44079040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fae4c00b3b0 tx=0x7fae4c009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:25.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fcf70 msgr2=0x7fae5c10dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.763 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fcf70 0x7fae5c10dec0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fae5000c8c0 tx=0x7fae500044c0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 shutdown_connections 2024-04-25T15:53:25.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae5c0fbcc0 0x7fae5c10d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fae44076b90 0x7fae44079040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 --2- 172.21.15.110:0/3379871578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae5c0fcf70 0x7fae5c10dec0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 >> 172.21.15.110:0/3379871578 conn(0x7fae5c0f7420 msgr2=0x7fae5c066850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:25.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 shutdown_connections 2024-04-25T15:53:25.764 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.762+0000 7fae61ca1700 1 -- 172.21.15.110:0/3379871578 wait complete. 2024-04-25T15:53:25.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:25 smithi110 ceph-mon[90991]: pgmap v57: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 383 B/s wr, 0 op/s 2024-04-25T15:53:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.863+0000 7fbaa504e700 1 -- 172.21.15.110:0/1812967752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 msgr2=0x7fbaa00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.863+0000 7fbaa504e700 1 --2- 172.21.15.110:0/1812967752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 0x7fbaa00fd3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fba94004940 tx=0x7fba94009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.863+0000 7fbaa504e700 1 -- 172.21.15.110:0/1812967752 shutdown_connections 2024-04-25T15:53:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.863+0000 7fbaa504e700 1 --2- 172.21.15.110:0/1812967752 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 0x7fbaa00fd3d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.863+0000 7fbaa504e700 1 --2- 172.21.15.110:0/1812967752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaa00fbcb0 0x7fbaa00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.863+0000 7fbaa504e700 1 -- 172.21.15.110:0/1812967752 >> 172.21.15.110:0/1812967752 conn(0x7fbaa00f7430 msgr2=0x7fbaa00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.863+0000 7fbaa504e700 1 -- 172.21.15.110:0/1812967752 shutdown_connections 2024-04-25T15:53:25.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.863+0000 7fbaa504e700 1 -- 172.21.15.110:0/1812967752 wait complete. 2024-04-25T15:53:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fbaa504e700 1 Processor -- start 2024-04-25T15:53:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fbaa504e700 1 -- start start 2024-04-25T15:53:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fbaa504e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaa00fbcb0 0x7fbaa010c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fbaa504e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 0x7fbaa010ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fbaa504e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa010d470 con 0x7fbaa00fcf60 2024-04-25T15:53:25.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fbaa504e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa010dcc0 con 0x7fbaa00fbcb0 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fba9e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 0x7fbaa010ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fba9e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 0x7fbaa010ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50338/0 (socket says 172.21.15.110:50338) 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fba9e59c700 1 -- 172.21.15.110:0/3963252302 learned_addr learned my addr 172.21.15.110:0/3963252302 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fba9ed9d700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaa00fbcb0 0x7fbaa010c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fba9e59c700 1 -- 172.21.15.110:0/3963252302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaa00fbcb0 msgr2=0x7fbaa010c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fba9e59c700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaa00fbcb0 0x7fbaa010c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fba9e59c700 1 -- 172.21.15.110:0/3963252302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba94005040 con 0x7fbaa00fcf60 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.864+0000 7fba9ed9d700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaa00fbcb0 0x7fbaa010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.865+0000 7fba9e59c700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 0x7fbaa010ce70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba94009fd0 tx=0x7fba94009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.865+0000 7fba877fe700 1 -- 172.21.15.110:0/3963252302 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba9400e040 con 0x7fbaa00fcf60 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.865+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa010df40 con 0x7fbaa00fcf60 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.865+0000 7fba877fe700 1 -- 172.21.15.110:0/3963252302 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba9400bed0 con 0x7fbaa00fcf60 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.865+0000 7fba877fe700 1 -- 172.21.15.110:0/3963252302 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba940137c0 con 0x7fbaa00fcf60 2024-04-25T15:53:25.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.865+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa010e480 con 0x7fbaa00fcf60 2024-04-25T15:53:25.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.866+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa005f5c0 con 0x7fbaa00fcf60 2024-04-25T15:53:25.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.866+0000 7fba877fe700 1 -- 172.21.15.110:0/3963252302 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fba9401fbc0 con 0x7fbaa00fcf60 2024-04-25T15:53:25.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.867+0000 7fba877fe700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fba88076b90 0x7fba88079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:25.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.867+0000 7fba877fe700 1 -- 172.21.15.110:0/3963252302 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fba9409bc90 con 0x7fbaa00fcf60 2024-04-25T15:53:25.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.867+0000 7fba9ed9d700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fba88076b90 0x7fba88079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:25.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.867+0000 7fba9ed9d700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fba88076b90 0x7fba88079040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba98009fd0 tx=0x7fba98009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:25.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:25.872+0000 7fba877fe700 1 -- 172.21.15.110:0/3963252302 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fba940648c0 con 0x7fbaa00fcf60 2024-04-25T15:53:26.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.078+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaa010e810 con 0x7fba88076b90 2024-04-25T15:53:26.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.088+0000 7fba877fe700 1 -- 172.21.15.110:0/3963252302 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fbaa010e810 con 0x7fba88076b90 2024-04-25T15:53:26.089 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:53:26.089 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (3m) 11s ago 12m 26.3M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:53:26.089 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (44s) 11s ago 12m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:53:26.089 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (38s) 35s ago 11m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:53:26.089 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (3m) 11s ago 12m 77.8M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (6m) 11s ago 6m 19.5M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (7m) 11s ago 7m 150M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (7m) 35s ago 7m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (6m) 35s ago 6m 16.7M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (3m) 11s ago 13m 567M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (2m) 35s ago 11m 517M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (108s) 11s ago 13m 111M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (85s) 35s ago 11m 98.5M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (4m) 11s ago 12m 15.6M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (3m) 35s ago 11m 15.7M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (17s) 11s ago 10m 82.2M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (10m) 11s ago 10m 218M 1557M 17.2.7-902-gc61472c8 0de407c1b3af 1f9895a414c8 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (9m) 11s ago 9m 178M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:53:26.090 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (9m) 35s ago 9m 239M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:53:26.091 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (8m) 35s ago 8m 206M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:53:26.091 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (8m) 35s ago 8m 189M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:53:26.091 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (2m) 11s ago 11m 54.5M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fba88076b90 msgr2=0x7fba88079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fba88076b90 0x7fba88079040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fba98009fd0 tx=0x7fba98009ee0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 msgr2=0x7fbaa010ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 0x7fbaa010ce70 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fba94009fd0 tx=0x7fba94009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 shutdown_connections 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbaa00fbcb0 0x7fbaa010c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fba88076b90 0x7fba88079040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 --2- 172.21.15.110:0/3963252302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbaa00fcf60 0x7fbaa010ce70 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 >> 172.21.15.110:0/3963252302 conn(0x7fbaa00f7430 msgr2=0x7fbaa00f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:26.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 shutdown_connections 2024-04-25T15:53:26.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.091+0000 7fbaa504e700 1 -- 172.21.15.110:0/3963252302 wait complete. 2024-04-25T15:53:26.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.193+0000 7fcec1194700 1 -- 172.21.15.110:0/904555724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 msgr2=0x7fcebc0fe390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.193+0000 7fcec1194700 1 --2- 172.21.15.110:0/904555724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 0x7fcebc0fe390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fceb0004940 tx=0x7fceb0009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:26.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.193+0000 7fcec1194700 1 -- 172.21.15.110:0/904555724 shutdown_connections 2024-04-25T15:53:26.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.193+0000 7fcec1194700 1 --2- 172.21.15.110:0/904555724 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 0x7fcebc0fe390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.193+0000 7fcec1194700 1 --2- 172.21.15.110:0/904555724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc0fb1e0 0x7fcebc0fb5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.194 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.193+0000 7fcec1194700 1 -- 172.21.15.110:0/904555724 >> 172.21.15.110:0/904555724 conn(0x7fcebc0f6be0 msgr2=0x7fcebc0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:26.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.193+0000 7fcec1194700 1 -- 172.21.15.110:0/904555724 shutdown_connections 2024-04-25T15:53:26.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.193+0000 7fcec1194700 1 -- 172.21.15.110:0/904555724 wait complete. 2024-04-25T15:53:26.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fcec1194700 1 Processor -- start 2024-04-25T15:53:26.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fcec1194700 1 -- start start 2024-04-25T15:53:26.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fcec1194700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc0fb1e0 0x7fcebc10db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fcec1194700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 0x7fcebc10e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fcec1194700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc10e650 con 0x7fcebc0fbb30 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fcec1194700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc10e7c0 con 0x7fcebc0fb1e0 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fceba59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 0x7fcebc10e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fceba59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 0x7fcebc10e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50352/0 (socket says 172.21.15.110:50352) 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fceba59c700 1 -- 172.21.15.110:0/3169543385 learned_addr learned my addr 172.21.15.110:0/3169543385 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.194+0000 7fcebad9d700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc0fb1e0 0x7fcebc10db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fceba59c700 1 -- 172.21.15.110:0/3169543385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc0fb1e0 msgr2=0x7fcebc10db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fceba59c700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc0fb1e0 0x7fcebc10db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fceba59c700 1 -- 172.21.15.110:0/3169543385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb0005040 con 0x7fcebc0fbb30 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fcebad9d700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc0fb1e0 0x7fcebc10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fceba59c700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 0x7fcebc10e050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009150 tx=0x7fceb0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fcea37fe700 1 -- 172.21.15.110:0/3169543385 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceb000e040 con 0x7fcebc0fbb30 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcebc1131c0 con 0x7fcebc0fbb30 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc113780 con 0x7fcebc0fbb30 2024-04-25T15:53:26.197 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fcea37fe700 1 -- 172.21.15.110:0/3169543385 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fceb000bcc0 con 0x7fcebc0fbb30 2024-04-25T15:53:26.198 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.195+0000 7fcea37fe700 1 -- 172.21.15.110:0/3169543385 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fceb001c630 con 0x7fcebc0fbb30 2024-04-25T15:53:26.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.196+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcea80052e0 con 0x7fcebc0fbb30 2024-04-25T15:53:26.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.197+0000 7fcea37fe700 1 -- 172.21.15.110:0/3169543385 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fceb0013420 con 0x7fcebc0fbb30 2024-04-25T15:53:26.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.197+0000 7fcea37fe700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcea4076b90 0x7fcea4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.197+0000 7fcea37fe700 1 -- 172.21.15.110:0/3169543385 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fceb0098e30 con 0x7fcebc0fbb30 2024-04-25T15:53:26.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.197+0000 7fcebad9d700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcea4076b90 0x7fcea4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.199 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.198+0000 7fcebad9d700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcea4076b90 0x7fcea4079040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fceac0041d0 tx=0x7fceac009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:26.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.202+0000 7fcea37fe700 1 -- 172.21.15.110:0/3169543385 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fceb0061ad0 con 0x7fcebc0fbb30 2024-04-25T15:53:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:25 smithi193 ceph-mon[80071]: pgmap v57: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 383 B/s wr, 0 op/s 2024-04-25T15:53:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.484+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcea8005da0 con 0x7fcebc0fbb30 2024-04-25T15:53:26.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.484+0000 7fcea37fe700 1 -- 172.21.15.110:0/3169543385 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fceb0061470 con 0x7fcebc0fbb30 2024-04-25T15:53:26.485 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:53:26.485 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:53:26.485 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5, 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 9, 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 5 2024-04-25T15:53:26.486 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:53:26.487 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:53:26.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcea4076b90 msgr2=0x7fcea4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcea4076b90 0x7fcea4079040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fceac0041d0 tx=0x7fceac009040 comp rx=0 tx=0).stop 2024-04-25T15:53:26.487 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 msgr2=0x7fcebc10e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 0x7fcebc10e050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fceb0009150 tx=0x7fceb0009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:26.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 shutdown_connections 2024-04-25T15:53:26.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcebc0fb1e0 0x7fcebc10db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcea4076b90 0x7fcea4079040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 --2- 172.21.15.110:0/3169543385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcebc0fbb30 0x7fcebc10e050 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 >> 172.21.15.110:0/3169543385 conn(0x7fcebc0f6be0 msgr2=0x7fcebc0f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:26.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 shutdown_connections 2024-04-25T15:53:26.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.486+0000 7fcec1194700 1 -- 172.21.15.110:0/3169543385 wait complete. 2024-04-25T15:53:26.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.592+0000 7f6fce48d700 1 -- 172.21.15.110:0/2705021093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 msgr2=0x7f6fc80fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.592+0000 7f6fce48d700 1 --2- 172.21.15.110:0/2705021093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 0x7f6fc80fe440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8004940 tx=0x7f6fb8009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:26.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.592+0000 7f6fce48d700 1 -- 172.21.15.110:0/2705021093 shutdown_connections 2024-04-25T15:53:26.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.592+0000 7f6fce48d700 1 --2- 172.21.15.110:0/2705021093 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 0x7f6fc80fe440 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.592+0000 7f6fce48d700 1 --2- 172.21.15.110:0/2705021093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80fce60 0x7f6fc80fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.592+0000 7f6fce48d700 1 -- 172.21.15.110:0/2705021093 >> 172.21.15.110:0/2705021093 conn(0x7f6fc80f8540 msgr2=0x7f6fc80fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:26.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.592+0000 7f6fce48d700 1 -- 172.21.15.110:0/2705021093 shutdown_connections 2024-04-25T15:53:26.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.592+0000 7f6fce48d700 1 -- 172.21.15.110:0/2705021093 wait complete. 2024-04-25T15:53:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fce48d700 1 Processor -- start 2024-04-25T15:53:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fce48d700 1 -- start start 2024-04-25T15:53:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fce48d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80fce60 0x7f6fc8071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fce48d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 0x7f6fc80702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fce48d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc8072200 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fce48d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80707f0 con 0x7f6fc80fce60 2024-04-25T15:53:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fc77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 0x7f6fc80702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.595 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fc77fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 0x7f6fc80702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50362/0 (socket says 172.21.15.110:50362) 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fc77fe700 1 -- 172.21.15.110:0/20219198 learned_addr learned my addr 172.21.15.110:0/20219198 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fc7fff700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80fce60 0x7f6fc8071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fc77fe700 1 -- 172.21.15.110:0/20219198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80fce60 msgr2=0x7f6fc8071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fc77fe700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80fce60 0x7f6fc8071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fc77fe700 1 -- 172.21.15.110:0/20219198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb8005040 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.593+0000 7f6fc77fe700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 0x7f6fc80702b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800b3b0 tx=0x7f6fb8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.594+0000 7f6fc4ff9700 1 -- 172.21.15.110:0/20219198 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fb800e040 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.594+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc8070a40 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.594+0000 7f6fc4ff9700 1 -- 172.21.15.110:0/20219198 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fb800bbf0 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.594+0000 7f6fc4ff9700 1 -- 172.21.15.110:0/20219198 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fb801c630 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.596 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.594+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc8071000 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.595+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc804c710 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.595+0000 7f6fc4ff9700 1 -- 172.21.15.110:0/20219198 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f6fb800bd60 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.596+0000 7f6fc4ff9700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6fb0076b90 0x7f6fb0079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.596+0000 7f6fc7fff700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6fb0076b90 0x7f6fb0079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.596+0000 7f6fc4ff9700 1 -- 172.21.15.110:0/20219198 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6fb8098f30 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.597 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.596+0000 7f6fc7fff700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6fb0076b90 0x7f6fb0079040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0041d0 tx=0x7f6fbc0043c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:26.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.600+0000 7f6fc4ff9700 1 -- 172.21.15.110:0/20219198 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6fb8061ad0 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.851+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6fc81058c0 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.852 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.851+0000 7f6fc4ff9700 1 -- 172.21.15.110:0/20219198 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f6fb8061470 con 0x7f6fc80fdfd0 2024-04-25T15:53:26.852 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:53:26.852 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:53:26.852 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:53:26.852 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:53:26.852 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:53:26.853 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:53:26.854 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:53:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:53:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:53:26.855 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:53:26.855 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:53:26.855 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:53:26.855 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:53:26.855 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6fb0076b90 msgr2=0x7f6fb0079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6fb0076b90 0x7f6fb0079040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0041d0 tx=0x7f6fbc0043c0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 msgr2=0x7f6fc80702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 0x7f6fc80702b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6fb800b3b0 tx=0x7f6fb8009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 shutdown_connections 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80fce60 0x7f6fc8071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6fb0076b90 0x7f6fb0079040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 --2- 172.21.15.110:0/20219198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6fc80fdfd0 0x7f6fc80702b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 >> 172.21.15.110:0/20219198 conn(0x7f6fc80f8540 msgr2=0x7f6fc80f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 shutdown_connections 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.854+0000 7f6fce48d700 1 -- 172.21.15.110:0/20219198 wait complete. 2024-04-25T15:53:26.856 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:53:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 -- 172.21.15.110:0/58885501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 msgr2=0x7f8b8c102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 --2- 172.21.15.110:0/58885501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 0x7f8b8c102370 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8b80004940 tx=0x7f8b80009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 -- 172.21.15.110:0/58885501 shutdown_connections 2024-04-25T15:53:26.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 --2- 172.21.15.110:0/58885501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 0x7f8b8c102370 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 --2- 172.21.15.110:0/58885501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8c066ed0 0x7f8b8c0ffa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 -- 172.21.15.110:0/58885501 >> 172.21.15.110:0/58885501 conn(0x7f8b8c0f7480 msgr2=0x7f8b8c0f9890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:26.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 -- 172.21.15.110:0/58885501 shutdown_connections 2024-04-25T15:53:26.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 -- 172.21.15.110:0/58885501 wait complete. 2024-04-25T15:53:26.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.956+0000 7f8b930ef700 1 Processor -- start 2024-04-25T15:53:26.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b930ef700 1 -- start start 2024-04-25T15:53:26.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b930ef700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8c066ed0 0x7f8b8c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b930ef700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 0x7f8b8c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b930ef700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c10e4b0 con 0x7f8b8c0fff90 2024-04-25T15:53:26.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b930ef700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8c10e620 con 0x7f8b8c066ed0 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b8bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 0x7f8b8c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b8bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 0x7f8b8c10deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50378/0 (socket says 172.21.15.110:50378) 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b8bfff700 1 -- 172.21.15.110:0/2928586068 learned_addr learned my addr 172.21.15.110:0/2928586068 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b90e8b700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8c066ed0 0x7f8b8c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b8bfff700 1 -- 172.21.15.110:0/2928586068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8c066ed0 msgr2=0x7f8b8c10d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b8bfff700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8c066ed0 0x7f8b8c10d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b8bfff700 1 -- 172.21.15.110:0/2928586068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b80005040 con 0x7f8b8c0fff90 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.957+0000 7f8b8bfff700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 0x7f8b8c10deb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000b3b0 tx=0x7f8b80009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.958+0000 7f8b897fa700 1 -- 172.21.15.110:0/2928586068 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b8000e040 con 0x7f8b8c0fff90 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.958+0000 7f8b897fa700 1 -- 172.21.15.110:0/2928586068 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b8000bcc0 con 0x7f8b8c0fff90 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.958+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c112fd0 con 0x7f8b8c0fff90 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.958+0000 7f8b897fa700 1 -- 172.21.15.110:0/2928586068 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b8001c630 con 0x7f8b8c0fff90 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.958+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8c113560 con 0x7f8b8c0fff90 2024-04-25T15:53:26.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.959+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8c04c710 con 0x7f8b8c0fff90 2024-04-25T15:53:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.959+0000 7f8b897fa700 1 -- 172.21.15.110:0/2928586068 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f8b80013980 con 0x7f8b8c0fff90 2024-04-25T15:53:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.960+0000 7f8b897fa700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8b74076b90 0x7f8b74079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.960+0000 7f8b897fa700 1 -- 172.21.15.110:0/2928586068 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8b80098fc0 con 0x7f8b8c0fff90 2024-04-25T15:53:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.960+0000 7f8b90e8b700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8b74076b90 0x7f8b74079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:26.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.960+0000 7f8b90e8b700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8b74076b90 0x7f8b74079040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c066c10 tx=0x7f8b8400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:26.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:26.965+0000 7f8b897fa700 1 -- 172.21.15.110:0/2928586068 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8b80061b40 con 0x7f8b8c0fff90 2024-04-25T15:53:27.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:26 smithi110 ceph-mon[90991]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:27.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:26 smithi110 ceph-mon[90991]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:27.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:26 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3169543385' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:27.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.183+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b8c0f8f70 con 0x7f8b74076b90 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.184+0000 7f8b897fa700 1 -- 172.21.15.110:0/2928586068 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f8b8c0f8f70 con 0x7f8b74076b90 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:53:27.185 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:53:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "7/21 daemons upgraded", 2024-04-25T15:53:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T15:53:27.186 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:53:27.186 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:53:27.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8b74076b90 msgr2=0x7f8b74079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:27.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8b74076b90 0x7f8b74079040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c066c10 tx=0x7f8b8400b040 comp rx=0 tx=0).stop 2024-04-25T15:53:27.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 msgr2=0x7f8b8c10deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:27.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 0x7f8b8c10deb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000b3b0 tx=0x7f8b80009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 shutdown_connections 2024-04-25T15:53:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8c066ed0 0x7f8b8c10d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8b74076b90 0x7f8b74079040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 --2- 172.21.15.110:0/2928586068 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b8c0fff90 0x7f8b8c10deb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 >> 172.21.15.110:0/2928586068 conn(0x7f8b8c0f7480 msgr2=0x7f8b8c0f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 shutdown_connections 2024-04-25T15:53:27.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.186+0000 7f8b930ef700 1 -- 172.21.15.110:0/2928586068 wait complete. 2024-04-25T15:53:27.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:26 smithi193 ceph-mon[80071]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:27.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:26 smithi193 ceph-mon[80071]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:27.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:26 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3169543385' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/1400782284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 msgr2=0x7fe6f40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 --2- 172.21.15.110:0/1400782284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 0x7fe6f40fc0c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6e8004940 tx=0x7fe6e8009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/1400782284 shutdown_connections 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 --2- 172.21.15.110:0/1400782284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f40fcf60 0x7fe6f40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 --2- 172.21.15.110:0/1400782284 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 0x7fe6f40fc0c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/1400782284 >> 172.21.15.110:0/1400782284 conn(0x7fe6f40f7430 msgr2=0x7fe6f40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/1400782284 shutdown_connections 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/1400782284 wait complete. 2024-04-25T15:53:27.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.287+0000 7fe6fa5c6700 1 Processor -- start 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6fa5c6700 1 -- start start 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6fa5c6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 0x7fe6f4110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6fa5c6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f40fcf60 0x7fe6f4112c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6fa5c6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4066330 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6fa5c6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f4066470 con 0x7fe6f40fcf60 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 0x7fe6f4110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f3fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 0x7fe6f4110720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50402/0 (socket says 172.21.15.110:50402) 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f3fff700 1 -- 172.21.15.110:0/3987323254 learned_addr learned my addr 172.21.15.110:0/3987323254 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f37fe700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f40fcf60 0x7fe6f4112c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f3fff700 1 -- 172.21.15.110:0/3987323254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f40fcf60 msgr2=0x7fe6f4112c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f3fff700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f40fcf60 0x7fe6f4112c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f3fff700 1 -- 172.21.15.110:0/3987323254 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e8005040 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f37fe700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f40fcf60 0x7fe6f4112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:53:27.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f3fff700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 0x7fe6f4110720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800b3b0 tx=0x7fe6e8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:27.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6f0ff9700 1 -- 172.21.15.110:0/3987323254 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6e800e040 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.288+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f41131b0 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.289+0000 7fe6f0ff9700 1 -- 172.21.15.110:0/3987323254 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe6e800bcc0 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.289+0000 7fe6f0ff9700 1 -- 172.21.15.110:0/3987323254 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6e801c630 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.289+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f4113770 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.289+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f405f5c0 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.290+0000 7fe6f0ff9700 1 -- 172.21.15.110:0/3987323254 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fe6e8013980 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.291+0000 7fe6f0ff9700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe6dc076b90 0x7fe6dc079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:27.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.291+0000 7fe6f0ff9700 1 -- 172.21.15.110:0/3987323254 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fe6e8099710 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.291+0000 7fe6f37fe700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe6dc076b90 0x7fe6dc079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:27.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.291+0000 7fe6f37fe700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe6dc076b90 0x7fe6dc079040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40041d0 tx=0x7fe6e4009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:27.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.295+0000 7fe6f0ff9700 1 -- 172.21.15.110:0/3987323254 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe6e8062310 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.574+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe6f404bed0 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.575+0000 7fe6f0ff9700 1 -- 172.21.15.110:0/3987323254 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe6e8061cb0 con 0x7fe6f40fbcb0 2024-04-25T15:53:27.576 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:53:27.576 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:53:27.576 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe6dc076b90 msgr2=0x7fe6dc079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe6dc076b90 0x7fe6dc079040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe6e40041d0 tx=0x7fe6e4009040 comp rx=0 tx=0).stop 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 msgr2=0x7fe6f4110720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 0x7fe6f4110720 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800b3b0 tx=0x7fe6e8009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 shutdown_connections 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe6dc076b90 0x7fe6dc079040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe6f40fbcb0 0x7fe6f4110720 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 --2- 172.21.15.110:0/3987323254 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6f40fcf60 0x7fe6f4112c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:27.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 >> 172.21.15.110:0/3987323254 conn(0x7fe6f40f7430 msgr2=0x7fe6f40f8b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:27.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 shutdown_connections 2024-04-25T15:53:27.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:27.577+0000 7fe6fa5c6700 1 -- 172.21.15.110:0/3987323254 wait complete. 2024-04-25T15:53:27.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:27 smithi110 ceph-mon[90991]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:27.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:27 smithi110 ceph-mon[90991]: pgmap v58: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 321 B/s wr, 0 op/s 2024-04-25T15:53:27.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:27 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/20219198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:53:27.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:27 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3987323254' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:53:28.140 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:27 smithi193 ceph-mon[80071]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:28.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:27 smithi193 ceph-mon[80071]: pgmap v58: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 321 B/s wr, 0 op/s 2024-04-25T15:53:28.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:27 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/20219198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:53:28.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:27 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3987323254' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:53:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:28 smithi110 ceph-mon[90991]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:28 smithi193 ceph-mon[80071]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:29 smithi110 ceph-mon[90991]: pgmap v59: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:29 smithi193 ceph-mon[80071]: pgmap v59: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:31 smithi110 ceph-mon[90991]: pgmap v60: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:53:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:31 smithi193 ceph-mon[80071]: pgmap v60: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:53:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:33 smithi110 ceph-mon[90991]: pgmap v61: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:33 smithi193 ceph-mon[80071]: pgmap v61: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:35.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:34 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T15:53:35.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:34 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T15:53:36.100 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:35 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T15:53:36.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:35 smithi110 ceph-mon[90991]: Upgrade: osd.1 is safe to restart 2024-04-25T15:53:36.101 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:35 smithi110 ceph-mon[90991]: pgmap v62: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:36.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:35 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-25T15:53:36.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:35 smithi193 ceph-mon[80071]: Upgrade: osd.1 is safe to restart 2024-04-25T15:53:36.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:35 smithi193 ceph-mon[80071]: pgmap v62: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:37.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:37 smithi193 ceph-mon[80071]: Upgrade: Updating osd.1 2024-04-25T15:53:37.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:37.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T15:53:37.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:53:37.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:37 smithi193 ceph-mon[80071]: Deploying daemon osd.1 on smithi110 2024-04-25T15:53:37.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:37 smithi193 ceph-mon[80071]: pgmap v63: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:37 smithi110 ceph-mon[90991]: Upgrade: Updating osd.1 2024-04-25T15:53:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T15:53:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:53:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:37 smithi110 ceph-mon[90991]: Deploying daemon osd.1 on smithi110 2024-04-25T15:53:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:37 smithi110 ceph-mon[90991]: pgmap v63: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:39.679 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:39 smithi110 systemd[1]: Stopping Ceph osd.1 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:53:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:39 smithi110 ceph-mon[90991]: pgmap v64: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:39 smithi110 ceph-mon[90991]: osd.1 marked itself down and dead 2024-04-25T15:53:40.113 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:39 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[53015]: 2024-04-25T15:53:39.676+0000 7fdf68493700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:53:40.113 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:39 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[53015]: 2024-04-25T15:53:39.676+0000 7fdf68493700 -1 osd.1 54 *** Got signal Terminated *** 2024-04-25T15:53:40.113 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:39 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[53015]: 2024-04-25T15:53:39.676+0000 7fdf68493700 -1 osd.1 54 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:53:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:39 smithi193 ceph-mon[80071]: pgmap v64: 65 pgs: 65 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:39 smithi193 ceph-mon[80071]: osd.1 marked itself down and dead 2024-04-25T15:53:40.380 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:40 smithi110 podman[104900]: 2024-04-25 15:53:40.1273035 +0000 UTC m=+0.562671669 container died 1f9895a414c88fb5ef78b3db761b19bac0dc70e0ad3ad8735c7f10e656920d7c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T15:53:40.638 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:40 smithi110 podman[104900]: 2024-04-25 15:53:40.453626239 +0000 UTC m=+0.888994406 container cleanup 1f9895a414c88fb5ef78b3db761b19bac0dc70e0ad3ad8735c7f10e656920d7c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:53:40.638 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:40 smithi110 bash[104900]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1 2024-04-25T15:53:41.072 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:40 smithi110 ceph-mon[90991]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:53:41.072 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:40 smithi110 ceph-mon[90991]: osdmap e55: 6 total, 5 up, 6 in 2024-04-25T15:53:41.072 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:40 smithi110 ceph-mon[90991]: osdmap e56: 6 total, 5 up, 6 in 2024-04-25T15:53:41.073 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:40 smithi110 podman[104917]: 2024-04-25 15:53:40.638409824 +0000 UTC m=+0.509388272 container remove 1f9895a414c88fb5ef78b3db761b19bac0dc70e0ad3ad8735c7f10e656920d7c (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T15:53:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:40 smithi193 ceph-mon[80071]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:53:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:40 smithi193 ceph-mon[80071]: osdmap e55: 6 total, 5 up, 6 in 2024-04-25T15:53:41.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:40 smithi193 ceph-mon[80071]: osdmap e56: 6 total, 5 up, 6 in 2024-04-25T15:53:41.363 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:41 smithi110 podman[105011]: 2024-04-25 15:53:40.975683492 +0000 UTC m=+0.017930482 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:53:41.363 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:41 smithi110 podman[105011]: 2024-04-25 15:53:41.114592989 +0000 UTC m=+0.156839962 container create 0ecc660bc3d12d3f31e6b2fc23b9d951648730e1a9e01f22c3a00e9e9668d56f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T15:53:41.363 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:41 smithi110 podman[105011]: 2024-04-25 15:53:41.308728617 +0000 UTC m=+0.350975592 container init 0ecc660bc3d12d3f31e6b2fc23b9d951648730e1a9e01f22c3a00e9e9668d56f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T15:53:41.363 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:41 smithi110 podman[105011]: 2024-04-25 15:53:41.311721741 +0000 UTC m=+0.353968722 container start 0ecc660bc3d12d3f31e6b2fc23b9d951648730e1a9e01f22c3a00e9e9668d56f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T15:53:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:41 smithi110 ceph-mon[90991]: pgmap v66: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:41 smithi110 ceph-mon[90991]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-25T15:53:41.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:41 smithi110 podman[105011]: 2024-04-25 15:53:41.608991658 +0000 UTC m=+0.651238645 container attach 0ecc660bc3d12d3f31e6b2fc23b9d951648730e1a9e01f22c3a00e9e9668d56f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T15:53:41.864 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:41 smithi110 podman[105011]: 2024-04-25 15:53:41.609363102 +0000 UTC m=+0.651610076 container died 0ecc660bc3d12d3f31e6b2fc23b9d951648730e1a9e01f22c3a00e9e9668d56f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T15:53:42.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:41 smithi193 ceph-mon[80071]: pgmap v66: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:42.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:41 smithi193 ceph-mon[80071]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-25T15:53:42.611 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 podman[105037]: 2024-04-25 15:53:42.254697284 +0000 UTC m=+0.643142143 container remove 0ecc660bc3d12d3f31e6b2fc23b9d951648730e1a9e01f22c3a00e9e9668d56f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T15:53:42.611 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-25T15:53:42.611 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 systemd[1]: Stopped Ceph osd.1 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:53:42.611 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.1.service: Consumed 54.386s CPU time. 2024-04-25T15:53:42.611 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 systemd[1]: Starting Ceph osd.1 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:53:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:42 smithi110 ceph-mon[90991]: Health check failed: Degraded data redundancy: 32/267 objects degraded (11.985%), 11 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:42.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 podman[105142]: 2024-04-25 15:53:42.609808117 +0000 UTC m=+0.017934787 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:53:42.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 podman[105142]: 2024-04-25 15:53:42.726050464 +0000 UTC m=+0.134177118 container create 05256029e44554e3a53b383402c72c370e3166dc9a59072af0407752caed8b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T15:53:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:42 smithi193 ceph-mon[80071]: Health check failed: Degraded data redundancy: 32/267 objects degraded (11.985%), 11 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:43.363 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 podman[105142]: 2024-04-25 15:53:42.966995367 +0000 UTC m=+0.375122025 container init 05256029e44554e3a53b383402c72c370e3166dc9a59072af0407752caed8b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:53:43.363 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:42 smithi110 podman[105142]: 2024-04-25 15:53:42.970271543 +0000 UTC m=+0.378398199 container start 05256029e44554e3a53b383402c72c370e3166dc9a59072af0407752caed8b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T15:53:43.363 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 podman[105142]: 2024-04-25 15:53:43.066997421 +0000 UTC m=+0.475124089 container attach 05256029e44554e3a53b383402c72c370e3166dc9a59072af0407752caed8b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T15:53:43.820 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:53:43.820 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:53:43.820 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:53:43.820 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:53:43.820 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T15:53:43.820 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-25T15:53:43.820 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 bash[105142]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate[105157]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-25T15:53:43.821 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:43 smithi110 podman[105142]: 2024-04-25 15:53:43.578324772 +0000 UTC m=+0.986451420 container died 05256029e44554e3a53b383402c72c370e3166dc9a59072af0407752caed8b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:53:44.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:43 smithi110 ceph-mon[90991]: pgmap v68: 65 pgs: 11 active+undersized+degraded, 5 peering, 18 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 32/267 objects degraded (11.985%) 2024-04-25T15:53:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:43 smithi193 ceph-mon[80071]: pgmap v68: 65 pgs: 11 active+undersized+degraded, 5 peering, 18 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 32/267 objects degraded (11.985%) 2024-04-25T15:53:44.494 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:44 smithi110 podman[105302]: 2024-04-25 15:53:44.171466521 +0000 UTC m=+0.583966984 container remove 05256029e44554e3a53b383402c72c370e3166dc9a59072af0407752caed8b13 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2) 2024-04-25T15:53:44.495 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:44 smithi110 podman[105364]: 2024-04-25 15:53:44.287036104 +0000 UTC m=+0.017820612 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:53:44.495 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:44 smithi110 podman[105364]: 2024-04-25 15:53:44.388998611 +0000 UTC m=+0.119783105 container create 9b8b51eaafaca75c507aba8a81ecf5f5b7f75cc5932cb01f3820c3542d8dd0a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T15:53:44.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:44 smithi110 podman[105364]: 2024-04-25 15:53:44.633569925 +0000 UTC m=+0.364354424 container init 9b8b51eaafaca75c507aba8a81ecf5f5b7f75cc5932cb01f3820c3542d8dd0a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T15:53:44.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:44 smithi110 podman[105364]: 2024-04-25 15:53:44.637006637 +0000 UTC m=+0.367791135 container start 9b8b51eaafaca75c507aba8a81ecf5f5b7f75cc5932cb01f3820c3542d8dd0a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T15:53:44.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:44 smithi110 ceph-osd[105383]: -- 172.21.15.110:0/862917156 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56013b153680 con 0x56013b1fad00 2024-04-25T15:53:44.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:44 smithi110 bash[105364]: 9b8b51eaafaca75c507aba8a81ecf5f5b7f75cc5932cb01f3820c3542d8dd0a6 2024-04-25T15:53:44.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:44 smithi110 systemd[1]: Started Ceph osd.1 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:53:45.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:45 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[105379]: 2024-04-25T15:53:45.196+0000 7fb2d12387c0 -1 Falling back to public interface 2024-04-25T15:53:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:45 smithi110 ceph-mon[90991]: pgmap v69: 65 pgs: 11 active+undersized+degraded, 5 peering, 18 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 32/267 objects degraded (11.985%) 2024-04-25T15:53:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:45 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:45 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:45 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:53:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:45 smithi193 ceph-mon[80071]: pgmap v69: 65 pgs: 11 active+undersized+degraded, 5 peering, 18 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 32/267 objects degraded (11.985%) 2024-04-25T15:53:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:45 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:45 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:45 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:53:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:47 smithi110 ceph-mon[90991]: pgmap v70: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:53:47.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:47 smithi110 ceph-mon[90991]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-25T15:53:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:47 smithi193 ceph-mon[80071]: pgmap v70: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:48.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:53:48.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:47 smithi193 ceph-mon[80071]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-25T15:53:49.035 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:48 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:49.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:48 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 13 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:49 smithi110 ceph-mon[90991]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:49 smithi193 ceph-mon[80071]: pgmap v71: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:50.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:50 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[105379]: 2024-04-25T15:53:50.505+0000 7fb2d12387c0 -1 osd.1 54 log_to_monitors true 2024-04-25T15:53:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:51 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:51.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:51 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:51.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:51 smithi193 ceph-mon[80071]: from='osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T15:53:51.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:51 smithi193 ceph-mon[80071]: pgmap v72: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:51 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:51 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:51.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:51 smithi110 ceph-mon[90991]: from='osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T15:53:51.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:51 smithi110 ceph-mon[90991]: pgmap v72: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:52.363 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 15:53:52 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[105379]: 2024-04-25T15:53:52.074+0000 7fb2be64a700 -1 osd.1 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:53:52.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:52 smithi110 ceph-mon[90991]: from='osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T15:53:52.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:52 smithi110 ceph-mon[90991]: osdmap e57: 6 total, 5 up, 6 in 2024-04-25T15:53:52.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:52 smithi110 ceph-mon[90991]: from='osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:53:52.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:52 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:52.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:52 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:52.669 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:52 smithi110 ceph-mon[90991]: from='osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194]' entity='osd.1' 2024-04-25T15:53:52.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:52 smithi193 ceph-mon[80071]: from='osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T15:53:52.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:52 smithi193 ceph-mon[80071]: osdmap e57: 6 total, 5 up, 6 in 2024-04-25T15:53:52.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:52 smithi193 ceph-mon[80071]: from='osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:53:52.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:52 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:52.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:52 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:52.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:52 smithi193 ceph-mon[80071]: from='osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194]' entity='osd.1' 2024-04-25T15:53:53.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:53 smithi110 ceph-mon[90991]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:53.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:53 smithi110 ceph-mon[90991]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:53:53.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:53 smithi110 ceph-mon[90991]: osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194] boot 2024-04-25T15:53:53.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:53 smithi110 ceph-mon[90991]: osdmap e58: 6 total, 6 up, 6 in 2024-04-25T15:53:53.707 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:53 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:53:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:53 smithi193 ceph-mon[80071]: pgmap v74: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:53 smithi193 ceph-mon[80071]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:53:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:53 smithi193 ceph-mon[80071]: osd.1 [v2:172.21.15.110:6810/4003194194,v1:172.21.15.110:6811/4003194194] boot 2024-04-25T15:53:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:53 smithi193 ceph-mon[80071]: osdmap e58: 6 total, 6 up, 6 in 2024-04-25T15:53:53.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:53 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T15:53:55.225 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:55 smithi110 ceph-mon[90991]: osdmap e59: 6 total, 6 up, 6 in 2024-04-25T15:53:55.226 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:55 smithi110 ceph-mon[90991]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:55.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:55 smithi193 ceph-mon[80071]: osdmap e59: 6 total, 6 up, 6 in 2024-04-25T15:53:55.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:55 smithi193 ceph-mon[80071]: pgmap v77: 65 pgs: 13 active+undersized+degraded, 21 active+undersized, 31 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:53:56.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:56.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:56.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:53:56.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:53:56.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:56.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:53:56.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:56.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:56.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:56.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T15:53:56.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T15:53:56.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:56 smithi110 ceph-mon[90991]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-25T15:53:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:53:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:53:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:53:56.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:53:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T15:53:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T15:53:56.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:56 smithi193 ceph-mon[80071]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-25T15:53:57.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:57 smithi110 ceph-mon[90991]: pgmap v78: 65 pgs: 6 active+undersized+degraded, 14 active+undersized, 45 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 16/267 objects degraded (5.993%) 2024-04-25T15:53:57.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.693+0000 7fdf17008700 1 -- 172.21.15.110:0/4267913796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fce60 msgr2=0x7fdf100fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:57.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.693+0000 7fdf17008700 1 --2- 172.21.15.110:0/4267913796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fce60 0x7fdf100fd270 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdf04004940 tx=0x7fdf04009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:57.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.693+0000 7fdf17008700 1 -- 172.21.15.110:0/4267913796 shutdown_connections 2024-04-25T15:53:57.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.693+0000 7fdf17008700 1 --2- 172.21.15.110:0/4267913796 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf100fdfd0 0x7fdf100fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:57.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.693+0000 7fdf17008700 1 --2- 172.21.15.110:0/4267913796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fce60 0x7fdf100fd270 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:57.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.693+0000 7fdf17008700 1 -- 172.21.15.110:0/4267913796 >> 172.21.15.110:0/4267913796 conn(0x7fdf100f8540 msgr2=0x7fdf100fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:57.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.694+0000 7fdf17008700 1 -- 172.21.15.110:0/4267913796 shutdown_connections 2024-04-25T15:53:57.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.694+0000 7fdf17008700 1 -- 172.21.15.110:0/4267913796 wait complete. 2024-04-25T15:53:57.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.694+0000 7fdf17008700 1 Processor -- start 2024-04-25T15:53:57.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.694+0000 7fdf17008700 1 -- start start 2024-04-25T15:53:57.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf17008700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf100fce60 0x7fdf1010d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:57.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf17008700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fdfd0 0x7fdf1010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:57.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf17008700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1010e480 con 0x7fdf100fdfd0 2024-04-25T15:53:57.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf17008700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1010e5f0 con 0x7fdf100fce60 2024-04-25T15:53:57.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf0ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fdfd0 0x7fdf1010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:57.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf0ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fdfd0 0x7fdf1010de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44148/0 (socket says 172.21.15.110:44148) 2024-04-25T15:53:57.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf0ffff700 1 -- 172.21.15.110:0/1605812573 learned_addr learned my addr 172.21.15.110:0/1605812573 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:57.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf14da4700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf100fce60 0x7fdf1010d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf0ffff700 1 -- 172.21.15.110:0/1605812573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf100fce60 msgr2=0x7fdf1010d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf0ffff700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf100fce60 0x7fdf1010d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf0ffff700 1 -- 172.21.15.110:0/1605812573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04005040 con 0x7fdf100fdfd0 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.695+0000 7fdf0ffff700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fdfd0 0x7fdf1010de80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c830 tx=0x7fdefc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.696+0000 7fdf0d7fa700 1 -- 172.21.15.110:0/1605812573 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdefc0106f0 con 0x7fdf100fdfd0 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.696+0000 7fdf0d7fa700 1 -- 172.21.15.110:0/1605812573 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdefc010d20 con 0x7fdf100fdfd0 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.696+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf10113050 con 0x7fdf100fdfd0 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.696+0000 7fdf0d7fa700 1 -- 172.21.15.110:0/1605812573 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdefc0113a0 con 0x7fdf100fdfd0 2024-04-25T15:53:57.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.696+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101135e0 con 0x7fdf100fdfd0 2024-04-25T15:53:57.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.697+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1004c710 con 0x7fdf100fdfd0 2024-04-25T15:53:57.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.697+0000 7fdf0d7fa700 1 -- 172.21.15.110:0/1605812573 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fdefc012400 con 0x7fdf100fdfd0 2024-04-25T15:53:57.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.698+0000 7fdf0d7fa700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdf00076b80 0x7fdf00079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:57.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.698+0000 7fdf14da4700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdf00076b80 0x7fdf00079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:57.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.698+0000 7fdf0d7fa700 1 -- 172.21.15.110:0/1605812573 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdefc098280 con 0x7fdf100fdfd0 2024-04-25T15:53:57.702 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.698+0000 7fdf14da4700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdf00076b80 0x7fdf00079030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009fd0 tx=0x7fdf04009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:57.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.703+0000 7fdf0d7fa700 1 -- 172.21.15.110:0/1605812573 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdefc060e80 con 0x7fdf100fdfd0 2024-04-25T15:53:57.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:57 smithi193 ceph-mon[80071]: pgmap v78: 65 pgs: 6 active+undersized+degraded, 14 active+undersized, 45 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 16/267 objects degraded (5.993%) 2024-04-25T15:53:57.922 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.921+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf10102330 con 0x7fdf00076b80 2024-04-25T15:53:57.923 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.922+0000 7fdf0d7fa700 1 -- 172.21.15.110:0/1605812573 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fdf10102330 con 0x7fdf00076b80 2024-04-25T15:53:57.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdf00076b80 msgr2=0x7fdf00079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:57.925 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdf00076b80 0x7fdf00079030 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009fd0 tx=0x7fdf04009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fdfd0 msgr2=0x7fdf1010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fdfd0 0x7fdf1010de80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdefc00c830 tx=0x7fdefc004470 comp rx=0 tx=0).stop 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 shutdown_connections 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf100fce60 0x7fdf1010d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdf00076b80 0x7fdf00079030 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 --2- 172.21.15.110:0/1605812573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf100fdfd0 0x7fdf1010de80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 >> 172.21.15.110:0/1605812573 conn(0x7fdf100f8540 msgr2=0x7fdf100f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 shutdown_connections 2024-04-25T15:53:57.926 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:57.924+0000 7fdf17008700 1 -- 172.21.15.110:0/1605812573 wait complete. 2024-04-25T15:53:57.936 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:53:58.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.026+0000 7f396738a700 1 -- 172.21.15.110:0/2330487099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 msgr2=0x7f39600fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.026+0000 7f396738a700 1 --2- 172.21.15.110:0/2330487099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 0x7f39600fc0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f395c004940 tx=0x7f395c009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:58.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 -- 172.21.15.110:0/2330487099 shutdown_connections 2024-04-25T15:53:58.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 --2- 172.21.15.110:0/2330487099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39600fcf70 0x7f39600fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 --2- 172.21.15.110:0/2330487099 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 0x7f39600fc0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 -- 172.21.15.110:0/2330487099 >> 172.21.15.110:0/2330487099 conn(0x7f39600f7440 msgr2=0x7f39600f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:58.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 -- 172.21.15.110:0/2330487099 shutdown_connections 2024-04-25T15:53:58.028 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 -- 172.21.15.110:0/2330487099 wait complete. 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 Processor -- start 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 -- start start 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 0x7f396010fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39600fcf70 0x7f39601100d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601106d0 con 0x7f39600fbcc0 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.027+0000 7f396738a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3960110840 con 0x7f39600fcf70 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3965126700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 0x7f396010fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3965126700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 0x7f396010fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44168/0 (socket says 172.21.15.110:44168) 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3965126700 1 -- 172.21.15.110:0/2362638302 learned_addr learned my addr 172.21.15.110:0/2362638302 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3964925700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39600fcf70 0x7f39601100d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3965126700 1 -- 172.21.15.110:0/2362638302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39600fcf70 msgr2=0x7f39601100d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3965126700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39600fcf70 0x7f39601100d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.030 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3965126700 1 -- 172.21.15.110:0/2362638302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f395c005040 con 0x7f39600fbcc0 2024-04-25T15:53:58.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3965126700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 0x7f396010fb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f395c00b970 tx=0x7f395c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:58.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3955ffb700 1 -- 172.21.15.110:0/2362638302 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f395c00e040 con 0x7f39600fbcc0 2024-04-25T15:53:58.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3955ffb700 1 -- 172.21.15.110:0/2362638302 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f395c00bbf0 con 0x7f39600fbcc0 2024-04-25T15:53:58.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39601151a0 con 0x7f39600fbcc0 2024-04-25T15:53:58.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f3955ffb700 1 -- 172.21.15.110:0/2362638302 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f395c01c630 con 0x7f39600fbcc0 2024-04-25T15:53:58.031 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.028+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3960115760 con 0x7f39600fbcc0 2024-04-25T15:53:58.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.030+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3960101900 con 0x7f39600fbcc0 2024-04-25T15:53:58.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.030+0000 7f3955ffb700 1 -- 172.21.15.110:0/2362638302 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f395c00bd60 con 0x7f39600fbcc0 2024-04-25T15:53:58.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.030+0000 7f3955ffb700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f394c076b90 0x7f394c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.031+0000 7f3955ffb700 1 -- 172.21.15.110:0/2362638302 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f395c099500 con 0x7f39600fbcc0 2024-04-25T15:53:58.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.031+0000 7f3964925700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f394c076b90 0x7f394c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.032 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.031+0000 7f3964925700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f394c076b90 0x7f394c079040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3950004f40 tx=0x7f395000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:58.036 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.035+0000 7f3955ffb700 1 -- 172.21.15.110:0/2362638302 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f395c0621a0 con 0x7f39600fbcc0 2024-04-25T15:53:58.255 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.254+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f396002b340 con 0x7f394c076b90 2024-04-25T15:53:58.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.255+0000 7f3955ffb700 1 -- 172.21.15.110:0/2362638302 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f396002b340 con 0x7f394c076b90 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.257+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f394c076b90 msgr2=0x7f394c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.257+0000 7f396738a700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f394c076b90 0x7f394c079040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3950004f40 tx=0x7f395000b040 comp rx=0 tx=0).stop 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.257+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 msgr2=0x7f396010fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.257+0000 7f396738a700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 0x7f396010fb90 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f395c00b970 tx=0x7f395c009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.258+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 shutdown_connections 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.258+0000 7f396738a700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f394c076b90 0x7f394c079040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.258+0000 7f396738a700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39600fbcc0 0x7f396010fb90 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.258+0000 7f396738a700 1 --2- 172.21.15.110:0/2362638302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f39600fcf70 0x7f39601100d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.258+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 >> 172.21.15.110:0/2362638302 conn(0x7f39600f7440 msgr2=0x7f39600667d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.258+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 shutdown_connections 2024-04-25T15:53:58.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.258+0000 7f396738a700 1 -- 172.21.15.110:0/2362638302 wait complete. 2024-04-25T15:53:58.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.359+0000 7f2a82582700 1 -- 172.21.15.110:0/188476947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fcf60 msgr2=0x7f2a7c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.359+0000 7f2a82582700 1 --2- 172.21.15.110:0/188476947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fcf60 0x7f2a7c0fd3d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2a70004940 tx=0x7f2a70009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:58.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.359+0000 7f2a82582700 1 -- 172.21.15.110:0/188476947 shutdown_connections 2024-04-25T15:53:58.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.359+0000 7f2a82582700 1 --2- 172.21.15.110:0/188476947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fcf60 0x7f2a7c0fd3d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.359+0000 7f2a82582700 1 --2- 172.21.15.110:0/188476947 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0fbcb0 0x7f2a7c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.359+0000 7f2a82582700 1 -- 172.21.15.110:0/188476947 >> 172.21.15.110:0/188476947 conn(0x7f2a7c0f7430 msgr2=0x7f2a7c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:58.361 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:58 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 16/267 objects degraded (5.993%), 6 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.359+0000 7f2a82582700 1 -- 172.21.15.110:0/188476947 shutdown_connections 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.359+0000 7f2a82582700 1 -- 172.21.15.110:0/188476947 wait complete. 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a82582700 1 Processor -- start 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a82582700 1 -- start start 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a82582700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fbcb0 0x7f2a7c112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a82582700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0fcf60 0x7f2a7c115060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a82582700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c066220 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a82582700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c066360 con 0x7f2a7c0fcf60 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a7bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fbcb0 0x7f2a7c112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a7bfff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fbcb0 0x7f2a7c112b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44182/0 (socket says 172.21.15.110:44182) 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a7bfff700 1 -- 172.21.15.110:0/1617185238 learned_addr learned my addr 172.21.15.110:0/1617185238 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.360+0000 7f2a7b7fe700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0fcf60 0x7f2a7c115060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a7bfff700 1 -- 172.21.15.110:0/1617185238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0fcf60 msgr2=0x7f2a7c115060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a7bfff700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0fcf60 0x7f2a7c115060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a7bfff700 1 -- 172.21.15.110:0/1617185238 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70005040 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a7bfff700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fbcb0 0x7f2a7c112b10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b790 tx=0x7f2a6c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a78ff9700 1 -- 172.21.15.110:0/1617185238 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a6c00f6f0 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a78ff9700 1 -- 172.21.15.110:0/1617185238 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a6c00fd20 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a78ff9700 1 -- 172.21.15.110:0/1617185238 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a6c0103a0 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c1155a0 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.361+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c115a20 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.362+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c05f5c0 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.363+0000 7f2a78ff9700 1 -- 172.21.15.110:0/1617185238 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f2a6c011400 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.363+0000 7f2a78ff9700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2a64076b90 0x7f2a64079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.363+0000 7f2a78ff9700 1 -- 172.21.15.110:0/1617185238 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f2a6c0980c0 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.363+0000 7f2a7b7fe700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2a64076b90 0x7f2a64079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.364+0000 7f2a7b7fe700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2a64076b90 0x7f2a64079040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b970 tx=0x7f2a70009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:58.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.369+0000 7f2a78ff9700 1 -- 172.21.15.110:0/1617185238 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2a6c060cc0 con 0x7f2a7c0fbcb0 2024-04-25T15:53:58.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.579+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a7c101200 con 0x7f2a64076b90 2024-04-25T15:53:58.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.589+0000 7f2a78ff9700 1 -- 172.21.15.110:0/1617185238 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2a7c101200 con 0x7f2a64076b90 2024-04-25T15:53:58.590 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (4m) 8s ago 12m 26.3M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (76s) 8s ago 13m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (71s) 68s ago 12m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (4m) 8s ago 12m 78.9M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (7m) 8s ago 7m 19.7M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (7m) 8s ago 7m 150M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (7m) 68s ago 7m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (7m) 68s ago 7m 16.7M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (3m) 8s ago 14m 569M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (2m) 68s ago 11m 517M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (2m) 8s ago 14m 113M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (117s) 68s ago 11m 98.5M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (4m) 8s ago 13m 15.7M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:53:58.591 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (3m) 68s ago 12m 15.7M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:53:58.592 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (49s) 8s ago 11m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:53:58.592 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (14s) 8s ago 10m 80.0M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:53:58.592 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (10m) 8s ago 10m 178M 1557M 17.2.7-902-gc61472c8 0de407c1b3af edc31bf2dac1 2024-04-25T15:53:58.592 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (9m) 68s ago 9m 239M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:53:58.592 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (9m) 68s ago 9m 206M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:53:58.592 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (9m) 68s ago 9m 189M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:53:58.592 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (2m) 8s ago 12m 54.6M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2a64076b90 msgr2=0x7f2a64079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2a64076b90 0x7f2a64079040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000b970 tx=0x7f2a70009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fbcb0 msgr2=0x7f2a7c112b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fbcb0 0x7f2a7c112b10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00b790 tx=0x7f2a6c009250 comp rx=0 tx=0).stop 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 shutdown_connections 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2a64076b90 0x7f2a64079040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2a7c0fbcb0 0x7f2a7c112b10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 --2- 172.21.15.110:0/1617185238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0fcf60 0x7f2a7c115060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 >> 172.21.15.110:0/1617185238 conn(0x7f2a7c0f7430 msgr2=0x7f2a7c0f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:58.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 shutdown_connections 2024-04-25T15:53:58.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.592+0000 7f2a82582700 1 -- 172.21.15.110:0/1617185238 wait complete. 2024-04-25T15:53:58.640 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:58 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 16/267 objects degraded (5.993%), 6 pgs degraded (PG_DEGRADED) 2024-04-25T15:53:58.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.694+0000 7f9ebf539700 1 -- 172.21.15.110:0/2950525890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fed80 msgr2=0x7f9eb80faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.694+0000 7f9ebf539700 1 --2- 172.21.15.110:0/2950525890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fed80 0x7f9eb80faec0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8004940 tx=0x7f9ea8009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:58.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.694+0000 7f9ebf539700 1 -- 172.21.15.110:0/2950525890 shutdown_connections 2024-04-25T15:53:58.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.694+0000 7f9ebf539700 1 --2- 172.21.15.110:0/2950525890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fed80 0x7f9eb80faec0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.694+0000 7f9ebf539700 1 --2- 172.21.15.110:0/2950525890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb80fe430 0x7f9eb80fe840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.694+0000 7f9ebf539700 1 -- 172.21.15.110:0/2950525890 >> 172.21.15.110:0/2950525890 conn(0x7f9eb80f6bd0 msgr2=0x7f9eb80f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:58.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebf539700 1 -- 172.21.15.110:0/2950525890 shutdown_connections 2024-04-25T15:53:58.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebf539700 1 -- 172.21.15.110:0/2950525890 wait complete. 2024-04-25T15:53:58.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebf539700 1 Processor -- start 2024-04-25T15:53:58.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebf539700 1 -- start start 2024-04-25T15:53:58.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebf539700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fe430 0x7f9eb80fda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebf539700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb80fed80 0x7f9eb80fc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.697 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebf539700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80fe040 con 0x7f9eb80fe430 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebf539700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80fc630 con 0x7f9eb80fed80 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebd2d5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fe430 0x7f9eb80fda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebd2d5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fe430 0x7f9eb80fda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44200/0 (socket says 172.21.15.110:44200) 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebd2d5700 1 -- 172.21.15.110:0/3106503942 learned_addr learned my addr 172.21.15.110:0/3106503942 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.695+0000 7f9ebcad4700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb80fed80 0x7f9eb80fc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9ebd2d5700 1 -- 172.21.15.110:0/3106503942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb80fed80 msgr2=0x7f9eb80fc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9ebd2d5700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb80fed80 0x7f9eb80fc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9ebd2d5700 1 -- 172.21.15.110:0/3106503942 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea8005040 con 0x7f9eb80fe430 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9ebcad4700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb80fed80 0x7f9eb80fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9ebd2d5700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fe430 0x7f9eb80fda40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400c8c0 tx=0x7f9eb40044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9eadffb700 1 -- 172.21.15.110:0/3106503942 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb400a980 con 0x7f9eb80fe430 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb80fc8e0 con 0x7f9eb80fe430 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9eadffb700 1 -- 172.21.15.110:0/3106503942 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9eb400ce60 con 0x7f9eb80fe430 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9eadffb700 1 -- 172.21.15.110:0/3106503942 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9eb40115e0 con 0x7f9eb80fe430 2024-04-25T15:53:58.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.696+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb80fcea0 con 0x7f9eb80fe430 2024-04-25T15:53:58.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.697+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb804c710 con 0x7f9eb80fe430 2024-04-25T15:53:58.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.698+0000 7f9eadffb700 1 -- 172.21.15.110:0/3106503942 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f9eb4012400 con 0x7f9eb80fe430 2024-04-25T15:53:58.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.698+0000 7f9eadffb700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9ea4076b90 0x7f9ea4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:58.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.698+0000 7f9ebcad4700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9ea4076b90 0x7f9ea4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:58.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.698+0000 7f9eadffb700 1 -- 172.21.15.110:0/3106503942 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9eb40983f0 con 0x7f9eb80fe430 2024-04-25T15:53:58.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.699+0000 7f9ebcad4700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9ea4076b90 0x7f9ea4079040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800b3b0 tx=0x7f9ea8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:58.704 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.703+0000 7f9eadffb700 1 -- 172.21.15.110:0/3106503942 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9eb4060ff0 con 0x7f9eb80fe430 2024-04-25T15:53:58.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.987+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9eb8115bf0 con 0x7f9eb80fe430 2024-04-25T15:53:58.988 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.988+0000 7f9eadffb700 1 -- 172.21.15.110:0/3106503942 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7f9eb40126b0 con 0x7f9eb80fe430 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:53:58.989 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:53:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:53:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:53:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 8, 2024-04-25T15:53:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 6 2024-04-25T15:53:58.990 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:53:58.990 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9ea4076b90 msgr2=0x7f9ea4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9ea4076b90 0x7f9ea4079040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9ea800b3b0 tx=0x7f9ea8009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fe430 msgr2=0x7f9eb80fda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fe430 0x7f9eb80fda40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400c8c0 tx=0x7f9eb40044c0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 shutdown_connections 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9ea4076b90 0x7f9ea4079040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9eb80fe430 0x7f9eb80fda40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 --2- 172.21.15.110:0/3106503942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9eb80fed80 0x7f9eb80fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 >> 172.21.15.110:0/3106503942 conn(0x7f9eb80f6bd0 msgr2=0x7f9eb80f8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 shutdown_connections 2024-04-25T15:53:58.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:58.990+0000 7f9ebf539700 1 -- 172.21.15.110:0/3106503942 wait complete. 2024-04-25T15:53:59.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.091+0000 7ff937dde700 1 -- 172.21.15.110:0/1418043478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fbcb0 msgr2=0x7ff9300fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.091+0000 7ff937dde700 1 --2- 172.21.15.110:0/1418043478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fbcb0 0x7ff9300fc0c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff920004940 tx=0x7ff920009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:59.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.091+0000 7ff937dde700 1 -- 172.21.15.110:0/1418043478 shutdown_connections 2024-04-25T15:53:59.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.091+0000 7ff937dde700 1 --2- 172.21.15.110:0/1418043478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9300fcf60 0x7ff9300fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.091+0000 7ff937dde700 1 --2- 172.21.15.110:0/1418043478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fbcb0 0x7ff9300fc0c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.091+0000 7ff937dde700 1 -- 172.21.15.110:0/1418043478 >> 172.21.15.110:0/1418043478 conn(0x7ff9300f7430 msgr2=0x7ff9300f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:59.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.091+0000 7ff937dde700 1 -- 172.21.15.110:0/1418043478 shutdown_connections 2024-04-25T15:53:59.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.091+0000 7ff937dde700 1 -- 172.21.15.110:0/1418043478 wait complete. 2024-04-25T15:53:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff937dde700 1 Processor -- start 2024-04-25T15:53:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff937dde700 1 -- start start 2024-04-25T15:53:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff937dde700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9300fbcb0 0x7ff93010c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff937dde700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fcf60 0x7ff93010cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff937dde700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93010d3c0 con 0x7ff9300fcf60 2024-04-25T15:53:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff937dde700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93010dc10 con 0x7ff9300fbcb0 2024-04-25T15:53:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff935379700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fcf60 0x7ff93010cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff935379700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fcf60 0x7ff93010cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44230/0 (socket says 172.21.15.110:44230) 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff935379700 1 -- 172.21.15.110:0/3847256166 learned_addr learned my addr 172.21.15.110:0/3847256166 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff935b7a700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9300fbcb0 0x7ff93010c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff935379700 1 -- 172.21.15.110:0/3847256166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9300fbcb0 msgr2=0x7ff93010c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff935379700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9300fbcb0 0x7ff93010c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff935379700 1 -- 172.21.15.110:0/3847256166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920005040 con 0x7ff9300fcf60 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.092+0000 7ff935379700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fcf60 0x7ff93010cdc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff92c002a80 tx=0x7ff92c0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.093+0000 7ff9267fc700 1 -- 172.21.15.110:0/3847256166 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff92c007cc0 con 0x7ff9300fcf60 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.093+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff93010def0 con 0x7ff9300fcf60 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.093+0000 7ff9267fc700 1 -- 172.21.15.110:0/3847256166 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff92c00f040 con 0x7ff9300fcf60 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.093+0000 7ff9267fc700 1 -- 172.21.15.110:0/3847256166 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff92c00dac0 con 0x7ff9300fcf60 2024-04-25T15:53:59.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.093+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff93010e430 con 0x7ff9300fcf60 2024-04-25T15:53:59.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.093+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff93005f5c0 con 0x7ff9300fcf60 2024-04-25T15:53:59.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.094+0000 7ff9267fc700 1 -- 172.21.15.110:0/3847256166 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7ff92c00dc20 con 0x7ff9300fcf60 2024-04-25T15:53:59.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.095+0000 7ff9267fc700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff91c076b90 0x7ff91c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.095+0000 7ff935b7a700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff91c076b90 0x7ff91c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.095+0000 7ff9267fc700 1 -- 172.21.15.110:0/3847256166 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7ff92c097f60 con 0x7ff9300fcf60 2024-04-25T15:53:59.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.095+0000 7ff935b7a700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff91c076b90 0x7ff91c079040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b3b0 tx=0x7ff920009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:59.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.099+0000 7ff9267fc700 1 -- 172.21.15.110:0/3847256166 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff92c0ccff0 con 0x7ff9300fcf60 2024-04-25T15:53:59.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.345+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff93010e7b0 con 0x7ff9300fcf60 2024-04-25T15:53:59.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.345+0000 7ff9267fc700 1 -- 172.21.15.110:0/3847256166 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7ff92c060ae0 con 0x7ff9300fcf60 2024-04-25T15:53:59.346 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:53:59.346 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:53:59.347 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:53:59.348 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:53:59.349 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff91c076b90 msgr2=0x7ff91c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff91c076b90 0x7ff91c079040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff92000b3b0 tx=0x7ff920009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fcf60 msgr2=0x7ff93010cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fcf60 0x7ff93010cdc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff92c002a80 tx=0x7ff92c0046b0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 shutdown_connections 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9300fbcb0 0x7ff93010c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff91c076b90 0x7ff91c079040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 --2- 172.21.15.110:0/3847256166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff9300fcf60 0x7ff93010cdc0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.350 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.347+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 >> 172.21.15.110:0/3847256166 conn(0x7ff9300f7430 msgr2=0x7ff9300f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:59.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.348+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 shutdown_connections 2024-04-25T15:53:59.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.348+0000 7ff937dde700 1 -- 172.21.15.110:0/3847256166 wait complete. 2024-04-25T15:53:59.351 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:53:59.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 -- 172.21.15.110:0/3726624332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07980fdd00 msgr2=0x7f0798100180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 --2- 172.21.15.110:0/3726624332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07980fdd00 0x7f0798100180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f078c004940 tx=0x7f078c009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:59.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 -- 172.21.15.110:0/3726624332 shutdown_connections 2024-04-25T15:53:59.451 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 --2- 172.21.15.110:0/3726624332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f07980fdd00 0x7f0798100180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 --2- 172.21.15.110:0/3726624332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0798066b50 0x7f07980fd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 -- 172.21.15.110:0/3726624332 >> 172.21.15.110:0/3726624332 conn(0x7f07980f7440 msgr2=0x7f07980f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 -- 172.21.15.110:0/3726624332 shutdown_connections 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 -- 172.21.15.110:0/3726624332 wait complete. 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 Processor -- start 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 -- start start 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0798066b50 0x7f0798114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07980fdd00 0x7f07981145a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0798114ba0 con 0x7f0798066b50 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.449+0000 7f079cebc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07981153f0 con 0x7f07980fdd00 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0798066b50 0x7f0798114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0798066b50 0x7f0798114060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44248/0 (socket says 172.21.15.110:44248) 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079659c700 1 -- 172.21.15.110:0/210772949 learned_addr learned my addr 172.21.15.110:0/210772949 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:59.452 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f0795d9b700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07980fdd00 0x7f07981145a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079659c700 1 -- 172.21.15.110:0/210772949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07980fdd00 msgr2=0x7f07981145a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079659c700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07980fdd00 0x7f07981145a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079659c700 1 -- 172.21.15.110:0/210772949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c005040 con 0x7f0798066b50 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f0795d9b700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07980fdd00 0x7f07981145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079659c700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0798066b50 0x7f0798114060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f078800c800 tx=0x7f0788004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f0786ffd700 1 -- 172.21.15.110:0/210772949 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078800a760 con 0x7f0798066b50 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07981156d0 con 0x7f0798066b50 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f0786ffd700 1 -- 172.21.15.110:0/210772949 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f078800ad90 con 0x7f0798066b50 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0798115c10 con 0x7f0798066b50 2024-04-25T15:53:59.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.450+0000 7f0786ffd700 1 -- 172.21.15.110:0/210772949 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f078801b400 con 0x7f0798066b50 2024-04-25T15:53:59.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.451+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f079805f5c0 con 0x7f0798066b50 2024-04-25T15:53:59.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.452+0000 7f0786ffd700 1 -- 172.21.15.110:0/210772949 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f0788012400 con 0x7f0798066b50 2024-04-25T15:53:59.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.452+0000 7f0786ffd700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0780076b90 0x7f0780079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.452+0000 7f0786ffd700 1 -- 172.21.15.110:0/210772949 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f07880982e0 con 0x7f0798066b50 2024-04-25T15:53:59.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.452+0000 7f0795d9b700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0780076b90 0x7f0780079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.454 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.453+0000 7f0795d9b700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0780076b90 0x7f0780079040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f078c000c00 tx=0x7f078c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:59.458 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.457+0000 7f0786ffd700 1 -- 172.21.15.110:0/210772949 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0788060ee0 con 0x7f0798066b50 2024-04-25T15:53:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:59 smithi110 ceph-mon[90991]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:59 smithi110 ceph-mon[90991]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:59 smithi110 ceph-mon[90991]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:59 smithi110 ceph-mon[90991]: pgmap v79: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:59.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:53:59 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3106503942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:59.678 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.677+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0798116050 con 0x7f0780076b90 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.678+0000 7f0786ffd700 1 -- 172.21.15.110:0/210772949 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f0798116050 con 0x7f0780076b90 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:53:59.679 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:53:59.680 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:53:59.680 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "8/21 daemons upgraded", 2024-04-25T15:53:59.680 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T15:53:59.680 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:53:59.680 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:53:59.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.680+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0780076b90 msgr2=0x7f0780079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.680+0000 7f079cebc700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0780076b90 0x7f0780079040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f078c000c00 tx=0x7f078c009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.680+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0798066b50 msgr2=0x7f0798114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.680+0000 7f079cebc700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0798066b50 0x7f0798114060 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f078800c800 tx=0x7f0788004470 comp rx=0 tx=0).stop 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.680+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 shutdown_connections 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.680+0000 7f079cebc700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0780076b90 0x7f0780079040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.680+0000 7f079cebc700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0798066b50 0x7f0798114060 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.680+0000 7f079cebc700 1 --2- 172.21.15.110:0/210772949 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07980fdd00 0x7f07981145a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.681+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 >> 172.21.15.110:0/210772949 conn(0x7f07980f7440 msgr2=0x7f07980f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.681+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 shutdown_connections 2024-04-25T15:53:59.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.681+0000 7f079cebc700 1 -- 172.21.15.110:0/210772949 wait complete. 2024-04-25T15:53:59.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:59 smithi193 ceph-mon[80071]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:59.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:59 smithi193 ceph-mon[80071]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:59.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:59 smithi193 ceph-mon[80071]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:53:59.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:59 smithi193 ceph-mon[80071]: pgmap v79: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:53:59.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:53:59 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3106503942' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:53:59.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 -- 172.21.15.110:0/887418503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 msgr2=0x7f99300ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 --2- 172.21.15.110:0/887418503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 0x7f99300ff9b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f9928004940 tx=0x7f9928009e30 comp rx=0 tx=0).stop 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 -- 172.21.15.110:0/887418503 shutdown_connections 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 --2- 172.21.15.110:0/887418503 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99300ffef0 0x7f9930102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 --2- 172.21.15.110:0/887418503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 0x7f99300ff9b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 -- 172.21.15.110:0/887418503 >> 172.21.15.110:0/887418503 conn(0x7f99300f7420 msgr2=0x7f99300f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 -- 172.21.15.110:0/887418503 shutdown_connections 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 -- 172.21.15.110:0/887418503 wait complete. 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.782+0000 7f99375c5700 1 Processor -- start 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f99375c5700 1 -- start start 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f99375c5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 0x7f993010da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f99375c5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99300ffef0 0x7f993010dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f99375c5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993010e5b0 con 0x7f9930066b50 2024-04-25T15:53:59.785 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f99375c5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993010edb0 con 0x7f99300ffef0 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f9935361700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 0x7f993010da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f9935361700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 0x7f993010da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:44008/0 (socket says 172.21.15.110:44008) 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f9935361700 1 -- 172.21.15.110:0/2027397291 learned_addr learned my addr 172.21.15.110:0/2027397291 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f9934b60700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99300ffef0 0x7f993010dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f9935361700 1 -- 172.21.15.110:0/2027397291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99300ffef0 msgr2=0x7f993010dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f9935361700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99300ffef0 0x7f993010dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f9935361700 1 -- 172.21.15.110:0/2027397291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928005040 con 0x7f9930066b50 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f9935361700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 0x7f993010da70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f992800b3b0 tx=0x7f9928009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f992dffb700 1 -- 172.21.15.110:0/2027397291 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f992800e040 con 0x7f9930066b50 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.783+0000 7f992dffb700 1 -- 172.21.15.110:0/2027397291 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f992800bcc0 con 0x7f9930066b50 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.784+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f993010f030 con 0x7f9930066b50 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.784+0000 7f992dffb700 1 -- 172.21.15.110:0/2027397291 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f992801c630 con 0x7f9930066b50 2024-04-25T15:53:59.786 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.784+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993010f570 con 0x7f9930066b50 2024-04-25T15:53:59.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.785+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993005f5c0 con 0x7f9930066b50 2024-04-25T15:53:59.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.785+0000 7f992dffb700 1 -- 172.21.15.110:0/2027397291 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f99280139f0 con 0x7f9930066b50 2024-04-25T15:53:59.787 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.786+0000 7f992dffb700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9918076b90 0x7f9918079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:53:59.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.786+0000 7f992dffb700 1 -- 172.21.15.110:0/2027397291 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9928098e70 con 0x7f9930066b50 2024-04-25T15:53:59.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.786+0000 7f9934b60700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9918076b90 0x7f9918079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:53:59.788 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.786+0000 7f9934b60700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9918076b90 0x7f9918079040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99240041d0 tx=0x7f9924009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:53:59.791 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:53:59.791+0000 7f992dffb700 1 -- 172.21.15.110:0/2027397291 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9928061b10 con 0x7f9930066b50 2024-04-25T15:54:00.071 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.070+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f993004bed0 con 0x7f9930066b50 2024-04-25T15:54:00.072 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.071+0000 7f992dffb700 1 -- 172.21.15.110:0/2027397291 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f99280614b0 con 0x7f9930066b50 2024-04-25T15:54:00.072 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:54:00.072 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:54:00.072 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:54:00.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9918076b90 msgr2=0x7f9918079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:00.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9918076b90 0x7f9918079040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f99240041d0 tx=0x7f9924009040 comp rx=0 tx=0).stop 2024-04-25T15:54:00.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 msgr2=0x7f993010da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:00.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 0x7f993010da70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f992800b3b0 tx=0x7f9928009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:00.074 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 shutdown_connections 2024-04-25T15:54:00.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9918076b90 0x7f9918079040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:00.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9930066b50 0x7f993010da70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:00.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 --2- 172.21.15.110:0/2027397291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99300ffef0 0x7f993010dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:00.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 >> 172.21.15.110:0/2027397291 conn(0x7f99300f7420 msgr2=0x7f99300f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:00.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 shutdown_connections 2024-04-25T15:54:00.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:00.073+0000 7f99375c5700 1 -- 172.21.15.110:0/2027397291 wait complete. 2024-04-25T15:54:00.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:00 smithi110 ceph-mon[90991]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/267 objects degraded (5.993%), 6 pgs degraded) 2024-04-25T15:54:00.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:00 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3847256166' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:54:00.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:00 smithi110 ceph-mon[90991]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:00.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:00 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2027397291' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:54:00.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:00 smithi193 ceph-mon[80071]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/267 objects degraded (5.993%), 6 pgs degraded) 2024-04-25T15:54:00.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:00 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3847256166' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:54:00.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:00 smithi193 ceph-mon[80071]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:00.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:00 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2027397291' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:54:01.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:01 smithi110 ceph-mon[90991]: pgmap v80: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:01.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:01 smithi193 ceph-mon[80071]: pgmap v80: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:54:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:54:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:03 smithi110 ceph-mon[90991]: pgmap v81: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:04.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:03 smithi193 ceph-mon[80071]: pgmap v81: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:05 smithi110 ceph-mon[90991]: pgmap v82: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:05 smithi193 ceph-mon[80071]: pgmap v82: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:07 smithi110 ceph-mon[90991]: pgmap v83: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:08.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:07 smithi193 ceph-mon[80071]: pgmap v83: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:09 smithi110 ceph-mon[90991]: pgmap v84: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:09 smithi193 ceph-mon[80071]: pgmap v84: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:11.033 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:10 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T15:54:11.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:10 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T15:54:12.056 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:11 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T15:54:12.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:11 smithi110 ceph-mon[90991]: Upgrade: osd.2 is safe to restart 2024-04-25T15:54:12.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:11 smithi110 ceph-mon[90991]: pgmap v85: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:11 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-25T15:54:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:11 smithi193 ceph-mon[80071]: Upgrade: osd.2 is safe to restart 2024-04-25T15:54:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:11 smithi193 ceph-mon[80071]: pgmap v85: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:13 smithi193 ceph-mon[80071]: Upgrade: Updating osd.2 2024-04-25T15:54:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T15:54:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:54:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:13 smithi193 ceph-mon[80071]: Deploying daemon osd.2 on smithi110 2024-04-25T15:54:13.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:13 smithi193 ceph-mon[80071]: pgmap v86: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:13.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:13 smithi110 ceph-mon[90991]: Upgrade: Updating osd.2 2024-04-25T15:54:13.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:13.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T15:54:13.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:54:13.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:13 smithi110 ceph-mon[90991]: Deploying daemon osd.2 on smithi110 2024-04-25T15:54:13.505 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:13 smithi110 ceph-mon[90991]: pgmap v86: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:15 smithi110 ceph-mon[90991]: pgmap v87: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:15.864 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:15 smithi110 systemd[1]: Stopping Ceph osd.2 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:54:15.864 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:15 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[60352]: 2024-04-25T15:54:15.778+0000 7fcdc3aea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:54:15.864 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:15 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[60352]: 2024-04-25T15:54:15.778+0000 7fcdc3aea700 -1 osd.2 59 *** Got signal Terminated *** 2024-04-25T15:54:15.864 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:15 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[60352]: 2024-04-25T15:54:15.778+0000 7fcdc3aea700 -1 osd.2 59 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:54:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:15 smithi193 ceph-mon[80071]: pgmap v87: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:16 smithi110 ceph-mon[90991]: osd.2 marked itself down and dead 2024-04-25T15:54:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:54:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:16 smithi193 ceph-mon[80071]: osd.2 marked itself down and dead 2024-04-25T15:54:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:54:17.547 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:17 smithi110 podman[110415]: 2024-04-25 15:54:17.270360787 +0000 UTC m=+1.587967530 container died edc31bf2dac1d4d6e33fe968825a77b4e75d5720130b2f1e3a2ddaf88553f9ec (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T15:54:17.839 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:17 smithi110 ceph-mon[90991]: pgmap v88: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:17.839 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:17 smithi110 ceph-mon[90991]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:54:17.839 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:17 smithi110 ceph-mon[90991]: osdmap e60: 6 total, 5 up, 6 in 2024-04-25T15:54:17.839 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:17 smithi110 podman[110415]: 2024-04-25 15:54:17.620352641 +0000 UTC m=+1.937959383 container cleanup edc31bf2dac1d4d6e33fe968825a77b4e75d5720130b2f1e3a2ddaf88553f9ec (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T15:54:17.839 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:17 smithi110 bash[110415]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2 2024-04-25T15:54:18.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:17 smithi110 podman[110435]: 2024-04-25 15:54:17.83866774 +0000 UTC m=+0.560477632 container remove edc31bf2dac1d4d6e33fe968825a77b4e75d5720130b2f1e3a2ddaf88553f9ec (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, RELEASE=quincy-c61472c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-25T15:54:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:17 smithi193 ceph-mon[80071]: pgmap v88: 65 pgs: 65 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:17 smithi193 ceph-mon[80071]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:54:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:17 smithi193 ceph-mon[80071]: osdmap e60: 6 total, 5 up, 6 in 2024-04-25T15:54:18.542 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:18 smithi110 podman[110526]: 2024-04-25 15:54:18.144980243 +0000 UTC m=+0.018233543 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:54:18.542 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:18 smithi110 podman[110526]: 2024-04-25 15:54:18.281569515 +0000 UTC m=+0.154822812 container create 0e885836d4d9a2d010ce68a90d73a32e7291c7c92f274f7e24242f518d931028 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T15:54:18.543 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:18 smithi110 podman[110526]: 2024-04-25 15:54:18.542429714 +0000 UTC m=+0.415683013 container init 0e885836d4d9a2d010ce68a90d73a32e7291c7c92f274f7e24242f518d931028 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:54:18.837 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:18 smithi110 ceph-mon[90991]: osdmap e61: 6 total, 5 up, 6 in 2024-04-25T15:54:18.837 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:18 smithi110 podman[110526]: 2024-04-25 15:54:18.545721565 +0000 UTC m=+0.418974864 container start 0e885836d4d9a2d010ce68a90d73a32e7291c7c92f274f7e24242f518d931028 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T15:54:18.838 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:18 smithi110 podman[110526]: 2024-04-25 15:54:18.625739421 +0000 UTC m=+0.498992730 container attach 0e885836d4d9a2d010ce68a90d73a32e7291c7c92f274f7e24242f518d931028 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T15:54:19.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:18 smithi110 podman[110526]: 2024-04-25 15:54:18.837145376 +0000 UTC m=+0.710398672 container died 0e885836d4d9a2d010ce68a90d73a32e7291c7c92f274f7e24242f518d931028 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:54:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:18 smithi193 ceph-mon[80071]: osdmap e61: 6 total, 5 up, 6 in 2024-04-25T15:54:19.773 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:19 smithi110 podman[110552]: 2024-04-25 15:54:19.488348465 +0000 UTC m=+0.643071445 container remove 0e885836d4d9a2d010ce68a90d73a32e7291c7c92f274f7e24242f518d931028 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T15:54:19.773 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:19 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-25T15:54:19.773 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:19 smithi110 systemd[1]: Stopped Ceph osd.2 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:54:19.773 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:19 smithi110 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.2.service: Consumed 44.223s CPU time. 2024-04-25T15:54:19.773 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:19 smithi110 systemd[1]: Starting Ceph osd.2 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:54:20.103 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:19 smithi110 ceph-mon[90991]: pgmap v91: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:20.103 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:19 smithi110 ceph-mon[90991]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T15:54:20.103 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:19 smithi110 podman[110660]: 2024-04-25 15:54:19.849643219 +0000 UTC m=+0.019035261 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:54:20.103 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:19 smithi110 podman[110660]: 2024-04-25 15:54:19.973041958 +0000 UTC m=+0.142433990 container create a48ded3c9d235595aa714541a2f8bc48ba49c0cf4945cbba047e638c79aaa31c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T15:54:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:19 smithi193 ceph-mon[80071]: pgmap v91: 65 pgs: 12 peering, 7 stale+active+clean, 46 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2024-04-25T15:54:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:19 smithi193 ceph-mon[80071]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T15:54:20.363 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 podman[110660]: 2024-04-25 15:54:20.234181117 +0000 UTC m=+0.403573152 container init a48ded3c9d235595aa714541a2f8bc48ba49c0cf4945cbba047e638c79aaa31c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:54:20.363 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 podman[110660]: 2024-04-25 15:54:20.237281612 +0000 UTC m=+0.406673647 container start a48ded3c9d235595aa714541a2f8bc48ba49c0cf4945cbba047e638c79aaa31c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T15:54:20.363 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 podman[110660]: 2024-04-25 15:54:20.3089435 +0000 UTC m=+0.478335542 container attach a48ded3c9d235595aa714541a2f8bc48ba49c0cf4945cbba047e638c79aaa31c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T15:54:21.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-mon[90991]: Health check failed: Degraded data redundancy: 10/267 objects degraded (3.745%), 4 pgs degraded (PG_DEGRADED) 2024-04-25T15:54:21.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:54:21.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 bash[110660]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-25T15:54:21.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-25T15:54:21.115 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-25T15:54:21.115 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T15:54:21.115 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate[110675]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-25T15:54:21.115 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:20 smithi110 podman[110660]: 2024-04-25 15:54:20.850140301 +0000 UTC m=+1.019532334 container died a48ded3c9d235595aa714541a2f8bc48ba49c0cf4945cbba047e638c79aaa31c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T15:54:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:20 smithi193 ceph-mon[80071]: Health check failed: Degraded data redundancy: 10/267 objects degraded (3.745%), 4 pgs degraded (PG_DEGRADED) 2024-04-25T15:54:21.804 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:21 smithi110 podman[110814]: 2024-04-25 15:54:21.463314276 +0000 UTC m=+0.607656677 container remove a48ded3c9d235595aa714541a2f8bc48ba49c0cf4945cbba047e638c79aaa31c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T15:54:21.804 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:21 smithi110 podman[110880]: 2024-04-25 15:54:21.581739954 +0000 UTC m=+0.018102287 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:54:21.804 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:21 smithi110 podman[110880]: 2024-04-25 15:54:21.697693497 +0000 UTC m=+0.134055817 container create 4fba7cfd0663f27dca63e4b15f94fbc02dc82c7bed97b02aea8d3423cd27578c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T15:54:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:21 smithi110 ceph-mon[90991]: pgmap v92: 65 pgs: 4 active+undersized+degraded, 12 peering, 4 stale+active+clean, 4 active+undersized, 41 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 10/267 objects degraded (3.745%) 2024-04-25T15:54:22.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:21 smithi110 podman[110880]: 2024-04-25 15:54:21.950289655 +0000 UTC m=+0.386651978 container init 4fba7cfd0663f27dca63e4b15f94fbc02dc82c7bed97b02aea8d3423cd27578c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T15:54:22.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:21 smithi110 podman[110880]: 2024-04-25 15:54:21.953310305 +0000 UTC m=+0.389672626 container start 4fba7cfd0663f27dca63e4b15f94fbc02dc82c7bed97b02aea8d3423cd27578c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:54:22.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:21 smithi110 ceph-osd[110899]: -- 172.21.15.110:0/329726579 <== mon.0 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635ca3e7680 con 0x5635ca48ed00 2024-04-25T15:54:22.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:22 smithi110 bash[110880]: 4fba7cfd0663f27dca63e4b15f94fbc02dc82c7bed97b02aea8d3423cd27578c 2024-04-25T15:54:22.114 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:22 smithi110 systemd[1]: Started Ceph osd.2 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:54:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:21 smithi193 ceph-mon[80071]: pgmap v92: 65 pgs: 4 active+undersized+degraded, 12 peering, 4 stale+active+clean, 4 active+undersized, 41 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 10/267 objects degraded (3.745%) 2024-04-25T15:54:22.863 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:22 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[110895]: 2024-04-25T15:54:22.509+0000 7f4c53d617c0 -1 Falling back to public interface 2024-04-25T15:54:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:54:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:23 smithi110 ceph-mon[90991]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:54:23.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:23.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:23.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:54:23.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:23 smithi193 ceph-mon[80071]: pgmap v93: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:54:24.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:24 smithi110 ceph-mon[90991]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-25T15:54:24.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:24 smithi193 ceph-mon[80071]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-25T15:54:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:25 smithi110 ceph-mon[90991]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:54:25.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:25 smithi193 ceph-mon[80071]: pgmap v94: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:54:27.613 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:27 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[110895]: 2024-04-25T15:54:27.351+0000 7f4c53d617c0 -1 osd.2 59 log_to_monitors true 2024-04-25T15:54:28.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 15:54:27 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[110895]: 2024-04-25T15:54:27.782+0000 7f4c4517b700 -1 osd.2 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:54:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:27 smithi110 ceph-mon[90991]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:54:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:27 smithi110 ceph-mon[90991]: from='osd.2 [v2:172.21.15.110:6818/3018305145,v1:172.21.15.110:6819/3018305145]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T15:54:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:27 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:27 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:28.193 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:27 smithi193 ceph-mon[80071]: pgmap v95: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:54:28.193 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:27 smithi193 ceph-mon[80071]: from='osd.2 [v2:172.21.15.110:6818/3018305145,v1:172.21.15.110:6819/3018305145]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T15:54:28.193 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:27 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:28.193 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:27 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:28 smithi110 ceph-mon[90991]: from='osd.2 [v2:172.21.15.110:6818/3018305145,v1:172.21.15.110:6819/3018305145]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T15:54:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:28 smithi110 ceph-mon[90991]: osdmap e62: 6 total, 5 up, 6 in 2024-04-25T15:54:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:28 smithi110 ceph-mon[90991]: from='osd.2 [v2:172.21.15.110:6818/3018305145,v1:172.21.15.110:6819/3018305145]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:54:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:28 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded (PG_DEGRADED) 2024-04-25T15:54:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:28 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:28 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:28 smithi193 ceph-mon[80071]: from='osd.2 [v2:172.21.15.110:6818/3018305145,v1:172.21.15.110:6819/3018305145]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T15:54:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:28 smithi193 ceph-mon[80071]: osdmap e62: 6 total, 5 up, 6 in 2024-04-25T15:54:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:28 smithi193 ceph-mon[80071]: from='osd.2 [v2:172.21.15.110:6818/3018305145,v1:172.21.15.110:6819/3018305145]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-04-25T15:54:29.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:28 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 34/267 objects degraded (12.734%), 12 pgs degraded (PG_DEGRADED) 2024-04-25T15:54:29.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:28 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:29.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:28 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:29 smithi110 ceph-mon[90991]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:54:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:29 smithi110 ceph-mon[90991]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:54:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:29 smithi110 ceph-mon[90991]: osd.2 [v2:172.21.15.110:6818/3018305145,v1:172.21.15.110:6819/3018305145] boot 2024-04-25T15:54:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:29 smithi110 ceph-mon[90991]: osdmap e63: 6 total, 6 up, 6 in 2024-04-25T15:54:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:54:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:29 smithi110 ceph-mon[90991]: osdmap e64: 6 total, 6 up, 6 in 2024-04-25T15:54:30.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.180+0000 7fb27cda1700 1 -- 172.21.15.110:0/4018763661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 msgr2=0x7fb2780fd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.180+0000 7fb27cda1700 1 --2- 172.21.15.110:0/4018763661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 0x7fb2780fd210 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb260004940 tx=0x7fb260009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:30.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.180+0000 7fb27cda1700 1 -- 172.21.15.110:0/4018763661 shutdown_connections 2024-04-25T15:54:30.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.180+0000 7fb27cda1700 1 --2- 172.21.15.110:0/4018763661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 0x7fb2780fd210 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.180+0000 7fb27cda1700 1 --2- 172.21.15.110:0/4018763661 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2780fbaf0 0x7fb2780fbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.180+0000 7fb27cda1700 1 -- 172.21.15.110:0/4018763661 >> 172.21.15.110:0/4018763661 conn(0x7fb2780f7270 msgr2=0x7fb2780f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.180+0000 7fb27cda1700 1 -- 172.21.15.110:0/4018763661 shutdown_connections 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.180+0000 7fb27cda1700 1 -- 172.21.15.110:0/4018763661 wait complete. 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb27cda1700 1 Processor -- start 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb27cda1700 1 -- start start 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb27cda1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2780fbaf0 0x7fb278110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb27cda1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 0x7fb278112a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb27cda1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278066de0 con 0x7fb2780fcda0 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb27cda1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb278066f20 con 0x7fb2780fbaf0 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb276ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 0x7fb278112a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb276ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 0x7fb278112a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39458/0 (socket says 172.21.15.110:39458) 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb276ffd700 1 -- 172.21.15.110:0/3401480415 learned_addr learned my addr 172.21.15.110:0/3401480415 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:54:30.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb2777fe700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2780fbaf0 0x7fb278110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb276ffd700 1 -- 172.21.15.110:0/3401480415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2780fbaf0 msgr2=0x7fb278110500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb276ffd700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2780fbaf0 0x7fb278110500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb276ffd700 1 -- 172.21.15.110:0/3401480415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb260005040 con 0x7fb2780fcda0 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.181+0000 7fb276ffd700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 0x7fb278112a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb26000b3b0 tx=0x7fb260009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.182+0000 7fb26ffff700 1 -- 172.21.15.110:0/3401480415 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb26000e040 con 0x7fb2780fcda0 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.182+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb278112ff0 con 0x7fb2780fcda0 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.182+0000 7fb26ffff700 1 -- 172.21.15.110:0/3401480415 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb26000bbf0 con 0x7fb2780fcda0 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.182+0000 7fb26ffff700 1 -- 172.21.15.110:0/3401480415 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb26001c630 con 0x7fb2780fcda0 2024-04-25T15:54:30.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.182+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb278113580 con 0x7fb2780fcda0 2024-04-25T15:54:30.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.183+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2780faef0 con 0x7fb2780fcda0 2024-04-25T15:54:30.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.183+0000 7fb26ffff700 1 -- 172.21.15.110:0/3401480415 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fb26000bd60 con 0x7fb2780fcda0 2024-04-25T15:54:30.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.184+0000 7fb26ffff700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb264076b90 0x7fb264079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.184+0000 7fb26ffff700 1 -- 172.21.15.110:0/3401480415 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb260098d90 con 0x7fb2780fcda0 2024-04-25T15:54:30.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.184+0000 7fb2777fe700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb264076b90 0x7fb264079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.184+0000 7fb2777fe700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb264076b90 0x7fb264079040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb278066a60 tx=0x7fb27000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:30.190 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.189+0000 7fb26ffff700 1 -- 172.21.15.110:0/3401480415 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb260061a30 con 0x7fb2780fcda0 2024-04-25T15:54:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:29 smithi193 ceph-mon[80071]: pgmap v97: 65 pgs: 12 active+undersized+degraded, 15 active+undersized, 38 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:54:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:29 smithi193 ceph-mon[80071]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:54:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:29 smithi193 ceph-mon[80071]: osd.2 [v2:172.21.15.110:6818/3018305145,v1:172.21.15.110:6819/3018305145] boot 2024-04-25T15:54:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:29 smithi193 ceph-mon[80071]: osdmap e63: 6 total, 6 up, 6 in 2024-04-25T15:54:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T15:54:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:29 smithi193 ceph-mon[80071]: osdmap e64: 6 total, 6 up, 6 in 2024-04-25T15:54:30.422 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.421+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2781010c0 con 0x7fb264076b90 2024-04-25T15:54:30.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.422+0000 7fb26ffff700 1 -- 172.21.15.110:0/3401480415 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb2781010c0 con 0x7fb264076b90 2024-04-25T15:54:30.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb264076b90 msgr2=0x7fb264079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb264076b90 0x7fb264079040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb278066a60 tx=0x7fb27000b040 comp rx=0 tx=0).stop 2024-04-25T15:54:30.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 msgr2=0x7fb278112a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 0x7fb278112a50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb26000b3b0 tx=0x7fb260009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:30.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 shutdown_connections 2024-04-25T15:54:30.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2780fbaf0 0x7fb278110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb264076b90 0x7fb264079040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 --2- 172.21.15.110:0/3401480415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2780fcda0 0x7fb278112a50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.424+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 >> 172.21.15.110:0/3401480415 conn(0x7fb2780f7270 msgr2=0x7fb2780f8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:30.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.425+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 shutdown_connections 2024-04-25T15:54:30.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.425+0000 7fb27cda1700 1 -- 172.21.15.110:0/3401480415 wait complete. 2024-04-25T15:54:30.436 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.529+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1463997257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fdfd0 msgr2=0x7f41d40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.529+0000 7f41d9fdf700 1 --2- 172.21.15.110:0/1463997257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fdfd0 0x7f41d40fe440 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f41bc004940 tx=0x7f41bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.529+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1463997257 shutdown_connections 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.529+0000 7f41d9fdf700 1 --2- 172.21.15.110:0/1463997257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fdfd0 0x7f41d40fe440 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.529+0000 7f41d9fdf700 1 --2- 172.21.15.110:0/1463997257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d40fce60 0x7f41d40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.529+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1463997257 >> 172.21.15.110:0/1463997257 conn(0x7f41d40f8540 msgr2=0x7f41d40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.529+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1463997257 shutdown_connections 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.529+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1463997257 wait complete. 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d9fdf700 1 Processor -- start 2024-04-25T15:54:30.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d9fdf700 1 -- start start 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d9fdf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fce60 0x7f41d4071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d9fdf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d40fdfd0 0x7f41d40702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d9fdf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d4072200 con 0x7f41d40fce60 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d9fdf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d40707f0 con 0x7f41d40fdfd0 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fce60 0x7f41d4071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fce60 0x7f41d4071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39468/0 (socket says 172.21.15.110:39468) 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d37fe700 1 -- 172.21.15.110:0/1720808501 learned_addr learned my addr 172.21.15.110:0/1720808501 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d2ffd700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d40fdfd0 0x7f41d40702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d37fe700 1 -- 172.21.15.110:0/1720808501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d40fdfd0 msgr2=0x7f41d40702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d37fe700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d40fdfd0 0x7f41d40702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.530+0000 7f41d37fe700 1 -- 172.21.15.110:0/1720808501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc005040 con 0x7f41d40fce60 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.531+0000 7f41d37fe700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fce60 0x7f41d4071c00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41cc001d80 tx=0x7f41cc009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.531+0000 7f41cbfff700 1 -- 172.21.15.110:0/1720808501 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41cc007d10 con 0x7f41d40fce60 2024-04-25T15:54:30.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.531+0000 7f41cbfff700 1 -- 172.21.15.110:0/1720808501 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41cc001e80 con 0x7f41d40fce60 2024-04-25T15:54:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.531+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d4070aa0 con 0x7f41d40fce60 2024-04-25T15:54:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.531+0000 7f41cbfff700 1 -- 172.21.15.110:0/1720808501 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41cc014640 con 0x7f41d40fce60 2024-04-25T15:54:30.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.531+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d4071060 con 0x7f41d40fce60 2024-04-25T15:54:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.532+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d404c710 con 0x7f41d40fce60 2024-04-25T15:54:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.533+0000 7f41cbfff700 1 -- 172.21.15.110:0/1720808501 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f41cc00e960 con 0x7f41d40fce60 2024-04-25T15:54:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.533+0000 7f41cbfff700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f41c0076b90 0x7f41c0079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.533+0000 7f41cbfff700 1 -- 172.21.15.110:0/1720808501 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f41cc098060 con 0x7f41d40fce60 2024-04-25T15:54:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.533+0000 7f41d2ffd700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f41c0076b90 0x7f41c0079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.533+0000 7f41d2ffd700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f41c0076b90 0x7f41c0079040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00b3b0 tx=0x7f41bc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:30.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.538+0000 7f41cbfff700 1 -- 172.21.15.110:0/1720808501 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f41cc060c90 con 0x7f41d40fce60 2024-04-25T15:54:30.765 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.764+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f41d4102390 con 0x7f41c0076b90 2024-04-25T15:54:30.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.766+0000 7f41cbfff700 1 -- 172.21.15.110:0/1720808501 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f41d4102390 con 0x7f41c0076b90 2024-04-25T15:54:30.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f41c0076b90 msgr2=0x7f41c0079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f41c0076b90 0x7f41c0079040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f41bc00b3b0 tx=0x7f41bc009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:30.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fce60 msgr2=0x7f41d4071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fce60 0x7f41d4071c00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f41cc001d80 tx=0x7f41cc009ec0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 shutdown_connections 2024-04-25T15:54:30.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f41c0076b90 0x7f41c0079040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f41d40fce60 0x7f41d4071c00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 --2- 172.21.15.110:0/1720808501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41d40fdfd0 0x7f41d40702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 >> 172.21.15.110:0/1720808501 conn(0x7f41d40f8540 msgr2=0x7f41d40f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:30.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 shutdown_connections 2024-04-25T15:54:30.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.768+0000 7f41d9fdf700 1 -- 172.21.15.110:0/1720808501 wait complete. 2024-04-25T15:54:30.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 -- 172.21.15.110:0/3218349961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 msgr2=0x7f42d00fd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 --2- 172.21.15.110:0/3218349961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 0x7f42d00fd460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f42c0004940 tx=0x7f42c0009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:30.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 -- 172.21.15.110:0/3218349961 shutdown_connections 2024-04-25T15:54:30.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 --2- 172.21.15.110:0/3218349961 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 0x7f42d00fd460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 --2- 172.21.15.110:0/3218349961 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42d00fbd40 0x7f42d00fc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 -- 172.21.15.110:0/3218349961 >> 172.21.15.110:0/3218349961 conn(0x7f42d00f74c0 msgr2=0x7f42d00f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:30.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 -- 172.21.15.110:0/3218349961 shutdown_connections 2024-04-25T15:54:30.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 -- 172.21.15.110:0/3218349961 wait complete. 2024-04-25T15:54:30.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 Processor -- start 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.872+0000 7f42d7775700 1 -- start start 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d7775700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42d00fbd40 0x7f42d0110710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d7775700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 0x7f42d0112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d7775700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d0066bc0 con 0x7f42d00fcff0 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d7775700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d0066d00 con 0x7f42d00fbd40 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d4d10700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 0x7f42d0112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d4d10700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 0x7f42d0112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39486/0 (socket says 172.21.15.110:39486) 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d4d10700 1 -- 172.21.15.110:0/1897726552 learned_addr learned my addr 172.21.15.110:0/1897726552 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d5511700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42d00fbd40 0x7f42d0110710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d4d10700 1 -- 172.21.15.110:0/1897726552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42d00fbd40 msgr2=0x7f42d0110710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d4d10700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42d00fbd40 0x7f42d0110710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:30.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d4d10700 1 -- 172.21.15.110:0/1897726552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c0005040 con 0x7f42d00fcff0 2024-04-25T15:54:30.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42d4d10700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 0x7f42d0112c60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f42c000b970 tx=0x7f42c0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:30.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.873+0000 7f42c5ffb700 1 -- 172.21.15.110:0/1897726552 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c000e040 con 0x7f42d00fcff0 2024-04-25T15:54:30.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.874+0000 7f42c5ffb700 1 -- 172.21.15.110:0/1897726552 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f42c000bbf0 con 0x7f42d00fcff0 2024-04-25T15:54:30.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.874+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d01131a0 con 0x7f42d00fcff0 2024-04-25T15:54:30.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.874+0000 7f42c5ffb700 1 -- 172.21.15.110:0/1897726552 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42c001c630 con 0x7f42d00fcff0 2024-04-25T15:54:30.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.874+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d0113760 con 0x7f42d00fcff0 2024-04-25T15:54:30.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.875+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d005f5c0 con 0x7f42d00fcff0 2024-04-25T15:54:30.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.876+0000 7f42c5ffb700 1 -- 172.21.15.110:0/1897726552 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f42c000bd60 con 0x7f42d00fcff0 2024-04-25T15:54:30.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.876+0000 7f42c5ffb700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f42bc076b90 0x7f42bc079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:30.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.877+0000 7f42c5ffb700 1 -- 172.21.15.110:0/1897726552 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f42c009a020 con 0x7f42d00fcff0 2024-04-25T15:54:30.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.877+0000 7f42d5511700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f42bc076b90 0x7f42bc079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:30.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.877+0000 7f42d5511700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f42bc076b90 0x7f42bc079040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0041d0 tx=0x7f42cc0043c0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:30.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:30.880+0000 7f42c5ffb700 1 -- 172.21.15.110:0/1897726552 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f42c0062cc0 con 0x7f42d00fcff0 2024-04-25T15:54:31.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.097+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f42d0101360 con 0x7f42bc076b90 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.108+0000 7f42c5ffb700 1 -- 172.21.15.110:0/1897726552 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f42d0101360 con 0x7f42bc076b90 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (4m) 3s ago 13m 25.2M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (109s) 3s ago 13m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (104s) 100s ago 12m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (4m) 3s ago 13m 85.4M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (8m) 3s ago 8m 19.9M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (8m) 3s ago 8m 150M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (8m) 100s ago 8m 14.2M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (8m) 100s ago 7m 16.7M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:54:31.109 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (4m) 3s ago 14m 570M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (3m) 100s ago 12m 517M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (2m) 3s ago 14m 114M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (2m) 100s ago 12m 98.5M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (5m) 3s ago 13m 15.7M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (4m) 100s ago 12m 15.7M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (82s) 3s ago 11m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (46s) 3s ago 11m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (9s) 3s ago 10m 82.3M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (10m) 100s ago 10m 239M 2581M 17.2.7-902-gc61472c8 0de407c1b3af edda41138222 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (10m) 100s ago 10m 206M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (9m) 100s ago 9m 189M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:54:31.110 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (3m) 3s ago 12m 54.6M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:54:31.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.110+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f42bc076b90 msgr2=0x7f42bc079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.110+0000 7f42d7775700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f42bc076b90 0x7f42bc079040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0041d0 tx=0x7f42cc0043c0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.110+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 msgr2=0x7f42d0112c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.110+0000 7f42d7775700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 0x7f42d0112c60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f42c000b970 tx=0x7f42c0009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.111+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 shutdown_connections 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.111+0000 7f42d7775700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42d00fbd40 0x7f42d0110710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.111+0000 7f42d7775700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f42bc076b90 0x7f42bc079040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.111+0000 7f42d7775700 1 --2- 172.21.15.110:0/1897726552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f42d00fcff0 0x7f42d0112c60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.111+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 >> 172.21.15.110:0/1897726552 conn(0x7f42d00f74c0 msgr2=0x7f42d00f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.111+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 shutdown_connections 2024-04-25T15:54:31.112 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.111+0000 7f42d7775700 1 -- 172.21.15.110:0/1897726552 wait complete. 2024-04-25T15:54:31.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.216+0000 7fc4696a6700 1 -- 172.21.15.110:0/419484032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fd3d0 msgr2=0x7fc4640fd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.216+0000 7fc4696a6700 1 --2- 172.21.15.110:0/419484032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fd3d0 0x7fc4640fd7e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc454004940 tx=0x7fc454009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:31.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.216+0000 7fc4696a6700 1 -- 172.21.15.110:0/419484032 shutdown_connections 2024-04-25T15:54:31.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.216+0000 7fc4696a6700 1 --2- 172.21.15.110:0/419484032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4640fdd20 0x7fc464104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.216+0000 7fc4696a6700 1 --2- 172.21.15.110:0/419484032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fd3d0 0x7fc4640fd7e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.216+0000 7fc4696a6700 1 -- 172.21.15.110:0/419484032 >> 172.21.15.110:0/419484032 conn(0x7fc4640f6b80 msgr2=0x7fc4640f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:31.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.216+0000 7fc4696a6700 1 -- 172.21.15.110:0/419484032 shutdown_connections 2024-04-25T15:54:31.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.216+0000 7fc4696a6700 1 -- 172.21.15.110:0/419484032 wait complete. 2024-04-25T15:54:31.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4696a6700 1 Processor -- start 2024-04-25T15:54:31.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4696a6700 1 -- start start 2024-04-25T15:54:31.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4696a6700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4640fd3d0 0x7fc46410fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4696a6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fdd20 0x7fc464110120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4696a6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464110720 con 0x7fc4640fdd20 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4696a6700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc464110f20 con 0x7fc4640fd3d0 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4627fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fdd20 0x7fc464110120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4627fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fdd20 0x7fc464110120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39498/0 (socket says 172.21.15.110:39498) 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc4627fc700 1 -- 172.21.15.110:0/1334445685 learned_addr learned my addr 172.21.15.110:0/1334445685 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.217+0000 7fc462ffd700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4640fd3d0 0x7fc46410fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc4627fc700 1 -- 172.21.15.110:0/1334445685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4640fd3d0 msgr2=0x7fc46410fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc4627fc700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4640fd3d0 0x7fc46410fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc4627fc700 1 -- 172.21.15.110:0/1334445685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc454005040 con 0x7fc4640fdd20 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc4627fc700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fdd20 0x7fc464110120 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc45800eb40 tx=0x7fc45800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc44b7fe700 1 -- 172.21.15.110:0/1334445685 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc458007cc0 con 0x7fc4640fdd20 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc44b7fe700 1 -- 172.21.15.110:0/1334445685 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc45800f040 con 0x7fc4640fdd20 2024-04-25T15:54:31.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4641111a0 con 0x7fc4640fdd20 2024-04-25T15:54:31.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc44b7fe700 1 -- 172.21.15.110:0/1334445685 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc458009620 con 0x7fc4640fdd20 2024-04-25T15:54:31.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.218+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4641116e0 con 0x7fc4640fdd20 2024-04-25T15:54:31.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.219+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46405f5c0 con 0x7fc4640fdd20 2024-04-25T15:54:31.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.220+0000 7fc44b7fe700 1 -- 172.21.15.110:0/1334445685 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fc458009780 con 0x7fc4640fdd20 2024-04-25T15:54:31.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.221+0000 7fc44b7fe700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc44c076b90 0x7fc44c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:31.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.221+0000 7fc44b7fe700 1 -- 172.21.15.110:0/1334445685 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc458097f10 con 0x7fc4640fdd20 2024-04-25T15:54:31.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.221+0000 7fc462ffd700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc44c076b90 0x7fc44c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:31.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.221+0000 7fc462ffd700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc44c076b90 0x7fc44c079040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc454009b30 tx=0x7fc454009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:31.226 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.226+0000 7fc44b7fe700 1 -- 172.21.15.110:0/1334445685 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc458060b40 con 0x7fc4640fdd20 2024-04-25T15:54:31.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.529+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc46404bed0 con 0x7fc4640fdd20 2024-04-25T15:54:31.530 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.529+0000 7fc44b7fe700 1 -- 172.21.15.110:0/1334445685 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fc4580604e0 con 0x7fc4640fdd20 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 3 2024-04-25T15:54:31.531 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:54:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:54:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:54:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:54:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:54:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 7, 2024-04-25T15:54:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 7 2024-04-25T15:54:31.532 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:54:31.532 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:54:31.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc44c076b90 msgr2=0x7fc44c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc44c076b90 0x7fc44c079040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc454009b30 tx=0x7fc454009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:31.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fdd20 msgr2=0x7fc464110120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fdd20 0x7fc464110120 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc45800eb40 tx=0x7fc45800b040 comp rx=0 tx=0).stop 2024-04-25T15:54:31.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 shutdown_connections 2024-04-25T15:54:31.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4640fd3d0 0x7fc46410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.533 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc44c076b90 0x7fc44c079040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 --2- 172.21.15.110:0/1334445685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4640fdd20 0x7fc464110120 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 >> 172.21.15.110:0/1334445685 conn(0x7fc4640f6b80 msgr2=0x7fc4640f6fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:31.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 shutdown_connections 2024-04-25T15:54:31.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.532+0000 7fc4696a6700 1 -- 172.21.15.110:0/1334445685 wait complete. 2024-04-25T15:54:31.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.634+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/3751746747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fbce0 msgr2=0x7f3b480fc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.634+0000 7f3b4d2d5700 1 --2- 172.21.15.110:0/3751746747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fbce0 0x7f3b480fc0f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3b30004940 tx=0x7f3b30009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:31.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.634+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/3751746747 shutdown_connections 2024-04-25T15:54:31.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.634+0000 7f3b4d2d5700 1 --2- 172.21.15.110:0/3751746747 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480fcf90 0x7f3b480fd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.634+0000 7f3b4d2d5700 1 --2- 172.21.15.110:0/3751746747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fbce0 0x7f3b480fc0f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.634+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/3751746747 >> 172.21.15.110:0/3751746747 conn(0x7f3b480f7420 msgr2=0x7f3b480f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:31.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.634+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/3751746747 shutdown_connections 2024-04-25T15:54:31.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.634+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/3751746747 wait complete. 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b4d2d5700 1 Processor -- start 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b4d2d5700 1 -- start start 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b4d2d5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480fbce0 0x7f3b48066c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b4d2d5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fcf90 0x7f3b481126a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b4d2d5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b48112be0 con 0x7f3b480fcf90 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b4d2d5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b48112d20 con 0x7f3b480fbce0 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b467fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fcf90 0x7f3b481126a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b467fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fcf90 0x7f3b481126a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39514/0 (socket says 172.21.15.110:39514) 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b467fc700 1 -- 172.21.15.110:0/1491952835 learned_addr learned my addr 172.21.15.110:0/1491952835 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:54:31.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.635+0000 7f3b46ffd700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480fbce0 0x7f3b48066c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b467fc700 1 -- 172.21.15.110:0/1491952835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480fbce0 msgr2=0x7f3b48066c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b467fc700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480fbce0 0x7f3b48066c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b467fc700 1 -- 172.21.15.110:0/1491952835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b30005040 con 0x7f3b480fcf90 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b467fc700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fcf90 0x7f3b481126a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3b38002a80 tx=0x7f3b380046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b3f7fe700 1 -- 172.21.15.110:0/1491952835 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b38007cc0 con 0x7f3b480fcf90 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b3f7fe700 1 -- 172.21.15.110:0/1491952835 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b3800f040 con 0x7f3b480fcf90 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b48112f80 con 0x7f3b480fcf90 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b3f7fe700 1 -- 172.21.15.110:0/1491952835 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b3800dac0 con 0x7f3b480fcf90 2024-04-25T15:54:31.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.636+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b48113510 con 0x7f3b480fcf90 2024-04-25T15:54:31.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.637+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b4804c710 con 0x7f3b480fcf90 2024-04-25T15:54:31.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.638+0000 7f3b3f7fe700 1 -- 172.21.15.110:0/1491952835 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f3b3800dc20 con 0x7f3b480fcf90 2024-04-25T15:54:31.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.638+0000 7f3b3f7fe700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3b34076b90 0x7f3b34079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:31.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.638+0000 7f3b3f7fe700 1 -- 172.21.15.110:0/1491952835 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f3b38097f30 con 0x7f3b480fcf90 2024-04-25T15:54:31.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.638+0000 7f3b46ffd700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3b34076b90 0x7f3b34079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:31.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.639+0000 7f3b46ffd700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3b34076b90 0x7f3b34079040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000b970 tx=0x7f3b30009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:31.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.643+0000 7f3b3f7fe700 1 -- 172.21.15.110:0/1491952835 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3b380ccff0 con 0x7f3b480fcf90 2024-04-25T15:54:31.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.898+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3b481139f0 con 0x7f3b480fcf90 2024-04-25T15:54:31.900 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.899+0000 7f3b3f7fe700 1 -- 172.21.15.110:0/1491952835 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f3b38060ab0 con 0x7f3b480fcf90 2024-04-25T15:54:31.900 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:54:31.900 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:54:31.900 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:54:31.900 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:54:31.900 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:54:31.901 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:54:31.902 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:54:31.903 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:54:31.903 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:54:31.903 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:54:31.903 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:54:31.903 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:54:31.903 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:54:31.903 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:54:31.903 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3b34076b90 msgr2=0x7f3b34079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3b34076b90 0x7f3b34079040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000b970 tx=0x7f3b30009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fcf90 msgr2=0x7f3b481126a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fcf90 0x7f3b481126a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3b38002a80 tx=0x7f3b380046b0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 shutdown_connections 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480fbce0 0x7f3b48066c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3b34076b90 0x7f3b34079040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 --2- 172.21.15.110:0/1491952835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b480fcf90 0x7f3b481126a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:31.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 >> 172.21.15.110:0/1491952835 conn(0x7f3b480f7420 msgr2=0x7f3b480f8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:31.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 shutdown_connections 2024-04-25T15:54:31.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:31.901+0000 7f3b4d2d5700 1 -- 172.21.15.110:0/1491952835 wait complete. 2024-04-25T15:54:31.905 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:54:32.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.004+0000 7f159f5c5700 1 -- 172.21.15.110:0/1841444738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fe3e0 msgr2=0x7f15980fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:32.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.004+0000 7f159f5c5700 1 --2- 172.21.15.110:0/1841444738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fe3e0 0x7f15980fe7f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1594009ce0 tx=0x7f1594004d10 comp rx=0 tx=0).stop 2024-04-25T15:54:32.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.004+0000 7f159f5c5700 1 -- 172.21.15.110:0/1841444738 shutdown_connections 2024-04-25T15:54:32.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.004+0000 7f159f5c5700 1 --2- 172.21.15.110:0/1841444738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15980fed30 0x7f15980fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.004+0000 7f159f5c5700 1 --2- 172.21.15.110:0/1841444738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fe3e0 0x7f15980fe7f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.004+0000 7f159f5c5700 1 -- 172.21.15.110:0/1841444738 >> 172.21.15.110:0/1841444738 conn(0x7f15980f6b80 msgr2=0x7f15980f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:32.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.004+0000 7f159f5c5700 1 -- 172.21.15.110:0/1841444738 shutdown_connections 2024-04-25T15:54:32.006 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.004+0000 7f159f5c5700 1 -- 172.21.15.110:0/1841444738 wait complete. 2024-04-25T15:54:32.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159f5c5700 1 Processor -- start 2024-04-25T15:54:32.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159f5c5700 1 -- start start 2024-04-25T15:54:32.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159f5c5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15980fe3e0 0x7f159810db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:32.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159f5c5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fed30 0x7f159810e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:32.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159f5c5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159810e680 con 0x7f15980fed30 2024-04-25T15:54:32.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159f5c5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159810eed0 con 0x7f15980fe3e0 2024-04-25T15:54:32.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159cb60700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fed30 0x7f159810e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:32.008 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159cb60700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fed30 0x7f159810e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39538/0 (socket says 172.21.15.110:39538) 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159cb60700 1 -- 172.21.15.110:0/2682593629 learned_addr learned my addr 172.21.15.110:0/2682593629 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159d361700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15980fe3e0 0x7f159810db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159cb60700 1 -- 172.21.15.110:0/2682593629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15980fe3e0 msgr2=0x7f159810db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159cb60700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15980fe3e0 0x7f159810db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159cb60700 1 -- 172.21.15.110:0/2682593629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1594009940 con 0x7f15980fed30 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.005+0000 7f159cb60700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fed30 0x7f159810e080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1588002a50 tx=0x7f15880046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.006+0000 7f158dffb700 1 -- 172.21.15.110:0/2682593629 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1588007cc0 con 0x7f15980fed30 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.006+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159810f1b0 con 0x7f15980fed30 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.006+0000 7f158dffb700 1 -- 172.21.15.110:0/2682593629 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f158800f040 con 0x7f15980fed30 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.006+0000 7f158dffb700 1 -- 172.21.15.110:0/2682593629 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f158800da30 con 0x7f15980fed30 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.006+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159810f6f0 con 0x7f15980fed30 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.007+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f159805f5c0 con 0x7f15980fed30 2024-04-25T15:54:32.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.007+0000 7f158dffb700 1 -- 172.21.15.110:0/2682593629 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f158800db90 con 0x7f15980fed30 2024-04-25T15:54:32.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.008+0000 7f158dffb700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1584076b90 0x7f1584079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:32.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.008+0000 7f158dffb700 1 -- 172.21.15.110:0/2682593629 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f1588098060 con 0x7f15980fed30 2024-04-25T15:54:32.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.008+0000 7f159d361700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1584076b90 0x7f1584079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:32.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.008+0000 7f159d361700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1584076b90 0x7f1584079040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1594015040 tx=0x7f1594004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:32.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.013+0000 7f158dffb700 1 -- 172.21.15.110:0/2682593629 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1588060be0 con 0x7f15980fed30 2024-04-25T15:54:32.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:31 smithi110 ceph-mon[90991]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:32.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:31 smithi110 ceph-mon[90991]: pgmap v100: 65 pgs: 10 active+undersized+degraded, 12 active+undersized, 43 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/267 objects degraded (11.236%) 2024-04-25T15:54:32.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:31 smithi110 ceph-mon[90991]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:32.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:31 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1334445685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:54:32.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:32.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:54:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:31 smithi193 ceph-mon[80071]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:31 smithi193 ceph-mon[80071]: pgmap v100: 65 pgs: 10 active+undersized+degraded, 12 active+undersized, 43 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 30/267 objects degraded (11.236%) 2024-04-25T15:54:32.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:31 smithi193 ceph-mon[80071]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:32.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:31 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1334445685' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:54:32.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:32.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:54:32.243 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.242+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f159810fa20 con 0x7f1584076b90 2024-04-25T15:54:32.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.244+0000 7f158dffb700 1 -- 172.21.15.110:0/2682593629 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f159810fa20 con 0x7f1584076b90 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "9/21 daemons upgraded", 2024-04-25T15:54:32.245 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T15:54:32.246 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:54:32.246 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:54:32.247 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1584076b90 msgr2=0x7f1584079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1584076b90 0x7f1584079040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1594015040 tx=0x7f1594004d10 comp rx=0 tx=0).stop 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fed30 msgr2=0x7f159810e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fed30 0x7f159810e080 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1588002a50 tx=0x7f15880046b0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 shutdown_connections 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15980fe3e0 0x7f159810db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1584076b90 0x7f1584079040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 --2- 172.21.15.110:0/2682593629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15980fed30 0x7f159810e080 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 >> 172.21.15.110:0/2682593629 conn(0x7f15980f6b80 msgr2=0x7f15980f8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 shutdown_connections 2024-04-25T15:54:32.248 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.246+0000 7f159f5c5700 1 -- 172.21.15.110:0/2682593629 wait complete. 2024-04-25T15:54:32.354 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.351+0000 7f40c53d3700 1 -- 172.21.15.110:0/265505206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fcf60 msgr2=0x7f40c00fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.351+0000 7f40c53d3700 1 --2- 172.21.15.110:0/265505206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fcf60 0x7f40c00fd3d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40b0004940 tx=0x7f40b0009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 -- 172.21.15.110:0/265505206 shutdown_connections 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 --2- 172.21.15.110:0/265505206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fcf60 0x7f40c00fd3d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 --2- 172.21.15.110:0/265505206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c00fbcb0 0x7f40c00fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 -- 172.21.15.110:0/265505206 >> 172.21.15.110:0/265505206 conn(0x7f40c00f7430 msgr2=0x7f40c00f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 -- 172.21.15.110:0/265505206 shutdown_connections 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 -- 172.21.15.110:0/265505206 wait complete. 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 Processor -- start 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 -- start start 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fbcb0 0x7f40c010fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c00fcf60 0x7f40c01101f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c01107f0 con 0x7f40c00fbcb0 2024-04-25T15:54:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.352+0000 7f40c53d3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40c0110ff0 con 0x7f40c00fcf60 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40beffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fbcb0 0x7f40c010fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40beffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fbcb0 0x7f40c010fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39562/0 (socket says 172.21.15.110:39562) 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40beffd700 1 -- 172.21.15.110:0/2508337024 learned_addr learned my addr 172.21.15.110:0/2508337024 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40be7fc700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c00fcf60 0x7f40c01101f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40beffd700 1 -- 172.21.15.110:0/2508337024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c00fcf60 msgr2=0x7f40c01101f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40beffd700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c00fcf60 0x7f40c01101f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40beffd700 1 -- 172.21.15.110:0/2508337024 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b0005040 con 0x7f40c00fbcb0 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40beffd700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fbcb0 0x7f40c010fcb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40b4002a50 tx=0x7f40b40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40a77fe700 1 -- 172.21.15.110:0/2508337024 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b4007cc0 con 0x7f40c00fbcb0 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40a77fe700 1 -- 172.21.15.110:0/2508337024 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f40b400f040 con 0x7f40c00fbcb0 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40c01112d0 con 0x7f40c00fbcb0 2024-04-25T15:54:32.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40a77fe700 1 -- 172.21.15.110:0/2508337024 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40b400da30 con 0x7f40c00fbcb0 2024-04-25T15:54:32.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.353+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40c0111810 con 0x7f40c00fbcb0 2024-04-25T15:54:32.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.354+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40c01018f0 con 0x7f40c00fbcb0 2024-04-25T15:54:32.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.355+0000 7f40a77fe700 1 -- 172.21.15.110:0/2508337024 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f40b400db90 con 0x7f40c00fbcb0 2024-04-25T15:54:32.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.355+0000 7f40a77fe700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f40a8076b90 0x7f40a8079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:54:32.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.355+0000 7f40a77fe700 1 -- 172.21.15.110:0/2508337024 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f40b4098060 con 0x7f40c00fbcb0 2024-04-25T15:54:32.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.355+0000 7f40be7fc700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f40a8076b90 0x7f40a8079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:54:32.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.356+0000 7f40be7fc700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f40a8076b90 0x7f40a8079040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b970 tx=0x7f40b0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:54:32.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.360+0000 7f40a77fe700 1 -- 172.21.15.110:0/2508337024 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f40b4060be0 con 0x7f40c00fbcb0 2024-04-25T15:54:32.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.644+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f40c0104920 con 0x7f40c00fbcb0 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.644+0000 7f40a77fe700 1 -- 172.21.15.110:0/2508337024 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+929 (secure 0 0 0) 0x7f40b4060580 con 0x7f40c00fbcb0 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 30/267 objects degraded (11.236%), 10 pgs degraded 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 30/267 objects degraded (11.236%), 10 pgs degraded 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.c is active+undersized+degraded, acting [3,0] 2024-04-25T15:54:32.646 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-04-25T15:54:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-04-25T15:54:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-04-25T15:54:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-04-25T15:54:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-04-25T15:54:32.647 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f40a8076b90 msgr2=0x7f40a8079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f40a8076b90 0x7f40a8079040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40b000b970 tx=0x7f40b0009e30 comp rx=0 tx=0).stop 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fbcb0 msgr2=0x7f40c010fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fbcb0 0x7f40c010fcb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40b4002a50 tx=0x7f40b40046b0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 shutdown_connections 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f40a8076b90 0x7f40a8079040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f40c00fbcb0 0x7f40c010fcb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 --2- 172.21.15.110:0/2508337024 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f40c00fcf60 0x7f40c01101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 >> 172.21.15.110:0/2508337024 conn(0x7f40c00f7430 msgr2=0x7f40c0065c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 shutdown_connections 2024-04-25T15:54:32.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:54:32.647+0000 7f40c53d3700 1 -- 172.21.15.110:0/2508337024 wait complete. 2024-04-25T15:54:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1491952835' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:54:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:54:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:54:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:54:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:54:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:54:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:54:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T15:54:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:32 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2508337024' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:54:33.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:33.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1491952835' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T15:54:33.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:32 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2508337024' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:54:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:33 smithi110 ceph-mon[90991]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:33 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T15:54:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:33 smithi110 ceph-mon[90991]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-04-25T15:54:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:33 smithi110 ceph-mon[90991]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 17/267 objects degraded (6.367%) 2024-04-25T15:54:34.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:33 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 17/267 objects degraded (6.367%), 5 pgs degraded (PG_DEGRADED) 2024-04-25T15:54:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:33 smithi193 ceph-mon[80071]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:54:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:33 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T15:54:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:33 smithi193 ceph-mon[80071]: Upgrade: unsafe to stop osd(s) at this time (10 PGs are or would become offline) 2024-04-25T15:54:34.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:33 smithi193 ceph-mon[80071]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 4 active+undersized, 56 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 17/267 objects degraded (6.367%) 2024-04-25T15:54:34.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:33 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 17/267 objects degraded (6.367%), 5 pgs degraded (PG_DEGRADED) 2024-04-25T15:54:35.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:34 smithi110 ceph-mon[90991]: pgmap v102: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:35.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:34 smithi110 ceph-mon[90991]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/267 objects degraded (6.367%), 5 pgs degraded) 2024-04-25T15:54:35.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:34 smithi193 ceph-mon[80071]: pgmap v102: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:35.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:34 smithi193 ceph-mon[80071]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 17/267 objects degraded (6.367%), 5 pgs degraded) 2024-04-25T15:54:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:37 smithi110 ceph-mon[90991]: pgmap v103: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:37 smithi193 ceph-mon[80071]: pgmap v103: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:39 smithi110 ceph-mon[90991]: pgmap v104: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:39 smithi193 ceph-mon[80071]: pgmap v104: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:41 smithi110 ceph-mon[90991]: pgmap v105: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:41 smithi193 ceph-mon[80071]: pgmap v105: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:44.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:43 smithi110 ceph-mon[90991]: pgmap v106: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:43 smithi193 ceph-mon[80071]: pgmap v106: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:45 smithi110 ceph-mon[90991]: pgmap v107: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:45 smithi193 ceph-mon[80071]: pgmap v107: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:48.027 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:47 smithi193 ceph-mon[80071]: pgmap v108: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:48.027 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:48.027 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:54:48.027 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T15:54:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:47 smithi110 ceph-mon[90991]: pgmap v108: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:54:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T15:54:49.037 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:48 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T15:54:49.038 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:48 smithi193 ceph-mon[80071]: Upgrade: osd.3 is safe to restart 2024-04-25T15:54:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:48 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-25T15:54:49.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:48 smithi110 ceph-mon[90991]: Upgrade: osd.3 is safe to restart 2024-04-25T15:54:50.053 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:49 smithi193 ceph-mon[80071]: pgmap v109: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:50.053 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:49 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:50.053 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:49 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T15:54:50.053 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:49 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:54:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:49 smithi110 ceph-mon[90991]: pgmap v109: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:49 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:49 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T15:54:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:49 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:54:50.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:50 smithi193 ceph-mon[80071]: Upgrade: Updating osd.3 2024-04-25T15:54:50.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:50 smithi193 ceph-mon[80071]: Deploying daemon osd.3 on smithi193 2024-04-25T15:54:51.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:50 smithi110 ceph-mon[90991]: Upgrade: Updating osd.3 2024-04-25T15:54:51.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:50 smithi110 ceph-mon[90991]: Deploying daemon osd.3 on smithi193 2024-04-25T15:54:51.973 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:51 smithi193 ceph-mon[80071]: pgmap v110: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:52.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:51 smithi110 ceph-mon[90991]: pgmap v110: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:52.603 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:52 smithi193 systemd[1]: Stopping Ceph osd.3 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:54:52.603 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:52 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[39361]: 2024-04-25T15:54:52.559+0000 7f2d6a355700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:54:52.603 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:52 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[39361]: 2024-04-25T15:54:52.559+0000 7f2d6a355700 -1 osd.3 64 *** Got signal Terminated *** 2024-04-25T15:54:52.603 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:52 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[39361]: 2024-04-25T15:54:52.559+0000 7f2d6a355700 -1 osd.3 64 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:54:52.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:52 smithi110 ceph-mon[90991]: osd.3 marked itself down and dead 2024-04-25T15:54:52.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:52 smithi193 ceph-mon[80071]: osd.3 marked itself down and dead 2024-04-25T15:54:53.477 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:53 smithi193 podman[88533]: 2024-04-25 15:54:53.182884241 +0000 UTC m=+0.723634891 container died edda41138222b16ba1d9dcfacc07e6560df3aae84204c23399f4b6724294bc07 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=quincy-c61472c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T15:54:53.739 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:53 smithi193 podman[88533]: 2024-04-25 15:54:53.584905799 +0000 UTC m=+1.125656431 container cleanup edda41138222b16ba1d9dcfacc07e6560df3aae84204c23399f4b6724294bc07 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T15:54:53.739 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:53 smithi193 bash[88533]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3 2024-04-25T15:54:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:53 smithi110 ceph-mon[90991]: pgmap v111: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:53 smithi110 ceph-mon[90991]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:54:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:53 smithi110 ceph-mon[90991]: osdmap e65: 6 total, 5 up, 6 in 2024-04-25T15:54:54.149 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:53 smithi193 ceph-mon[80071]: pgmap v111: 65 pgs: 65 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:54.149 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:53 smithi193 ceph-mon[80071]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:54:54.149 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:53 smithi193 ceph-mon[80071]: osdmap e65: 6 total, 5 up, 6 in 2024-04-25T15:54:54.149 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:53 smithi193 podman[88552]: 2024-04-25 15:54:53.827932522 +0000 UTC m=+0.640565022 container remove edda41138222b16ba1d9dcfacc07e6560df3aae84204c23399f4b6724294bc07 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, RELEASE=quincy-c61472c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T15:54:54.448 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:54 smithi193 podman[88642]: 2024-04-25 15:54:54.052037264 +0000 UTC m=+0.017208673 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:54:54.448 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:54 smithi193 podman[88642]: 2024-04-25 15:54:54.162216685 +0000 UTC m=+0.127388137 container create 7f89f1393af7fc0b513d6c6a240219a22c375fbc8ad05d9b8870b0847e41ccb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0) 2024-04-25T15:54:54.448 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:54 smithi193 podman[88642]: 2024-04-25 15:54:54.389557902 +0000 UTC m=+0.354729308 container init 7f89f1393af7fc0b513d6c6a240219a22c375fbc8ad05d9b8870b0847e41ccb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T15:54:54.448 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:54 smithi193 podman[88642]: 2024-04-25 15:54:54.392662705 +0000 UTC m=+0.357834115 container start 7f89f1393af7fc0b513d6c6a240219a22c375fbc8ad05d9b8870b0847e41ccb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T15:54:54.448 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:54 smithi193 podman[88642]: 2024-04-25 15:54:54.44783227 +0000 UTC m=+0.413003680 container attach 7f89f1393af7fc0b513d6c6a240219a22c375fbc8ad05d9b8870b0847e41ccb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-25T15:54:54.739 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:54 smithi193 podman[88642]: 2024-04-25 15:54:54.679237355 +0000 UTC m=+0.644408753 container died 7f89f1393af7fc0b513d6c6a240219a22c375fbc8ad05d9b8870b0847e41ccb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:54:55.034 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:54 smithi193 ceph-mon[80071]: osdmap e66: 6 total, 5 up, 6 in 2024-04-25T15:54:55.038 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:54 smithi110 ceph-mon[90991]: osdmap e66: 6 total, 5 up, 6 in 2024-04-25T15:54:55.720 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 podman[88668]: 2024-04-25 15:54:55.30988838 +0000 UTC m=+0.624992595 container remove 7f89f1393af7fc0b513d6c6a240219a22c375fbc8ad05d9b8870b0847e41ccb0 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T15:54:55.720 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-25T15:54:55.720 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 systemd[1]: Stopped Ceph osd.3 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:54:55.720 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 202ms CPU time. 2024-04-25T15:54:55.720 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 systemd[1]: Starting Ceph osd.3 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:54:55.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:55 smithi193 ceph-mon[80071]: pgmap v114: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:55.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:55 smithi193 ceph-mon[80071]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T15:54:55.990 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 podman[88771]: 2024-04-25 15:54:55.719250899 +0000 UTC m=+0.092945708 container create 75e05c359e0ccce379d8d33e5f9aa70964892d8cd5e85e34c1917fb8d3aa05d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T15:54:55.990 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 podman[88771]: 2024-04-25 15:54:55.644445363 +0000 UTC m=+0.018140178 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:54:55.990 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 podman[88771]: 2024-04-25 15:54:55.938863542 +0000 UTC m=+0.312558356 container init 75e05c359e0ccce379d8d33e5f9aa70964892d8cd5e85e34c1917fb8d3aa05d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T15:54:55.990 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:55 smithi193 podman[88771]: 2024-04-25 15:54:55.942250884 +0000 UTC m=+0.315945695 container start 75e05c359e0ccce379d8d33e5f9aa70964892d8cd5e85e34c1917fb8d3aa05d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T15:54:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:55 smithi110 ceph-mon[90991]: pgmap v114: 65 pgs: 4 peering, 15 stale+active+clean, 46 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:54:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:55 smithi110 ceph-mon[90991]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T15:54:56.489 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 podman[88771]: 2024-04-25 15:54:56.004995065 +0000 UTC m=+0.378689886 container attach 75e05c359e0ccce379d8d33e5f9aa70964892d8cd5e85e34c1917fb8d3aa05d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:54:56.821 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:54:56.821 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:54:56.821 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 bash[88771]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-25T15:54:56.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-25T15:54:56.823 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T15:54:56.823 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate[88786]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-25T15:54:56.823 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:56 smithi193 podman[88771]: 2024-04-25 15:54:56.556314102 +0000 UTC m=+0.930008903 container died 75e05c359e0ccce379d8d33e5f9aa70964892d8cd5e85e34c1917fb8d3aa05d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5) 2024-04-25T15:54:57.108 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:56 smithi193 ceph-mon[80071]: Health check failed: Degraded data redundancy: 24/267 objects degraded (8.989%), 8 pgs degraded (PG_DEGRADED) 2024-04-25T15:54:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:56 smithi110 ceph-mon[90991]: Health check failed: Degraded data redundancy: 24/267 objects degraded (8.989%), 8 pgs degraded (PG_DEGRADED) 2024-04-25T15:54:57.422 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:57 smithi193 podman[88922]: 2024-04-25 15:54:57.108920194 +0000 UTC m=+0.544990897 container remove 75e05c359e0ccce379d8d33e5f9aa70964892d8cd5e85e34c1917fb8d3aa05d4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-25T15:54:57.422 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:57 smithi193 podman[88982]: 2024-04-25 15:54:57.318213686 +0000 UTC m=+0.112560226 container create daab482bd8fad060f798eedc27b37e402e8f5d64f09d72d153f5e31e524bfd08 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:54:57.422 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:57 smithi193 podman[88982]: 2024-04-25 15:54:57.223541622 +0000 UTC m=+0.017888175 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:54:57.822 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:57 smithi193 podman[88982]: 2024-04-25 15:54:57.537212076 +0000 UTC m=+0.331558616 container init daab482bd8fad060f798eedc27b37e402e8f5d64f09d72d153f5e31e524bfd08 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T15:54:57.823 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:57 smithi193 podman[88982]: 2024-04-25 15:54:57.540291248 +0000 UTC m=+0.334637789 container start daab482bd8fad060f798eedc27b37e402e8f5d64f09d72d153f5e31e524bfd08 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T15:54:57.823 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:57 smithi193 ceph-osd[89003]: -- 172.21.15.193:0/3268343911 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557a0084d680 con 0x557a008f4d00 2024-04-25T15:54:57.823 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:57 smithi193 bash[88982]: daab482bd8fad060f798eedc27b37e402e8f5d64f09d72d153f5e31e524bfd08 2024-04-25T15:54:57.823 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:57 smithi193 systemd[1]: Started Ceph osd.3 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:54:58.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:57 smithi193 ceph-mon[80071]: pgmap v115: 65 pgs: 8 active+undersized+degraded, 4 peering, 10 stale+active+clean, 8 active+undersized, 35 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 24/267 objects degraded (8.989%) 2024-04-25T15:54:58.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:58.100 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:58.101 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:54:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:57 smithi110 ceph-mon[90991]: pgmap v115: 65 pgs: 8 active+undersized+degraded, 4 peering, 10 stale+active+clean, 8 active+undersized, 35 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 24/267 objects degraded (8.989%) 2024-04-25T15:54:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:54:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:54:58.476 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:54:58 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[88999]: 2024-04-25T15:54:58.099+0000 7fc28921b7c0 -1 Falling back to public interface 2024-04-25T15:54:59.177 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:58 smithi193 ceph-mon[80071]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:54:59.177 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:54:58 smithi193 ceph-mon[80071]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T15:54:59.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:58 smithi110 ceph-mon[90991]: pgmap v116: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:54:59.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:54:58 smithi110 ceph-mon[90991]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T15:55:01.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:01.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:01.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:01 smithi193 ceph-mon[80071]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:55:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:01 smithi110 ceph-mon[90991]: pgmap v117: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 -- 172.21.15.110:0/688999638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 msgr2=0x7fb330102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 --2- 172.21.15.110:0/688999638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 0x7fb330102350 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb324004940 tx=0x7fb324009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 -- 172.21.15.110:0/688999638 shutdown_connections 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 --2- 172.21.15.110:0/688999638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 0x7fb330102350 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 --2- 172.21.15.110:0/688999638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb330066eb0 0x7fb3300ffa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 -- 172.21.15.110:0/688999638 >> 172.21.15.110:0/688999638 conn(0x7fb3300f7440 msgr2=0x7fb3300f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 -- 172.21.15.110:0/688999638 shutdown_connections 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 -- 172.21.15.110:0/688999638 wait complete. 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.765+0000 7fb335628700 1 Processor -- start 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb335628700 1 -- start start 2024-04-25T15:55:02.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb335628700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb330066eb0 0x7fb33010d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb335628700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 0x7fb33010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb335628700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33010e460 con 0x7fb3300fff70 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb335628700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33010e5d0 con 0x7fb330066eb0 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 0x7fb33010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32effd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb330066eb0 0x7fb33010d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32e7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 0x7fb33010de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56152/0 (socket says 172.21.15.110:56152) 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32e7fc700 1 -- 172.21.15.110:0/3618992380 learned_addr learned my addr 172.21.15.110:0/3618992380 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32effd700 1 -- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 msgr2=0x7fb33010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32effd700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 0x7fb33010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32effd700 1 -- 172.21.15.110:0/3618992380 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb324005040 con 0x7fb330066eb0 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32e7fc700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 0x7fb33010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.766+0000 7fb32effd700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb330066eb0 0x7fb33010d920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb32800ebf0 tx=0x7fb32800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.767+0000 7fb3177fe700 1 -- 172.21.15.110:0/3618992380 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb328007c00 con 0x7fb330066eb0 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.767+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb330112fe0 con 0x7fb330066eb0 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.767+0000 7fb3177fe700 1 -- 172.21.15.110:0/3618992380 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb32800f040 con 0x7fb330066eb0 2024-04-25T15:55:02.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.767+0000 7fb3177fe700 1 -- 172.21.15.110:0/3618992380 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb32800cbf0 con 0x7fb330066eb0 2024-04-25T15:55:02.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.767+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb330113490 con 0x7fb330066eb0 2024-04-25T15:55:02.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.768+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33004c710 con 0x7fb330066eb0 2024-04-25T15:55:02.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.768+0000 7fb3177fe700 1 -- 172.21.15.110:0/3618992380 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fb328009110 con 0x7fb330066eb0 2024-04-25T15:55:02.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.769+0000 7fb3177fe700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb318076b80 0x7fb318079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:02.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.769+0000 7fb32e7fc700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb318076b80 0x7fb318079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:02.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.769+0000 7fb3177fe700 1 -- 172.21.15.110:0/3618992380 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb328098370 con 0x7fb330066eb0 2024-04-25T15:55:02.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.769+0000 7fb32e7fc700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb318076b80 0x7fb318079030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb324013740 tx=0x7fb324009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:02.775 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.774+0000 7fb3177fe700 1 -- 172.21.15.110:0/3618992380 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb328060f00 con 0x7fb330066eb0 2024-04-25T15:55:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:55:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:02.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.995+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3300f8f30 con 0x7fb318076b80 2024-04-25T15:55:02.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.996+0000 7fb3177fe700 1 -- 172.21.15.110:0/3618992380 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fb3300f8f30 con 0x7fb318076b80 2024-04-25T15:55:02.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb318076b80 msgr2=0x7fb318079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:02.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb318076b80 0x7fb318079030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb324013740 tx=0x7fb324009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:02.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb330066eb0 msgr2=0x7fb33010d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:02.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb330066eb0 0x7fb33010d920 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fb32800ebf0 tx=0x7fb32800b040 comp rx=0 tx=0).stop 2024-04-25T15:55:02.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 shutdown_connections 2024-04-25T15:55:02.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb330066eb0 0x7fb33010d920 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb318076b80 0x7fb318079030 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 --2- 172.21.15.110:0/3618992380 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3300fff70 0x7fb33010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 >> 172.21.15.110:0/3618992380 conn(0x7fb3300f7440 msgr2=0x7fb3300f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:03.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 shutdown_connections 2024-04-25T15:55:03.000 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:02.998+0000 7fb335628700 1 -- 172.21.15.110:0/3618992380 wait complete. 2024-04-25T15:55:03.009 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.100+0000 7f720e319700 1 -- 172.21.15.110:0/2136777876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fdfd0 msgr2=0x7f72080fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.100+0000 7f720e319700 1 --2- 172.21.15.110:0/2136777876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fdfd0 0x7f72080fe440 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f71f8004940 tx=0x7f71f8009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.100+0000 7f720e319700 1 -- 172.21.15.110:0/2136777876 shutdown_connections 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.100+0000 7f720e319700 1 --2- 172.21.15.110:0/2136777876 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fdfd0 0x7f72080fe440 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.100+0000 7f720e319700 1 --2- 172.21.15.110:0/2136777876 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72080fce60 0x7f72080fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.100+0000 7f720e319700 1 -- 172.21.15.110:0/2136777876 >> 172.21.15.110:0/2136777876 conn(0x7f72080f8540 msgr2=0x7f72080fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.100+0000 7f720e319700 1 -- 172.21.15.110:0/2136777876 shutdown_connections 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.100+0000 7f720e319700 1 -- 172.21.15.110:0/2136777876 wait complete. 2024-04-25T15:55:03.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f720e319700 1 Processor -- start 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f720e319700 1 -- start start 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f720e319700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fce60 0x7f720810d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f720e319700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72080fdfd0 0x7f720810deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f720e319700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720810e4b0 con 0x7f72080fce60 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f720e319700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720810ed00 con 0x7f72080fdfd0 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f7207fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fce60 0x7f720810d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f7207fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fce60 0x7f720810d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56156/0 (socket says 172.21.15.110:56156) 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f7207fff700 1 -- 172.21.15.110:0/1880603504 learned_addr learned my addr 172.21.15.110:0/1880603504 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f72077fe700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72080fdfd0 0x7f720810deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f7207fff700 1 -- 172.21.15.110:0/1880603504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72080fdfd0 msgr2=0x7f720810deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f7207fff700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72080fdfd0 0x7f720810deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.101+0000 7f7207fff700 1 -- 172.21.15.110:0/1880603504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f8005040 con 0x7f72080fce60 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.102+0000 7f7207fff700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fce60 0x7f720810d970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00eb40 tx=0x7f71fc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:03.104 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.102+0000 7f7204ff9700 1 -- 172.21.15.110:0/1880603504 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71fc007cc0 con 0x7f72080fce60 2024-04-25T15:55:03.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.102+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720810ef80 con 0x7f72080fce60 2024-04-25T15:55:03.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.102+0000 7f7204ff9700 1 -- 172.21.15.110:0/1880603504 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f71fc00f040 con 0x7f72080fce60 2024-04-25T15:55:03.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.102+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f720810f4c0 con 0x7f72080fce60 2024-04-25T15:55:03.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.102+0000 7f7204ff9700 1 -- 172.21.15.110:0/1880603504 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71fc00c9b0 con 0x7f72080fce60 2024-04-25T15:55:03.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.103+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f720805f5c0 con 0x7f72080fce60 2024-04-25T15:55:03.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.103+0000 7f7204ff9700 1 -- 172.21.15.110:0/1880603504 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f71fc00cbf0 con 0x7f72080fce60 2024-04-25T15:55:03.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.104+0000 7f7204ff9700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f71f0076b90 0x7f71f0079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.104+0000 7f7204ff9700 1 -- 172.21.15.110:0/1880603504 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f71fc097e30 con 0x7f72080fce60 2024-04-25T15:55:03.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.104+0000 7f72077fe700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f71f0076b90 0x7f71f0079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.104+0000 7f72077fe700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f71f0076b90 0x7f71f0079040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009b30 tx=0x7f71f8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:03.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.109+0000 7f7204ff9700 1 -- 172.21.15.110:0/1880603504 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f71fc060a60 con 0x7f72080fce60 2024-04-25T15:55:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:55:03.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:03.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:03.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.327+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f720810f7a0 con 0x7f71f0076b90 2024-04-25T15:55:03.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.328+0000 7f7204ff9700 1 -- 172.21.15.110:0/1880603504 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f720810f7a0 con 0x7f71f0076b90 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f71f0076b90 msgr2=0x7f71f0079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f71f0076b90 0x7f71f0079040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009b30 tx=0x7f71f8009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fce60 msgr2=0x7f720810d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fce60 0x7f720810d970 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00eb40 tx=0x7f71fc00b040 comp rx=0 tx=0).stop 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 shutdown_connections 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f71f0076b90 0x7f71f0079040 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72080fce60 0x7f720810d970 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 --2- 172.21.15.110:0/1880603504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72080fdfd0 0x7f720810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.330+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 >> 172.21.15.110:0/1880603504 conn(0x7f72080f8540 msgr2=0x7f72080f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.331+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 shutdown_connections 2024-04-25T15:55:03.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.331+0000 7f720e319700 1 -- 172.21.15.110:0/1880603504 wait complete. 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.431+0000 7fc04a59f700 1 -- 172.21.15.110:0/3336290061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 msgr2=0x7fc0440fbf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.431+0000 7fc04a59f700 1 --2- 172.21.15.110:0/3336290061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 0x7fc0440fbf00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc040004940 tx=0x7fc040009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.431+0000 7fc04a59f700 1 -- 172.21.15.110:0/3336290061 shutdown_connections 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.431+0000 7fc04a59f700 1 --2- 172.21.15.110:0/3336290061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0440fcda0 0x7fc0440fd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.431+0000 7fc04a59f700 1 --2- 172.21.15.110:0/3336290061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 0x7fc0440fbf00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.431+0000 7fc04a59f700 1 -- 172.21.15.110:0/3336290061 >> 172.21.15.110:0/3336290061 conn(0x7fc0440f7250 msgr2=0x7fc0440f9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.431+0000 7fc04a59f700 1 -- 172.21.15.110:0/3336290061 shutdown_connections 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.431+0000 7fc04a59f700 1 -- 172.21.15.110:0/3336290061 wait complete. 2024-04-25T15:55:03.434 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04a59f700 1 Processor -- start 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04a59f700 1 -- start start 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04a59f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 0x7fc04410c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04a59f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0440fcda0 0x7fc04410cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04a59f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04410d1f0 con 0x7fc0440fbaf0 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04a59f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc04410d9d0 con 0x7fc0440fcda0 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04959d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 0x7fc04410c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04959d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 0x7fc04410c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56176/0 (socket says 172.21.15.110:56176) 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04959d700 1 -- 172.21.15.110:0/309049167 learned_addr learned my addr 172.21.15.110:0/309049167 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc048d9c700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0440fcda0 0x7fc04410cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04959d700 1 -- 172.21.15.110:0/309049167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0440fcda0 msgr2=0x7fc04410cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04959d700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0440fcda0 0x7fc04410cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04959d700 1 -- 172.21.15.110:0/309049167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc040005040 con 0x7fc0440fbaf0 2024-04-25T15:55:03.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc04959d700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 0x7fc04410c6b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc040013870 tx=0x7fc040009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:03.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.432+0000 7fc039ffb700 1 -- 172.21.15.110:0/309049167 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc040021070 con 0x7fc0440fbaf0 2024-04-25T15:55:03.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.433+0000 7fc039ffb700 1 -- 172.21.15.110:0/309049167 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc040009430 con 0x7fc0440fbaf0 2024-04-25T15:55:03.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.433+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc04410dc50 con 0x7fc0440fbaf0 2024-04-25T15:55:03.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.433+0000 7fc039ffb700 1 -- 172.21.15.110:0/309049167 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc04000e070 con 0x7fc0440fbaf0 2024-04-25T15:55:03.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.433+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc04410e190 con 0x7fc0440fbaf0 2024-04-25T15:55:03.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.434+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc04405f5c0 con 0x7fc0440fbaf0 2024-04-25T15:55:03.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.435+0000 7fc039ffb700 1 -- 172.21.15.110:0/309049167 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fc04001c750 con 0x7fc0440fbaf0 2024-04-25T15:55:03.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.435+0000 7fc039ffb700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc030076b10 0x7fc030078fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.435+0000 7fc039ffb700 1 -- 172.21.15.110:0/309049167 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fc0400994b0 con 0x7fc0440fbaf0 2024-04-25T15:55:03.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.435+0000 7fc048d9c700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc030076b10 0x7fc030078fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.437 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.435+0000 7fc048d9c700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc030076b10 0x7fc030078fc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc034009b80 tx=0x7fc03400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:03.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.440+0000 7fc039ffb700 1 -- 172.21.15.110:0/309049167 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc0400620b0 con 0x7fc0440fbaf0 2024-04-25T15:55:03.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.649+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc04410e4c0 con 0x7fc030076b10 2024-04-25T15:55:03.659 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.659+0000 7fc039ffb700 1 -- 172.21.15.110:0/309049167 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fc04410e4c0 con 0x7fc030076b10 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (5m) 36s ago 13m 25.2M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (2m) 36s ago 14m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (2m) 2s ago 13m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (5m) 36s ago 13m 85.4M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (8m) 36s ago 8m 19.9M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (8m) 36s ago 8m 150M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (8m) 2s ago 8m 14.7M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (8m) 2s ago 8m 17.2M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (4m) 36s ago 15m 570M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (3m) 2s ago 13m 518M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (3m) 36s ago 15m 114M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:55:03.660 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (3m) 2s ago 13m 103M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (5m) 36s ago 14m 15.7M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (4m) 2s ago 13m 15.3M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (114s) 36s ago 12m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (79s) 36s ago 11m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (42s) 36s ago 11m 82.3M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (6s) 2s ago 10m 79.8M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (10m) 2s ago 10m 208M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 46158a874415 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (10m) 2s ago 10m 191M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:55:03.661 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (3m) 36s ago 13m 54.6M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc030076b10 msgr2=0x7fc030078fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc030076b10 0x7fc030078fc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc034009b80 tx=0x7fc03400b040 comp rx=0 tx=0).stop 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 msgr2=0x7fc04410c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 0x7fc04410c6b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc040013870 tx=0x7fc040009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 shutdown_connections 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc030076b10 0x7fc030078fc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc0440fbaf0 0x7fc04410c6b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 --2- 172.21.15.110:0/309049167 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0440fcda0 0x7fc04410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 >> 172.21.15.110:0/309049167 conn(0x7fc0440f7250 msgr2=0x7fc0440f8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 shutdown_connections 2024-04-25T15:55:03.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.661+0000 7fc04a59f700 1 -- 172.21.15.110:0/309049167 wait complete. 2024-04-25T15:55:03.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.761+0000 7fb5975d9700 1 -- 172.21.15.110:0/4076952226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 msgr2=0x7fb5900fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.761+0000 7fb5975d9700 1 --2- 172.21.15.110:0/4076952226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 0x7fb5900fe440 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb580004940 tx=0x7fb580009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:03.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.761+0000 7fb5975d9700 1 -- 172.21.15.110:0/4076952226 shutdown_connections 2024-04-25T15:55:03.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.761+0000 7fb5975d9700 1 --2- 172.21.15.110:0/4076952226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 0x7fb5900fe440 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.761+0000 7fb5975d9700 1 --2- 172.21.15.110:0/4076952226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5900fce60 0x7fb5900fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.767 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.761+0000 7fb5975d9700 1 -- 172.21.15.110:0/4076952226 >> 172.21.15.110:0/4076952226 conn(0x7fb5900f8540 msgr2=0x7fb5900fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.761+0000 7fb5975d9700 1 -- 172.21.15.110:0/4076952226 shutdown_connections 2024-04-25T15:55:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.761+0000 7fb5975d9700 1 -- 172.21.15.110:0/4076952226 wait complete. 2024-04-25T15:55:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb5975d9700 1 Processor -- start 2024-04-25T15:55:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb5975d9700 1 -- start start 2024-04-25T15:55:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb5975d9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5900fce60 0x7fb59010d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb5975d9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 0x7fb59010de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb5975d9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59010e400 con 0x7fb5900fdfd0 2024-04-25T15:55:03.768 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb5975d9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59010e570 con 0x7fb5900fce60 2024-04-25T15:55:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb594b74700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 0x7fb59010de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb594b74700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 0x7fb59010de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56188/0 (socket says 172.21.15.110:56188) 2024-04-25T15:55:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb594b74700 1 -- 172.21.15.110:0/500844850 learned_addr learned my addr 172.21.15.110:0/500844850 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb595375700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5900fce60 0x7fb59010d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb594b74700 1 -- 172.21.15.110:0/500844850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5900fce60 msgr2=0x7fb59010d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb594b74700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5900fce60 0x7fb59010d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:03.769 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb594b74700 1 -- 172.21.15.110:0/500844850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb580005040 con 0x7fb5900fdfd0 2024-04-25T15:55:03.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.762+0000 7fb594b74700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 0x7fb59010de00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb580013870 tx=0x7fb580009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:03.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.763+0000 7fb585ffb700 1 -- 172.21.15.110:0/500844850 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb580021070 con 0x7fb5900fdfd0 2024-04-25T15:55:03.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.763+0000 7fb585ffb700 1 -- 172.21.15.110:0/500844850 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb580009430 con 0x7fb5900fdfd0 2024-04-25T15:55:03.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.763+0000 7fb585ffb700 1 -- 172.21.15.110:0/500844850 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58000e070 con 0x7fb5900fdfd0 2024-04-25T15:55:03.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.763+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb590112f70 con 0x7fb5900fdfd0 2024-04-25T15:55:03.770 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.763+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb590113480 con 0x7fb5900fdfd0 2024-04-25T15:55:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.764+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59004c710 con 0x7fb5900fdfd0 2024-04-25T15:55:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.765+0000 7fb585ffb700 1 -- 172.21.15.110:0/500844850 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fb58001c750 con 0x7fb5900fdfd0 2024-04-25T15:55:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.765+0000 7fb585ffb700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb57c076b20 0x7fb57c078fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.765+0000 7fb585ffb700 1 -- 172.21.15.110:0/500844850 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fb580099450 con 0x7fb5900fdfd0 2024-04-25T15:55:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.765+0000 7fb595375700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb57c076b20 0x7fb57c078fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:03.771 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.765+0000 7fb595375700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb57c076b20 0x7fb57c078fd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009b80 tx=0x7fb58c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:03.772 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:03.770+0000 7fb585ffb700 1 -- 172.21.15.110:0/500844850 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb580062050 con 0x7fb5900fdfd0 2024-04-25T15:55:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:03 smithi193 ceph-mon[80071]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:55:03.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:03 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded (PG_DEGRADED) 2024-04-25T15:55:04.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.051+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb5901058c0 con 0x7fb5900fdfd0 2024-04-25T15:55:04.053 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:03 smithi110 ceph-mon[90991]: pgmap v118: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:55:04.053 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:03 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded (PG_DEGRADED) 2024-04-25T15:55:04.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.052+0000 7fb585ffb700 1 -- 172.21.15.110:0/500844850 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fb5800619f0 con 0x7fb5900fdfd0 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 2, 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 3 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:55:04.054 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:55:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:55:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:55:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:55:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 6, 2024-04-25T15:55:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 7 2024-04-25T15:55:04.055 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:55:04.055 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:55:04.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb57c076b20 msgr2=0x7fb57c078fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb57c076b20 0x7fb57c078fd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb58c009b80 tx=0x7fb58c00b040 comp rx=0 tx=0).stop 2024-04-25T15:55:04.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 msgr2=0x7fb59010de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 0x7fb59010de00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb580013870 tx=0x7fb580009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:04.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 shutdown_connections 2024-04-25T15:55:04.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5900fce60 0x7fb59010d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb57c076b20 0x7fb57c078fd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 --2- 172.21.15.110:0/500844850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb5900fdfd0 0x7fb59010de00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 >> 172.21.15.110:0/500844850 conn(0x7fb5900f8540 msgr2=0x7fb5900f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:04.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 shutdown_connections 2024-04-25T15:55:04.057 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.055+0000 7fb5975d9700 1 -- 172.21.15.110:0/500844850 wait complete. 2024-04-25T15:55:04.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.155+0000 7f0c0a308700 1 -- 172.21.15.110:0/3197328958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c04066fb0 msgr2=0x7f0c040ff9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 --2- 172.21.15.110:0/3197328958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c04066fb0 0x7f0c040ff9b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4004940 tx=0x7f0bf4009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 -- 172.21.15.110:0/3197328958 shutdown_connections 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 --2- 172.21.15.110:0/3197328958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c040ffef0 0x7f0c04102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 --2- 172.21.15.110:0/3197328958 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c04066fb0 0x7f0c040ff9b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 -- 172.21.15.110:0/3197328958 >> 172.21.15.110:0/3197328958 conn(0x7f0c040f7420 msgr2=0x7f0c040f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 -- 172.21.15.110:0/3197328958 shutdown_connections 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 -- 172.21.15.110:0/3197328958 wait complete. 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 Processor -- start 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 -- start start 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c04066fb0 0x7f0c040fd200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c040ffef0 0x7f0c040fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040fd800 con 0x7f0c040ffef0 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.156+0000 7f0c0a308700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c040fbdf0 con 0x7f0c04066fb0 2024-04-25T15:55:04.159 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c037fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c040ffef0 0x7f0c040fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c037fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c040ffef0 0x7f0c040fb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56200/0 (socket says 172.21.15.110:56200) 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c037fe700 1 -- 172.21.15.110:0/2557975194 learned_addr learned my addr 172.21.15.110:0/2557975194 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c03fff700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c04066fb0 0x7f0c040fd200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c037fe700 1 -- 172.21.15.110:0/2557975194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c04066fb0 msgr2=0x7f0c040fd200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c037fe700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c04066fb0 0x7f0c040fd200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c037fe700 1 -- 172.21.15.110:0/2557975194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4005040 con 0x7f0c040ffef0 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c037fe700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c040ffef0 0x7f0c040fb8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800eb40 tx=0x7f0bf800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c00ff9700 1 -- 172.21.15.110:0/2557975194 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf8007cc0 con 0x7f0c040ffef0 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c040fc040 con 0x7f0c040ffef0 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c00ff9700 1 -- 172.21.15.110:0/2557975194 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0bf800f040 con 0x7f0c040ffef0 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c00ff9700 1 -- 172.21.15.110:0/2557975194 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bf800ca10 con 0x7f0c040ffef0 2024-04-25T15:55:04.160 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.157+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c040fc580 con 0x7f0c040ffef0 2024-04-25T15:55:04.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.158+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0404c710 con 0x7f0c040ffef0 2024-04-25T15:55:04.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.159+0000 7f0c00ff9700 1 -- 172.21.15.110:0/2557975194 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f0bf800cc80 con 0x7f0c040ffef0 2024-04-25T15:55:04.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.159+0000 7f0c00ff9700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0bec076b90 0x7f0bec079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.160+0000 7f0c03fff700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0bec076b90 0x7f0bec079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.161 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.160+0000 7f0c00ff9700 1 -- 172.21.15.110:0/2557975194 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f0bf8097ea0 con 0x7f0c040ffef0 2024-04-25T15:55:04.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.160+0000 7f0c03fff700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0bec076b90 0x7f0bec079040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4004890 tx=0x7f0bf4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:04.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.164+0000 7f0c00ff9700 1 -- 172.21.15.110:0/2557975194 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0bf8060ad0 con 0x7f0c040ffef0 2024-04-25T15:55:04.411 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.410+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0c041058b0 con 0x7f0c040ffef0 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.410+0000 7f0c00ff9700 1 -- 172.21.15.110:0/2557975194 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f0bf8060470 con 0x7f0c040ffef0 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:55:04.412 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:55:04.413 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:55:04.414 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:55:04.415 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:55:04.415 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:55:04.415 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:55:04.415 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:55:04.415 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0bec076b90 msgr2=0x7f0bec079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0bec076b90 0x7f0bec079040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4004890 tx=0x7f0bf4009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c040ffef0 msgr2=0x7f0c040fb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c040ffef0 0x7f0c040fb8b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800eb40 tx=0x7f0bf800b040 comp rx=0 tx=0).stop 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 shutdown_connections 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c04066fb0 0x7f0c040fd200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0bec076b90 0x7f0bec079040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 --2- 172.21.15.110:0/2557975194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c040ffef0 0x7f0c040fb8b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 >> 172.21.15.110:0/2557975194 conn(0x7f0c040f7420 msgr2=0x7f0c040f7840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 shutdown_connections 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.413+0000 7f0c0a308700 1 -- 172.21.15.110:0/2557975194 wait complete. 2024-04-25T15:55:04.416 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:55:04.489 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:55:04 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[88999]: 2024-04-25T15:55:04.068+0000 7fc28921b7c0 -1 osd.3 64 log_to_monitors true 2024-04-25T15:55:04.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.515+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3894762613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28070830 msgr2=0x7f9a28070ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.515+0000 7f9a2ddfc700 1 --2- 172.21.15.110:0/3894762613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28070830 0x7f9a28070ca0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c004940 tx=0x7f9a1c009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:04.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.515+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3894762613 shutdown_connections 2024-04-25T15:55:04.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.515+0000 7f9a2ddfc700 1 --2- 172.21.15.110:0/3894762613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28070830 0x7f9a28070ca0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.515+0000 7f9a2ddfc700 1 --2- 172.21.15.110:0/3894762613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a28071de0 0x7f9a280702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.515+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3894762613 >> 172.21.15.110:0/3894762613 conn(0x7f9a280f8540 msgr2=0x7f9a280fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:04.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.515+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3894762613 shutdown_connections 2024-04-25T15:55:04.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.515+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3894762613 wait complete. 2024-04-25T15:55:04.517 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a2ddfc700 1 Processor -- start 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a2ddfc700 1 -- start start 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a2ddfc700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a28070830 0x7f9a2810fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a2ddfc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28071de0 0x7f9a28110050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a2ddfc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a28110650 con 0x7f9a28071de0 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a2ddfc700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a281107c0 con 0x7f9a28070830 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28071de0 0x7f9a28110050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a26ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28071de0 0x7f9a28110050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56220/0 (socket says 172.21.15.110:56220) 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a26ffd700 1 -- 172.21.15.110:0/3725786795 learned_addr learned my addr 172.21.15.110:0/3725786795 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a277fe700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a28070830 0x7f9a2810fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a26ffd700 1 -- 172.21.15.110:0/3725786795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a28070830 msgr2=0x7f9a2810fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.519 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a26ffd700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a28070830 0x7f9a2810fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.516+0000 7f9a26ffd700 1 -- 172.21.15.110:0/3725786795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a1c005040 con 0x7f9a28071de0 2024-04-25T15:55:04.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.517+0000 7f9a26ffd700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28071de0 0x7f9a28110050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b3b0 tx=0x7f9a1c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:04.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.517+0000 7f9a0ffff700 1 -- 172.21.15.110:0/3725786795 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a1c00e040 con 0x7f9a28071de0 2024-04-25T15:55:04.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.517+0000 7f9a0ffff700 1 -- 172.21.15.110:0/3725786795 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a1c00bbf0 con 0x7f9a28071de0 2024-04-25T15:55:04.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.517+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a281151c0 con 0x7f9a28071de0 2024-04-25T15:55:04.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.517+0000 7f9a0ffff700 1 -- 172.21.15.110:0/3725786795 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a1c01c630 con 0x7f9a28071de0 2024-04-25T15:55:04.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.517+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a28115780 con 0x7f9a28071de0 2024-04-25T15:55:04.520 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.518+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a2804c710 con 0x7f9a28071de0 2024-04-25T15:55:04.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.519+0000 7f9a0ffff700 1 -- 172.21.15.110:0/3725786795 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f9a1c00bd60 con 0x7f9a28071de0 2024-04-25T15:55:04.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.519+0000 7f9a0ffff700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9a10076b90 0x7f9a10079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.519+0000 7f9a0ffff700 1 -- 172.21.15.110:0/3725786795 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9a1c098f30 con 0x7f9a28071de0 2024-04-25T15:55:04.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.519+0000 7f9a277fe700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9a10076b90 0x7f9a10079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.520+0000 7f9a277fe700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9a10076b90 0x7f9a10079040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a28071b20 tx=0x7f9a1800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:04.525 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.524+0000 7f9a0ffff700 1 -- 172.21.15.110:0/3725786795 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9a1c061ad0 con 0x7f9a28071de0 2024-04-25T15:55:04.743 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.741+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a28103eb0 con 0x7f9a10076b90 2024-04-25T15:55:04.744 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.743+0000 7f9a0ffff700 1 -- 172.21.15.110:0/3725786795 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f9a28103eb0 con 0x7f9a10076b90 2024-04-25T15:55:04.744 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:55:04.744 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:55:04.744 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:55:04.744 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:55:04.744 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:55:04.744 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:55:04.744 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:55:04.745 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:55:04.745 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:55:04.745 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "10/21 daemons upgraded", 2024-04-25T15:55:04.745 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T15:55:04.745 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:55:04.745 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:55:04.746 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.745+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9a10076b90 msgr2=0x7f9a10079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.745+0000 7f9a2ddfc700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9a10076b90 0x7f9a10079040 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9a28071b20 tx=0x7f9a1800b040 comp rx=0 tx=0).stop 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.745+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28071de0 msgr2=0x7f9a28110050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.745+0000 7f9a2ddfc700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28071de0 0x7f9a28110050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c00b3b0 tx=0x7f9a1c009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.745+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 shutdown_connections 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.746+0000 7f9a2ddfc700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a28070830 0x7f9a2810fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.746+0000 7f9a2ddfc700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9a10076b90 0x7f9a10079040 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.746+0000 7f9a2ddfc700 1 --2- 172.21.15.110:0/3725786795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9a28071de0 0x7f9a28110050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.746+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 >> 172.21.15.110:0/3725786795 conn(0x7f9a280f8540 msgr2=0x7f9a280f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.746+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 shutdown_connections 2024-04-25T15:55:04.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.746+0000 7f9a2ddfc700 1 -- 172.21.15.110:0/3725786795 wait complete. 2024-04-25T15:55:04.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.849+0000 7fd9d97bb700 1 -- 172.21.15.110:0/1078777129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 msgr2=0x7fd9d4102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.849+0000 7fd9d97bb700 1 --2- 172.21.15.110:0/1078777129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 0x7fd9d4102300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8004940 tx=0x7fd9c8009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.849+0000 7fd9d97bb700 1 -- 172.21.15.110:0/1078777129 shutdown_connections 2024-04-25T15:55:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.849+0000 7fd9d97bb700 1 --2- 172.21.15.110:0/1078777129 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 0x7fd9d4102300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.849+0000 7fd9d97bb700 1 --2- 172.21.15.110:0/1078777129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9d40662c0 0x7fd9d40ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.849+0000 7fd9d97bb700 1 -- 172.21.15.110:0/1078777129 >> 172.21.15.110:0/1078777129 conn(0x7fd9d40f7430 msgr2=0x7fd9d40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.849+0000 7fd9d97bb700 1 -- 172.21.15.110:0/1078777129 shutdown_connections 2024-04-25T15:55:04.851 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.849+0000 7fd9d97bb700 1 -- 172.21.15.110:0/1078777129 wait complete. 2024-04-25T15:55:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d97bb700 1 Processor -- start 2024-04-25T15:55:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d97bb700 1 -- start start 2024-04-25T15:55:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d97bb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9d40662c0 0x7fd9d410d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d97bb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 0x7fd9d410df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d97bb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d410e510 con 0x7fd9d40ffee0 2024-04-25T15:55:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d97bb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d410ed10 con 0x7fd9d40662c0 2024-04-25T15:55:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d27fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 0x7fd9d410df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d27fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 0x7fd9d410df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56228/0 (socket says 172.21.15.110:56228) 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.850+0000 7fd9d27fc700 1 -- 172.21.15.110:0/3163434394 learned_addr learned my addr 172.21.15.110:0/3163434394 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9d2ffd700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9d40662c0 0x7fd9d410d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9d27fc700 1 -- 172.21.15.110:0/3163434394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9d40662c0 msgr2=0x7fd9d410d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9d27fc700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9d40662c0 0x7fd9d410d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9d27fc700 1 -- 172.21.15.110:0/3163434394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c8005040 con 0x7fd9d40ffee0 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9d27fc700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 0x7fd9d410df10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800b3b0 tx=0x7fd9c8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9bb7fe700 1 -- 172.21.15.110:0/3163434394 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9c800e040 con 0x7fd9d40ffee0 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9bb7fe700 1 -- 172.21.15.110:0/3163434394 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9c8004290 con 0x7fd9d40ffee0 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d410ef90 con 0x7fd9d40ffee0 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9bb7fe700 1 -- 172.21.15.110:0/3163434394 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9c801c3f0 con 0x7fd9d40ffee0 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.851+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d410f4d0 con 0x7fd9d40ffee0 2024-04-25T15:55:04.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.852+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d405f5c0 con 0x7fd9d40ffee0 2024-04-25T15:55:04.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.853+0000 7fd9bb7fe700 1 -- 172.21.15.110:0/3163434394 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fd9c800bbf0 con 0x7fd9d40ffee0 2024-04-25T15:55:04.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.853+0000 7fd9bb7fe700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd9bc076b80 0x7fd9bc079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:04.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.853+0000 7fd9bb7fe700 1 -- 172.21.15.110:0/3163434394 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fd9c80995e0 con 0x7fd9d40ffee0 2024-04-25T15:55:04.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.853+0000 7fd9d2ffd700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd9bc076b80 0x7fd9bc079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:04.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.854+0000 7fd9d2ffd700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd9bc076b80 0x7fd9bc079030 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:04.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:04.858+0000 7fd9bb7fe700 1 -- 172.21.15.110:0/3163434394 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd9c80621e0 con 0x7fd9d40ffee0 2024-04-25T15:55:05.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:04 smithi110 ceph-mon[90991]: from='client.44209 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:05.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:04 smithi110 ceph-mon[90991]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:05.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:04 smithi110 ceph-mon[90991]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:05.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:04 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/500844850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:05.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:04 smithi110 ceph-mon[90991]: from='osd.3 [v2:172.21.15.193:6800/1773058821,v1:172.21.15.193:6801/1773058821]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T15:55:05.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:04 smithi110 ceph-mon[90991]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T15:55:05.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:04 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2557975194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:55:05.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.139+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd9d404bed0 con 0x7fd9d40ffee0 2024-04-25T15:55:05.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.140+0000 7fd9bb7fe700 1 -- 172.21.15.110:0/3163434394 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1461 (secure 0 0 0) 0x7fd9c8061b80 con 0x7fd9d40ffee0 2024-04-25T15:55:05.141 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-04-25T15:55:05.141 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:55:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:55:05.141 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-25T15:55:05.141 INFO:teuthology.orchestra.run.smithi110.stdout: osd.3 (root=default,host=smithi193) is down 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 60/267 objects degraded (22.472%), 18 pgs degraded 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.c is active+undersized+degraded, acting [2,0] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-04-25T15:55:05.142 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-04-25T15:55:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-04-25T15:55:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-04-25T15:55:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-04-25T15:55:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-04-25T15:55:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-04-25T15:55:05.143 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd9bc076b80 msgr2=0x7fd9bc079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd9bc076b80 0x7fd9bc079030 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c400b040 comp rx=0 tx=0).stop 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 msgr2=0x7fd9d410df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 0x7fd9d410df10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd9c800b3b0 tx=0x7fd9c8009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 shutdown_connections 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9d40662c0 0x7fd9d410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd9bc076b80 0x7fd9bc079030 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 --2- 172.21.15.110:0/3163434394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd9d40ffee0 0x7fd9d410df10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 >> 172.21.15.110:0/3163434394 conn(0x7fd9d40f7430 msgr2=0x7fd9d40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 shutdown_connections 2024-04-25T15:55:05.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:05.142+0000 7fd9d97bb700 1 -- 172.21.15.110:0/3163434394 wait complete. 2024-04-25T15:55:05.196 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:04 smithi193 ceph-mon[80071]: from='client.44209 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:05.196 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:04 smithi193 ceph-mon[80071]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:05.196 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:04 smithi193 ceph-mon[80071]: from='client.34320 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:05.197 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:04 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/500844850' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:05.197 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:04 smithi193 ceph-mon[80071]: from='osd.3 [v2:172.21.15.193:6800/1773058821,v1:172.21.15.193:6801/1773058821]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T15:55:05.197 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:04 smithi193 ceph-mon[80071]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T15:55:05.197 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:04 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2557975194' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='osd.3 [v2:172.21.15.193:6800/1773058821,v1:172.21.15.193:6801/1773058821]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: osdmap e67: 6 total, 5 up, 6 in 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3163434394' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:55:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-25T15:55:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:05 smithi110 ceph-mon[90991]: from='osd.3 ' entity='osd.3' 2024-04-25T15:55:06.239 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[88999]: 2024-04-25T15:55:05.766+0000 7fc27662d700 -1 osd.3 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: pgmap v119: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='osd.3 [v2:172.21.15.193:6800/1773058821,v1:172.21.15.193:6801/1773058821]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: osdmap e67: 6 total, 5 up, 6 in 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3163434394' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T15:55:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T15:55:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-25T15:55:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:05 smithi193 ceph-mon[80071]: from='osd.3 ' entity='osd.3' 2024-04-25T15:55:07.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:06 smithi110 ceph-mon[90991]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:55:07.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:06 smithi110 ceph-mon[90991]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:55:07.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:06 smithi110 ceph-mon[90991]: osd.3 [v2:172.21.15.193:6800/1773058821,v1:172.21.15.193:6801/1773058821] boot 2024-04-25T15:55:07.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:06 smithi110 ceph-mon[90991]: osdmap e68: 6 total, 6 up, 6 in 2024-04-25T15:55:07.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:06 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:55:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:06 smithi193 ceph-mon[80071]: pgmap v121: 65 pgs: 18 active+undersized+degraded, 20 active+undersized, 27 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 60/267 objects degraded (22.472%) 2024-04-25T15:55:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:06 smithi193 ceph-mon[80071]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:55:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:06 smithi193 ceph-mon[80071]: osd.3 [v2:172.21.15.193:6800/1773058821,v1:172.21.15.193:6801/1773058821] boot 2024-04-25T15:55:07.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:06 smithi193 ceph-mon[80071]: osdmap e68: 6 total, 6 up, 6 in 2024-04-25T15:55:07.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:06 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T15:55:08.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:08 smithi193 ceph-mon[80071]: osdmap e69: 6 total, 6 up, 6 in 2024-04-25T15:55:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:08 smithi110 ceph-mon[90991]: osdmap e69: 6 total, 6 up, 6 in 2024-04-25T15:55:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:09 smithi110 ceph-mon[90991]: pgmap v124: 65 pgs: 3 activating, 8 peering, 14 active+undersized+degraded, 13 active+undersized, 27 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-04-25T15:55:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:09 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 46/267 objects degraded (17.228%), 14 pgs degraded (PG_DEGRADED) 2024-04-25T15:55:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:09 smithi193 ceph-mon[80071]: pgmap v124: 65 pgs: 3 activating, 8 peering, 14 active+undersized+degraded, 13 active+undersized, 27 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46/267 objects degraded (17.228%) 2024-04-25T15:55:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:09 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 46/267 objects degraded (17.228%), 14 pgs degraded (PG_DEGRADED) 2024-04-25T15:55:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:11 smithi110 ceph-mon[90991]: pgmap v125: 65 pgs: 3 activating, 8 peering, 4 active+undersized+degraded, 3 active+undersized, 47 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 14/267 objects degraded (5.243%) 2024-04-25T15:55:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:11 smithi193 ceph-mon[80071]: pgmap v125: 65 pgs: 3 activating, 8 peering, 4 active+undersized+degraded, 3 active+undersized, 47 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 14/267 objects degraded (5.243%) 2024-04-25T15:55:12.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:12 smithi110 ceph-mon[90991]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/267 objects degraded (5.243%), 4 pgs degraded) 2024-04-25T15:55:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:12 smithi193 ceph-mon[80071]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/267 objects degraded (5.243%), 4 pgs degraded) 2024-04-25T15:55:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:13 smithi110 ceph-mon[90991]: pgmap v126: 65 pgs: 3 activating, 62 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:13 smithi193 ceph-mon[80071]: pgmap v126: 65 pgs: 3 activating, 62 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:15 smithi110 ceph-mon[90991]: pgmap v127: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:15 smithi193 ceph-mon[80071]: pgmap v127: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:17 smithi110 ceph-mon[90991]: pgmap v128: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:55:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:17 smithi193 ceph-mon[80071]: pgmap v128: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:55:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:19 smithi110 ceph-mon[90991]: pgmap v129: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:19 smithi193 ceph-mon[80071]: pgmap v129: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:21.086 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:20 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T15:55:21.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:20 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T15:55:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:21 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T15:55:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:21 smithi110 ceph-mon[90991]: Upgrade: osd.4 is safe to restart 2024-04-25T15:55:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:21 smithi110 ceph-mon[90991]: pgmap v130: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:21 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-25T15:55:22.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:21 smithi193 ceph-mon[80071]: Upgrade: osd.4 is safe to restart 2024-04-25T15:55:22.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:21 smithi193 ceph-mon[80071]: pgmap v130: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:22 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:22 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:22 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T15:55:22.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:22 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:55:22.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:22 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:22.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:22 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:22.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:22 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T15:55:22.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:22 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:55:23.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:23 smithi193 ceph-mon[80071]: Upgrade: Updating osd.4 2024-04-25T15:55:23.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:23 smithi193 ceph-mon[80071]: Deploying daemon osd.4 on smithi193 2024-04-25T15:55:23.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:23 smithi193 ceph-mon[80071]: pgmap v131: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:24.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:23 smithi110 ceph-mon[90991]: Upgrade: Updating osd.4 2024-04-25T15:55:24.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:23 smithi110 ceph-mon[90991]: Deploying daemon osd.4 on smithi193 2024-04-25T15:55:24.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:23 smithi110 ceph-mon[90991]: pgmap v131: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:25.432 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:25 smithi193 systemd[1]: Stopping Ceph osd.4 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:55:25.740 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:25 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[46086]: 2024-04-25T15:55:25.430+0000 7f78b20e0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:55:25.740 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:25 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[46086]: 2024-04-25T15:55:25.430+0000 7f78b20e0700 -1 osd.4 69 *** Got signal Terminated *** 2024-04-25T15:55:25.740 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:25 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[46086]: 2024-04-25T15:55:25.430+0000 7f78b20e0700 -1 osd.4 69 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:55:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:25 smithi110 ceph-mon[90991]: pgmap v132: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:25 smithi110 ceph-mon[90991]: osd.4 marked itself down and dead 2024-04-25T15:55:26.233 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:25 smithi193 ceph-mon[80071]: pgmap v132: 65 pgs: 65 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:26.233 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:25 smithi193 ceph-mon[80071]: osd.4 marked itself down and dead 2024-04-25T15:55:26.489 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:26 smithi193 podman[93539]: 2024-04-25 15:55:26.233305799 +0000 UTC m=+0.910333867 container died 46158a874415c73380e777320aaa0d025908015145b43adb5a17c92569401089 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=quincy-c61472c) 2024-04-25T15:55:26.810 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:26 smithi193 podman[93539]: 2024-04-25 15:55:26.588707144 +0000 UTC m=+1.265735208 container cleanup 46158a874415c73380e777320aaa0d025908015145b43adb5a17c92569401089 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=quincy-c61472c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T15:55:26.810 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:26 smithi193 bash[93539]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4 2024-04-25T15:55:27.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:26 smithi110 ceph-mon[90991]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:55:27.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:26 smithi110 ceph-mon[90991]: osdmap e70: 6 total, 5 up, 6 in 2024-04-25T15:55:27.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:26 smithi110 ceph-mon[90991]: osdmap e71: 6 total, 5 up, 6 in 2024-04-25T15:55:27.202 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:26 smithi193 ceph-mon[80071]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:55:27.202 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:26 smithi193 ceph-mon[80071]: osdmap e70: 6 total, 5 up, 6 in 2024-04-25T15:55:27.202 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:26 smithi193 ceph-mon[80071]: osdmap e71: 6 total, 5 up, 6 in 2024-04-25T15:55:27.203 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:26 smithi193 podman[93557]: 2024-04-25 15:55:26.881663897 +0000 UTC m=+0.645842111 container remove 46158a874415c73380e777320aaa0d025908015145b43adb5a17c92569401089 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=quincy-c61472c, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T15:55:27.485 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:27 smithi193 podman[93652]: 2024-04-25 15:55:27.105378441 +0000 UTC m=+0.017564589 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:55:27.485 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:27 smithi193 podman[93652]: 2024-04-25 15:55:27.207873498 +0000 UTC m=+0.120059646 container create 0d2e4020a8297e627edde7dfbcb145b6e07c97acfcf7fa13bc01bbbbcbfa62b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T15:55:27.485 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:27 smithi193 podman[93652]: 2024-04-25 15:55:27.402092507 +0000 UTC m=+0.314278656 container init 0d2e4020a8297e627edde7dfbcb145b6e07c97acfcf7fa13bc01bbbbcbfa62b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T15:55:27.486 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:27 smithi193 podman[93652]: 2024-04-25 15:55:27.405227947 +0000 UTC m=+0.317414095 container start 0d2e4020a8297e627edde7dfbcb145b6e07c97acfcf7fa13bc01bbbbcbfa62b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T15:55:27.486 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:27 smithi193 podman[93652]: 2024-04-25 15:55:27.485140166 +0000 UTC m=+0.397326340 container attach 0d2e4020a8297e627edde7dfbcb145b6e07c97acfcf7fa13bc01bbbbcbfa62b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T15:55:27.739 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:27 smithi193 podman[93652]: 2024-04-25 15:55:27.695526843 +0000 UTC m=+0.607712992 container died 0d2e4020a8297e627edde7dfbcb145b6e07c97acfcf7fa13bc01bbbbcbfa62b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T15:55:28.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:27 smithi110 ceph-mon[90991]: pgmap v134: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:28.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:27 smithi110 ceph-mon[90991]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T15:55:28.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:27 smithi193 ceph-mon[80071]: pgmap v134: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:28.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:27 smithi193 ceph-mon[80071]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-25T15:55:28.515 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 podman[93677]: 2024-04-25 15:55:28.297020911 +0000 UTC m=+0.596147734 container remove 0d2e4020a8297e627edde7dfbcb145b6e07c97acfcf7fa13bc01bbbbcbfa62b4 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T15:55:28.515 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-25T15:55:28.515 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 systemd[1]: Stopped Ceph osd.4 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:55:28.515 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.4.service: Consumed 49.105s CPU time. 2024-04-25T15:55:28.848 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 systemd[1]: Starting Ceph osd.4 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:55:28.848 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 podman[93785]: 2024-04-25 15:55:28.635582974 +0000 UTC m=+0.017926578 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:55:28.848 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 podman[93785]: 2024-04-25 15:55:28.747771104 +0000 UTC m=+0.130114708 container create cac4c0e24d7e9ba9c491124853ece7bae316ae7be95f6264b909239c42e05410 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-25T15:55:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:28 smithi110 ceph-mon[90991]: pgmap v136: 65 pgs: 2 active+undersized+degraded, 8 peering, 3 stale+active+clean, 5 active+undersized, 47 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 8/267 objects degraded (2.996%) 2024-04-25T15:55:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:28 smithi110 ceph-mon[90991]: Health check failed: Degraded data redundancy: 8/267 objects degraded (2.996%), 2 pgs degraded (PG_DEGRADED) 2024-04-25T15:55:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:28 smithi193 ceph-mon[80071]: pgmap v136: 65 pgs: 2 active+undersized+degraded, 8 peering, 3 stale+active+clean, 5 active+undersized, 47 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 8/267 objects degraded (2.996%) 2024-04-25T15:55:29.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:28 smithi193 ceph-mon[80071]: Health check failed: Degraded data redundancy: 8/267 objects degraded (2.996%), 2 pgs degraded (PG_DEGRADED) 2024-04-25T15:55:29.240 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 podman[93785]: 2024-04-25 15:55:28.967178002 +0000 UTC m=+0.349521605 container init cac4c0e24d7e9ba9c491124853ece7bae316ae7be95f6264b909239c42e05410 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T15:55:29.240 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:28 smithi193 podman[93785]: 2024-04-25 15:55:28.970680168 +0000 UTC m=+0.353023775 container start cac4c0e24d7e9ba9c491124853ece7bae316ae7be95f6264b909239c42e05410 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T15:55:29.240 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 podman[93785]: 2024-04-25 15:55:29.050382531 +0000 UTC m=+0.432726147 container attach cac4c0e24d7e9ba9c491124853ece7bae316ae7be95f6264b909239c42e05410 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T15:55:29.879 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:55:29.879 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:55:29.879 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:55:29.879 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:55:29.879 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T15:55:29.879 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-25T15:55:29.879 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-25T15:55:29.879 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 bash[93785]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate[93800]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-25T15:55:29.880 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:29 smithi193 podman[93785]: 2024-04-25 15:55:29.584254171 +0000 UTC m=+0.966597780 container died cac4c0e24d7e9ba9c491124853ece7bae316ae7be95f6264b909239c42e05410 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T15:55:30.465 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:30 smithi193 podman[93931]: 2024-04-25 15:55:30.12903329 +0000 UTC m=+0.535763548 container remove cac4c0e24d7e9ba9c491124853ece7bae316ae7be95f6264b909239c42e05410 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-activate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T15:55:30.465 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:30 smithi193 podman[93994]: 2024-04-25 15:55:30.245730462 +0000 UTC m=+0.018416597 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:55:30.465 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:30 smithi193 podman[93994]: 2024-04-25 15:55:30.354847369 +0000 UTC m=+0.127533472 container create e67bdf50065ed84beb37e24b7b5c430c22f92ee68328b1f1e7f03c48a3fb3680 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-25T15:55:30.989 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:30 smithi193 podman[93994]: 2024-04-25 15:55:30.573723424 +0000 UTC m=+0.346409533 container init e67bdf50065ed84beb37e24b7b5c430c22f92ee68328b1f1e7f03c48a3fb3680 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:55:30.989 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:30 smithi193 podman[93994]: 2024-04-25 15:55:30.576965999 +0000 UTC m=+0.349652105 container start e67bdf50065ed84beb37e24b7b5c430c22f92ee68328b1f1e7f03c48a3fb3680 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-25T15:55:30.989 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:30 smithi193 ceph-osd[94013]: -- 172.21.15.193:0/1616315375 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e2b9003680 con 0x55e2b90aad00 2024-04-25T15:55:30.990 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:30 smithi193 bash[93994]: e67bdf50065ed84beb37e24b7b5c430c22f92ee68328b1f1e7f03c48a3fb3680 2024-04-25T15:55:30.990 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:30 smithi193 systemd[1]: Started Ceph osd.4 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:55:31.489 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:31 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T15:55:31.139+0000 7f147dffd7c0 -1 Falling back to public interface 2024-04-25T15:55:31.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:31.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:31.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:31 smithi193 ceph-mon[80071]: pgmap v137: 65 pgs: 11 active+undersized+degraded, 8 peering, 15 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40/267 objects degraded (14.981%) 2024-04-25T15:55:31.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:55:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:31 smithi110 ceph-mon[90991]: pgmap v137: 65 pgs: 11 active+undersized+degraded, 8 peering, 15 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40/267 objects degraded (14.981%) 2024-04-25T15:55:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:55:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:55:33.073 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:33.073 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:55:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:33 smithi110 ceph-mon[90991]: pgmap v138: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-25T15:55:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:33 smithi110 ceph-mon[90991]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T15:55:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:33 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:33 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:33 smithi193 ceph-mon[80071]: pgmap v138: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-25T15:55:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:33 smithi193 ceph-mon[80071]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-25T15:55:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:33 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:33 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 -- 172.21.15.110:0/796497017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fdfd0 msgr2=0x7feb6c0fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 --2- 172.21.15.110:0/796497017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fdfd0 0x7feb6c0fe440 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7feb5c004940 tx=0x7feb5c009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 -- 172.21.15.110:0/796497017 shutdown_connections 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 --2- 172.21.15.110:0/796497017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fdfd0 0x7feb6c0fe440 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 --2- 172.21.15.110:0/796497017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb6c0fce60 0x7feb6c0fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 -- 172.21.15.110:0/796497017 >> 172.21.15.110:0/796497017 conn(0x7feb6c0f8540 msgr2=0x7feb6c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 -- 172.21.15.110:0/796497017 shutdown_connections 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 -- 172.21.15.110:0/796497017 wait complete. 2024-04-25T15:55:35.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 Processor -- start 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 -- start start 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fce60 0x7feb6c10d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb6c0fdfd0 0x7feb6c10ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c10e3c0 con 0x7feb6c0fce60 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.259+0000 7feb747f9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb6c10e530 con 0x7feb6c0fdfd0 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb72595700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fce60 0x7feb6c10d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb72595700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fce60 0x7feb6c10d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58766/0 (socket says 172.21.15.110:58766) 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb72595700 1 -- 172.21.15.110:0/3535739955 learned_addr learned my addr 172.21.15.110:0/3535739955 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb71d94700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb6c0fdfd0 0x7feb6c10ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb72595700 1 -- 172.21.15.110:0/3535739955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb6c0fdfd0 msgr2=0x7feb6c10ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb72595700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb6c0fdfd0 0x7feb6c10ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb72595700 1 -- 172.21.15.110:0/3535739955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb5c005040 con 0x7feb6c0fce60 2024-04-25T15:55:35.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb72595700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fce60 0x7feb6c10d880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb6800eb40 tx=0x7feb6800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb62ffd700 1 -- 172.21.15.110:0/3535739955 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb68007cc0 con 0x7feb6c0fce60 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb62ffd700 1 -- 172.21.15.110:0/3535739955 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb6800f040 con 0x7feb6c0fce60 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb6c112f30 con 0x7feb6c0fce60 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb62ffd700 1 -- 172.21.15.110:0/3535739955 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb6800ca10 con 0x7feb6c0fce60 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.260+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb6c113460 con 0x7feb6c0fce60 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.261+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb6c04c710 con 0x7feb6c0fce60 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.263+0000 7feb62ffd700 1 -- 172.21.15.110:0/3535739955 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7feb6800cc80 con 0x7feb6c0fce60 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.263+0000 7feb62ffd700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7feb58076b90 0x7feb58079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.264 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.263+0000 7feb62ffd700 1 -- 172.21.15.110:0/3535739955 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6324+0+0 (secure 0 0 0) 0x7feb68098050 con 0x7feb6c0fce60 2024-04-25T15:55:35.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.263+0000 7feb71d94700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7feb58076b90 0x7feb58079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.265 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.264+0000 7feb71d94700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7feb58076b90 0x7feb58079040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00b0d0 tx=0x7feb5c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:35.268 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.267+0000 7feb62ffd700 1 -- 172.21.15.110:0/3535739955 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feb68060d00 con 0x7feb6c0fce60 2024-04-25T15:55:35.484 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.483+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb6c1022a0 con 0x7feb58076b90 2024-04-25T15:55:35.485 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.485+0000 7feb62ffd700 1 -- 172.21.15.110:0/3535739955 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7feb6c1022a0 con 0x7feb58076b90 2024-04-25T15:55:35.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7feb58076b90 msgr2=0x7feb58079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7feb58076b90 0x7feb58079040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00b0d0 tx=0x7feb5c009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:35.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fce60 msgr2=0x7feb6c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fce60 0x7feb6c10d880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb6800eb40 tx=0x7feb6800b040 comp rx=0 tx=0).stop 2024-04-25T15:55:35.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 shutdown_connections 2024-04-25T15:55:35.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7feb58076b90 0x7feb58079040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb6c0fce60 0x7feb6c10d880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 --2- 172.21.15.110:0/3535739955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb6c0fdfd0 0x7feb6c10ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 >> 172.21.15.110:0/3535739955 conn(0x7feb6c0f8540 msgr2=0x7feb6c0f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:35.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 shutdown_connections 2024-04-25T15:55:35.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.487+0000 7feb747f9700 1 -- 172.21.15.110:0/3535739955 wait complete. 2024-04-25T15:55:35.498 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:55:35.589 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.588+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/231091347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 msgr2=0x7fdb640fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.588+0000 7fdb6a1a1700 1 --2- 172.21.15.110:0/231091347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 0x7fdb640fe4f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdb58004940 tx=0x7fdb58009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:35.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.588+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/231091347 shutdown_connections 2024-04-25T15:55:35.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.588+0000 7fdb6a1a1700 1 --2- 172.21.15.110:0/231091347 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 0x7fdb640fe4f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.588+0000 7fdb6a1a1700 1 --2- 172.21.15.110:0/231091347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb640fce80 0x7fdb640fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.588+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/231091347 >> 172.21.15.110:0/231091347 conn(0x7fdb640f8590 msgr2=0x7fdb640fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:35.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.588+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/231091347 shutdown_connections 2024-04-25T15:55:35.590 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.588+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/231091347 wait complete. 2024-04-25T15:55:35.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.589+0000 7fdb6a1a1700 1 Processor -- start 2024-04-25T15:55:35.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.589+0000 7fdb6a1a1700 1 -- start start 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb6a1a1700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb640fce80 0x7fdb6410da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb6a1a1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 0x7fdb6410df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb6a1a1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6410e5e0 con 0x7fdb640fe080 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb6a1a1700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb6410edd0 con 0x7fdb640fce80 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb62ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 0x7fdb6410df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb637fe700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb640fce80 0x7fdb6410da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb62ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 0x7fdb6410df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58790/0 (socket says 172.21.15.110:58790) 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb62ffd700 1 -- 172.21.15.110:0/2470392195 learned_addr learned my addr 172.21.15.110:0/2470392195 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb62ffd700 1 -- 172.21.15.110:0/2470392195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb640fce80 msgr2=0x7fdb6410da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb62ffd700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb640fce80 0x7fdb6410da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb62ffd700 1 -- 172.21.15.110:0/2470392195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb58005040 con 0x7fdb640fe080 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb62ffd700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 0x7fdb6410df50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800b3b0 tx=0x7fdb58009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb4bfff700 1 -- 172.21.15.110:0/2470392195 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5800e040 con 0x7fdb640fe080 2024-04-25T15:55:35.592 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb4bfff700 1 -- 172.21.15.110:0/2470392195 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb5800bbf0 con 0x7fdb640fe080 2024-04-25T15:55:35.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb6410f050 con 0x7fdb640fe080 2024-04-25T15:55:35.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.590+0000 7fdb4bfff700 1 -- 172.21.15.110:0/2470392195 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb5801c630 con 0x7fdb640fe080 2024-04-25T15:55:35.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.591+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb6410f610 con 0x7fdb640fe080 2024-04-25T15:55:35.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.591+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb6405f5c0 con 0x7fdb640fe080 2024-04-25T15:55:35.593 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.592+0000 7fdb4bfff700 1 -- 172.21.15.110:0/2470392195 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fdb5800bd60 con 0x7fdb640fe080 2024-04-25T15:55:35.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.593+0000 7fdb4bfff700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb4c076b90 0x7fdb4c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.593+0000 7fdb637fe700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb4c076b90 0x7fdb4c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.593+0000 7fdb4bfff700 1 -- 172.21.15.110:0/2470392195 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fdb58099630 con 0x7fdb640fe080 2024-04-25T15:55:35.594 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.594+0000 7fdb637fe700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb4c076b90 0x7fdb4c079040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb640fdbf0 tx=0x7fdb540091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:35.598 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.597+0000 7fdb4bfff700 1 -- 172.21.15.110:0/2470392195 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdb58062210 con 0x7fdb640fe080 2024-04-25T15:55:35.822 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.821+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb6410f8d0 con 0x7fdb4c076b90 2024-04-25T15:55:35.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.822+0000 7fdb4bfff700 1 -- 172.21.15.110:0/2470392195 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fdb6410f8d0 con 0x7fdb4c076b90 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.824+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb4c076b90 msgr2=0x7fdb4c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb4c076b90 0x7fdb4c079040 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdb640fdbf0 tx=0x7fdb540091a0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 msgr2=0x7fdb6410df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 0x7fdb6410df50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800b3b0 tx=0x7fdb58009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 shutdown_connections 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb640fce80 0x7fdb6410da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb4c076b90 0x7fdb4c079040 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 --2- 172.21.15.110:0/2470392195 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb640fe080 0x7fdb6410df50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 >> 172.21.15.110:0/2470392195 conn(0x7fdb640f8590 msgr2=0x7fdb640f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:35.826 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 shutdown_connections 2024-04-25T15:55:35.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.825+0000 7fdb6a1a1700 1 -- 172.21.15.110:0/2470392195 wait complete. 2024-04-25T15:55:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:35 smithi110 ceph-mon[90991]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-25T15:55:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:35.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:35.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 -- 172.21.15.110:0/3422486334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 msgr2=0x7f02000fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 --2- 172.21.15.110:0/3422486334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 0x7f02000fd270 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f01fc004940 tx=0x7f01fc009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:35.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 -- 172.21.15.110:0/3422486334 shutdown_connections 2024-04-25T15:55:35.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 --2- 172.21.15.110:0/3422486334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02000fdfd0 0x7f02000fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.928 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 --2- 172.21.15.110:0/3422486334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 0x7f02000fd270 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 -- 172.21.15.110:0/3422486334 >> 172.21.15.110:0/3422486334 conn(0x7f02000f8540 msgr2=0x7f02000fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 -- 172.21.15.110:0/3422486334 shutdown_connections 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 -- 172.21.15.110:0/3422486334 wait complete. 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 Processor -- start 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 -- start start 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 0x7f020010d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02000fdfd0 0x7f020010ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020010e3f0 con 0x7f02000fce60 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.926+0000 7f02076c7700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020010e560 con 0x7f02000fdfd0 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f0205463700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 0x7f020010d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f0205463700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 0x7f020010d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58806/0 (socket says 172.21.15.110:58806) 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f0205463700 1 -- 172.21.15.110:0/3337436302 learned_addr learned my addr 172.21.15.110:0/3337436302 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f0204c62700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02000fdfd0 0x7f020010ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.929 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f0205463700 1 -- 172.21.15.110:0/3337436302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02000fdfd0 msgr2=0x7f020010ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:35.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f0205463700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02000fdfd0 0x7f020010ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:35.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f0205463700 1 -- 172.21.15.110:0/3337436302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc005040 con 0x7f02000fce60 2024-04-25T15:55:35.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f0205463700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 0x7f020010d8b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f01fc013870 tx=0x7f01fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:35.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f01f5ffb700 1 -- 172.21.15.110:0/3337436302 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01fc00e070 con 0x7f02000fce60 2024-04-25T15:55:35.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f01f5ffb700 1 -- 172.21.15.110:0/3337436302 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01fc013de0 con 0x7f02000fce60 2024-04-25T15:55:35.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0200112f60 con 0x7f02000fce60 2024-04-25T15:55:35.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f01f5ffb700 1 -- 172.21.15.110:0/3337436302 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01fc01cea0 con 0x7f02000fce60 2024-04-25T15:55:35.930 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.927+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02001133c0 con 0x7f02000fce60 2024-04-25T15:55:35.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.928+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020004c710 con 0x7f02000fce60 2024-04-25T15:55:35.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.929+0000 7f01f5ffb700 1 -- 172.21.15.110:0/3337436302 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f01fc01c690 con 0x7f02000fce60 2024-04-25T15:55:35.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.930+0000 7f01f5ffb700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f01ec076b80 0x7f01ec079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:35.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.930+0000 7f01f5ffb700 1 -- 172.21.15.110:0/3337436302 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f01fc09a620 con 0x7f02000fce60 2024-04-25T15:55:35.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.930+0000 7f0204c62700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f01ec076b80 0x7f01ec079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:35.931 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.930+0000 7f0204c62700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f01ec076b80 0x7f01ec079030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009b80 tx=0x7f01f000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:35.935 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:35.934+0000 7f01f5ffb700 1 -- 172.21.15.110:0/3337436302 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f01fc063200 con 0x7f02000fce60 2024-04-25T15:55:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:35 smithi193 ceph-mon[80071]: pgmap v139: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-25T15:55:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:35.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:36.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.141+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f02001022b0 con 0x7f01ec076b80 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.151+0000 7f01f5ffb700 1 -- 172.21.15.110:0/3337436302 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f02001022b0 con 0x7f01ec076b80 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (5m) 68s ago 14m 25.2M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (2m) 68s ago 14m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (2m) 2s ago 13m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (5m) 68s ago 14m 85.4M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (9m) 68s ago 9m 19.9M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (9m) 68s ago 9m 150M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (9m) 2s ago 9m 14.8M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (9m) 2s ago 9m 17.4M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (5m) 68s ago 15m 570M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:55:36.152 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (4m) 2s ago 13m 519M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (3m) 68s ago 16m 114M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (3m) 2s ago 13m 104M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (6m) 68s ago 14m 15.7M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (5m) 2s ago 13m 15.3M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (2m) 68s ago 12m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (111s) 68s ago 12m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (74s) 68s ago 11m 82.3M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (38s) 2s ago 11m 161M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (5s) 2s ago 11m 82.4M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (10m) 2s ago 10m 190M 2581M 17.2.7-902-gc61472c8 0de407c1b3af 988e985be7eb 2024-04-25T15:55:36.153 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (4m) 68s ago 14m 54.6M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:55:36.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f01ec076b80 msgr2=0x7f01ec079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f01ec076b80 0x7f01ec079030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f01f0009b80 tx=0x7f01f000b040 comp rx=0 tx=0).stop 2024-04-25T15:55:36.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 msgr2=0x7f020010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 0x7f020010d8b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f01fc013870 tx=0x7f01fc009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:36.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 shutdown_connections 2024-04-25T15:55:36.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f01ec076b80 0x7f01ec079030 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f02000fce60 0x7f020010d8b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 --2- 172.21.15.110:0/3337436302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02000fdfd0 0x7f020010ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 >> 172.21.15.110:0/3337436302 conn(0x7f02000f8540 msgr2=0x7f02000f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:36.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 shutdown_connections 2024-04-25T15:55:36.155 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.153+0000 7f02076c7700 1 -- 172.21.15.110:0/3337436302 wait complete. 2024-04-25T15:55:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 -- 172.21.15.110:0/4117909275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fcf70 msgr2=0x7fabf00fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 --2- 172.21.15.110:0/4117909275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fcf70 0x7fabf00fd3e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fabd8004940 tx=0x7fabd8009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 -- 172.21.15.110:0/4117909275 shutdown_connections 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 --2- 172.21.15.110:0/4117909275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fcf70 0x7fabf00fd3e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 --2- 172.21.15.110:0/4117909275 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf00fbcc0 0x7fabf00fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 -- 172.21.15.110:0/4117909275 >> 172.21.15.110:0/4117909275 conn(0x7fabf00f7440 msgr2=0x7fabf00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 -- 172.21.15.110:0/4117909275 shutdown_connections 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 -- 172.21.15.110:0/4117909275 wait complete. 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.253+0000 7fabf57ce700 1 Processor -- start 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabf57ce700 1 -- start start 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabf57ce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fbcc0 0x7fabf010c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabf57ce700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf00fcf70 0x7fabf010cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabf57ce700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf010d340 con 0x7fabf00fbcc0 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabf57ce700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf010d4b0 con 0x7fabf00fcf70 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabeeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fbcc0 0x7fabf010c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabeeffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fbcc0 0x7fabf010c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58822/0 (socket says 172.21.15.110:58822) 2024-04-25T15:55:36.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabeeffd700 1 -- 172.21.15.110:0/2040619420 learned_addr learned my addr 172.21.15.110:0/2040619420 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabe67fc700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf00fcf70 0x7fabf010cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabeeffd700 1 -- 172.21.15.110:0/2040619420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf00fcf70 msgr2=0x7fabf010cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabeeffd700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf00fcf70 0x7fabf010cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabeeffd700 1 -- 172.21.15.110:0/2040619420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8005040 con 0x7fabf00fbcc0 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.254+0000 7fabeeffd700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fbcc0 0x7fabf010c800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabe0002a80 tx=0x7fabe00046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.255+0000 7fabe7fff700 1 -- 172.21.15.110:0/2040619420 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe0007cc0 con 0x7fabf00fbcc0 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.255+0000 7fabe7fff700 1 -- 172.21.15.110:0/2040619420 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fabe000f040 con 0x7fabf00fbcc0 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.255+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf0111f10 con 0x7fabf00fbcc0 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.255+0000 7fabe7fff700 1 -- 172.21.15.110:0/2040619420 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabe000dac0 con 0x7fabf00fbcc0 2024-04-25T15:55:36.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.255+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf0112440 con 0x7fabf00fbcc0 2024-04-25T15:55:36.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.256+0000 7fabe7fff700 1 -- 172.21.15.110:0/2040619420 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fabe000dc20 con 0x7fabf00fbcc0 2024-04-25T15:55:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.257+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf004c710 con 0x7fabf00fbcc0 2024-04-25T15:55:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.257+0000 7fabe7fff700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fabdc076970 0x7fabdc078e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.257+0000 7fabe7fff700 1 -- 172.21.15.110:0/2040619420 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fabe0097cf0 con 0x7fabf00fbcc0 2024-04-25T15:55:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.257+0000 7fabe67fc700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fabdc076970 0x7fabdc078e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:36.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.257+0000 7fabe67fc700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fabdc076970 0x7fabdc078e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabd800b020 tx=0x7fabd8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:36.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.262+0000 7fabe7fff700 1 -- 172.21.15.110:0/2040619420 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fabe0060870 con 0x7fabf00fbcc0 2024-04-25T15:55:36.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.544+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fabf0112900 con 0x7fabf00fbcc0 2024-04-25T15:55:36.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.545+0000 7fabe7fff700 1 -- 172.21.15.110:0/2040619420 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+839 (secure 0 0 0) 0x7fabe0060210 con 0x7fabf00fbcc0 2024-04-25T15:55:36.546 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:55:36.546 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:55:36.546 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:55:36.546 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:55:36.546 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:55:36.546 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 4 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 5, 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 8 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:55:36.547 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fabdc076970 msgr2=0x7fabdc078e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fabdc076970 0x7fabdc078e20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fabd800b020 tx=0x7fabd8009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fbcc0 msgr2=0x7fabf010c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fbcc0 0x7fabf010c800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fabe0002a80 tx=0x7fabe00046b0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 shutdown_connections 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fabdc076970 0x7fabdc078e20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fabf00fbcc0 0x7fabf010c800 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 --2- 172.21.15.110:0/2040619420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fabf00fcf70 0x7fabf010cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.547+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 >> 172.21.15.110:0/2040619420 conn(0x7fabf00f7440 msgr2=0x7fabf00f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.548+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 shutdown_connections 2024-04-25T15:55:36.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.548+0000 7fabf57ce700 1 -- 172.21.15.110:0/2040619420 wait complete. 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.647+0000 7f49a4fc8700 1 -- 172.21.15.110:0/2475848680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 msgr2=0x7f49a00fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.647+0000 7f49a4fc8700 1 --2- 172.21.15.110:0/2475848680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 0x7f49a00fe440 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4994004940 tx=0x7f4994009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.647+0000 7f49a4fc8700 1 -- 172.21.15.110:0/2475848680 shutdown_connections 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.647+0000 7f49a4fc8700 1 --2- 172.21.15.110:0/2475848680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 0x7f49a00fe440 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.647+0000 7f49a4fc8700 1 --2- 172.21.15.110:0/2475848680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a00fce60 0x7f49a00fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.647+0000 7f49a4fc8700 1 -- 172.21.15.110:0/2475848680 >> 172.21.15.110:0/2475848680 conn(0x7f49a00f8540 msgr2=0x7f49a00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.647+0000 7f49a4fc8700 1 -- 172.21.15.110:0/2475848680 shutdown_connections 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.647+0000 7f49a4fc8700 1 -- 172.21.15.110:0/2475848680 wait complete. 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f49a4fc8700 1 Processor -- start 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f49a4fc8700 1 -- start start 2024-04-25T15:55:36.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f49a4fc8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a00fce60 0x7f49a010d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f49a4fc8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 0x7f49a010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f49a4fc8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a010e450 con 0x7f49a00fdfd0 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f49a4fc8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49a010e5c0 con 0x7f49a00fce60 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f499dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 0x7f49a010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f499dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 0x7f49a010de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58852/0 (socket says 172.21.15.110:58852) 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f499dd9b700 1 -- 172.21.15.110:0/4250027365 learned_addr learned my addr 172.21.15.110:0/4250027365 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f499e59c700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a00fce60 0x7f49a010d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f499dd9b700 1 -- 172.21.15.110:0/4250027365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a00fce60 msgr2=0x7f49a010d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f499dd9b700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a00fce60 0x7f49a010d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.648+0000 7f499dd9b700 1 -- 172.21.15.110:0/4250027365 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4994005040 con 0x7f49a00fdfd0 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.649+0000 7f499dd9b700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 0x7f49a010de50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f499400b3b0 tx=0x7f4994009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.649+0000 7f498effd700 1 -- 172.21.15.110:0/4250027365 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f499400e040 con 0x7f49a00fdfd0 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.649+0000 7f498effd700 1 -- 172.21.15.110:0/4250027365 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f499400bbf0 con 0x7f49a00fdfd0 2024-04-25T15:55:36.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.649+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49a0112fc0 con 0x7f49a00fdfd0 2024-04-25T15:55:36.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.649+0000 7f498effd700 1 -- 172.21.15.110:0/4250027365 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f499401c630 con 0x7f49a00fdfd0 2024-04-25T15:55:36.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.649+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49a0113550 con 0x7f49a00fdfd0 2024-04-25T15:55:36.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.650+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49a004c710 con 0x7f49a00fdfd0 2024-04-25T15:55:36.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.650+0000 7f498effd700 1 -- 172.21.15.110:0/4250027365 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f499400bd60 con 0x7f49a00fdfd0 2024-04-25T15:55:36.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.651+0000 7f498effd700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4988076b90 0x7f4988079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:36.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.651+0000 7f498effd700 1 -- 172.21.15.110:0/4250027365 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f4994098d80 con 0x7f49a00fdfd0 2024-04-25T15:55:36.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.651+0000 7f499e59c700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4988076b90 0x7f4988079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:36.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.651+0000 7f499e59c700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4988076b90 0x7f4988079040 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4990004f40 tx=0x7f499000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:36.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.656+0000 7f498effd700 1 -- 172.21.15.110:0/4250027365 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4994061a20 con 0x7f49a00fdfd0 2024-04-25T15:55:36.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.904+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f49a01058c0 con 0x7f49a00fdfd0 2024-04-25T15:55:36.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.904+0000 7f498effd700 1 -- 172.21.15.110:0/4250027365 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f49940613c0 con 0x7f49a00fdfd0 2024-04-25T15:55:36.905 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:55:36.905 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:55:36.906 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:55:36.907 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:55:36.908 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:55:36.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4988076b90 msgr2=0x7f4988079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4988076b90 0x7f4988079040 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4990004f40 tx=0x7f499000b040 comp rx=0 tx=0).stop 2024-04-25T15:55:36.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 msgr2=0x7f49a010de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:36.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 0x7f49a010de50 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f499400b3b0 tx=0x7f4994009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:36.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 shutdown_connections 2024-04-25T15:55:36.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49a00fce60 0x7f49a010d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4988076b90 0x7f4988079040 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 --2- 172.21.15.110:0/4250027365 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49a00fdfd0 0x7f49a010de50 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:36.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 >> 172.21.15.110:0/4250027365 conn(0x7f49a00f8540 msgr2=0x7f49a00f9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:36.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 shutdown_connections 2024-04-25T15:55:36.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:36.906+0000 7f49a4fc8700 1 -- 172.21.15.110:0/4250027365 wait complete. 2024-04-25T15:55:36.910 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:55:36.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:36 smithi193 ceph-mon[80071]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:36.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:36 smithi193 ceph-mon[80071]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:36.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:36 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2040619420' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:36.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:36 smithi193 ceph-mon[80071]: from='osd.4 [v2:172.21.15.193:6808/2972122544,v1:172.21.15.193:6809/2972122544]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T15:55:36.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:36 smithi193 ceph-mon[80071]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T15:55:36.990 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:36 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T15:55:36.704+0000 7f147dffd7c0 -1 osd.4 69 log_to_monitors true 2024-04-25T15:55:36.990 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 15:55:36 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T15:55:36.850+0000 7f146f417700 -1 osd.4 69 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 -- 172.21.15.110:0/3928194694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 msgr2=0x7f92d4100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 --2- 172.21.15.110:0/3928194694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 0x7f92d4100150 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f92c4004940 tx=0x7f92c4009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 -- 172.21.15.110:0/3928194694 shutdown_connections 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 --2- 172.21.15.110:0/3928194694 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 0x7f92d4100150 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 --2- 172.21.15.110:0/3928194694 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92d4066f30 0x7f92d40fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 -- 172.21.15.110:0/3928194694 >> 172.21.15.110:0/3928194694 conn(0x7f92d40f7410 msgr2=0x7f92d40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 -- 172.21.15.110:0/3928194694 shutdown_connections 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 -- 172.21.15.110:0/3928194694 wait complete. 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 Processor -- start 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 -- start start 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92d4066f30 0x7f92d4113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 0x7f92d4114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:37.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4114a40 con 0x7f92d40fdcd0 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.008+0000 7f92dc6e8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4114bb0 con 0x7f92d4066f30 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92d9c83700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 0x7f92d4114440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92d9c83700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 0x7f92d4114440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58870/0 (socket says 172.21.15.110:58870) 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92d9c83700 1 -- 172.21.15.110:0/151634087 learned_addr learned my addr 172.21.15.110:0/151634087 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92da484700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92d4066f30 0x7f92d4113f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92d9c83700 1 -- 172.21.15.110:0/151634087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92d4066f30 msgr2=0x7f92d4113f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92d9c83700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92d4066f30 0x7f92d4113f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92d9c83700 1 -- 172.21.15.110:0/151634087 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c4005040 con 0x7f92d40fdcd0 2024-04-25T15:55:37.011 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92d9c83700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 0x7f92d4114440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009150 tx=0x7f92c4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:37.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92caffd700 1 -- 172.21.15.110:0/151634087 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c400e040 con 0x7f92d40fdcd0 2024-04-25T15:55:37.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92caffd700 1 -- 172.21.15.110:0/151634087 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92c4004290 con 0x7f92d40fdcd0 2024-04-25T15:55:37.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d4119560 con 0x7f92d40fdcd0 2024-04-25T15:55:37.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92caffd700 1 -- 172.21.15.110:0/151634087 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c401c630 con 0x7f92d40fdcd0 2024-04-25T15:55:37.012 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.009+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d4119b20 con 0x7f92d40fdcd0 2024-04-25T15:55:37.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.010+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d404c710 con 0x7f92d40fdcd0 2024-04-25T15:55:37.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.011+0000 7f92caffd700 1 -- 172.21.15.110:0/151634087 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f92c400bbf0 con 0x7f92d40fdcd0 2024-04-25T15:55:37.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.011+0000 7f92caffd700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f92c0076b90 0x7f92c0079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:37.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.011+0000 7f92caffd700 1 -- 172.21.15.110:0/151634087 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f92c4098dc0 con 0x7f92d40fdcd0 2024-04-25T15:55:37.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.011+0000 7f92da484700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f92c0076b90 0x7f92c0079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:37.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.012+0000 7f92da484700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f92c0076b90 0x7f92c0079040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009fd0 tx=0x7f92d000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:37.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.016+0000 7f92caffd700 1 -- 172.21.15.110:0/151634087 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f92c4061a60 con 0x7f92d40fdcd0 2024-04-25T15:55:37.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:36 smithi110 ceph-mon[90991]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:37.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:36 smithi110 ceph-mon[90991]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:37.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:36 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2040619420' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:37.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:36 smithi110 ceph-mon[90991]: from='osd.4 [v2:172.21.15.193:6808/2972122544,v1:172.21.15.193:6809/2972122544]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T15:55:37.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:36 smithi110 ceph-mon[90991]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T15:55:37.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.237+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92d40f8f50 con 0x7f92c0076b90 2024-04-25T15:55:37.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.238+0000 7f92caffd700 1 -- 172.21.15.110:0/151634087 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f92d40f8f50 con 0x7f92c0076b90 2024-04-25T15:55:37.239 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:55:37.239 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:55:37.239 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "11/21 daemons upgraded", 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:55:37.240 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:55:37.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f92c0076b90 msgr2=0x7f92c0079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f92c0076b90 0x7f92c0079040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92d0009fd0 tx=0x7f92d000b040 comp rx=0 tx=0).stop 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 msgr2=0x7f92d4114440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 0x7f92d4114440 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009150 tx=0x7f92c4009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 shutdown_connections 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f92d4066f30 0x7f92d4113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f92c0076b90 0x7f92c0079040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 --2- 172.21.15.110:0/151634087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92d40fdcd0 0x7f92d4114440 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 >> 172.21.15.110:0/151634087 conn(0x7f92d40f7410 msgr2=0x7f92d40f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 shutdown_connections 2024-04-25T15:55:37.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.240+0000 7f92dc6e8700 1 -- 172.21.15.110:0/151634087 wait complete. 2024-04-25T15:55:37.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.340+0000 7fab3c878700 1 -- 172.21.15.110:0/3400710027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 msgr2=0x7fab380fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:37.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.340+0000 7fab3c878700 1 --2- 172.21.15.110:0/3400710027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 0x7fab380fe500 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fab28004940 tx=0x7fab28009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:37.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.340+0000 7fab3c878700 1 -- 172.21.15.110:0/3400710027 shutdown_connections 2024-04-25T15:55:37.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.340+0000 7fab3c878700 1 --2- 172.21.15.110:0/3400710027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 0x7fab380fe500 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.340+0000 7fab3c878700 1 --2- 172.21.15.110:0/3400710027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab380fce90 0x7fab380fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.340+0000 7fab3c878700 1 -- 172.21.15.110:0/3400710027 >> 172.21.15.110:0/3400710027 conn(0x7fab380f85c0 msgr2=0x7fab380fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:37.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.340+0000 7fab3c878700 1 -- 172.21.15.110:0/3400710027 shutdown_connections 2024-04-25T15:55:37.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.340+0000 7fab3c878700 1 -- 172.21.15.110:0/3400710027 wait complete. 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab3c878700 1 Processor -- start 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab3c878700 1 -- start start 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab3c878700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab380fce90 0x7fab3810da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab3c878700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 0x7fab3810df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab3c878700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3810e580 con 0x7fab380fe090 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab3c878700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3810edd0 con 0x7fab380fce90 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab35d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 0x7fab3810df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab35d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 0x7fab3810df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58886/0 (socket says 172.21.15.110:58886) 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab35d9b700 1 -- 172.21.15.110:0/1605582122 learned_addr learned my addr 172.21.15.110:0/1605582122 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab3659c700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab380fce90 0x7fab3810da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab35d9b700 1 -- 172.21.15.110:0/1605582122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab380fce90 msgr2=0x7fab3810da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab35d9b700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab380fce90 0x7fab3810da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.341+0000 7fab35d9b700 1 -- 172.21.15.110:0/1605582122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28005040 con 0x7fab380fe090 2024-04-25T15:55:37.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.342+0000 7fab35d9b700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 0x7fab3810df80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fab2800b3b0 tx=0x7fab28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:37.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.342+0000 7fab26ffd700 1 -- 172.21.15.110:0/1605582122 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2800e040 con 0x7fab380fe090 2024-04-25T15:55:37.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.342+0000 7fab26ffd700 1 -- 172.21.15.110:0/1605582122 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab2800bbf0 con 0x7fab380fe090 2024-04-25T15:55:37.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.342+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab3810f050 con 0x7fab380fe090 2024-04-25T15:55:37.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.342+0000 7fab26ffd700 1 -- 172.21.15.110:0/1605582122 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab2801c630 con 0x7fab380fe090 2024-04-25T15:55:37.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.342+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3810f590 con 0x7fab380fe090 2024-04-25T15:55:37.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.343+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab3805f5c0 con 0x7fab380fe090 2024-04-25T15:55:37.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.343+0000 7fab26ffd700 1 -- 172.21.15.110:0/1605582122 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fab2800bd60 con 0x7fab380fe090 2024-04-25T15:55:37.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.344+0000 7fab26ffd700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fab20076b90 0x7fab20079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:55:37.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.344+0000 7fab26ffd700 1 -- 172.21.15.110:0/1605582122 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6324+0+0 (secure 0 0 0) 0x7fab28099570 con 0x7fab380fe090 2024-04-25T15:55:37.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.344+0000 7fab3659c700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fab20076b90 0x7fab20079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:55:37.348 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.344+0000 7fab3659c700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fab20076b90 0x7fab20079040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fab380fdc00 tx=0x7fab2c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:55:37.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.348+0000 7fab26ffd700 1 -- 172.21.15.110:0/1605582122 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fab28062210 con 0x7fab380fe090 2024-04-25T15:55:37.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.631+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fab3804bed0 con 0x7fab380fe090 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.631+0000 7fab26ffd700 1 -- 172.21.15.110:0/1605582122 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1296 (secure 0 0 0) 0x7fab28061bb0 con 0x7fab380fe090 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: osd.4 (root=default,host=smithi193) is down 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-04-25T15:55:37.633 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-04-25T15:55:37.634 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-04-25T15:55:37.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.633+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fab20076b90 msgr2=0x7fab20079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:37.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.633+0000 7fab3c878700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fab20076b90 0x7fab20079040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fab380fdc00 tx=0x7fab2c00b040 comp rx=0 tx=0).stop 2024-04-25T15:55:37.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.633+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 msgr2=0x7fab3810df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:55:37.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.633+0000 7fab3c878700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 0x7fab3810df80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fab2800b3b0 tx=0x7fab28009e30 comp rx=0 tx=0).stop 2024-04-25T15:55:37.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.634+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 shutdown_connections 2024-04-25T15:55:37.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.634+0000 7fab3c878700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab380fce90 0x7fab3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.634+0000 7fab3c878700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fab20076b90 0x7fab20079040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.634+0000 7fab3c878700 1 --2- 172.21.15.110:0/1605582122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab380fe090 0x7fab3810df80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:55:37.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.634+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 >> 172.21.15.110:0/1605582122 conn(0x7fab380f85c0 msgr2=0x7fab380f9cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:55:37.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.634+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 shutdown_connections 2024-04-25T15:55:37.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:55:37.634+0000 7fab3c878700 1 -- 172.21.15.110:0/1605582122 wait complete. 2024-04-25T15:55:37.946 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:37 smithi193 ceph-mon[80071]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:37.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:37 smithi193 ceph-mon[80071]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-25T15:55:37.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:37 smithi193 ceph-mon[80071]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T15:55:37.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:37 smithi193 ceph-mon[80071]: osdmap e72: 6 total, 5 up, 6 in 2024-04-25T15:55:37.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:37 smithi193 ceph-mon[80071]: from='osd.4 [v2:172.21.15.193:6808/2972122544,v1:172.21.15.193:6809/2972122544]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:55:37.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:37 smithi193 ceph-mon[80071]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:55:37.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:37 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/4250027365' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:55:37.947 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:37 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1605582122' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:55:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:37 smithi110 ceph-mon[90991]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:37 smithi110 ceph-mon[90991]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-25T15:55:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:37 smithi110 ceph-mon[90991]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T15:55:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:37 smithi110 ceph-mon[90991]: osdmap e72: 6 total, 5 up, 6 in 2024-04-25T15:55:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:37 smithi110 ceph-mon[90991]: from='osd.4 [v2:172.21.15.193:6808/2972122544,v1:172.21.15.193:6809/2972122544]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:55:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:37 smithi110 ceph-mon[90991]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:55:38.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:37 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/4250027365' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:55:38.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:37 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1605582122' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:55:39.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: osd.4 [v2:172.21.15.193:6808/2972122544,v1:172.21.15.193:6809/2972122544] boot 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: osdmap e73: 6 total, 6 up, 6 in 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:39.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T15:55:39.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:38 smithi110 ceph-mon[90991]: osdmap e74: 6 total, 6 up, 6 in 2024-04-25T15:55:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: osd.4 [v2:172.21.15.193:6808/2972122544,v1:172.21.15.193:6809/2972122544] boot 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: osdmap e73: 6 total, 6 up, 6 in 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 47/267 objects degraded (17.603%), 15 pgs degraded (PG_DEGRADED) 2024-04-25T15:55:39.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:55:39.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:39.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:39.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:55:39.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T15:55:39.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:38 smithi193 ceph-mon[80071]: osdmap e74: 6 total, 6 up, 6 in 2024-04-25T15:55:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:39 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T15:55:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:39 smithi193 ceph-mon[80071]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-25T15:55:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:39 smithi193 ceph-mon[80071]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-25T15:55:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:39 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T15:55:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:39 smithi110 ceph-mon[90991]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-25T15:55:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:39 smithi110 ceph-mon[90991]: pgmap v143: 65 pgs: 15 active+undersized+degraded, 19 active+undersized, 31 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 47/267 objects degraded (17.603%) 2024-04-25T15:55:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:41 smithi110 ceph-mon[90991]: pgmap v145: 65 pgs: 7 active+undersized+degraded, 11 active+undersized, 47 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18/267 objects degraded (6.742%) 2024-04-25T15:55:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:41 smithi193 ceph-mon[80071]: pgmap v145: 65 pgs: 7 active+undersized+degraded, 11 active+undersized, 47 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 18/267 objects degraded (6.742%) 2024-04-25T15:55:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:42 smithi193 ceph-mon[80071]: pgmap v146: 65 pgs: 1 active+undersized, 64 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:42 smithi193 ceph-mon[80071]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded) 2024-04-25T15:55:43.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:42 smithi110 ceph-mon[90991]: pgmap v146: 65 pgs: 1 active+undersized, 64 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:43.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:42 smithi110 ceph-mon[90991]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 18/267 objects degraded (6.742%), 7 pgs degraded) 2024-04-25T15:55:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:45 smithi110 ceph-mon[90991]: pgmap v147: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:45 smithi193 ceph-mon[80071]: pgmap v147: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:47 smithi110 ceph-mon[90991]: pgmap v148: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:55:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:47 smithi193 ceph-mon[80071]: pgmap v148: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:55:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:49 smithi110 ceph-mon[90991]: pgmap v149: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:49 smithi193 ceph-mon[80071]: pgmap v149: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:52.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:51 smithi110 ceph-mon[90991]: pgmap v150: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:51 smithi193 ceph-mon[80071]: pgmap v150: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:52.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:52 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:52 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:53 smithi110 ceph-mon[90991]: pgmap v151: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:54.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:53 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T15:55:54.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:53 smithi193 ceph-mon[80071]: pgmap v151: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:54.121 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:53 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T15:55:54.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:54 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T15:55:54.953 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:54 smithi193 ceph-mon[80071]: Upgrade: osd.5 is safe to restart 2024-04-25T15:55:55.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:54 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-25T15:55:55.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:54 smithi110 ceph-mon[90991]: Upgrade: osd.5 is safe to restart 2024-04-25T15:55:55.906 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:55 smithi193 ceph-mon[80071]: pgmap v152: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:55.906 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:55 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:55.906 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:55 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T15:55:55.906 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:55 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:55:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:55 smithi110 ceph-mon[90991]: pgmap v152: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:55:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T15:55:56.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:55:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:56 smithi110 ceph-mon[90991]: Upgrade: Updating osd.5 2024-04-25T15:55:57.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:56 smithi110 ceph-mon[90991]: Deploying daemon osd.5 on smithi193 2024-04-25T15:55:57.174 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:56 smithi193 ceph-mon[80071]: Upgrade: Updating osd.5 2024-04-25T15:55:57.174 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:56 smithi193 ceph-mon[80071]: Deploying daemon osd.5 on smithi193 2024-04-25T15:55:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:57 smithi110 ceph-mon[90991]: pgmap v153: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:58.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:57 smithi193 ceph-mon[80071]: pgmap v153: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:55:58.388 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:55:58 smithi193 systemd[1]: Stopping Ceph osd.5 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:55:58.739 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:55:58 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[52847]: 2024-04-25T15:55:58.386+0000 7fc9e5c3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T15:55:58.739 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:55:58 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[52847]: 2024-04-25T15:55:58.386+0000 7fc9e5c3e700 -1 osd.5 74 *** Got signal Terminated *** 2024-04-25T15:55:58.739 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:55:58 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[52847]: 2024-04-25T15:55:58.386+0000 7fc9e5c3e700 -1 osd.5 74 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T15:55:59.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:58 smithi110 ceph-mon[90991]: osd.5 marked itself down and dead 2024-04-25T15:55:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:58 smithi193 ceph-mon[80071]: osd.5 marked itself down and dead 2024-04-25T15:55:59.571 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:55:59 smithi193 podman[98675]: 2024-04-25 15:55:59.277471912 +0000 UTC m=+1.006429242 container died 988e985be7eb6e7f579c110ea8b66db747b9ace34176c336fc229856bfe1b832 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, ceph=True, io.buildah.version=1.33.5, RELEASE=quincy-c61472c, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:55:59.825 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:55:59 smithi193 podman[98675]: 2024-04-25 15:55:59.661646659 +0000 UTC m=+1.390603992 container cleanup 988e985be7eb6e7f579c110ea8b66db747b9ace34176c336fc229856bfe1b832 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=quincy-c61472c) 2024-04-25T15:55:59.825 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:55:59 smithi193 bash[98675]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5 2024-04-25T15:56:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:59 smithi110 ceph-mon[90991]: pgmap v154: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:56:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:59 smithi110 ceph-mon[90991]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:56:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:59 smithi110 ceph-mon[90991]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-25T15:56:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:55:59 smithi110 ceph-mon[90991]: osdmap e75: 6 total, 5 up, 6 in 2024-04-25T15:56:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:59 smithi193 ceph-mon[80071]: pgmap v154: 65 pgs: 65 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-25T15:56:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:59 smithi193 ceph-mon[80071]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-25T15:56:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:59 smithi193 ceph-mon[80071]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-25T15:56:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:55:59 smithi193 ceph-mon[80071]: osdmap e75: 6 total, 5 up, 6 in 2024-04-25T15:56:00.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:55:59 smithi193 podman[98695]: 2024-04-25 15:55:59.971638804 +0000 UTC m=+0.693474189 container remove 988e985be7eb6e7f579c110ea8b66db747b9ace34176c336fc229856bfe1b832 (image=quay.ceph.io/ceph-ci/ceph@sha256:27d43f80a23d3545096a7bb490598a6f37852da2940e69e5a8f3e3d7fb58fd17, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=quincy-c61472c, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-25T15:56:00.617 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:00 smithi193 podman[98787]: 2024-04-25 15:56:00.23753422 +0000 UTC m=+0.017950831 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:56:00.617 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:00 smithi193 podman[98787]: 2024-04-25 15:56:00.339260467 +0000 UTC m=+0.119677124 container create 51dd2b9b42b1b2cc4d5308da01647614194d455b3e6696324419e3ce5b2ce006 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T15:56:00.617 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:00 smithi193 podman[98787]: 2024-04-25 15:56:00.550484884 +0000 UTC m=+0.330901495 container init 51dd2b9b42b1b2cc4d5308da01647614194d455b3e6696324419e3ce5b2ce006 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T15:56:00.617 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:00 smithi193 podman[98787]: 2024-04-25 15:56:00.553647589 +0000 UTC m=+0.334064208 container start 51dd2b9b42b1b2cc4d5308da01647614194d455b3e6696324419e3ce5b2ce006 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T15:56:00.618 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:00 smithi193 podman[98787]: 2024-04-25 15:56:00.616341087 +0000 UTC m=+0.396757724 container attach 51dd2b9b42b1b2cc4d5308da01647614194d455b3e6696324419e3ce5b2ce006 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T15:56:00.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:00 smithi193 ceph-mon[80071]: osdmap e76: 6 total, 5 up, 6 in 2024-04-25T15:56:00.990 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:00 smithi193 podman[98787]: 2024-04-25 15:56:00.847036187 +0000 UTC m=+0.627452789 container died 51dd2b9b42b1b2cc4d5308da01647614194d455b3e6696324419e3ce5b2ce006 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:56:01.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:00 smithi110 ceph-mon[90991]: osdmap e76: 6 total, 5 up, 6 in 2024-04-25T15:56:01.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:01 smithi193 podman[98813]: 2024-04-25 15:56:01.403192663 +0000 UTC m=+0.546649397 container remove 51dd2b9b42b1b2cc4d5308da01647614194d455b3e6696324419e3ce5b2ce006 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True) 2024-04-25T15:56:01.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:01 smithi193 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-25T15:56:01.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:01 smithi193 systemd[1]: Stopped Ceph osd.5 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:56:01.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:01 smithi193 systemd[1]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.5.service: Consumed 48.730s CPU time. 2024-04-25T15:56:01.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:01 smithi193 systemd[1]: Starting Ceph osd.5 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T15:56:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:01 smithi110 ceph-mon[90991]: pgmap v157: 65 pgs: 6 active+undersized+degraded, 8 stale+active+clean, 7 active+undersized, 44 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24/267 objects degraded (8.989%) 2024-04-25T15:56:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:01 smithi110 ceph-mon[90991]: Health check failed: Degraded data redundancy: 24/267 objects degraded (8.989%), 6 pgs degraded (PG_DEGRADED) 2024-04-25T15:56:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:56:02.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:01 smithi193 ceph-mon[80071]: pgmap v157: 65 pgs: 6 active+undersized+degraded, 8 stale+active+clean, 7 active+undersized, 44 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 24/267 objects degraded (8.989%) 2024-04-25T15:56:02.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:01 smithi193 ceph-mon[80071]: Health check failed: Degraded data redundancy: 24/267 objects degraded (8.989%), 6 pgs degraded (PG_DEGRADED) 2024-04-25T15:56:02.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:02.123 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:56:02.124 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:01 smithi193 podman[98918]: 2024-04-25 15:56:01.730751578 +0000 UTC m=+0.017926191 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:56:02.124 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:01 smithi193 podman[98918]: 2024-04-25 15:56:01.854347486 +0000 UTC m=+0.141522071 container create 8e4d916ec81a313870bb93fdc6f607209e60997d2513cf7c2ab376a1f9f85e97 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T15:56:02.124 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 podman[98918]: 2024-04-25 15:56:02.040266218 +0000 UTC m=+0.327440812 container init 8e4d916ec81a313870bb93fdc6f607209e60997d2513cf7c2ab376a1f9f85e97 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T15:56:02.124 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 podman[98918]: 2024-04-25 15:56:02.043505762 +0000 UTC m=+0.330680356 container start 8e4d916ec81a313870bb93fdc6f607209e60997d2513cf7c2ab376a1f9f85e97 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T15:56:02.124 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 podman[98918]: 2024-04-25 15:56:02.123220826 +0000 UTC m=+0.410395429 container attach 8e4d916ec81a313870bb93fdc6f607209e60997d2513cf7c2ab376a1f9f85e97 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T15:56:02.952 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 bash[98918]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-25T15:56:02.953 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-25T15:56:02.954 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-25T15:56:02.954 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T15:56:02.954 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate[98933]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-25T15:56:02.954 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:02 smithi193 podman[98918]: 2024-04-25 15:56:02.653309042 +0000 UTC m=+0.940483632 container died 8e4d916ec81a313870bb93fdc6f607209e60997d2513cf7c2ab376a1f9f85e97 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T15:56:03.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:03 smithi193 podman[99074]: 2024-04-25 15:56:03.226833388 +0000 UTC m=+0.564023704 container remove 8e4d916ec81a313870bb93fdc6f607209e60997d2513cf7c2ab376a1f9f85e97 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T15:56:03.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:03 smithi193 podman[99141]: 2024-04-25 15:56:03.345608236 +0000 UTC m=+0.017866659 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T15:56:03.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:03 smithi193 podman[99141]: 2024-04-25 15:56:03.452705797 +0000 UTC m=+0.124964211 container create 31c4aca488869588257725d278251a739e88eba2dae3c1a8c63a39b83f2f039f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T15:56:03.840 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:03 smithi193 podman[99141]: 2024-04-25 15:56:03.65524599 +0000 UTC m=+0.327504402 container init 31c4aca488869588257725d278251a739e88eba2dae3c1a8c63a39b83f2f039f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-25T15:56:03.840 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:03 smithi193 podman[99141]: 2024-04-25 15:56:03.658173241 +0000 UTC m=+0.330431655 container start 31c4aca488869588257725d278251a739e88eba2dae3c1a8c63a39b83f2f039f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T15:56:03.840 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:03 smithi193 ceph-osd[99160]: -- 172.21.15.193:0/1519577098 <== mon.1 v2:172.21.15.193:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3b43b3680 con 0x55c3b445ad00 2024-04-25T15:56:03.840 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:03 smithi193 bash[99141]: 31c4aca488869588257725d278251a739e88eba2dae3c1a8c63a39b83f2f039f 2024-04-25T15:56:03.840 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:03 smithi193 systemd[1]: Started Ceph osd.5 for e79cc6fc-0319-11ef-bc93-c7b262605968. 2024-04-25T15:56:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:03 smithi110 ceph-mon[90991]: pgmap v158: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 12 active+undersized, 39 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-04-25T15:56:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:04.213 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:03 smithi193 ceph-mon[80071]: pgmap v158: 65 pgs: 11 active+undersized+degraded, 3 stale+active+clean, 12 active+undersized, 39 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 38/267 objects degraded (14.232%) 2024-04-25T15:56:04.213 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:04.213 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:04.213 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:04.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:04 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T15:56:04.212+0000 7f8f3c02f7c0 -1 Falling back to public interface 2024-04-25T15:56:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:05 smithi110 ceph-mon[90991]: pgmap v159: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-04-25T15:56:06.205 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:05 smithi193 ceph-mon[80071]: pgmap v159: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-04-25T15:56:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 -- 172.21.15.110:0/1791827112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 msgr2=0x7efc180fd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 --2- 172.21.15.110:0/1791827112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 0x7efc180fd3e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efc08004940 tx=0x7efc08009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:07.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 -- 172.21.15.110:0/1791827112 shutdown_connections 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 --2- 172.21.15.110:0/1791827112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 0x7efc180fd3e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 --2- 172.21.15.110:0/1791827112 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc180fbcc0 0x7efc180fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 -- 172.21.15.110:0/1791827112 >> 172.21.15.110:0/1791827112 conn(0x7efc180f7440 msgr2=0x7efc180f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 -- 172.21.15.110:0/1791827112 shutdown_connections 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 -- 172.21.15.110:0/1791827112 wait complete. 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 Processor -- start 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.747+0000 7efc201b2700 1 -- start start 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc201b2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc180fbcc0 0x7efc1810c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc201b2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 0x7efc1810cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc201b2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1810d300 con 0x7efc180fcf70 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc201b2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1810d470 con 0x7efc180fbcc0 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc1d74d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 0x7efc1810cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc1df4e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc180fbcc0 0x7efc1810c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc1d74d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 0x7efc1810cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55942/0 (socket says 172.21.15.110:55942) 2024-04-25T15:56:07.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc1d74d700 1 -- 172.21.15.110:0/4228174853 learned_addr learned my addr 172.21.15.110:0/4228174853 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc1d74d700 1 -- 172.21.15.110:0/4228174853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc180fbcc0 msgr2=0x7efc1810c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc1d74d700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc180fbcc0 0x7efc1810c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc1d74d700 1 -- 172.21.15.110:0/4228174853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc08005040 con 0x7efc180fcf70 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc1d74d700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 0x7efc1810cd00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc08013870 tx=0x7efc08009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc0e7fc700 1 -- 172.21.15.110:0/4228174853 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0800e070 con 0x7efc180fcf70 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc0e7fc700 1 -- 172.21.15.110:0/4228174853 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc08013de0 con 0x7efc180fcf70 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc18111e70 con 0x7efc180fcf70 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc0e7fc700 1 -- 172.21.15.110:0/4228174853 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc0801cea0 con 0x7efc180fcf70 2024-04-25T15:56:07.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.748+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc18112320 con 0x7efc180fcf70 2024-04-25T15:56:07.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.749+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc1804c710 con 0x7efc180fcf70 2024-04-25T15:56:07.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.750+0000 7efc0e7fc700 1 -- 172.21.15.110:0/4228174853 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7efc0801c690 con 0x7efc180fcf70 2024-04-25T15:56:07.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.751+0000 7efc0e7fc700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc04076b80 0x7efc04079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:07.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.751+0000 7efc0e7fc700 1 -- 172.21.15.110:0/4228174853 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6324+0+0 (secure 0 0 0) 0x7efc0809a510 con 0x7efc180fcf70 2024-04-25T15:56:07.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.751+0000 7efc1df4e700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc04076b80 0x7efc04079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:07.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.751+0000 7efc1df4e700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc04076b80 0x7efc04079030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc14009b80 tx=0x7efc1400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:07.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.755+0000 7efc0e7fc700 1 -- 172.21.15.110:0/4228174853 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efc08063130 con 0x7efc180fcf70 2024-04-25T15:56:07.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.984+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efc18101210 con 0x7efc04076b80 2024-04-25T15:56:07.986 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:07.986 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:07.986 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:07 smithi110 ceph-mon[90991]: pgmap v160: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-04-25T15:56:07.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.986+0000 7efc0e7fc700 1 -- 172.21.15.110:0/4228174853 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7efc18101210 con 0x7efc04076b80 2024-04-25T15:56:07.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc04076b80 msgr2=0x7efc04079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:07.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc04076b80 0x7efc04079030 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efc14009b80 tx=0x7efc1400b040 comp rx=0 tx=0).stop 2024-04-25T15:56:07.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 msgr2=0x7efc1810cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:07.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 0x7efc1810cd00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7efc08013870 tx=0x7efc08009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:07.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 shutdown_connections 2024-04-25T15:56:07.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc180fbcc0 0x7efc1810c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:07.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc04076b80 0x7efc04079030 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:07.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 --2- 172.21.15.110:0/4228174853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc180fcf70 0x7efc1810cd00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:07.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 >> 172.21.15.110:0/4228174853 conn(0x7efc180f7440 msgr2=0x7efc180f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:07.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 shutdown_connections 2024-04-25T15:56:07.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:07.988+0000 7efc201b2700 1 -- 172.21.15.110:0/4228174853 wait complete. 2024-04-25T15:56:07.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:07.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:07.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:07 smithi193 ceph-mon[80071]: pgmap v160: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-04-25T15:56:07.999 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:56:08.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.089+0000 7f6f1110b700 1 -- 172.21.15.110:0/2773999606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fdfe0 msgr2=0x7f6f0c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.089+0000 7f6f1110b700 1 --2- 172.21.15.110:0/2773999606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fdfe0 0x7f6f0c0fe450 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f6f00004940 tx=0x7f6f00009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:08.091 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.089+0000 7f6f1110b700 1 -- 172.21.15.110:0/2773999606 shutdown_connections 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.089+0000 7f6f1110b700 1 --2- 172.21.15.110:0/2773999606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fdfe0 0x7f6f0c0fe450 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.089+0000 7f6f1110b700 1 --2- 172.21.15.110:0/2773999606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f0c0fce70 0x7f6f0c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.089+0000 7f6f1110b700 1 -- 172.21.15.110:0/2773999606 >> 172.21.15.110:0/2773999606 conn(0x7f6f0c0f8550 msgr2=0x7f6f0c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.089+0000 7f6f1110b700 1 -- 172.21.15.110:0/2773999606 shutdown_connections 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.089+0000 7f6f1110b700 1 -- 172.21.15.110:0/2773999606 wait complete. 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f1110b700 1 Processor -- start 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f1110b700 1 -- start start 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f1110b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fce70 0x7f6f0c071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f1110b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f0c0fdfe0 0x7f6f0c0702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f1110b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c072200 con 0x7f6f0c0fce70 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f1110b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0707f0 con 0x7f6f0c0fdfe0 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f0ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fce70 0x7f6f0c071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f0ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fce70 0x7f6f0c071c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55962/0 (socket says 172.21.15.110:55962) 2024-04-25T15:56:08.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f0ad9d700 1 -- 172.21.15.110:0/2133012544 learned_addr learned my addr 172.21.15.110:0/2133012544 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:08.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f0a59c700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f0c0fdfe0 0x7f6f0c0702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f0ad9d700 1 -- 172.21.15.110:0/2133012544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f0c0fdfe0 msgr2=0x7f6f0c0702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f0ad9d700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f0c0fdfe0 0x7f6f0c0702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.090+0000 7f6f0ad9d700 1 -- 172.21.15.110:0/2133012544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f00005040 con 0x7f6f0c0fce70 2024-04-25T15:56:08.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.091+0000 7f6f0ad9d700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fce70 0x7f6f0c071c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f04001d80 tx=0x7f6f04009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:08.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.091+0000 7f6ef37fe700 1 -- 172.21.15.110:0/2133012544 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f04007d10 con 0x7f6f0c0fce70 2024-04-25T15:56:08.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.091+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f0c070aa0 con 0x7f6f0c0fce70 2024-04-25T15:56:08.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.091+0000 7f6ef37fe700 1 -- 172.21.15.110:0/2133012544 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f04001e80 con 0x7f6f0c0fce70 2024-04-25T15:56:08.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.091+0000 7f6ef37fe700 1 -- 172.21.15.110:0/2133012544 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f04014640 con 0x7f6f0c0fce70 2024-04-25T15:56:08.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.091+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c071060 con 0x7f6f0c0fce70 2024-04-25T15:56:08.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.092+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c04c710 con 0x7f6f0c0fce70 2024-04-25T15:56:08.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.092+0000 7f6ef37fe700 1 -- 172.21.15.110:0/2133012544 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f6f0400e960 con 0x7f6f0c0fce70 2024-04-25T15:56:08.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.093+0000 7f6ef37fe700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6ef4076b90 0x7f6ef4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.093+0000 7f6ef37fe700 1 -- 172.21.15.110:0/2133012544 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6f04098060 con 0x7f6f0c0fce70 2024-04-25T15:56:08.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.093+0000 7f6f0a59c700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6ef4076b90 0x7f6ef4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.093+0000 7f6f0a59c700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6ef4076b90 0x7f6ef4079040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000b3b0 tx=0x7f6f00009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:08.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.098+0000 7f6ef37fe700 1 -- 172.21.15.110:0/2133012544 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6f04060c90 con 0x7f6f0c0fce70 2024-04-25T15:56:08.320 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.319+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f0c102350 con 0x7f6ef4076b90 2024-04-25T15:56:08.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.320+0000 7f6ef37fe700 1 -- 172.21.15.110:0/2133012544 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6f0c102350 con 0x7f6ef4076b90 2024-04-25T15:56:08.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6ef4076b90 msgr2=0x7f6ef4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6ef4076b90 0x7f6ef4079040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000b3b0 tx=0x7f6f00009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:08.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fce70 msgr2=0x7f6f0c071c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fce70 0x7f6f0c071c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6f04001d80 tx=0x7f6f04009ec0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 shutdown_connections 2024-04-25T15:56:08.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6ef4076b90 0x7f6ef4079040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6f0c0fce70 0x7f6f0c071c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 --2- 172.21.15.110:0/2133012544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f0c0fdfe0 0x7f6f0c0702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 >> 172.21.15.110:0/2133012544 conn(0x7f6f0c0f8550 msgr2=0x7f6f0c0f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:08.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 shutdown_connections 2024-04-25T15:56:08.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.322+0000 7f6f1110b700 1 -- 172.21.15.110:0/2133012544 wait complete. 2024-04-25T15:56:08.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 -- 172.21.15.110:0/932082075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 msgr2=0x7f6b340fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 --2- 172.21.15.110:0/932082075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 0x7f6b340fe4f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6b28004940 tx=0x7f6b28009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:08.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 -- 172.21.15.110:0/932082075 shutdown_connections 2024-04-25T15:56:08.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 --2- 172.21.15.110:0/932082075 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 0x7f6b340fe4f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 --2- 172.21.15.110:0/932082075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b340fce80 0x7f6b340fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.425 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 -- 172.21.15.110:0/932082075 >> 172.21.15.110:0/932082075 conn(0x7f6b340f8590 msgr2=0x7f6b340fa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 -- 172.21.15.110:0/932082075 shutdown_connections 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 -- 172.21.15.110:0/932082075 wait complete. 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 Processor -- start 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 -- start start 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.423+0000 7f6b3b7da700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b340fce80 0x7f6b3410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b3b7da700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 0x7f6b3410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b3b7da700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3410e570 con 0x7f6b340fe080 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b3b7da700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3410edc0 con 0x7f6b340fce80 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b38d75700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 0x7f6b3410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b38d75700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 0x7f6b3410df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:55984/0 (socket says 172.21.15.110:55984) 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b38d75700 1 -- 172.21.15.110:0/603295977 learned_addr learned my addr 172.21.15.110:0/603295977 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b39576700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b340fce80 0x7f6b3410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b38d75700 1 -- 172.21.15.110:0/603295977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b340fce80 msgr2=0x7f6b3410da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.426 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b38d75700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b340fce80 0x7f6b3410da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b38d75700 1 -- 172.21.15.110:0/603295977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28005040 con 0x7f6b340fe080 2024-04-25T15:56:08.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b38d75700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 0x7f6b3410df70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800b3b0 tx=0x7f6b28009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:08.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b31ffb700 1 -- 172.21.15.110:0/603295977 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b2800e040 con 0x7f6b340fe080 2024-04-25T15:56:08.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b3410f040 con 0x7f6b340fe080 2024-04-25T15:56:08.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b31ffb700 1 -- 172.21.15.110:0/603295977 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b2800bcc0 con 0x7f6b340fe080 2024-04-25T15:56:08.427 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b31ffb700 1 -- 172.21.15.110:0/603295977 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b2801c630 con 0x7f6b340fe080 2024-04-25T15:56:08.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.424+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3410f580 con 0x7f6b340fe080 2024-04-25T15:56:08.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.425+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3405f5c0 con 0x7f6b340fe080 2024-04-25T15:56:08.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.426+0000 7f6b31ffb700 1 -- 172.21.15.110:0/603295977 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f6b28013980 con 0x7f6b340fe080 2024-04-25T15:56:08.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.427+0000 7f6b31ffb700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6b1c076b90 0x7f6b1c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.427+0000 7f6b31ffb700 1 -- 172.21.15.110:0/603295977 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f6b28098ff0 con 0x7f6b340fe080 2024-04-25T15:56:08.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.427+0000 7f6b39576700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6b1c076b90 0x7f6b1c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.428 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.427+0000 7f6b39576700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6b1c076b90 0x7f6b1c079040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b340fdbf0 tx=0x7f6b2400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:08.432 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.432+0000 7f6b31ffb700 1 -- 172.21.15.110:0/603295977 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6b28061b70 con 0x7f6b340fe080 2024-04-25T15:56:08.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.640+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b3410f970 con 0x7f6b1c076b90 2024-04-25T15:56:08.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.650+0000 7f6b31ffb700 1 -- 172.21.15.110:0/603295977 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f6b3410f970 con 0x7f6b1c076b90 2024-04-25T15:56:08.651 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (6m) 101s ago 15m 25.2M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (3m) 101s ago 15m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (3m) 1s ago 14m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (6m) 101s ago 14m 85.4M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (9m) 101s ago 9m 19.9M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (9m) 101s ago 9m 150M - 17.2.7-902-gc61472c8 0de407c1b3af 5da6e2fab13e 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (9m) 1s ago 9m 15.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (9m) 1s ago 9m 17.5M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (5m) 101s ago 16m 570M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (4m) 1s ago 14m 519M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (4m) 101s ago 16m 114M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (4m) 1s ago 14m 106M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (7m) 101s ago 15m 15.7M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:56:08.652 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 1s ago 14m 15.4M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:56:08.653 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (2m) 101s ago 13m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:56:08.653 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (2m) 101s ago 12m 156M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:56:08.653 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (107s) 101s ago 12m 82.3M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:56:08.653 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (71s) 1s ago 12m 162M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:56:08.653 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (38s) 1s ago 11m 152M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T15:56:08.653 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (5s) 1s ago 11m 80.0M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 31c4aca48886 2024-04-25T15:56:08.653 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (4m) 101s ago 14m 54.6M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6b1c076b90 msgr2=0x7f6b1c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6b1c076b90 0x7f6b1c079040 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6b340fdbf0 tx=0x7f6b2400b040 comp rx=0 tx=0).stop 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 msgr2=0x7f6b3410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 0x7f6b3410df70 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b2800b3b0 tx=0x7f6b28009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 shutdown_connections 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6b340fce80 0x7f6b3410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6b1c076b90 0x7f6b1c079040 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 --2- 172.21.15.110:0/603295977 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6b340fe080 0x7f6b3410df70 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.652+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 >> 172.21.15.110:0/603295977 conn(0x7f6b340f8590 msgr2=0x7f6b340f9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.653+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 shutdown_connections 2024-04-25T15:56:08.654 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.653+0000 7f6b3b7da700 1 -- 172.21.15.110:0/603295977 wait complete. 2024-04-25T15:56:08.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 -- 172.21.15.110:0/3403695448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14066240 msgr2=0x7f9d140ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 --2- 172.21.15.110:0/3403695448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14066240 0x7f9d140ffa20 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9d10004940 tx=0x7f9d10009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 -- 172.21.15.110:0/3403695448 shutdown_connections 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 --2- 172.21.15.110:0/3403695448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d140fff60 0x7f9d14102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 --2- 172.21.15.110:0/3403695448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d14066240 0x7f9d140ffa20 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 -- 172.21.15.110:0/3403695448 >> 172.21.15.110:0/3403695448 conn(0x7f9d140f7430 msgr2=0x7f9d140f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 -- 172.21.15.110:0/3403695448 shutdown_connections 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 -- 172.21.15.110:0/3403695448 wait complete. 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.752+0000 7f9d1b870700 1 Processor -- start 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d1b870700 1 -- start start 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d1b870700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14066240 0x7f9d1410da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d1b870700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d140fff60 0x7f9d1410df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d1b870700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1410e590 con 0x7f9d140fff60 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d1b870700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1410ede0 con 0x7f9d14066240 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d18e0b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d140fff60 0x7f9d1410df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d18e0b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d140fff60 0x7f9d1410df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56012/0 (socket says 172.21.15.110:56012) 2024-04-25T15:56:08.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d18e0b700 1 -- 172.21.15.110:0/276160646 learned_addr learned my addr 172.21.15.110:0/276160646 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d1960c700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14066240 0x7f9d1410da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d18e0b700 1 -- 172.21.15.110:0/276160646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14066240 msgr2=0x7f9d1410da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d18e0b700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14066240 0x7f9d1410da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d18e0b700 1 -- 172.21.15.110:0/276160646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d10005040 con 0x7f9d140fff60 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d18e0b700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d140fff60 0x7f9d1410df90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9d04002a50 tx=0x7f9d040046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d09ffb700 1 -- 172.21.15.110:0/276160646 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d04007cc0 con 0x7f9d140fff60 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1410f0c0 con 0x7f9d140fff60 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d09ffb700 1 -- 172.21.15.110:0/276160646 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d0400f040 con 0x7f9d140fff60 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.753+0000 7f9d09ffb700 1 -- 172.21.15.110:0/276160646 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d0400da30 con 0x7f9d140fff60 2024-04-25T15:56:08.756 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.754+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d1410f600 con 0x7f9d140fff60 2024-04-25T15:56:08.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.755+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d1405f5c0 con 0x7f9d140fff60 2024-04-25T15:56:08.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.755+0000 7f9d09ffb700 1 -- 172.21.15.110:0/276160646 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f9d0400db90 con 0x7f9d140fff60 2024-04-25T15:56:08.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.755+0000 7f9d09ffb700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9d00076b90 0x7f9d00079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:08.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.756+0000 7f9d09ffb700 1 -- 172.21.15.110:0/276160646 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f9d04098060 con 0x7f9d140fff60 2024-04-25T15:56:08.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.756+0000 7f9d1960c700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9d00076b90 0x7f9d00079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:08.757 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.756+0000 7f9d1960c700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9d00076b90 0x7f9d00079040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d10004890 tx=0x7f9d10009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:08.761 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:08.761+0000 7f9d09ffb700 1 -- 172.21.15.110:0/276160646 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9d04060be0 con 0x7f9d140fff60 2024-04-25T15:56:09.045 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.044+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9d1410f8c0 con 0x7f9d140fff60 2024-04-25T15:56:09.045 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:08 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:09.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:08 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:09.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:08 smithi110 ceph-mon[90991]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:09.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:08 smithi110 ceph-mon[90991]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:09.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:08 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 42/267 objects degraded (15.730%), 13 pgs degraded (PG_DEGRADED) 2024-04-25T15:56:09.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:08 smithi110 ceph-mon[90991]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:09.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:08 smithi110 ceph-mon[90991]: pgmap v161: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-04-25T15:56:09.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.045+0000 7f9d09ffb700 1 -- 172.21.15.110:0/276160646 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+732 (secure 0 0 0) 0x7f9d04060580 con 0x7f9d140fff60 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 5 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:56:09.047 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:56:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4 2024-04-25T15:56:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:56:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:56:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 4, 2024-04-25T15:56:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 9 2024-04-25T15:56:09.048 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:56:09.048 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9d00076b90 msgr2=0x7f9d00079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9d00076b90 0x7f9d00079040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d10004890 tx=0x7f9d10009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d140fff60 msgr2=0x7f9d1410df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d140fff60 0x7f9d1410df90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9d04002a50 tx=0x7f9d040046b0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 shutdown_connections 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14066240 0x7f9d1410da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9d00076b90 0x7f9d00079040 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 --2- 172.21.15.110:0/276160646 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9d140fff60 0x7f9d1410df90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 >> 172.21.15.110:0/276160646 conn(0x7f9d140f7430 msgr2=0x7f9d140f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 shutdown_connections 2024-04-25T15:56:09.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.047+0000 7f9d1b870700 1 -- 172.21.15.110:0/276160646 wait complete. 2024-04-25T15:56:09.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.148+0000 7f2795d6a700 1 -- 172.21.15.110:0/1996063741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27900ffef0 msgr2=0x7f2790102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.148+0000 7f2795d6a700 1 --2- 172.21.15.110:0/1996063741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27900ffef0 0x7f2790102310 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2778004940 tx=0x7f2778009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:09.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.148+0000 7f2795d6a700 1 -- 172.21.15.110:0/1996063741 shutdown_connections 2024-04-25T15:56:09.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.148+0000 7f2795d6a700 1 --2- 172.21.15.110:0/1996063741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27900ffef0 0x7f2790102310 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.148+0000 7f2795d6a700 1 --2- 172.21.15.110:0/1996063741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2790066f50 0x7f27900ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.148+0000 7f2795d6a700 1 -- 172.21.15.110:0/1996063741 >> 172.21.15.110:0/1996063741 conn(0x7f27900f7440 msgr2=0x7f27900f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.148+0000 7f2795d6a700 1 -- 172.21.15.110:0/1996063741 shutdown_connections 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.148+0000 7f2795d6a700 1 -- 172.21.15.110:0/1996063741 wait complete. 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f2795d6a700 1 Processor -- start 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f2795d6a700 1 -- start start 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f2795d6a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2790066f50 0x7f279010d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f2795d6a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27900ffef0 0x7f279010def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f2795d6a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279010e4f0 con 0x7f2790066f50 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f2795d6a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279010e660 con 0x7f27900ffef0 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f278f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2790066f50 0x7f279010d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f278f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2790066f50 0x7f279010d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56038/0 (socket says 172.21.15.110:56038) 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f278f7fe700 1 -- 172.21.15.110:0/3353626104 learned_addr learned my addr 172.21.15.110:0/3353626104 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:09.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f278effd700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27900ffef0 0x7f279010def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f278f7fe700 1 -- 172.21.15.110:0/3353626104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27900ffef0 msgr2=0x7f279010def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f278f7fe700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27900ffef0 0x7f279010def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.149+0000 7f278f7fe700 1 -- 172.21.15.110:0/3353626104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778005040 con 0x7f2790066f50 2024-04-25T15:56:09.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.150+0000 7f278f7fe700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2790066f50 0x7f279010d9b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f278800c890 tx=0x7f27880044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:09.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.150+0000 7f2787fff700 1 -- 172.21.15.110:0/3353626104 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f278800a700 con 0x7f2790066f50 2024-04-25T15:56:09.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.150+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27901130c0 con 0x7f2790066f50 2024-04-25T15:56:09.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.150+0000 7f2787fff700 1 -- 172.21.15.110:0/3353626104 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f278800ad30 con 0x7f2790066f50 2024-04-25T15:56:09.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.150+0000 7f2787fff700 1 -- 172.21.15.110:0/3353626104 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27880113a0 con 0x7f2790066f50 2024-04-25T15:56:09.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.150+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2790113680 con 0x7f2790066f50 2024-04-25T15:56:09.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.151+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279004c710 con 0x7f2790066f50 2024-04-25T15:56:09.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.151+0000 7f2787fff700 1 -- 172.21.15.110:0/3353626104 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f2788012400 con 0x7f2790066f50 2024-04-25T15:56:09.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.152+0000 7f2787fff700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f277c076b90 0x7f277c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.152+0000 7f2787fff700 1 -- 172.21.15.110:0/3353626104 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f27880983c0 con 0x7f2790066f50 2024-04-25T15:56:09.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.152+0000 7f278effd700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f277c076b90 0x7f277c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.152+0000 7f278effd700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f277c076b90 0x7f277c079040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f277800b3b0 tx=0x7f2778009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:09.158 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.157+0000 7f2787fff700 1 -- 172.21.15.110:0/3353626104 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2788060fc0 con 0x7f2790066f50 2024-04-25T15:56:09.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:09 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T15:56:09.051+0000 7f8f3c02f7c0 -1 osd.5 74 log_to_monitors true 2024-04-25T15:56:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:08 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:08 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:08 smithi193 ceph-mon[80071]: from='client.34374 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:08 smithi193 ceph-mon[80071]: from='client.34378 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:08 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 42/267 objects degraded (15.730%), 13 pgs degraded (PG_DEGRADED) 2024-04-25T15:56:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:08 smithi193 ceph-mon[80071]: from='client.34382 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:09.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:08 smithi193 ceph-mon[80071]: pgmap v161: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-04-25T15:56:09.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.399+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2790113ae0 con 0x7f2790066f50 2024-04-25T15:56:09.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.399+0000 7f2787fff700 1 -- 172.21.15.110:0/3353626104 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1878 (secure 0 0 0) 0x7f2788060960 con 0x7f2790066f50 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:e13 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 13 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:56:09.401 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:46:35.394851+0000 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 0 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:56:09.402 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14468} 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:14468} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.110:6826/123050770,v1:172.21.15.110:6827/123050770] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:56:09.403 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:56:09.404 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:56:09.404 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:56:09.404 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:56:09.404 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:14478} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:56:09.404 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:56:09.404 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.401+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f277c076b90 msgr2=0x7f277c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.401+0000 7f2795d6a700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f277c076b90 0x7f277c079040 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f277800b3b0 tx=0x7f2778009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.401+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2790066f50 msgr2=0x7f279010d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.401+0000 7f2795d6a700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2790066f50 0x7f279010d9b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f278800c890 tx=0x7f27880044c0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.402+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 shutdown_connections 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.402+0000 7f2795d6a700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f277c076b90 0x7f277c079040 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.402+0000 7f2795d6a700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2790066f50 0x7f279010d9b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.402+0000 7f2795d6a700 1 --2- 172.21.15.110:0/3353626104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27900ffef0 0x7f279010def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.402+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 >> 172.21.15.110:0/3353626104 conn(0x7f27900f7440 msgr2=0x7f27900f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:09.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.402+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 shutdown_connections 2024-04-25T15:56:09.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.402+0000 7f2795d6a700 1 -- 172.21.15.110:0/3353626104 wait complete. 2024-04-25T15:56:09.406 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 13 2024-04-25T15:56:09.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 -- 172.21.15.110:0/3004142496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fce60 msgr2=0x7f8cd00fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 --2- 172.21.15.110:0/3004142496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fce60 0x7f8cd00fd270 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0004940 tx=0x7f8cc0009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 -- 172.21.15.110:0/3004142496 shutdown_connections 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 --2- 172.21.15.110:0/3004142496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd00fdfd0 0x7f8cd00fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 --2- 172.21.15.110:0/3004142496 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fce60 0x7f8cd00fd270 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 -- 172.21.15.110:0/3004142496 >> 172.21.15.110:0/3004142496 conn(0x7f8cd00f8540 msgr2=0x7f8cd00fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 -- 172.21.15.110:0/3004142496 shutdown_connections 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 -- 172.21.15.110:0/3004142496 wait complete. 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 Processor -- start 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.503+0000 7f8cd6694700 1 -- start start 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cd6694700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd00fce60 0x7f8cd0071c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cd6694700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fdfd0 0x7f8cd00702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.505 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cd6694700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd0072200 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cd6694700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd00707f0 con 0x7f8cd00fce60 2024-04-25T15:56:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fdfd0 0x7f8cd00702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8ccf7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fdfd0 0x7f8cd00702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49564/0 (socket says 172.21.15.110:49564) 2024-04-25T15:56:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8ccf7fe700 1 -- 172.21.15.110:0/3558483402 learned_addr learned my addr 172.21.15.110:0/3558483402 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:09.506 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8ccffff700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd00fce60 0x7f8cd0071c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8ccf7fe700 1 -- 172.21.15.110:0/3558483402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd00fce60 msgr2=0x7f8cd0071c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8ccf7fe700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd00fce60 0x7f8cd0071c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8ccf7fe700 1 -- 172.21.15.110:0/3558483402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc0005040 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8ccf7fe700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fdfd0 0x7f8cd00702b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4002a50 tx=0x7f8cc40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:09.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cccff9700 1 -- 172.21.15.110:0/3558483402 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cc4007cc0 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cccff9700 1 -- 172.21.15.110:0/3558483402 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cc400f040 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd0070aa0 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cccff9700 1 -- 172.21.15.110:0/3558483402 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cc400da30 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.504+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd0071060 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.505+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd004c710 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.506+0000 7f8cccff9700 1 -- 172.21.15.110:0/3558483402 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f8cc400dbb0 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.507+0000 7f8cccff9700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8cb8076b90 0x7f8cb8079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.507+0000 7f8ccffff700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8cb8076b90 0x7f8cb8079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.507+0000 7f8cccff9700 1 -- 172.21.15.110:0/3558483402 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f8cc4098040 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.507+0000 7f8ccffff700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8cb8076b90 0x7f8cb8079040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000b970 tx=0x7f8cc0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:09.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.511+0000 7f8cccff9700 1 -- 172.21.15.110:0/3558483402 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f8cc4060bc0 con 0x7f8cd00fdfd0 2024-04-25T15:56:09.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.727+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cd0102330 con 0x7f8cb8076b90 2024-04-25T15:56:09.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.728+0000 7f8cccff9700 1 -- 172.21.15.110:0/3558483402 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8cd0102330 con 0x7f8cb8076b90 2024-04-25T15:56:09.729 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "12/21 daemons upgraded", 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading osd daemons", 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:56:09.730 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.730+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8cb8076b90 msgr2=0x7f8cb8079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.730+0000 7f8cd6694700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8cb8076b90 0x7f8cb8079040 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8cc000b970 tx=0x7f8cc0009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.730+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fdfd0 msgr2=0x7f8cd00702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.730+0000 7f8cd6694700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fdfd0 0x7f8cd00702b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4002a50 tx=0x7f8cc40046b0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.730+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 shutdown_connections 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.730+0000 7f8cd6694700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd00fce60 0x7f8cd0071c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.730+0000 7f8cd6694700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8cb8076b90 0x7f8cb8079040 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.730+0000 7f8cd6694700 1 --2- 172.21.15.110:0/3558483402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8cd00fdfd0 0x7f8cd00702b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.731+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 >> 172.21.15.110:0/3558483402 conn(0x7f8cd00f8540 msgr2=0x7f8cd00f9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.731+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 shutdown_connections 2024-04-25T15:56:09.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.731+0000 7f8cd6694700 1 -- 172.21.15.110:0/3558483402 wait complete. 2024-04-25T15:56:09.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1668380983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 msgr2=0x7f13b8102300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 --2- 172.21.15.110:0/1668380983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 0x7f13b8102300 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f13ac004940 tx=0x7f13ac009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1668380983 shutdown_connections 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 --2- 172.21.15.110:0/1668380983 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 0x7f13b8102300 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 --2- 172.21.15.110:0/1668380983 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b80662c0 0x7f13b80ff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1668380983 >> 172.21.15.110:0/1668380983 conn(0x7f13b80f7430 msgr2=0x7f13b80f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1668380983 shutdown_connections 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1668380983 wait complete. 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 Processor -- start 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.832+0000 7f13bd9f9700 1 -- start start 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13bd9f9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b80662c0 0x7f13b80fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13bd9f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 0x7f13b80fb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13bd9f9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b80fbe30 con 0x7f13b80ffee0 2024-04-25T15:56:09.835 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13bd9f9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b80fbf70 con 0x7f13b80662c0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13b67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 0x7f13b80fb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13b67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 0x7f13b80fb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:49580/0 (socket says 172.21.15.110:49580) 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13b67fc700 1 -- 172.21.15.110:0/1963968287 learned_addr learned my addr 172.21.15.110:0/1963968287 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13b6ffd700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b80662c0 0x7f13b80fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13b67fc700 1 -- 172.21.15.110:0/1963968287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b80662c0 msgr2=0x7f13b80fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13b67fc700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b80662c0 0x7f13b80fd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13b67fc700 1 -- 172.21.15.110:0/1963968287 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13ac005040 con 0x7f13b80ffee0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f13b67fc700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 0x7f13b80fb8a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13ac013870 tx=0x7f13ac009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f139f7fe700 1 -- 172.21.15.110:0/1963968287 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13ac021070 con 0x7f13b80ffee0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.833+0000 7f139f7fe700 1 -- 172.21.15.110:0/1963968287 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13ac01c400 con 0x7f13b80ffee0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.834+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b80fc1f0 con 0x7f13b80ffee0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.834+0000 7f139f7fe700 1 -- 172.21.15.110:0/1963968287 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13ac00e070 con 0x7f13b80ffee0 2024-04-25T15:56:09.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.834+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b8115a00 con 0x7f13b80ffee0 2024-04-25T15:56:09.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.835+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b805f5c0 con 0x7f13b80ffee0 2024-04-25T15:56:09.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.835+0000 7f139f7fe700 1 -- 172.21.15.110:0/1963968287 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f13ac01ca20 con 0x7f13b80ffee0 2024-04-25T15:56:09.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.836+0000 7f139f7fe700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13a0076b90 0x7f13a0079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:09.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.836+0000 7f13b6ffd700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13a0076b90 0x7f13a0079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:09.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.836+0000 7f139f7fe700 1 -- 172.21.15.110:0/1963968287 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6324+0+0 (secure 0 0 0) 0x7f13ac099510 con 0x7f13b80ffee0 2024-04-25T15:56:09.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.836+0000 7f13b6ffd700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13a0076b90 0x7f13a0079040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f13a8004600 tx=0x7f13a800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:09.842 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:09.841+0000 7f139f7fe700 1 -- 172.21.15.110:0/1963968287 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f13ac062110 con 0x7f13b80ffee0 2024-04-25T15:56:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:09 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/276160646' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:09 smithi110 ceph-mon[90991]: from='osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T15:56:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:09 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3353626104' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:56:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:09 smithi110 ceph-mon[90991]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:10.130 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.129+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f13b81058b0 con 0x7f13b80ffee0 2024-04-25T15:56:10.131 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.130+0000 7f139f7fe700 1 -- 172.21.15.110:0/1963968287 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1424 (secure 0 0 0) 0x7f13ac004490 con 0x7f13b80ffee0 2024-04-25T15:56:10.131 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 42/267 objects degraded (15.730%), 13 pgs degraded 2024-04-25T15:56:10.131 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:56:10.131 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:56:10.131 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-25T15:56:10.131 INFO:teuthology.orchestra.run.smithi110.stdout: osd.5 (root=default,host=smithi193) is down 2024-04-25T15:56:10.131 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-04-25T15:56:10.131 INFO:teuthology.orchestra.run.smithi110.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 42/267 objects degraded (15.730%), 13 pgs degraded 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-04-25T15:56:10.132 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-04-25T15:56:10.133 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-04-25T15:56:10.133 INFO:teuthology.orchestra.run.smithi110.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-04-25T15:56:10.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13a0076b90 msgr2=0x7f13a0079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13a0076b90 0x7f13a0079040 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f13a8004600 tx=0x7f13a800b040 comp rx=0 tx=0).stop 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 msgr2=0x7f13b80fb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 0x7f13b80fb8a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f13ac013870 tx=0x7f13ac009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 shutdown_connections 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13b80662c0 0x7f13b80fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13a0076b90 0x7f13a0079040 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 --2- 172.21.15.110:0/1963968287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13b80ffee0 0x7f13b80fb8a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 >> 172.21.15.110:0/1963968287 conn(0x7f13b80f7430 msgr2=0x7f13b80f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 shutdown_connections 2024-04-25T15:56:10.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:10.132+0000 7f13bd9f9700 1 -- 172.21.15.110:0/1963968287 wait complete. 2024-04-25T15:56:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:09 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/276160646' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:09 smithi193 ceph-mon[80071]: from='osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T15:56:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:09 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3353626104' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:56:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:09 smithi193 ceph-mon[80071]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:11.112 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 15:56:10 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T15:56:10.828+0000 7f8f29441700 -1 osd.5 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T15:56:11.112 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:10 smithi193 ceph-mon[80071]: from='osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T15:56:11.112 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:10 smithi193 ceph-mon[80071]: osdmap e77: 6 total, 5 up, 6 in 2024-04-25T15:56:11.113 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:10 smithi193 ceph-mon[80071]: from='osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:56:11.113 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:10 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1963968287' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:56:11.113 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:10 smithi193 ceph-mon[80071]: pgmap v163: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1011 MiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-04-25T15:56:11.113 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:10 smithi193 ceph-mon[80071]: from='osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987]' entity='osd.5' 2024-04-25T15:56:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:10 smithi110 ceph-mon[90991]: from='osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987]' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T15:56:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:10 smithi110 ceph-mon[90991]: osdmap e77: 6 total, 5 up, 6 in 2024-04-25T15:56:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:10 smithi110 ceph-mon[90991]: from='osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi193", "root=default"]}]: dispatch 2024-04-25T15:56:11.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:10 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1963968287' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:56:11.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:10 smithi110 ceph-mon[90991]: pgmap v163: 65 pgs: 13 active+undersized+degraded, 15 active+undersized, 37 active+clean; 259 MiB data, 1011 MiB used, 535 GiB / 536 GiB avail; 42/267 objects degraded (15.730%) 2024-04-25T15:56:11.364 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:10 smithi110 ceph-mon[90991]: from='osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987]' entity='osd.5' 2024-04-25T15:56:12.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:12.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:12.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:56:12.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:56:12.680 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all osd 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-25T15:56:12.681 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: Upgrade: Setting require_osd_release to 19 squid 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987] boot 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: osdmap e78: 6 total, 6 up, 6 in 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:56:12.682 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi110.mvvbtv"]}]: dispatch 2024-04-25T15:56:12.683 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:12 smithi110 ceph-mon[90991]: Upgrade: It appears safe to stop mds.cephfs.smithi110.mvvbtv 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:12.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all osd 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-25T15:56:12.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: Upgrade: Setting require_osd_release to 19 squid 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: osd.5 [v2:172.21.15.193:6816/479474987,v1:172.21.15.193:6817/479474987] boot 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: osdmap e78: 6 total, 6 up, 6 in 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi110.mvvbtv"]}]: dispatch 2024-04-25T15:56:12.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:12 smithi193 ceph-mon[80071]: Upgrade: It appears safe to stop mds.cephfs.smithi110.mvvbtv 2024-04-25T15:56:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:13 smithi110 ceph-mon[90991]: pgmap v165: 65 pgs: 3 peering, 11 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 1011 MiB used, 535 GiB / 536 GiB avail; 37/267 objects degraded (13.858%) 2024-04-25T15:56:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:13 smithi110 ceph-mon[90991]: osdmap e79: 6 total, 6 up, 6 in 2024-04-25T15:56:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:13 smithi110 ceph-mon[90991]: Health check update: Degraded data redundancy: 37/267 objects degraded (13.858%), 11 pgs degraded (PG_DEGRADED) 2024-04-25T15:56:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.mvvbtv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:56:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:56:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:13 smithi193 ceph-mon[80071]: pgmap v165: 65 pgs: 3 peering, 11 active+undersized+degraded, 14 active+undersized, 37 active+clean; 259 MiB data, 1011 MiB used, 535 GiB / 536 GiB avail; 37/267 objects degraded (13.858%) 2024-04-25T15:56:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:13 smithi193 ceph-mon[80071]: osdmap e79: 6 total, 6 up, 6 in 2024-04-25T15:56:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:13 smithi193 ceph-mon[80071]: Health check update: Degraded data redundancy: 37/267 objects degraded (13.858%), 11 pgs degraded (PG_DEGRADED) 2024-04-25T15:56:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:14.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.mvvbtv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:56:14.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:56:15.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:14 smithi110 ceph-mon[90991]: Upgrade: Updating mds.cephfs.smithi110.mvvbtv 2024-04-25T15:56:15.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:14 smithi110 ceph-mon[90991]: Deploying daemon mds.cephfs.smithi110.mvvbtv on smithi110 2024-04-25T15:56:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:14 smithi193 ceph-mon[80071]: Upgrade: Updating mds.cephfs.smithi110.mvvbtv 2024-04-25T15:56:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:14 smithi193 ceph-mon[80071]: Deploying daemon mds.cephfs.smithi110.mvvbtv on smithi110 2024-04-25T15:56:15.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[90987]: 2024-04-25T15:56:15.604+0000 7effa00b9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:56:15.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-mon[90991]: pgmap v167: 65 pgs: 5 peering, 10 active+undersized+degraded, 13 active+undersized, 37 active+clean; 259 MiB data, 1011 MiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:56:15.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-mon[90991]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:56:15.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-mon[90991]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:56:15.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-mon[90991]: osdmap e80: 6 total, 6 up, 6 in 2024-04-25T15:56:15.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-mon[90991]: Standby daemon mds.cephfs.smithi110.kdhjqx assigned to filesystem cephfs as rank 0 2024-04-25T15:56:15.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-mon[90991]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:56:15.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-mon[90991]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-25T15:56:15.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:15 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi110.kdhjqx=up:replay} 2 up:standby 2024-04-25T15:56:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:15 smithi193 ceph-mon[80071]: pgmap v167: 65 pgs: 5 peering, 10 active+undersized+degraded, 13 active+undersized, 37 active+clean; 259 MiB data, 1011 MiB used, 535 GiB / 536 GiB avail; 34/267 objects degraded (12.734%) 2024-04-25T15:56:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:15 smithi193 ceph-mon[80071]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:56:16.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:15 smithi193 ceph-mon[80071]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:56:16.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:15 smithi193 ceph-mon[80071]: osdmap e80: 6 total, 6 up, 6 in 2024-04-25T15:56:16.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:15 smithi193 ceph-mon[80071]: Standby daemon mds.cephfs.smithi110.kdhjqx assigned to filesystem cephfs as rank 0 2024-04-25T15:56:16.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:15 smithi193 ceph-mon[80071]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:56:16.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:15 smithi193 ceph-mon[80071]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-25T15:56:16.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:15 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi110.kdhjqx=up:replay} 2 up:standby 2024-04-25T15:56:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:17 smithi110 ceph-mon[90991]: pgmap v169: 65 pgs: 5 peering, 60 active+clean; 259 MiB data, 1015 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-04-25T15:56:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:56:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:17 smithi110 ceph-mon[90991]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/267 objects degraded (12.734%), 10 pgs degraded) 2024-04-25T15:56:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:56:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:17 smithi193 ceph-mon[80071]: pgmap v169: 65 pgs: 5 peering, 60 active+clean; 259 MiB data, 1015 MiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 0 op/s 2024-04-25T15:56:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:56:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:17 smithi193 ceph-mon[80071]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/267 objects degraded (12.734%), 10 pgs degraded) 2024-04-25T15:56:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:56:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:19 smithi110 ceph-mon[90991]: pgmap v170: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s rd, 1 op/s 2024-04-25T15:56:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:19 smithi193 ceph-mon[80071]: pgmap v170: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 4.1 MiB/s rd, 1 op/s 2024-04-25T15:56:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:21 smithi110 ceph-mon[90991]: pgmap v171: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-25T15:56:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:21 smithi193 ceph-mon[80071]: pgmap v171: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-25T15:56:24.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:23 smithi110 ceph-mon[90991]: pgmap v172: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-25T15:56:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:23 smithi193 ceph-mon[80071]: pgmap v172: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 5 op/s 2024-04-25T15:56:26.057 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:25 smithi110 ceph-mon[90991]: pgmap v173: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-25T15:56:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:25 smithi193 ceph-mon[80071]: pgmap v173: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-25T15:56:27.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:26 smithi110 ceph-mon[90991]: reconnect by client.14526 192.168.0.1:0/2988371434 after 0.000999984 2024-04-25T15:56:27.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:26 smithi110 ceph-mon[90991]: reconnect by client.24297 192.168.0.1:0/4279727202 after 0.000999984 2024-04-25T15:56:27.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:26 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:reconnect 2024-04-25T15:56:27.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:26 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi110.kdhjqx=up:reconnect} 2 up:standby 2024-04-25T15:56:27.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:26 smithi193 ceph-mon[80071]: reconnect by client.14526 192.168.0.1:0/2988371434 after 0.000999984 2024-04-25T15:56:27.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:26 smithi193 ceph-mon[80071]: reconnect by client.24297 192.168.0.1:0/4279727202 after 0.000999984 2024-04-25T15:56:27.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:26 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:reconnect 2024-04-25T15:56:27.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:26 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi110.kdhjqx=up:reconnect} 2 up:standby 2024-04-25T15:56:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:27 smithi110 ceph-mon[90991]: pgmap v174: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-25T15:56:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:27 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:rejoin 2024-04-25T15:56:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:27 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi110.kdhjqx=up:rejoin} 2 up:standby 2024-04-25T15:56:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:27 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:28.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:27 smithi110 ceph-mon[90991]: daemon mds.cephfs.smithi110.kdhjqx is now active in filesystem cephfs as rank 0 2024-04-25T15:56:28.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:27 smithi193 ceph-mon[80071]: pgmap v174: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-25T15:56:28.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:27 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:rejoin 2024-04-25T15:56:28.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:27 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi110.kdhjqx=up:rejoin} 2 up:standby 2024-04-25T15:56:28.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:27 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:28.141 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:27 smithi193 ceph-mon[80071]: daemon mds.cephfs.smithi110.kdhjqx is now active in filesystem cephfs as rank 0 2024-04-25T15:56:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:28 smithi110 ceph-mon[90991]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:56:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:28 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:active 2024-04-25T15:56:29.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:28 smithi110 ceph-mon[90991]: fsmap cephfs:1 {0=cephfs.smithi110.kdhjqx=up:active} 2 up:standby 2024-04-25T15:56:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:28 smithi193 ceph-mon[80071]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:56:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:28 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] up:active 2024-04-25T15:56:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:28 smithi193 ceph-mon[80071]: fsmap cephfs:1 {0=cephfs.smithi110.kdhjqx=up:active} 2 up:standby 2024-04-25T15:56:29.883 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:29 smithi110 ceph-mon[90991]: pgmap v175: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-04-25T15:56:29.883 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:29 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] up:boot 2024-04-25T15:56:29.883 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:29 smithi110 ceph-mon[90991]: fsmap cephfs:1 {0=cephfs.smithi110.kdhjqx=up:active} 3 up:standby 2024-04-25T15:56:29.883 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:56:29.883 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:29.883 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:29.883 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:29 smithi193 ceph-mon[80071]: pgmap v175: 65 pgs: 65 active+clean; 259 MiB data, 1016 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 5 op/s 2024-04-25T15:56:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:29 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] up:boot 2024-04-25T15:56:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:29 smithi193 ceph-mon[80071]: fsmap cephfs:1 {0=cephfs.smithi110.kdhjqx=up:active} 3 up:standby 2024-04-25T15:56:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:56:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:30.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:31 smithi110 ceph-mon[90991]: pgmap v176: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T15:56:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:32.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:56:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:31 smithi193 ceph-mon[80071]: pgmap v176: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 19 MiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-04-25T15:56:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:32.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:56:33.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:33 smithi110 ceph-mon[90991]: pgmap v177: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-25T15:56:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:33 smithi193 ceph-mon[80071]: pgmap v177: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 10 MiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-04-25T15:56:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:35 smithi110 ceph-mon[90991]: pgmap v178: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-04-25T15:56:35.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:35.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:35.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:35 smithi193 ceph-mon[80071]: pgmap v178: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 6.6 MiB/s rd, 4.9 KiB/s wr, 7 op/s 2024-04-25T15:56:36.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:36 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:36.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:36 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:36 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:36.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:36 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:37.456 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:37 smithi110 ceph-mon[90991]: pgmap v179: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-04-25T15:56:37.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:37 smithi193 ceph-mon[80071]: pgmap v179: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s rd, 4.9 KiB/s wr, 6 op/s 2024-04-25T15:56:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: pgmap v180: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-25T15:56:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:56:39.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:56:39.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:39.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:39.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:39.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:39.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:39.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:39.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi110.kdhjqx"]}]: dispatch 2024-04-25T15:56:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: pgmap v180: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 345 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-25T15:56:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi110.kdhjqx"]}]: dispatch 2024-04-25T15:56:40.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.240+0000 7f3454303700 1 -- 172.21.15.110:0/81261044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 msgr2=0x7f344c0fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.240+0000 7f3454303700 1 --2- 172.21.15.110:0/81261044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 0x7f344c0fd320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f343c009ce0 tx=0x7f343c004d10 comp rx=0 tx=0).stop 2024-04-25T15:56:40.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.240+0000 7f3454303700 1 -- 172.21.15.110:0/81261044 shutdown_connections 2024-04-25T15:56:40.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.240+0000 7f3454303700 1 --2- 172.21.15.110:0/81261044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344c0fe080 0x7f344c0fe4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.240+0000 7f3454303700 1 --2- 172.21.15.110:0/81261044 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 0x7f344c0fd320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.240+0000 7f3454303700 1 -- 172.21.15.110:0/81261044 >> 172.21.15.110:0/81261044 conn(0x7f344c0f8560 msgr2=0x7f344c0fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:40.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.240+0000 7f3454303700 1 -- 172.21.15.110:0/81261044 shutdown_connections 2024-04-25T15:56:40.242 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.240+0000 7f3454303700 1 -- 172.21.15.110:0/81261044 wait complete. 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f3454303700 1 Processor -- start 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f3454303700 1 -- start start 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f3454303700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 0x7f344c10dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f3454303700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344c0fe080 0x7f344c10e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f3454303700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c10e610 con 0x7f344c0fcf10 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f3454303700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c10ee60 con 0x7f344c0fe080 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f345209f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 0x7f344c10dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f345209f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 0x7f344c10dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56304/0 (socket says 172.21.15.110:56304) 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f345209f700 1 -- 172.21.15.110:0/1144773878 learned_addr learned my addr 172.21.15.110:0/1144773878 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.241+0000 7f345189e700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344c0fe080 0x7f344c10e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f345209f700 1 -- 172.21.15.110:0/1144773878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344c0fe080 msgr2=0x7f344c10e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f345209f700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344c0fe080 0x7f344c10e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f345209f700 1 -- 172.21.15.110:0/1144773878 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c009940 con 0x7f344c0fcf10 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f345209f700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 0x7f344c10dad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f343c009380 tx=0x7f343c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:40.244 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f344affd700 1 -- 172.21.15.110:0/1144773878 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f343c00e040 con 0x7f344c0fcf10 2024-04-25T15:56:40.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c10f0e0 con 0x7f344c0fcf10 2024-04-25T15:56:40.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f344affd700 1 -- 172.21.15.110:0/1144773878 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f343c0128e0 con 0x7f344c0fcf10 2024-04-25T15:56:40.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f344affd700 1 -- 172.21.15.110:0/1144773878 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f343c01ca10 con 0x7f344c0fcf10 2024-04-25T15:56:40.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.242+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c10f620 con 0x7f344c0fcf10 2024-04-25T15:56:40.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.243+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c05f5c0 con 0x7f344c0fcf10 2024-04-25T15:56:40.245 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.244+0000 7f344affd700 1 -- 172.21.15.110:0/1144773878 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f343c012a50 con 0x7f344c0fcf10 2024-04-25T15:56:40.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.244+0000 7f344affd700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3434076b90 0x7f3434079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.244+0000 7f344affd700 1 -- 172.21.15.110:0/1144773878 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f343c099f40 con 0x7f344c0fcf10 2024-04-25T15:56:40.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.244+0000 7f345189e700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3434076b90 0x7f3434079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.246 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.244+0000 7f345189e700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3434076b90 0x7f3434079040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3440009fd0 tx=0x7f3440009f60 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:40.250 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.250+0000 7f344affd700 1 -- 172.21.15.110:0/1144773878 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f343c0ceff0 con 0x7f344c0fcf10 2024-04-25T15:56:40.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.470+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f344c10f9a0 con 0x7f3434076b90 2024-04-25T15:56:40.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.471+0000 7f344affd700 1 -- 172.21.15.110:0/1144773878 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f344c10f9a0 con 0x7f3434076b90 2024-04-25T15:56:40.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3434076b90 msgr2=0x7f3434079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3434076b90 0x7f3434079040 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3440009fd0 tx=0x7f3440009f60 comp rx=0 tx=0).stop 2024-04-25T15:56:40.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 msgr2=0x7f344c10dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 0x7f344c10dad0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f343c009380 tx=0x7f343c004d10 comp rx=0 tx=0).stop 2024-04-25T15:56:40.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 shutdown_connections 2024-04-25T15:56:40.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3434076b90 0x7f3434079040 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f344c0fcf10 0x7f344c10dad0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 --2- 172.21.15.110:0/1144773878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f344c0fe080 0x7f344c10e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 >> 172.21.15.110:0/1144773878 conn(0x7f344c0f8560 msgr2=0x7f344c0f9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:40.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 shutdown_connections 2024-04-25T15:56:40.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.473+0000 7f3454303700 1 -- 172.21.15.110:0/1144773878 wait complete. 2024-04-25T15:56:40.485 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:56:40.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.582+0000 7fe836020700 1 -- 172.21.15.110:0/1686189178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fe090 msgr2=0x7fe8300fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.582+0000 7fe836020700 1 --2- 172.21.15.110:0/1686189178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fe090 0x7fe8300fe500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe820004940 tx=0x7fe820009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.582+0000 7fe836020700 1 -- 172.21.15.110:0/1686189178 shutdown_connections 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.582+0000 7fe836020700 1 --2- 172.21.15.110:0/1686189178 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fe090 0x7fe8300fe500 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.582+0000 7fe836020700 1 --2- 172.21.15.110:0/1686189178 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8300fce90 0x7fe8300fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.582+0000 7fe836020700 1 -- 172.21.15.110:0/1686189178 >> 172.21.15.110:0/1686189178 conn(0x7fe8300f85c0 msgr2=0x7fe8300fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.582+0000 7fe836020700 1 -- 172.21.15.110:0/1686189178 shutdown_connections 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.582+0000 7fe836020700 1 -- 172.21.15.110:0/1686189178 wait complete. 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe836020700 1 Processor -- start 2024-04-25T15:56:40.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe836020700 1 -- start start 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe836020700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fce90 0x7fe830071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe836020700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8300fe090 0x7fe8300702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe836020700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830070860 con 0x7fe8300fce90 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe836020700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300709d0 con 0x7fe8300fe090 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe82f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fce90 0x7fe830071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe82f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fce90 0x7fe830071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56332/0 (socket says 172.21.15.110:56332) 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe82f7fe700 1 -- 172.21.15.110:0/2229862395 learned_addr learned my addr 172.21.15.110:0/2229862395 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe82effd700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8300fe090 0x7fe8300702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe82f7fe700 1 -- 172.21.15.110:0/2229862395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8300fe090 msgr2=0x7fe8300702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe82f7fe700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8300fe090 0x7fe8300702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.583+0000 7fe82f7fe700 1 -- 172.21.15.110:0/2229862395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820005040 con 0x7fe8300fce90 2024-04-25T15:56:40.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.584+0000 7fe82f7fe700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fce90 0x7fe830071de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe82400eb70 tx=0x7fe82400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:40.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.584+0000 7fe817fff700 1 -- 172.21.15.110:0/2229862395 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe824007cc0 con 0x7fe8300fce90 2024-04-25T15:56:40.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.584+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe830070c50 con 0x7fe8300fce90 2024-04-25T15:56:40.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.584+0000 7fe817fff700 1 -- 172.21.15.110:0/2229862395 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe82400f040 con 0x7fe8300fce90 2024-04-25T15:56:40.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.584+0000 7fe817fff700 1 -- 172.21.15.110:0/2229862395 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe82400cb70 con 0x7fe8300fce90 2024-04-25T15:56:40.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.584+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8301159e0 con 0x7fe8300fce90 2024-04-25T15:56:40.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.585+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83005f5c0 con 0x7fe8300fce90 2024-04-25T15:56:40.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.585+0000 7fe817fff700 1 -- 172.21.15.110:0/2229862395 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fe82400ccd0 con 0x7fe8300fce90 2024-04-25T15:56:40.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.586+0000 7fe817fff700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe818076b90 0x7fe818079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.586+0000 7fe817fff700 1 -- 172.21.15.110:0/2229862395 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7fe824097dc0 con 0x7fe8300fce90 2024-04-25T15:56:40.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.586+0000 7fe82effd700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe818076b90 0x7fe818079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.588 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.586+0000 7fe82effd700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe818076b90 0x7fe818079040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe820009b30 tx=0x7fe820009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:40.591 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.591+0000 7fe817fff700 1 -- 172.21.15.110:0/2229862395 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe8240609a0 con 0x7fe8300fce90 2024-04-25T15:56:40.823 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.822+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe830102330 con 0x7fe818076b90 2024-04-25T15:56:40.823 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:40 smithi110 ceph-mon[90991]: Detected new or changed devices on smithi110 2024-04-25T15:56:40.823 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:40 smithi110 ceph-mon[90991]: Upgrade: It appears safe to stop mds.cephfs.smithi110.kdhjqx 2024-04-25T15:56:40.823 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:40.823 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.kdhjqx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:56:40.823 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:56:40.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.823+0000 7fe817fff700 1 -- 172.21.15.110:0/2229862395 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe830102330 con 0x7fe818076b90 2024-04-25T15:56:40.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe818076b90 msgr2=0x7fe818079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe818076b90 0x7fe818079040 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe820009b30 tx=0x7fe820009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fce90 msgr2=0x7fe830071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fce90 0x7fe830071de0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe82400eb70 tx=0x7fe82400b040 comp rx=0 tx=0).stop 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 shutdown_connections 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe818076b90 0x7fe818079040 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe8300fce90 0x7fe830071de0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 --2- 172.21.15.110:0/2229862395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8300fe090 0x7fe8300702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 >> 172.21.15.110:0/2229862395 conn(0x7fe8300f85c0 msgr2=0x7fe8300f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 shutdown_connections 2024-04-25T15:56:40.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.827+0000 7fe836020700 1 -- 172.21.15.110:0/2229862395 wait complete. 2024-04-25T15:56:40.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.935+0000 7f727a623700 1 -- 172.21.15.110:0/2058711165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fcf60 msgr2=0x7f72740fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.935+0000 7f727a623700 1 --2- 172.21.15.110:0/2058711165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fcf60 0x7f72740fd3d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f7268004940 tx=0x7f7268009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.935+0000 7f727a623700 1 -- 172.21.15.110:0/2058711165 shutdown_connections 2024-04-25T15:56:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.935+0000 7f727a623700 1 --2- 172.21.15.110:0/2058711165 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fcf60 0x7f72740fd3d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.935+0000 7f727a623700 1 --2- 172.21.15.110:0/2058711165 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72740fbcb0 0x7f72740fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.935+0000 7f727a623700 1 -- 172.21.15.110:0/2058711165 >> 172.21.15.110:0/2058711165 conn(0x7f72740f7430 msgr2=0x7f72740f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.935+0000 7f727a623700 1 -- 172.21.15.110:0/2058711165 shutdown_connections 2024-04-25T15:56:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.935+0000 7f727a623700 1 -- 172.21.15.110:0/2058711165 wait complete. 2024-04-25T15:56:40.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f727a623700 1 Processor -- start 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f727a623700 1 -- start start 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f727a623700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fbcb0 0x7f727410c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f727a623700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72740fcf60 0x7f727410cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f727a623700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727410d3e0 con 0x7f72740fbcb0 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f727a623700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727410dc30 con 0x7f72740fcf60 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f7273fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fbcb0 0x7f727410c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f7273fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fbcb0 0x7f727410c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56354/0 (socket says 172.21.15.110:56354) 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f7273fff700 1 -- 172.21.15.110:0/3489932744 learned_addr learned my addr 172.21.15.110:0/3489932744 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.936+0000 7f72737fe700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72740fcf60 0x7f727410cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f7273fff700 1 -- 172.21.15.110:0/3489932744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72740fcf60 msgr2=0x7f727410cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:40.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f7273fff700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72740fcf60 0x7f727410cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f7273fff700 1 -- 172.21.15.110:0/3489932744 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7268005040 con 0x7f72740fbcb0 2024-04-25T15:56:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f7273fff700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fbcb0 0x7f727410c8a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7264002a80 tx=0x7f72640046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f7270ff9700 1 -- 172.21.15.110:0/3489932744 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7264007cc0 con 0x7f72740fbcb0 2024-04-25T15:56:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f7270ff9700 1 -- 172.21.15.110:0/3489932744 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f726400f040 con 0x7f72740fbcb0 2024-04-25T15:56:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f7270ff9700 1 -- 172.21.15.110:0/3489932744 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f726400dac0 con 0x7f72740fbcb0 2024-04-25T15:56:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727410df10 con 0x7f72740fbcb0 2024-04-25T15:56:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.937+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727410e400 con 0x7f72740fbcb0 2024-04-25T15:56:40.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.938+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727405f5c0 con 0x7f72740fbcb0 2024-04-25T15:56:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.939+0000 7f7270ff9700 1 -- 172.21.15.110:0/3489932744 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f726400dc20 con 0x7f72740fbcb0 2024-04-25T15:56:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.939+0000 7f7270ff9700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f725c076b80 0x7f725c079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.939+0000 7f7270ff9700 1 -- 172.21.15.110:0/3489932744 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7264098530 con 0x7f72740fbcb0 2024-04-25T15:56:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.940+0000 7f72737fe700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f725c076b80 0x7f725c079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:40.941 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.940+0000 7f72737fe700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f725c076b80 0x7f725c079030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f726800b3b0 tx=0x7f7268009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:40.945 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:40.944+0000 7f7270ff9700 1 -- 172.21.15.110:0/3489932744 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f72640610e0 con 0x7f72740fbcb0 2024-04-25T15:56:41.167 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.166+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f727410e7f0 con 0x7f725c076b80 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.176+0000 7f7270ff9700 1 -- 172.21.15.110:0/3489932744 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f727410e7f0 con 0x7f725c076b80 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (7m) 7s ago 15m 25.1M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (3m) 7s ago 15m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (3m) 34s ago 14m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (6m) 7s ago 15m 85.4M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (10m) 7s ago 10m 164M - 17.2.7-902-gc61472c8 0de407c1b3af 3bebf88406b6 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (14s) 7s ago 10m 84.8M - 19.0.0-3291-g994e2f20 d53c96779a29 763d1dda41b7 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (10m) 34s ago 10m 15.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:56:41.177 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (10m) 34s ago 10m 17.5M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (6m) 7s ago 17m 577M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (5m) 34s ago 14m 519M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (5m) 7s ago 17m 118M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (4m) 34s ago 14m 106M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (7m) 7s ago 15m 16.0M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (6m) 34s ago 14m 15.4M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (3m) 7s ago 13m 166M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (2m) 7s ago 13m 168M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (2m) 7s ago 12m 154M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (103s) 34s ago 12m 162M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (70s) 34s ago 12m 152M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (37s) 34s ago 11m 80.0M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 31c4aca48886 2024-04-25T15:56:41.178 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (5m) 7s ago 15m 63.0M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f725c076b80 msgr2=0x7f725c079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f725c076b80 0x7f725c079030 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f726800b3b0 tx=0x7f7268009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fbcb0 msgr2=0x7f727410c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fbcb0 0x7f727410c8a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7264002a80 tx=0x7f72640046b0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 shutdown_connections 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f725c076b80 0x7f725c079030 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72740fbcb0 0x7f727410c8a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 --2- 172.21.15.110:0/3489932744 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72740fcf60 0x7f727410cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 >> 172.21.15.110:0/3489932744 conn(0x7f72740f7430 msgr2=0x7f72740f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 shutdown_connections 2024-04-25T15:56:41.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.179+0000 7f727a623700 1 -- 172.21.15.110:0/3489932744 wait complete. 2024-04-25T15:56:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:40 smithi193 ceph-mon[80071]: Detected new or changed devices on smithi110 2024-04-25T15:56:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:40 smithi193 ceph-mon[80071]: Upgrade: It appears safe to stop mds.cephfs.smithi110.kdhjqx 2024-04-25T15:56:41.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:41.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi110.kdhjqx", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:56:41.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:56:41.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 -- 172.21.15.110:0/3636604442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 msgr2=0x7f12e40fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 --2- 172.21.15.110:0/3636604442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 0x7f12e40fe440 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f12d4004940 tx=0x7f12d4009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:41.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 -- 172.21.15.110:0/3636604442 shutdown_connections 2024-04-25T15:56:41.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 --2- 172.21.15.110:0/3636604442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 0x7f12e40fe440 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 --2- 172.21.15.110:0/3636604442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12e40fce60 0x7f12e40fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 -- 172.21.15.110:0/3636604442 >> 172.21.15.110:0/3636604442 conn(0x7f12e40f8540 msgr2=0x7f12e40fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:41.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 -- 172.21.15.110:0/3636604442 shutdown_connections 2024-04-25T15:56:41.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 -- 172.21.15.110:0/3636604442 wait complete. 2024-04-25T15:56:41.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 Processor -- start 2024-04-25T15:56:41.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.281+0000 7f12ea2c8700 1 -- start start 2024-04-25T15:56:41.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12ea2c8700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12e40fce60 0x7f12e410d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:41.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12ea2c8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 0x7f12e410de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:41.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12ea2c8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e410e400 con 0x7f12e40fdfd0 2024-04-25T15:56:41.284 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12ea2c8700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e410e570 con 0x7f12e40fce60 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 0x7f12e410de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 0x7f12e410de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56374/0 (socket says 172.21.15.110:56374) 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e37fe700 1 -- 172.21.15.110:0/7760730 learned_addr learned my addr 172.21.15.110:0/7760730 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e3fff700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12e40fce60 0x7f12e410d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e37fe700 1 -- 172.21.15.110:0/7760730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12e40fce60 msgr2=0x7f12e410d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e37fe700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12e40fce60 0x7f12e410d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e37fe700 1 -- 172.21.15.110:0/7760730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d4005040 con 0x7f12e40fdfd0 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e37fe700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 0x7f12e410de00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f12d4013870 tx=0x7f12d4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.282+0000 7f12e0ff9700 1 -- 172.21.15.110:0/7760730 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d400e070 con 0x7f12e40fdfd0 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.283+0000 7f12e0ff9700 1 -- 172.21.15.110:0/7760730 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12d4013de0 con 0x7f12e40fdfd0 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.283+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e4112f70 con 0x7f12e40fdfd0 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.283+0000 7f12e0ff9700 1 -- 172.21.15.110:0/7760730 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d401cea0 con 0x7f12e40fdfd0 2024-04-25T15:56:41.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.283+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e4113420 con 0x7f12e40fdfd0 2024-04-25T15:56:41.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.283+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e404c710 con 0x7f12e40fdfd0 2024-04-25T15:56:41.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.284+0000 7f12e0ff9700 1 -- 172.21.15.110:0/7760730 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f12d401c690 con 0x7f12e40fdfd0 2024-04-25T15:56:41.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.285+0000 7f12e0ff9700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f12cc076b80 0x7f12cc079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:41.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.285+0000 7f12e0ff9700 1 -- 172.21.15.110:0/7760730 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f12d4099a20 con 0x7f12e40fdfd0 2024-04-25T15:56:41.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.285+0000 7f12e3fff700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f12cc076b80 0x7f12cc079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:41.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.285+0000 7f12e3fff700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f12cc076b80 0x7f12cc079030 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009b80 tx=0x7f12d800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:41.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.290+0000 7f12e0ff9700 1 -- 172.21.15.110:0/7760730 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f12d40625d0 con 0x7f12e40fdfd0 2024-04-25T15:56:41.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.582+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f12e41058c0 con 0x7f12e40fdfd0 2024-04-25T15:56:41.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.582+0000 7f12e0ff9700 1 -- 172.21.15.110:0/7760730 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7f12d4004490 con 0x7f12e40fdfd0 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 6 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 1 2024-04-25T15:56:41.584 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:56:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:56:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 3, 2024-04-25T15:56:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 11 2024-04-25T15:56:41.585 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:56:41.585 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:56:41.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f12cc076b80 msgr2=0x7f12cc079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f12cc076b80 0x7f12cc079030 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009b80 tx=0x7f12d800b040 comp rx=0 tx=0).stop 2024-04-25T15:56:41.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 msgr2=0x7f12e410de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 0x7f12e410de00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f12d4013870 tx=0x7f12d4009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:41.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 shutdown_connections 2024-04-25T15:56:41.586 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12e40fce60 0x7f12e410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f12cc076b80 0x7f12cc079030 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 --2- 172.21.15.110:0/7760730 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12e40fdfd0 0x7f12e410de00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 >> 172.21.15.110:0/7760730 conn(0x7f12e40f8540 msgr2=0x7f12e40f9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:41.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 shutdown_connections 2024-04-25T15:56:41.587 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.585+0000 7f12ea2c8700 1 -- 172.21.15.110:0/7760730 wait complete. 2024-04-25T15:56:41.688 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.687+0000 7f7e46900700 1 -- 172.21.15.110:0/47633634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fcf90 msgr2=0x7f7e400fd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.687+0000 7f7e46900700 1 --2- 172.21.15.110:0/47633634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fcf90 0x7f7e400fd400 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f7e30004940 tx=0x7f7e30009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.687+0000 7f7e46900700 1 -- 172.21.15.110:0/47633634 shutdown_connections 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.687+0000 7f7e46900700 1 --2- 172.21.15.110:0/47633634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fcf90 0x7f7e400fd400 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.687+0000 7f7e46900700 1 --2- 172.21.15.110:0/47633634 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e400fbce0 0x7f7e400fc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.687+0000 7f7e46900700 1 -- 172.21.15.110:0/47633634 >> 172.21.15.110:0/47633634 conn(0x7f7e400f7420 msgr2=0x7f7e400f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.687+0000 7f7e46900700 1 -- 172.21.15.110:0/47633634 shutdown_connections 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.687+0000 7f7e46900700 1 -- 172.21.15.110:0/47633634 wait complete. 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e46900700 1 Processor -- start 2024-04-25T15:56:41.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e46900700 1 -- start start 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e46900700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fbce0 0x7f7e4010fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e46900700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e400fcf90 0x7f7e401100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e46900700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e401106e0 con 0x7f7e400fbce0 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e46900700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e40110850 con 0x7f7e400fcf90 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e3ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fbce0 0x7f7e4010fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e3ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fbce0 0x7f7e4010fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56390/0 (socket says 172.21.15.110:56390) 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e3ffff700 1 -- 172.21.15.110:0/131673263 learned_addr learned my addr 172.21.15.110:0/131673263 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e3f7fe700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e400fcf90 0x7f7e401100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:41.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e3ffff700 1 -- 172.21.15.110:0/131673263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e400fcf90 msgr2=0x7f7e401100e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e3ffff700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e400fcf90 0x7f7e401100e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.688+0000 7f7e3ffff700 1 -- 172.21.15.110:0/131673263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e30005040 con 0x7f7e400fbce0 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.689+0000 7f7e3f7fe700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e400fcf90 0x7f7e401100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.689+0000 7f7e3ffff700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fbce0 0x7f7e4010fba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400c830 tx=0x7f7e34004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.689+0000 7f7e3cff9700 1 -- 172.21.15.110:0/131673263 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e3400a760 con 0x7f7e400fbce0 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.689+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e40115260 con 0x7f7e400fbce0 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.689+0000 7f7e3cff9700 1 -- 172.21.15.110:0/131673263 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e3400ad90 con 0x7f7e400fbce0 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.689+0000 7f7e3cff9700 1 -- 172.21.15.110:0/131673263 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e340113a0 con 0x7f7e400fbce0 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.689+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e40115820 con 0x7f7e400fbce0 2024-04-25T15:56:41.692 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.690+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e40101920 con 0x7f7e400fbce0 2024-04-25T15:56:41.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.691+0000 7f7e3cff9700 1 -- 172.21.15.110:0/131673263 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f7e34012400 con 0x7f7e400fbce0 2024-04-25T15:56:41.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.691+0000 7f7e3cff9700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7e28076b90 0x7f7e28079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:41.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.691+0000 7f7e3cff9700 1 -- 172.21.15.110:0/131673263 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f7e34098e10 con 0x7f7e400fbce0 2024-04-25T15:56:41.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.691+0000 7f7e3f7fe700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7e28076b90 0x7f7e28079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:41.693 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.692+0000 7f7e3f7fe700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7e28076b90 0x7f7e28079040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000b970 tx=0x7f7e30009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:41.696 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.695+0000 7f7e3cff9700 1 -- 172.21.15.110:0/131673263 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7e34061a60 con 0x7f7e400fbce0 2024-04-25T15:56:41.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.948+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7e401047d0 con 0x7f7e400fbce0 2024-04-25T15:56:41.950 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:41 smithi110 ceph-mon[90991]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:41.950 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:41 smithi110 ceph-mon[90991]: Upgrade: Updating mds.cephfs.smithi110.kdhjqx 2024-04-25T15:56:41.950 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:41 smithi110 ceph-mon[90991]: Deploying daemon mds.cephfs.smithi110.kdhjqx on smithi110 2024-04-25T15:56:41.950 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:41 smithi110 ceph-mon[90991]: pgmap v181: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-25T15:56:41.950 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:41 smithi110 ceph-mon[90991]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:41.950 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:41 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/7760730' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.949+0000 7f7e3cff9700 1 -- 172.21.15.110:0/131673263 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) v1 ==== 76+0+1893 (secure 0 0 0) 0x7f7e34061400 con 0x7f7e400fbce0 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:e19 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 18 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:56:41.951 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:56:27.808558+0000 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 80 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=14478} 2024-04-25T15:56:41.952 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 14478 members: 14478 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{0:14478} state up:active seq 153 join_fscid=1 addr [v2:172.21.15.110:6828/164855346,v1:172.21.15.110:6829/164855346] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:24251} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:24265} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:56:41.953 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:56:41.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7e28076b90 msgr2=0x7f7e28079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7e28076b90 0x7f7e28079040 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7e3000b970 tx=0x7f7e30009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:41.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fbce0 msgr2=0x7f7e4010fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:41.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fbce0 0x7f7e4010fba0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f7e3400c830 tx=0x7f7e34004470 comp rx=0 tx=0).stop 2024-04-25T15:56:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 shutdown_connections 2024-04-25T15:56:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7e28076b90 0x7f7e28079040 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e400fbce0 0x7f7e4010fba0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 --2- 172.21.15.110:0/131673263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e400fcf90 0x7f7e401100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 >> 172.21.15.110:0/131673263 conn(0x7f7e400f7420 msgr2=0x7f7e400667c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 shutdown_connections 2024-04-25T15:56:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:41.951+0000 7f7e46900700 1 -- 172.21.15.110:0/131673263 wait complete. 2024-04-25T15:56:41.955 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 19 2024-04-25T15:56:42.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 -- 172.21.15.110:0/3597859081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c0fff10 msgr2=0x7f362c102330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:42.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 --2- 172.21.15.110:0/3597859081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c0fff10 0x7f362c102330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3618004940 tx=0x7f3618009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:42.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 -- 172.21.15.110:0/3597859081 shutdown_connections 2024-04-25T15:56:42.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 --2- 172.21.15.110:0/3597859081 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c0fff10 0x7f362c102330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 --2- 172.21.15.110:0/3597859081 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f362c066ed0 0x7f362c0ff9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 -- 172.21.15.110:0/3597859081 >> 172.21.15.110:0/3597859081 conn(0x7f362c0f7420 msgr2=0x7f362c0f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:42.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 -- 172.21.15.110:0/3597859081 shutdown_connections 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 -- 172.21.15.110:0/3597859081 wait complete. 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 Processor -- start 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 -- start start 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.055+0000 7f3632d89700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c066ed0 0x7f362c0fd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3632d89700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f362c0fff10 0x7f362c0fb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3632d89700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0fd820 con 0x7f362c066ed0 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3632d89700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362c0fbe10 con 0x7f362c0fff10 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3630b25700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c066ed0 0x7f362c0fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3630b25700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c066ed0 0x7f362c0fd220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56396/0 (socket says 172.21.15.110:56396) 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3630b25700 1 -- 172.21.15.110:0/1765433208 learned_addr learned my addr 172.21.15.110:0/1765433208 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f362bfff700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f362c0fff10 0x7f362c0fb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3630b25700 1 -- 172.21.15.110:0/1765433208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f362c0fff10 msgr2=0x7f362c0fb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3630b25700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f362c0fff10 0x7f362c0fb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3630b25700 1 -- 172.21.15.110:0/1765433208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3618005040 con 0x7f362c066ed0 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3630b25700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c066ed0 0x7f362c0fd220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f362000c890 tx=0x7f36200044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:42.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f36297fa700 1 -- 172.21.15.110:0/1765433208 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f362000a700 con 0x7f362c066ed0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f36297fa700 1 -- 172.21.15.110:0/1765433208 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f362000ad30 con 0x7f362c066ed0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362c0fc0c0 con 0x7f362c066ed0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f36297fa700 1 -- 172.21.15.110:0/1765433208 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36200113a0 con 0x7f362c066ed0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.056+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362c0fc680 con 0x7f362c066ed0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.057+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f362c04c710 con 0x7f362c066ed0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.058+0000 7f36297fa700 1 -- 172.21.15.110:0/1765433208 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f3620012400 con 0x7f362c066ed0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.058+0000 7f36297fa700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f361c076b90 0x7f361c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.059+0000 7f362bfff700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f361c076b90 0x7f361c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.059+0000 7f36297fa700 1 -- 172.21.15.110:0/1765433208 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f3620098480 con 0x7f362c066ed0 2024-04-25T15:56:42.062 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.059+0000 7f362bfff700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f361c076b90 0x7f361c079040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f361800b3b0 tx=0x7f3618009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:42.064 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.063+0000 7f36297fa700 1 -- 172.21.15.110:0/1765433208 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3620060f90 con 0x7f362c066ed0 2024-04-25T15:56:42.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:41 smithi193 ceph-mon[80071]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:42.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:41 smithi193 ceph-mon[80071]: Upgrade: Updating mds.cephfs.smithi110.kdhjqx 2024-04-25T15:56:42.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:41 smithi193 ceph-mon[80071]: Deploying daemon mds.cephfs.smithi110.kdhjqx on smithi110 2024-04-25T15:56:42.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:41 smithi193 ceph-mon[80071]: pgmap v181: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.4 KiB/s rd, 4.9 KiB/s wr, 5 op/s 2024-04-25T15:56:42.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:41 smithi193 ceph-mon[80071]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:42.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:41 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/7760730' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:56:42.290 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.289+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f362c0f8f80 con 0x7f361c076b90 2024-04-25T15:56:42.291 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.291+0000 7f36297fa700 1 -- 172.21.15.110:0/1765433208 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f362c0f8f80 con 0x7f361c076b90 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "13/21 daemons upgraded", 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:56:42.292 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f361c076b90 msgr2=0x7f361c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f361c076b90 0x7f361c079040 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f361800b3b0 tx=0x7f3618009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c066ed0 msgr2=0x7f362c0fd220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c066ed0 0x7f362c0fd220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f362000c890 tx=0x7f36200044c0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 shutdown_connections 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f361c076b90 0x7f361c079040 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f362c066ed0 0x7f362c0fd220 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 --2- 172.21.15.110:0/1765433208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f362c0fff10 0x7f362c0fb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.293+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 >> 172.21.15.110:0/1765433208 conn(0x7f362c0f7420 msgr2=0x7f362c0f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.294+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 shutdown_connections 2024-04-25T15:56:42.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.294+0000 7f3632d89700 1 -- 172.21.15.110:0/1765433208 wait complete. 2024-04-25T15:56:42.405 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.403+0000 7f22dc732700 1 -- 172.21.15.110:0/2125127121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 msgr2=0x7f22d40fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.403+0000 7f22dc732700 1 --2- 172.21.15.110:0/2125127121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 0x7f22d40fc0c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f22d0004940 tx=0x7f22d0009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.403+0000 7f22dc732700 1 -- 172.21.15.110:0/2125127121 shutdown_connections 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.403+0000 7f22dc732700 1 --2- 172.21.15.110:0/2125127121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d40fcf60 0x7f22d40fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.403+0000 7f22dc732700 1 --2- 172.21.15.110:0/2125127121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 0x7f22d40fc0c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.403+0000 7f22dc732700 1 -- 172.21.15.110:0/2125127121 >> 172.21.15.110:0/2125127121 conn(0x7f22d40f7430 msgr2=0x7f22d40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.403+0000 7f22dc732700 1 -- 172.21.15.110:0/2125127121 shutdown_connections 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.403+0000 7f22dc732700 1 -- 172.21.15.110:0/2125127121 wait complete. 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22dc732700 1 Processor -- start 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22dc732700 1 -- start start 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22dc732700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 0x7f22d410d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:42.406 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22dc732700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d40fcf60 0x7f22d410ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22dc732700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d410e4d0 con 0x7f22d40fbcb0 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22dc732700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d410ed20 con 0x7f22d40fcf60 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22da4ce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 0x7f22d410d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22da4ce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 0x7f22d410d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:56414/0 (socket says 172.21.15.110:56414) 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22da4ce700 1 -- 172.21.15.110:0/3033439079 learned_addr learned my addr 172.21.15.110:0/3033439079 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22d9ccd700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d40fcf60 0x7f22d410ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22da4ce700 1 -- 172.21.15.110:0/3033439079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d40fcf60 msgr2=0x7f22d410ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22da4ce700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d40fcf60 0x7f22d410ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22da4ce700 1 -- 172.21.15.110:0/3033439079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0005040 con 0x7f22d40fbcb0 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.404+0000 7f22da4ce700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 0x7f22d410d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22d0013870 tx=0x7f22d0009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.405+0000 7f22caffd700 1 -- 172.21.15.110:0/3033439079 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22d0021070 con 0x7f22d40fbcb0 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.405+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d410efa0 con 0x7f22d40fbcb0 2024-04-25T15:56:42.407 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.405+0000 7f22caffd700 1 -- 172.21.15.110:0/3033439079 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f22d0009430 con 0x7f22d40fbcb0 2024-04-25T15:56:42.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.405+0000 7f22caffd700 1 -- 172.21.15.110:0/3033439079 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22d000e070 con 0x7f22d40fbcb0 2024-04-25T15:56:42.408 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.405+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d410f4e0 con 0x7f22d40fbcb0 2024-04-25T15:56:42.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.406+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d405f5c0 con 0x7f22d40fbcb0 2024-04-25T15:56:42.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.407+0000 7f22caffd700 1 -- 172.21.15.110:0/3033439079 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f22d001c750 con 0x7f22d40fbcb0 2024-04-25T15:56:42.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.407+0000 7f22caffd700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f22c007af20 0x7f22c007d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:56:42.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.407+0000 7f22caffd700 1 -- 172.21.15.110:0/3033439079 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6410+0+0 (secure 0 0 0) 0x7f22d0099460 con 0x7f22d40fbcb0 2024-04-25T15:56:42.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.407+0000 7f22d9ccd700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f22c007af20 0x7f22c007d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:56:42.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.408+0000 7f22d9ccd700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f22c007af20 0x7f22c007d3d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009b80 tx=0x7f22c400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:56:42.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.412+0000 7f22caffd700 1 -- 172.21.15.110:0/3033439079 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f22d0062010 con 0x7f22d40fbcb0 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.736+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f22d404bed0 con 0x7f22d40fbcb0 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.737+0000 7f22caffd700 1 -- 172.21.15.110:0/3033439079 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f22d00619b0 con 0x7f22d40fbcb0 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs is degraded 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-25T15:56:42.739 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-25T15:56:42.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f22c007af20 msgr2=0x7f22c007d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:42.740 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f22c007af20 0x7f22c007d3d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f22c4009b80 tx=0x7f22c400b040 comp rx=0 tx=0).stop 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 msgr2=0x7f22d410d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 0x7f22d410d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f22d0013870 tx=0x7f22d0009e30 comp rx=0 tx=0).stop 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 shutdown_connections 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f22c007af20 0x7f22c007d3d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22d40fbcb0 0x7f22d410d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 --2- 172.21.15.110:0/3033439079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22d40fcf60 0x7f22d410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 >> 172.21.15.110:0/3033439079 conn(0x7f22d40f7430 msgr2=0x7f22d4065b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 shutdown_connections 2024-04-25T15:56:42.741 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:56:42.739+0000 7f22dc732700 1 -- 172.21.15.110:0/3033439079 wait complete. 2024-04-25T15:56:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[90987]: 2024-04-25T15:56:42.731+0000 7effa00b9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:56:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/131673263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:56:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:56:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:56:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: osdmap e81: 6 total, 6 up, 6 in 2024-04-25T15:56:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: Standby daemon mds.cephfs.smithi193.ewxwfw assigned to filesystem cephfs as rank 0 2024-04-25T15:56:42.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:56:42.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-25T15:56:42.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3033439079' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:56:42.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:42 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi193.ewxwfw=up:replay} 2 up:standby 2024-04-25T15:56:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: from='client.34416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/131673263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:56:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:56:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:56:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: osdmap e81: 6 total, 6 up, 6 in 2024-04-25T15:56:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: Standby daemon mds.cephfs.smithi193.ewxwfw assigned to filesystem cephfs as rank 0 2024-04-25T15:56:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:56:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-25T15:56:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3033439079' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:56:43.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:42 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi193.ewxwfw=up:replay} 2 up:standby 2024-04-25T15:56:44.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:43 smithi110 ceph-mon[90991]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:44.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:43 smithi110 ceph-mon[90991]: pgmap v183: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 204 B/s wr, 1 op/s 2024-04-25T15:56:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:43 smithi193 ceph-mon[80071]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:56:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:43 smithi193 ceph-mon[80071]: pgmap v183: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 204 B/s wr, 1 op/s 2024-04-25T15:56:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:45 smithi110 ceph-mon[90991]: pgmap v184: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 410 KiB/s rd, 0 op/s 2024-04-25T15:56:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:45 smithi193 ceph-mon[80071]: pgmap v184: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 410 KiB/s rd, 0 op/s 2024-04-25T15:56:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:47 smithi110 ceph-mon[90991]: pgmap v185: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-04-25T15:56:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:56:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:47 smithi193 ceph-mon[80071]: pgmap v185: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-04-25T15:56:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:56:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:49 smithi110 ceph-mon[90991]: pgmap v186: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-25T15:56:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:49 smithi193 ceph-mon[80071]: pgmap v186: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-25T15:56:52.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:51 smithi110 ceph-mon[90991]: pgmap v187: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-25T15:56:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:51 smithi193 ceph-mon[80071]: pgmap v187: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-25T15:56:54.045 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:53 smithi110 ceph-mon[90991]: pgmap v188: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-25T15:56:54.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:53 smithi110 ceph-mon[90991]: reconnect by client.24297 192.168.0.1:0/4279727202 after 0.000999985 2024-04-25T15:56:54.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:53 smithi110 ceph-mon[90991]: reconnect by client.14526 192.168.0.1:0/2988371434 after 0.000999985 2024-04-25T15:56:54.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:53 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:reconnect 2024-04-25T15:56:54.046 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:53 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi193.ewxwfw=up:reconnect} 2 up:standby 2024-04-25T15:56:54.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:53 smithi193 ceph-mon[80071]: pgmap v188: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-25T15:56:54.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:53 smithi193 ceph-mon[80071]: reconnect by client.24297 192.168.0.1:0/4279727202 after 0.000999985 2024-04-25T15:56:54.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:53 smithi193 ceph-mon[80071]: reconnect by client.14526 192.168.0.1:0/2988371434 after 0.000999985 2024-04-25T15:56:54.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:53 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:reconnect 2024-04-25T15:56:54.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:53 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi193.ewxwfw=up:reconnect} 2 up:standby 2024-04-25T15:56:55.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:54 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:rejoin 2024-04-25T15:56:55.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:54 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi193.ewxwfw=up:rejoin} 2 up:standby 2024-04-25T15:56:55.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:54 smithi110 ceph-mon[90991]: daemon mds.cephfs.smithi193.ewxwfw is now active in filesystem cephfs as rank 0 2024-04-25T15:56:55.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:55.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:55.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:54 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:55.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:54 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:rejoin 2024-04-25T15:56:55.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:54 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi193.ewxwfw=up:rejoin} 2 up:standby 2024-04-25T15:56:55.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:54 smithi193 ceph-mon[80071]: daemon mds.cephfs.smithi193.ewxwfw is now active in filesystem cephfs as rank 0 2024-04-25T15:56:55.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:54 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:55.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:54 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:56:55.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:54 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:56:56.069 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:55 smithi110 ceph-mon[90991]: pgmap v189: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-25T15:56:56.069 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:55 smithi110 ceph-mon[90991]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:56:56.070 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:55 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:active 2024-04-25T15:56:56.070 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:55 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.110:6828/1518950103,v1:172.21.15.110:6829/1518950103] up:boot 2024-04-25T15:56:56.070 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:55 smithi110 ceph-mon[90991]: fsmap cephfs:1 {0=cephfs.smithi193.ewxwfw=up:active} 3 up:standby 2024-04-25T15:56:56.070 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:56:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:55 smithi193 ceph-mon[80071]: pgmap v189: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-25T15:56:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:55 smithi193 ceph-mon[80071]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:56:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:55 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.193:6824/746795019,v1:172.21.15.193:6825/746795019] up:active 2024-04-25T15:56:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:55 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.110:6828/1518950103,v1:172.21.15.110:6829/1518950103] up:boot 2024-04-25T15:56:56.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:55 smithi193 ceph-mon[80071]: fsmap cephfs:1 {0=cephfs.smithi193.ewxwfw=up:active} 3 up:standby 2024-04-25T15:56:56.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:55 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:56:57.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:57 smithi193 ceph-mon[80071]: pgmap v190: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T15:56:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:57 smithi110 ceph-mon[90991]: pgmap v190: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T15:57:00.060 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:59 smithi193 ceph-mon[80071]: pgmap v191: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T15:57:00.060 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:00.060 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:56:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:59 smithi110 ceph-mon[90991]: pgmap v191: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T15:57:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:56:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:01 smithi193 ceph-mon[80071]: pgmap v192: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T15:57:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:01 smithi110 ceph-mon[90991]: pgmap v192: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 9.6 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T15:57:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:02.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:57:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:57:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: pgmap v193: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-25T15:57:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:57:04.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:04.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:04.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:04.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:04.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:04.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:04.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi193.ewxwfw"]}]: dispatch 2024-04-25T15:57:04.220 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: pgmap v193: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-25T15:57:04.220 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:04.220 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:04.220 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:04.220 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:57:04.220 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:04.221 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:04.221 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:04.221 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:04.221 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:04.221 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:04.221 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi193.ewxwfw"]}]: dispatch 2024-04-25T15:57:04.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:04 smithi193 ceph-mon[80071]: Upgrade: It appears safe to stop mds.cephfs.smithi193.ewxwfw 2024-04-25T15:57:05.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:04 smithi110 ceph-mon[90991]: Upgrade: It appears safe to stop mds.cephfs.smithi193.ewxwfw 2024-04-25T15:57:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:05 smithi110 ceph-mon[90991]: pgmap v194: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ewxwfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:57:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:06.118 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:05 smithi193 ceph-mon[80071]: pgmap v194: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:06.118 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:06.118 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.ewxwfw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:57:06.119 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:06.881 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:06 smithi193 ceph-mon[80071]: Upgrade: Updating mds.cephfs.smithi193.ewxwfw 2024-04-25T15:57:06.881 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:06 smithi193 ceph-mon[80071]: Deploying daemon mds.cephfs.smithi193.ewxwfw on smithi193 2024-04-25T15:57:07.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:06 smithi110 ceph-mon[90991]: Upgrade: Updating mds.cephfs.smithi193.ewxwfw 2024-04-25T15:57:07.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:06 smithi110 ceph-mon[90991]: Deploying daemon mds.cephfs.smithi193.ewxwfw on smithi193 2024-04-25T15:57:07.810 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[90987]: 2024-04-25T15:57:07.401+0000 7effa00b9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:57:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: pgmap v195: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 326 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:57:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:57:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: osdmap e82: 6 total, 6 up, 6 in 2024-04-25T15:57:08.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: Standby daemon mds.cephfs.smithi193.jtyzhl assigned to filesystem cephfs as rank 0 2024-04-25T15:57:08.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:57:08.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-25T15:57:08.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi193.jtyzhl=up:replay} 2 up:standby 2024-04-25T15:57:08.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: pgmap v195: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 326 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: osdmap e82: 6 total, 6 up, 6 in 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: Standby daemon mds.cephfs.smithi193.jtyzhl assigned to filesystem cephfs as rank 0 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi193.jtyzhl=up:replay} 2 up:standby 2024-04-25T15:57:08.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:57:10.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:09 smithi110 ceph-mon[90991]: pgmap v197: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 409 KiB/s rd, 4.9 KiB/s wr, 2 op/s 2024-04-25T15:57:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:09 smithi193 ceph-mon[80071]: pgmap v197: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 409 KiB/s rd, 4.9 KiB/s wr, 2 op/s 2024-04-25T15:57:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:11 smithi110 ceph-mon[90991]: pgmap v198: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 8.5 MiB/s rd, 2 op/s 2024-04-25T15:57:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:11 smithi193 ceph-mon[80071]: pgmap v198: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 8.5 MiB/s rd, 2 op/s 2024-04-25T15:57:12.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.864+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3956920327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fcf60 msgr2=0x7f13c40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:12.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.864+0000 7f13c8cc3700 1 --2- 172.21.15.110:0/3956920327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fcf60 0x7f13c40fd3d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f13b8004940 tx=0x7f13b8009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:12.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.864+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3956920327 shutdown_connections 2024-04-25T15:57:12.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.864+0000 7f13c8cc3700 1 --2- 172.21.15.110:0/3956920327 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fcf60 0x7f13c40fd3d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:12.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.864+0000 7f13c8cc3700 1 --2- 172.21.15.110:0/3956920327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13c40fbcb0 0x7f13c40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.864+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3956920327 >> 172.21.15.110:0/3956920327 conn(0x7f13c40f7430 msgr2=0x7f13c40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.864+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3956920327 shutdown_connections 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.864+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3956920327 wait complete. 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c8cc3700 1 Processor -- start 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c8cc3700 1 -- start start 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c8cc3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fbcb0 0x7f13c41106c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c8cc3700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13c40fcf60 0x7f13c4112c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c8cc3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4066330 con 0x7f13c40fbcb0 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c8cc3700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13c4066470 con 0x7f13c40fcf60 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fbcb0 0x7f13c41106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c1d9b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13c40fcf60 0x7f13c4112c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c259c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fbcb0 0x7f13c41106c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38608/0 (socket says 172.21.15.110:38608) 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c259c700 1 -- 172.21.15.110:0/3206678599 learned_addr learned my addr 172.21.15.110:0/3206678599 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:12.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c1d9b700 1 -- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fbcb0 msgr2=0x7f13c41106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c1d9b700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fbcb0 0x7f13c41106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c1d9b700 1 -- 172.21.15.110:0/3206678599 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b8005040 con 0x7f13c40fcf60 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.865+0000 7f13c259c700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fbcb0 0x7f13c41106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.866+0000 7f13c1d9b700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13c40fcf60 0x7f13c4112c10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13b8000c00 tx=0x7f13b8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.866+0000 7f13b2ffd700 1 -- 172.21.15.110:0/3206678599 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13b800e030 con 0x7f13c40fcf60 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.866+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13c4113150 con 0x7f13c40fcf60 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.866+0000 7f13b2ffd700 1 -- 172.21.15.110:0/3206678599 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f13b801c400 con 0x7f13c40fcf60 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.866+0000 7f13b2ffd700 1 -- 172.21.15.110:0/3206678599 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13b800b650 con 0x7f13c40fcf60 2024-04-25T15:57:12.869 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.866+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13c4113690 con 0x7f13c40fcf60 2024-04-25T15:57:12.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.867+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c405f5c0 con 0x7f13c40fcf60 2024-04-25T15:57:12.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.868+0000 7f13b2ffd700 1 -- 172.21.15.110:0/3206678599 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f13b801ca90 con 0x7f13c40fcf60 2024-04-25T15:57:12.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.868+0000 7f13b2ffd700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13ac076b90 0x7f13ac079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:12.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.868+0000 7f13b2ffd700 1 -- 172.21.15.110:0/3206678599 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f13b8099fe0 con 0x7f13c40fcf60 2024-04-25T15:57:12.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.868+0000 7f13c259c700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13ac076b90 0x7f13ac079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:12.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.869+0000 7f13c259c700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13ac076b90 0x7f13ac079040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13c4065fb0 tx=0x7f13b400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:12.878 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:12.877+0000 7f13b2ffd700 1 -- 172.21.15.110:0/3206678599 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f13b8062bc0 con 0x7f13c40fcf60 2024-04-25T15:57:13.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.097+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f13c4101230 con 0x7f13ac076b90 2024-04-25T15:57:13.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.099+0000 7f13b2ffd700 1 -- 172.21.15.110:0/3206678599 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f13c4101230 con 0x7f13ac076b90 2024-04-25T15:57:13.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13ac076b90 msgr2=0x7f13ac079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13ac076b90 0x7f13ac079040 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13c4065fb0 tx=0x7f13b400b040 comp rx=0 tx=0).stop 2024-04-25T15:57:13.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13c40fcf60 msgr2=0x7f13c4112c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13c40fcf60 0x7f13c4112c10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13b8000c00 tx=0x7f13b8009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:13.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 shutdown_connections 2024-04-25T15:57:13.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f13ac076b90 0x7f13ac079040 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13c40fbcb0 0x7f13c41106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 --2- 172.21.15.110:0/3206678599 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13c40fcf60 0x7f13c4112c10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 >> 172.21.15.110:0/3206678599 conn(0x7f13c40f7430 msgr2=0x7f13c40f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:13.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 shutdown_connections 2024-04-25T15:57:13.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.101+0000 7f13c8cc3700 1 -- 172.21.15.110:0/3206678599 wait complete. 2024-04-25T15:57:13.112 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:57:13.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.203+0000 7fa0c9a69700 1 -- 172.21.15.110:0/2954685525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 msgr2=0x7fa0c40faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.203+0000 7fa0c9a69700 1 --2- 172.21.15.110:0/2954685525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 0x7fa0c40faea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa0b8004940 tx=0x7fa0b8009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:13.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.203+0000 7fa0c9a69700 1 -- 172.21.15.110:0/2954685525 shutdown_connections 2024-04-25T15:57:13.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.203+0000 7fa0c9a69700 1 --2- 172.21.15.110:0/2954685525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 0x7fa0c40faea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.203+0000 7fa0c9a69700 1 --2- 172.21.15.110:0/2954685525 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0c40fe410 0x7fa0c40fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.203+0000 7fa0c9a69700 1 -- 172.21.15.110:0/2954685525 >> 172.21.15.110:0/2954685525 conn(0x7fa0c40f6bd0 msgr2=0x7fa0c40f8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:13.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.203+0000 7fa0c9a69700 1 -- 172.21.15.110:0/2954685525 shutdown_connections 2024-04-25T15:57:13.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.203+0000 7fa0c9a69700 1 -- 172.21.15.110:0/2954685525 wait complete. 2024-04-25T15:57:13.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.204+0000 7fa0c9a69700 1 Processor -- start 2024-04-25T15:57:13.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.204+0000 7fa0c9a69700 1 -- start start 2024-04-25T15:57:13.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c9a69700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0c40fe410 0x7fa0c410d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c9a69700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 0x7fa0c410dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c9a69700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c410e3a0 con 0x7fa0c40fed60 2024-04-25T15:57:13.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c9a69700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c410e510 con 0x7fa0c40fe410 2024-04-25T15:57:13.207 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 0x7fa0c410dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 0x7fa0c410dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38634/0 (socket says 172.21.15.110:38634) 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c2ffd700 1 -- 172.21.15.110:0/1794888186 learned_addr learned my addr 172.21.15.110:0/1794888186 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c37fe700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0c40fe410 0x7fa0c410d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c37fe700 1 -- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 msgr2=0x7fa0c410dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c37fe700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 0x7fa0c410dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c37fe700 1 -- 172.21.15.110:0/1794888186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b8005040 con 0x7fa0c40fe410 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c2ffd700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 0x7fa0c410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0c37fe700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0c40fe410 0x7fa0c410d860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009fd0 tx=0x7fa0b400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.205+0000 7fa0abfff700 1 -- 172.21.15.110:0/1794888186 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0b4004e90 con 0x7fa0c40fe410 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.206+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4112f10 con 0x7fa0c40fe410 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.206+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c41133c0 con 0x7fa0c40fe410 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.206+0000 7fa0abfff700 1 -- 172.21.15.110:0/1794888186 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0b400f040 con 0x7fa0c40fe410 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.206+0000 7fa0abfff700 1 -- 172.21.15.110:0/1794888186 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0b400ca50 con 0x7fa0c40fe410 2024-04-25T15:57:13.208 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.206+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0b00052e0 con 0x7fa0c40fe410 2024-04-25T15:57:13.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.208+0000 7fa0abfff700 1 -- 172.21.15.110:0/1794888186 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fa0b4007cc0 con 0x7fa0c40fe410 2024-04-25T15:57:13.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.208+0000 7fa0abfff700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0ac076b90 0x7fa0ac079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.208+0000 7fa0abfff700 1 -- 172.21.15.110:0/1794888186 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fa0b4097de0 con 0x7fa0c40fe410 2024-04-25T15:57:13.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.208+0000 7fa0c2ffd700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0ac076b90 0x7fa0ac079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.209+0000 7fa0c2ffd700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0ac076b90 0x7fa0ac079040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b0d0 tx=0x7fa0b8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:13.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.217+0000 7fa0abfff700 1 -- 172.21.15.110:0/1794888186 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa0b4060a30 con 0x7fa0c40fe410 2024-04-25T15:57:13.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.434+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0b0000bf0 con 0x7fa0ac076b90 2024-04-25T15:57:13.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.436+0000 7fa0abfff700 1 -- 172.21.15.110:0/1794888186 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa0b0000bf0 con 0x7fa0ac076b90 2024-04-25T15:57:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0ac076b90 msgr2=0x7fa0ac079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0ac076b90 0x7fa0ac079040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800b0d0 tx=0x7fa0b8009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0c40fe410 msgr2=0x7fa0c410d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0c40fe410 0x7fa0c410d860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009fd0 tx=0x7fa0b400b040 comp rx=0 tx=0).stop 2024-04-25T15:57:13.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 shutdown_connections 2024-04-25T15:57:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa0c40fe410 0x7fa0c410d860 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0ac076b90 0x7fa0ac079040 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 --2- 172.21.15.110:0/1794888186 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa0c40fed60 0x7fa0c410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 >> 172.21.15.110:0/1794888186 conn(0x7fa0c40f6bd0 msgr2=0x7fa0c40f8170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 shutdown_connections 2024-04-25T15:57:13.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.438+0000 7fa0c9a69700 1 -- 172.21.15.110:0/1794888186 wait complete. 2024-04-25T15:57:13.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 -- 172.21.15.110:0/893066366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fe3e0 msgr2=0x7f7f640fe7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 --2- 172.21.15.110:0/893066366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fe3e0 0x7f7f640fe7f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c004940 tx=0x7f7f5c009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 -- 172.21.15.110:0/893066366 shutdown_connections 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 --2- 172.21.15.110:0/893066366 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f640fed30 0x7f7f640fae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 --2- 172.21.15.110:0/893066366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fe3e0 0x7f7f640fe7f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 -- 172.21.15.110:0/893066366 >> 172.21.15.110:0/893066366 conn(0x7f7f640f6b80 msgr2=0x7f7f640f8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 -- 172.21.15.110:0/893066366 shutdown_connections 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 -- 172.21.15.110:0/893066366 wait complete. 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 Processor -- start 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.548+0000 7f7f69dd9700 1 -- start start 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f69dd9700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f640fe3e0 0x7f7f6410d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f69dd9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fed30 0x7f7f6410deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f69dd9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6410e4b0 con 0x7f7f640fed30 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f69dd9700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6410ed00 con 0x7f7f640fe3e0 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f62ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fed30 0x7f7f6410deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.551 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f62ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fed30 0x7f7f6410deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38648/0 (socket says 172.21.15.110:38648) 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f62ffd700 1 -- 172.21.15.110:0/1480358268 learned_addr learned my addr 172.21.15.110:0/1480358268 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f637fe700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f640fe3e0 0x7f7f6410d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f62ffd700 1 -- 172.21.15.110:0/1480358268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f640fe3e0 msgr2=0x7f7f6410d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f62ffd700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f640fe3e0 0x7f7f6410d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f62ffd700 1 -- 172.21.15.110:0/1480358268 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c005040 con 0x7f7f640fed30 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f62ffd700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fed30 0x7f7f6410deb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f58001db0 tx=0x7f7f58009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f4bfff700 1 -- 172.21.15.110:0/1480358268 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f5800ed50 con 0x7f7f640fed30 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f4bfff700 1 -- 172.21.15.110:0/1480358268 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f58007df0 con 0x7f7f640fed30 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6410efe0 con 0x7f7f640fed30 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f4bfff700 1 -- 172.21.15.110:0/1480358268 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f58014640 con 0x7f7f640fed30 2024-04-25T15:57:13.552 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.549+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6410f520 con 0x7f7f640fed30 2024-04-25T15:57:13.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.550+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6405f5c0 con 0x7f7f640fed30 2024-04-25T15:57:13.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.551+0000 7f7f4bfff700 1 -- 172.21.15.110:0/1480358268 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f7f58007920 con 0x7f7f640fed30 2024-04-25T15:57:13.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.551+0000 7f7f4bfff700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7f4c076be0 0x7f7f4c079090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.552+0000 7f7f637fe700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7f4c076be0 0x7f7f4c079090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.553 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.552+0000 7f7f4bfff700 1 -- 172.21.15.110:0/1480358268 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f7f58098100 con 0x7f7f640fed30 2024-04-25T15:57:13.554 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.552+0000 7f7f637fe700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7f4c076be0 0x7f7f4c079090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00b3b0 tx=0x7f7f5c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:13.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.557+0000 7f7f4bfff700 1 -- 172.21.15.110:0/1480358268 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7f58060c30 con 0x7f7f640fed30 2024-04-25T15:57:13.766 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.764+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f6410f970 con 0x7f7f4c076be0 2024-04-25T15:57:13.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.775+0000 7f7f4bfff700 1 -- 172.21.15.110:0/1480358268 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f7f6410f970 con 0x7f7f4c076be0 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (7m) 14s ago 16m 25.1M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (4m) 14s ago 16m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (4m) 67s ago 15m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (7m) 14s ago 15m 85.4M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (19s) 14s ago 10m 82.5M - 19.0.0-3291-g994e2f20 d53c96779a29 051262111b6b 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (47s) 14s ago 10m 85.0M - 19.0.0-3291-g994e2f20 d53c96779a29 763d1dda41b7 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (10m) 67s ago 10m 15.0M - 17.2.7-902-gc61472c8 0de407c1b3af 1086d23d9e85 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (10m) 67s ago 10m 17.5M - 17.2.7-902-gc61472c8 0de407c1b3af 6ef51522d4b8 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (7m) 14s ago 17m 578M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (5m) 67s ago 15m 519M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (5m) 14s ago 17m 119M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:57:13.777 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (5m) 67s ago 15m 106M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (8m) 14s ago 16m 16.0M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 67s ago 15m 15.4M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (4m) 14s ago 14m 166M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (3m) 14s ago 13m 169M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (2m) 14s ago 13m 155M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (2m) 67s ago 13m 162M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (103s) 67s ago 12m 152M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (70s) 67s ago 12m 80.0M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 31c4aca48886 2024-04-25T15:57:13.778 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (5m) 14s ago 15m 63.0M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:57:13.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.777+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7f4c076be0 msgr2=0x7f7f4c079090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.777+0000 7f7f69dd9700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7f4c076be0 0x7f7f4c079090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00b3b0 tx=0x7f7f5c009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:13.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.777+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fed30 msgr2=0x7f7f6410deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.777+0000 7f7f69dd9700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fed30 0x7f7f6410deb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7f58001db0 tx=0x7f7f58009ee0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.778+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 shutdown_connections 2024-04-25T15:57:13.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.778+0000 7f7f69dd9700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f640fe3e0 0x7f7f6410d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.778+0000 7f7f69dd9700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7f4c076be0 0x7f7f4c079090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.778+0000 7f7f69dd9700 1 --2- 172.21.15.110:0/1480358268 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7f640fed30 0x7f7f6410deb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.778+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 >> 172.21.15.110:0/1480358268 conn(0x7f7f640f6b80 msgr2=0x7f7f640f81f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:13.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.778+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 shutdown_connections 2024-04-25T15:57:13.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.778+0000 7f7f69dd9700 1 -- 172.21.15.110:0/1480358268 wait complete. 2024-04-25T15:57:13.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.880+0000 7ff901049700 1 -- 172.21.15.110:0/1664678396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 msgr2=0x7ff8fc070c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.880+0000 7ff901049700 1 --2- 172.21.15.110:0/1664678396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 0x7ff8fc070c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0004940 tx=0x7ff8f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:13.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.880+0000 7ff901049700 1 -- 172.21.15.110:0/1664678396 shutdown_connections 2024-04-25T15:57:13.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.880+0000 7ff901049700 1 --2- 172.21.15.110:0/1664678396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 0x7ff8fc070c80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.883 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.880+0000 7ff901049700 1 --2- 172.21.15.110:0/1664678396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8fc071dc0 0x7ff8fc0702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.880+0000 7ff901049700 1 -- 172.21.15.110:0/1664678396 >> 172.21.15.110:0/1664678396 conn(0x7ff8fc0f8520 msgr2=0x7ff8fc0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff901049700 1 -- 172.21.15.110:0/1664678396 shutdown_connections 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff901049700 1 -- 172.21.15.110:0/1664678396 wait complete. 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff901049700 1 Processor -- start 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff901049700 1 -- start start 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff901049700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 0x7ff8fc10fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff901049700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8fc071dc0 0x7ff8fc10ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff901049700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc1105c0 con 0x7ff8fc070810 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff901049700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8fc110730 con 0x7ff8fc071dc0 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff8fad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 0x7ff8fc10fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff8fad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 0x7ff8fc10fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38660/0 (socket says 172.21.15.110:38660) 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.881+0000 7ff8fad9d700 1 -- 172.21.15.110:0/2635089530 learned_addr learned my addr 172.21.15.110:0/2635089530 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8fa59c700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8fc071dc0 0x7ff8fc10ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8fa59c700 1 -- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 msgr2=0x7ff8fc10fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8fa59c700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 0x7ff8fc10fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8fa59c700 1 -- 172.21.15.110:0/2635089530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f0005040 con 0x7ff8fc071dc0 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8fad9d700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 0x7ff8fc10fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8fa59c700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8fc071dc0 0x7ff8fc10ffc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0000c00 tx=0x7ff8f0009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8e37fe700 1 -- 172.21.15.110:0/2635089530 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8f000e030 con 0x7ff8fc071dc0 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8fc115130 con 0x7ff8fc071dc0 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8fc115670 con 0x7ff8fc071dc0 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8e37fe700 1 -- 172.21.15.110:0/2635089530 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8f001c400 con 0x7ff8fc071dc0 2024-04-25T15:57:13.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.882+0000 7ff8e37fe700 1 -- 172.21.15.110:0/2635089530 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8f000b650 con 0x7ff8fc071dc0 2024-04-25T15:57:13.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.883+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8fc04c710 con 0x7ff8fc071dc0 2024-04-25T15:57:13.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.884+0000 7ff8e37fe700 1 -- 172.21.15.110:0/2635089530 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7ff8f001ca20 con 0x7ff8fc071dc0 2024-04-25T15:57:13.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.884+0000 7ff8e37fe700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff8e4076b90 0x7ff8e4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:13.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.885+0000 7ff8e37fe700 1 -- 172.21.15.110:0/2635089530 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7ff8f0099f80 con 0x7ff8fc071dc0 2024-04-25T15:57:13.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.885+0000 7ff8fad9d700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff8e4076b90 0x7ff8e4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:13.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.885+0000 7ff8fad9d700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff8e4076b90 0x7ff8e4079040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00f6a0 tx=0x7ff8ec00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:13.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:13.893+0000 7ff8e37fe700 1 -- 172.21.15.110:0/2635089530 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7ff8f0062be0 con 0x7ff8fc071dc0 2024-04-25T15:57:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:13 smithi110 ceph-mon[90991]: pgmap v199: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-25T15:57:14.179 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.178+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff8fc115b30 con 0x7ff8fc071dc0 2024-04-25T15:57:14.180 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.179+0000 7ff8e37fe700 1 -- 172.21.15.110:0/2635089530 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+837 (secure 0 0 0) 0x7ff8f0062580 con 0x7ff8fc071dc0 2024-04-25T15:57:14.180 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:57:14.180 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:57:14.180 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:57:14.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:57:14.180 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:57:14.180 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:57:14.180 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 6 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 17.2.7-902-gc61472c8 (c61472c83ce76869b0fe076c19ed2f4bcc3fb4af) quincy (stable)": 1, 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 12 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:57:14.181 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:57:14.182 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff8e4076b90 msgr2=0x7ff8e4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff8e4076b90 0x7ff8e4079040 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00f6a0 tx=0x7ff8ec00b040 comp rx=0 tx=0).stop 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8fc071dc0 msgr2=0x7ff8fc10ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8fc071dc0 0x7ff8fc10ffc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0000c00 tx=0x7ff8f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 shutdown_connections 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7ff8e4076b90 0x7ff8e4079040 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8fc070810 0x7ff8fc10fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 --2- 172.21.15.110:0/2635089530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8fc071dc0 0x7ff8fc10ffc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 >> 172.21.15.110:0/2635089530 conn(0x7ff8fc0f8520 msgr2=0x7ff8fc0f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 shutdown_connections 2024-04-25T15:57:14.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.181+0000 7ff901049700 1 -- 172.21.15.110:0/2635089530 wait complete. 2024-04-25T15:57:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:13 smithi193 ceph-mon[80071]: pgmap v199: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4 op/s 2024-04-25T15:57:14.285 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3716259128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 msgr2=0x7f2d480fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 --2- 172.21.15.110:0/3716259128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 0x7f2d480fe440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d38004940 tx=0x7f2d38009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3716259128 shutdown_connections 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 --2- 172.21.15.110:0/3716259128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 0x7f2d480fe440 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 --2- 172.21.15.110:0/3716259128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d480fce60 0x7f2d480fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3716259128 >> 172.21.15.110:0/3716259128 conn(0x7f2d480f8540 msgr2=0x7f2d480fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3716259128 shutdown_connections 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3716259128 wait complete. 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.283+0000 7f2d4dfdf700 1 Processor -- start 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d4dfdf700 1 -- start start 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d4dfdf700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d480fce60 0x7f2d4810d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d4dfdf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 0x7f2d4810de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d4dfdf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4810e480 con 0x7f2d480fdfd0 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d4dfdf700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4810ecd0 con 0x7f2d480fce60 2024-04-25T15:57:14.286 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d46ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 0x7f2d4810de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d46ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 0x7f2d4810de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38682/0 (socket says 172.21.15.110:38682) 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d477fe700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d480fce60 0x7f2d4810d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d46ffd700 1 -- 172.21.15.110:0/3233803855 learned_addr learned my addr 172.21.15.110:0/3233803855 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d46ffd700 1 -- 172.21.15.110:0/3233803855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d480fce60 msgr2=0x7f2d4810d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d46ffd700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d480fce60 0x7f2d4810d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d46ffd700 1 -- 172.21.15.110:0/3233803855 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d38005040 con 0x7f2d480fdfd0 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.284+0000 7f2d46ffd700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 0x7f2d4810de80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d38013870 tx=0x7f2d38009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.285+0000 7f2d2ffff700 1 -- 172.21.15.110:0/3233803855 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d3800e070 con 0x7f2d480fdfd0 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.285+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d4810ef50 con 0x7f2d480fdfd0 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.285+0000 7f2d2ffff700 1 -- 172.21.15.110:0/3233803855 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d38013de0 con 0x7f2d480fdfd0 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.285+0000 7f2d2ffff700 1 -- 172.21.15.110:0/3233803855 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d3801cea0 con 0x7f2d480fdfd0 2024-04-25T15:57:14.287 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.285+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d4810f430 con 0x7f2d480fdfd0 2024-04-25T15:57:14.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.285+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d4805f5c0 con 0x7f2d480fdfd0 2024-04-25T15:57:14.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.286+0000 7f2d2ffff700 1 -- 172.21.15.110:0/3233803855 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f2d3801c690 con 0x7f2d480fdfd0 2024-04-25T15:57:14.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.287+0000 7f2d2ffff700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d30076b80 0x7f2d30079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.287+0000 7f2d2ffff700 1 -- 172.21.15.110:0/3233803855 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f2d38099b00 con 0x7f2d480fdfd0 2024-04-25T15:57:14.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.287+0000 7f2d477fe700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d30076b80 0x7f2d30079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.288 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.287+0000 7f2d477fe700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d30076b80 0x7f2d30079030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009b80 tx=0x7f2d3c00b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:14.292 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.292+0000 7f2d2ffff700 1 -- 172.21.15.110:0/3233803855 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2d38062600 con 0x7f2d480fdfd0 2024-04-25T15:57:14.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.536+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2d481058c0 con 0x7f2d480fdfd0 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.537+0000 7f2d2ffff700 1 -- 172.21.15.110:0/3233803855 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1709 (secure 0 0 0) 0x7f2d38004490 con 0x7f2d480fdfd0 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:e26 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 26 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:57:14.538 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:57:07.408098+0000 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 82 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=24265} 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:57:14.539 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 0 members: 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{0:24265} state up:replay seq 2 join_fscid=1 addr [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] compat {c=[1],r=[1],i=[7ff]}] 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{-1:34404} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:57:14.540 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/1518950103,v1:172.21.15.110:6829/1518950103] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:57:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d30076b80 msgr2=0x7f2d30079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d30076b80 0x7f2d30079030 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c009b80 tx=0x7f2d3c00b040 comp rx=0 tx=0).stop 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 msgr2=0x7f2d4810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 0x7f2d4810de80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d38013870 tx=0x7f2d38009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 shutdown_connections 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d480fce60 0x7f2d4810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d30076b80 0x7f2d30079030 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 --2- 172.21.15.110:0/3233803855 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d480fdfd0 0x7f2d4810de80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 >> 172.21.15.110:0/3233803855 conn(0x7f2d480f8540 msgr2=0x7f2d480f9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 shutdown_connections 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.539+0000 7f2d4dfdf700 1 -- 172.21.15.110:0/3233803855 wait complete. 2024-04-25T15:57:14.542 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 26 2024-04-25T15:57:14.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.642+0000 7fdd61a2b700 1 -- 172.21.15.110:0/3329245388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fcf60 msgr2=0x7fdd5c0fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.642+0000 7fdd61a2b700 1 --2- 172.21.15.110:0/3329245388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fcf60 0x7fdd5c0fd3d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdd50004940 tx=0x7fdd50009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:14.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.642+0000 7fdd61a2b700 1 -- 172.21.15.110:0/3329245388 shutdown_connections 2024-04-25T15:57:14.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.642+0000 7fdd61a2b700 1 --2- 172.21.15.110:0/3329245388 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fcf60 0x7fdd5c0fd3d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.642+0000 7fdd61a2b700 1 --2- 172.21.15.110:0/3329245388 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c0fbcb0 0x7fdd5c0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.642+0000 7fdd61a2b700 1 -- 172.21.15.110:0/3329245388 >> 172.21.15.110:0/3329245388 conn(0x7fdd5c0f7430 msgr2=0x7fdd5c0f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:14.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.642+0000 7fdd61a2b700 1 -- 172.21.15.110:0/3329245388 shutdown_connections 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.642+0000 7fdd61a2b700 1 -- 172.21.15.110:0/3329245388 wait complete. 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd61a2b700 1 Processor -- start 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd61a2b700 1 -- start start 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd61a2b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fbcb0 0x7fdd5c066c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd5affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fbcb0 0x7fdd5c066c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd5affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fbcb0 0x7fdd5c066c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38706/0 (socket says 172.21.15.110:38706) 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd61a2b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c0fcf60 0x7fdd5c112680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c112bc0 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd5affd700 1 -- 172.21.15.110:0/2096767080 learned_addr learned my addr 172.21.15.110:0/2096767080 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.643+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5c112d00 con 0x7fdd5c0fcf60 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd5a7fc700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c0fcf60 0x7fdd5c112680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd5affd700 1 -- 172.21.15.110:0/2096767080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c0fcf60 msgr2=0x7fdd5c112680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd5affd700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c0fcf60 0x7fdd5c112680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd5affd700 1 -- 172.21.15.110:0/2096767080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd50005040 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd5affd700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fbcb0 0x7fdd5c066c50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00eb40 tx=0x7fdd4c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:14.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd437fe700 1 -- 172.21.15.110:0/2096767080 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4c007cc0 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd437fe700 1 -- 172.21.15.110:0/2096767080 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd4c00f040 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd5c112f00 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd437fe700 1 -- 172.21.15.110:0/2096767080 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd4c00ca10 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.644+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd5c1133b0 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.645+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5c04c710 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.646+0000 7fdd437fe700 1 -- 172.21.15.110:0/2096767080 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fdd4c00cc80 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.646+0000 7fdd437fe700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdd44076b80 0x7fdd44079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.646+0000 7fdd437fe700 1 -- 172.21.15.110:0/2096767080 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7fdd4c097f70 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.646+0000 7fdd5a7fc700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdd44076b80 0x7fdd44079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.647+0000 7fdd5a7fc700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdd44076b80 0x7fdd44079030 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000b0d0 tx=0x7fdd50009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:14.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.651+0000 7fdd437fe700 1 -- 172.21.15.110:0/2096767080 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdd4c0ccff0 con 0x7fdd5c0fbcb0 2024-04-25T15:57:14.876 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.875+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd5c101230 con 0x7fdd44076b80 2024-04-25T15:57:14.877 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.876+0000 7fdd437fe700 1 -- 172.21.15.110:0/2096767080 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdd5c101230 con 0x7fdd44076b80 2024-04-25T15:57:14.877 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "crash", 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr" 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "14/21 daemons upgraded", 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:57:14.878 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.878+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdd44076b80 msgr2=0x7fdd44079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.878+0000 7fdd61a2b700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdd44076b80 0x7fdd44079030 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdd5000b0d0 tx=0x7fdd50009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.878+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fbcb0 msgr2=0x7fdd5c066c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.878+0000 7fdd61a2b700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fbcb0 0x7fdd5c066c50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00eb40 tx=0x7fdd4c00b040 comp rx=0 tx=0).stop 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.879+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 shutdown_connections 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.879+0000 7fdd61a2b700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdd44076b80 0x7fdd44079030 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.879+0000 7fdd61a2b700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd5c0fbcb0 0x7fdd5c066c50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.879+0000 7fdd61a2b700 1 --2- 172.21.15.110:0/2096767080 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd5c0fcf60 0x7fdd5c112680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.879+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 >> 172.21.15.110:0/2096767080 conn(0x7fdd5c0f7430 msgr2=0x7fdd5c0f8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.879+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 shutdown_connections 2024-04-25T15:57:14.880 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.879+0000 7fdd61a2b700 1 -- 172.21.15.110:0/2096767080 wait complete. 2024-04-25T15:57:14.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 -- 172.21.15.110:0/2524830621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fce60 msgr2=0x7f9b6c0fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 --2- 172.21.15.110:0/2524830621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fce60 0x7f9b6c0fd270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c004940 tx=0x7f9b5c009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 -- 172.21.15.110:0/2524830621 shutdown_connections 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 --2- 172.21.15.110:0/2524830621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b6c0fdfd0 0x7f9b6c0fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 --2- 172.21.15.110:0/2524830621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fce60 0x7f9b6c0fd270 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 -- 172.21.15.110:0/2524830621 >> 172.21.15.110:0/2524830621 conn(0x7f9b6c0f8540 msgr2=0x7f9b6c0fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 -- 172.21.15.110:0/2524830621 shutdown_connections 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 -- 172.21.15.110:0/2524830621 wait complete. 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 Processor -- start 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 -- start start 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b6c0fce60 0x7f9b6c10d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fdfd0 0x7f9b6c10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c10e3f0 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.980+0000 7f9b70bf0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6c10e560 con 0x7f9b6c0fce60 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b69d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fdfd0 0x7f9b6c10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b6a59c700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b6c0fce60 0x7f9b6c10d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b69d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fdfd0 0x7f9b6c10ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:38718/0 (socket says 172.21.15.110:38718) 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b69d9b700 1 -- 172.21.15.110:0/1244152334 learned_addr learned my addr 172.21.15.110:0/1244152334 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b69d9b700 1 -- 172.21.15.110:0/1244152334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b6c0fce60 msgr2=0x7f9b6c10d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b69d9b700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b6c0fce60 0x7f9b6c10d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b69d9b700 1 -- 172.21.15.110:0/1244152334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c005040 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b6a59c700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b6c0fce60 0x7f9b6c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b69d9b700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fdfd0 0x7f9b6c10ddf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000eb40 tx=0x7f9b6000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b5affd700 1 -- 172.21.15.110:0/1244152334 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b60007cc0 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b5affd700 1 -- 172.21.15.110:0/1244152334 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b6000f040 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6c112f60 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b5affd700 1 -- 172.21.15.110:0/1244152334 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b6000ca10 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.981+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6c113470 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.982+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b6c04c710 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.983+0000 7f9b5affd700 1 -- 172.21.15.110:0/1244152334 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f9b6000cc80 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.983+0000 7f9b5affd700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9b54076b80 0x7f9b54079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:14.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.983+0000 7f9b5affd700 1 -- 172.21.15.110:0/1244152334 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6582+0+0 (secure 0 0 0) 0x7f9b60097f80 con 0x7f9b6c0fdfd0 2024-04-25T15:57:14.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.984+0000 7f9b6a59c700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9b54076b80 0x7f9b54079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:14.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.984+0000 7f9b6a59c700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9b54076b80 0x7f9b54079030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b0d0 tx=0x7f9b5c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:14.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:14.988+0000 7f9b5affd700 1 -- 172.21.15.110:0/1244152334 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9b60060ab0 con 0x7f9b6c0fdfd0 2024-04-25T15:57:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:14 smithi110 ceph-mon[90991]: from='client.44327 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:14 smithi110 ceph-mon[90991]: from='client.44331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:14 smithi110 ceph-mon[90991]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:14 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2635089530' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:15.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:14 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3233803855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:57:15.136 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:14 smithi193 ceph-mon[80071]: from='client.44327 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:15.136 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:14 smithi193 ceph-mon[80071]: from='client.44331 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:15.136 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:14 smithi193 ceph-mon[80071]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:15.136 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:14 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2635089530' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:15.136 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:14 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3233803855' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:57:15.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.268+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9b6c1058c0 con 0x7f9b6c0fdfd0 2024-04-25T15:57:15.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.268+0000 7f9b5affd700 1 -- 172.21.15.110:0/1244152334 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+297 (secure 0 0 0) 0x7f9b60067070 con 0x7f9b6c0fdfd0 2024-04-25T15:57:15.269 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data 2024-04-25T15:57:15.270 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-25T15:57:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs is degraded 2024-04-25T15:57:15.270 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:57:15.270 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:57:15.271 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9b54076b80 msgr2=0x7f9b54079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9b54076b80 0x7f9b54079030 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b0d0 tx=0x7f9b5c009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fdfd0 msgr2=0x7f9b6c10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fdfd0 0x7f9b6c10ddf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000eb40 tx=0x7f9b6000b040 comp rx=0 tx=0).stop 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 shutdown_connections 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b6c0fce60 0x7f9b6c10d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f9b54076b80 0x7f9b54079030 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 --2- 172.21.15.110:0/1244152334 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b6c0fdfd0 0x7f9b6c10ddf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.270+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 >> 172.21.15.110:0/1244152334 conn(0x7f9b6c0f8540 msgr2=0x7f9b6c0f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.271+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 shutdown_connections 2024-04-25T15:57:15.272 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:15.271+0000 7f9b70bf0700 1 -- 172.21.15.110:0/1244152334 wait complete. 2024-04-25T15:57:15.829 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:15 smithi193 ceph-mon[80071]: pgmap v200: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-04-25T15:57:15.829 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:15 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1244152334' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:57:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:15 smithi110 ceph-mon[90991]: pgmap v200: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 4 op/s 2024-04-25T15:57:16.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:15 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1244152334' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:57:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:16 smithi110 ceph-mon[90991]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:17.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:57:17.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:16 smithi193 ceph-mon[80071]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:57:17.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:17 smithi110 ceph-mon[90991]: pgmap v201: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-25T15:57:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:17 smithi110 ceph-mon[90991]: reconnect by client.24297 192.168.0.1:0/4279727202 after 0 2024-04-25T15:57:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:17 smithi110 ceph-mon[90991]: reconnect by client.14526 192.168.0.1:0/2988371434 after 0.000999984 2024-04-25T15:57:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:17 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.193:6824/1783080009,v1:172.21.15.193:6825/1783080009] up:boot 2024-04-25T15:57:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:17 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:reconnect 2024-04-25T15:57:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:17 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi193.jtyzhl=up:reconnect} 3 up:standby 2024-04-25T15:57:18.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:17 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:57:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:17 smithi193 ceph-mon[80071]: pgmap v201: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-25T15:57:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:17 smithi193 ceph-mon[80071]: reconnect by client.24297 192.168.0.1:0/4279727202 after 0 2024-04-25T15:57:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:17 smithi193 ceph-mon[80071]: reconnect by client.14526 192.168.0.1:0/2988371434 after 0.000999984 2024-04-25T15:57:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:17 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.193:6824/1783080009,v1:172.21.15.193:6825/1783080009] up:boot 2024-04-25T15:57:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:17 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:reconnect 2024-04-25T15:57:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:17 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi193.jtyzhl=up:reconnect} 3 up:standby 2024-04-25T15:57:18.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:17 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:57:19.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:18 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:rejoin 2024-04-25T15:57:19.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:18 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi193.jtyzhl=up:rejoin} 3 up:standby 2024-04-25T15:57:19.084 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:18 smithi193 ceph-mon[80071]: daemon mds.cephfs.smithi193.jtyzhl is now active in filesystem cephfs as rank 0 2024-04-25T15:57:19.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:18 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:rejoin 2024-04-25T15:57:19.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:18 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi193.jtyzhl=up:rejoin} 3 up:standby 2024-04-25T15:57:19.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:18 smithi110 ceph-mon[90991]: daemon mds.cephfs.smithi193.jtyzhl is now active in filesystem cephfs as rank 0 2024-04-25T15:57:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:19 smithi110 ceph-mon[90991]: pgmap v202: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-25T15:57:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:19 smithi110 ceph-mon[90991]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:57:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:19 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:active 2024-04-25T15:57:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:19 smithi110 ceph-mon[90991]: fsmap cephfs:1 {0=cephfs.smithi193.jtyzhl=up:active} 3 up:standby 2024-04-25T15:57:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:19 smithi193 ceph-mon[80071]: pgmap v202: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 6 op/s 2024-04-25T15:57:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:19 smithi193 ceph-mon[80071]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:57:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:19 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.193:6826/1539252929,v1:172.21.15.193:6827/1539252929] up:active 2024-04-25T15:57:20.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:19 smithi193 ceph-mon[80071]: fsmap cephfs:1 {0=cephfs.smithi193.jtyzhl=up:active} 3 up:standby 2024-04-25T15:57:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:21 smithi110 ceph-mon[90991]: pgmap v203: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T15:57:22.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:21 smithi193 ceph-mon[80071]: pgmap v203: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T15:57:24.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:23 smithi110 ceph-mon[90991]: pgmap v204: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T15:57:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:23 smithi193 ceph-mon[80071]: pgmap v204: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T15:57:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:25 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:25 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:25 smithi110 ceph-mon[90991]: pgmap v205: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T15:57:25.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:25 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:25.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:25 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:25.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:25 smithi193 ceph-mon[80071]: pgmap v205: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T15:57:26.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:26 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:26.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:26 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:26.567 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:26 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:26.567 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:26 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:27.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:27 smithi193 ceph-mon[80071]: pgmap v206: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 9.0 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T15:57:27.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:27 smithi110 ceph-mon[90991]: pgmap v206: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 9.0 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T15:57:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: Detected new or changed devices on smithi193 2024-04-25T15:57:29.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi193.jtyzhl"]}]: dispatch 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: Upgrade: It appears safe to stop mds.cephfs.smithi193.jtyzhl 2024-04-25T15:57:29.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:29 smithi110 ceph-mon[90991]: pgmap v207: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:29.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: Detected new or changed devices on smithi193 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds ok-to-stop", "ids": ["cephfs.smithi193.jtyzhl"]}]: dispatch 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: Upgrade: It appears safe to stop mds.cephfs.smithi193.jtyzhl 2024-04-25T15:57:29.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:29 smithi193 ceph-mon[80071]: pgmap v207: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:31.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:31 smithi193 ceph-mon[80071]: Upgrade: Updating mds.cephfs.smithi193.jtyzhl 2024-04-25T15:57:31.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:31.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.jtyzhl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:57:31.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:31.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:31 smithi193 ceph-mon[80071]: Deploying daemon mds.cephfs.smithi193.jtyzhl on smithi193 2024-04-25T15:57:31.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:31 smithi193 ceph-mon[80071]: pgmap v208: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:31.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:31 smithi110 ceph-mon[90991]: Upgrade: Updating mds.cephfs.smithi193.jtyzhl 2024-04-25T15:57:31.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:31.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi193.jtyzhl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T15:57:31.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:31.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:31 smithi110 ceph-mon[90991]: Deploying daemon mds.cephfs.smithi193.jtyzhl on smithi193 2024-04-25T15:57:31.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:31 smithi110 ceph-mon[90991]: pgmap v208: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[90987]: 2024-04-25T15:57:32.545+0000 7effa00b9700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:57:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:57:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:57:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:57:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: osdmap e83: 6 total, 6 up, 6 in 2024-04-25T15:57:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: Standby daemon mds.cephfs.smithi110.mvvbtv assigned to filesystem cephfs as rank 0 2024-04-25T15:57:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:57:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-25T15:57:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi110.mvvbtv=up:replay} 2 up:standby 2024-04-25T15:57:32.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: osdmap e83: 6 total, 6 up, 6 in 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: Standby daemon mds.cephfs.smithi110.mvvbtv assigned to filesystem cephfs as rank 0 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: fsmap cephfs:0/1 3 up:standby, 1 failed 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi110.mvvbtv=up:replay} 2 up:standby 2024-04-25T15:57:32.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:57:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:33 smithi110 ceph-mon[90991]: pgmap v210: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 4.9 KiB/s wr, 3 op/s 2024-04-25T15:57:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:33 smithi193 ceph-mon[80071]: pgmap v210: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 4.9 KiB/s wr, 3 op/s 2024-04-25T15:57:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:35 smithi110 ceph-mon[90991]: pgmap v211: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 410 KiB/s rd, 4.9 KiB/s wr, 2 op/s 2024-04-25T15:57:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:35 smithi193 ceph-mon[80071]: pgmap v211: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 410 KiB/s rd, 4.9 KiB/s wr, 2 op/s 2024-04-25T15:57:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:37 smithi110 ceph-mon[90991]: pgmap v212: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-04-25T15:57:38.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:37 smithi193 ceph-mon[80071]: pgmap v212: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 11 MiB/s rd, 3 op/s 2024-04-25T15:57:39.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:39 smithi193 ceph-mon[80071]: pgmap v213: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-25T15:57:40.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:39 smithi110 ceph-mon[90991]: pgmap v213: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 14 MiB/s rd, 4 op/s 2024-04-25T15:57:41.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:41 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:41.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:41 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:41.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:41 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:41.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:41 smithi193 ceph-mon[80071]: pgmap v214: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-25T15:57:41.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:41 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.193:6826/3103986931,v1:172.21.15.193:6827/3103986931] up:boot 2024-04-25T15:57:41.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:41 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi110.mvvbtv=up:replay} 3 up:standby 2024-04-25T15:57:41.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:41 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:57:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:41 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:41 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:41 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:41 smithi110 ceph-mon[90991]: pgmap v214: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 6 op/s 2024-04-25T15:57:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:41 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.193:6826/3103986931,v1:172.21.15.193:6827/3103986931] up:boot 2024-04-25T15:57:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:41 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi110.mvvbtv=up:replay} 3 up:standby 2024-04-25T15:57:41.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:41 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:57:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:43 smithi110 ceph-mon[90991]: reconnect by client.24297 192.168.0.1:0/4279727202 after 0.000999984 2024-04-25T15:57:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:43 smithi110 ceph-mon[90991]: reconnect by client.14526 192.168.0.1:0/2988371434 after 0.000999984 2024-04-25T15:57:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:43 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] up:reconnect 2024-04-25T15:57:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:43 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi110.mvvbtv=up:reconnect} 3 up:standby 2024-04-25T15:57:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:43 smithi110 ceph-mon[90991]: pgmap v215: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-25T15:57:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:43 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:43.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:43 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:43 smithi193 ceph-mon[80071]: reconnect by client.24297 192.168.0.1:0/4279727202 after 0.000999984 2024-04-25T15:57:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:43 smithi193 ceph-mon[80071]: reconnect by client.14526 192.168.0.1:0/2988371434 after 0.000999984 2024-04-25T15:57:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:43 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] up:reconnect 2024-04-25T15:57:43.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:43 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi110.mvvbtv=up:reconnect} 3 up:standby 2024-04-25T15:57:43.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:43 smithi193 ceph-mon[80071]: pgmap v215: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-25T15:57:43.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:43 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:43.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:43 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:44.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:44 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] up:rejoin 2024-04-25T15:57:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:44 smithi110 ceph-mon[90991]: fsmap cephfs:1/1 {0=cephfs.smithi110.mvvbtv=up:rejoin} 3 up:standby 2024-04-25T15:57:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:44 smithi110 ceph-mon[90991]: daemon mds.cephfs.smithi110.mvvbtv is now active in filesystem cephfs as rank 0 2024-04-25T15:57:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:44 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:44.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:44 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:44.876 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:44 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] up:rejoin 2024-04-25T15:57:44.876 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:44 smithi193 ceph-mon[80071]: fsmap cephfs:1/1 {0=cephfs.smithi110.mvvbtv=up:rejoin} 3 up:standby 2024-04-25T15:57:44.876 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:44 smithi193 ceph-mon[80071]: daemon mds.cephfs.smithi110.mvvbtv is now active in filesystem cephfs as rank 0 2024-04-25T15:57:44.876 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:44 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:44.876 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:44 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/692559653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 msgr2=0x7f6bd00ffa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 --2- 172.21.15.110:0/692559653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 0x7f6bd00ffa70 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4004940 tx=0x7f6bc4009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/692559653 shutdown_connections 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 --2- 172.21.15.110:0/692559653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bd00fffb0 0x7f6bd0102390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 --2- 172.21.15.110:0/692559653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 0x7f6bd00ffa70 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/692559653 >> 172.21.15.110:0/692559653 conn(0x7f6bd00f7440 msgr2=0x7f6bd00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/692559653 shutdown_connections 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/692559653 wait complete. 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 Processor -- start 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.385+0000 7f6bd6b5f700 1 -- start start 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd6b5f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 0x7f6bd010da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:45.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd6b5f700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bd00fffb0 0x7f6bd010df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd6b5f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd010e5f0 con 0x7f6bd0066e90 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd6b5f700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0112f90 con 0x7f6bd00fffb0 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd48fb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 0x7f6bd010da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd48fb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 0x7f6bd010da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60224/0 (socket says 172.21.15.110:60224) 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd48fb700 1 -- 172.21.15.110:0/3436406816 learned_addr learned my addr 172.21.15.110:0/3436406816 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bcffff700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bd00fffb0 0x7f6bd010df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd48fb700 1 -- 172.21.15.110:0/3436406816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bd00fffb0 msgr2=0x7f6bd010df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd48fb700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bd00fffb0 0x7f6bd010df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd48fb700 1 -- 172.21.15.110:0/3436406816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc4005040 con 0x7f6bd0066e90 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd48fb700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 0x7f6bd010da20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009fd0 tx=0x7f6bc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bcd7fa700 1 -- 172.21.15.110:0/3436406816 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bc400e040 con 0x7f6bd0066e90 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd0113190 con 0x7f6bd0066e90 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bcd7fa700 1 -- 172.21.15.110:0/3436406816 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bc400bbf0 con 0x7f6bd0066e90 2024-04-25T15:57:45.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bcd7fa700 1 -- 172.21.15.110:0/3436406816 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bc401c630 con 0x7f6bd0066e90 2024-04-25T15:57:45.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.386+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd0113700 con 0x7f6bd0066e90 2024-04-25T15:57:45.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.387+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd004c710 con 0x7f6bd0066e90 2024-04-25T15:57:45.390 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.388+0000 7f6bcd7fa700 1 -- 172.21.15.110:0/3436406816 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f6bc400bd60 con 0x7f6bd0066e90 2024-04-25T15:57:45.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.389+0000 7f6bcd7fa700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6bc0076b90 0x7f6bc0079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:45.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.389+0000 7f6bcffff700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6bc0076b90 0x7f6bc0079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:45.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.389+0000 7f6bcd7fa700 1 -- 172.21.15.110:0/3436406816 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f6bc40996c0 con 0x7f6bd0066e90 2024-04-25T15:57:45.391 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.389+0000 7f6bcffff700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6bc0076b90 0x7f6bc0079040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0067350 tx=0x7f6bbc0091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:45.394 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.394+0000 7f6bcd7fa700 1 -- 172.21.15.110:0/3436406816 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f6bc4062170 con 0x7f6bd0066e90 2024-04-25T15:57:45.616 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.615+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bd0101c20 con 0x7f6bc0076b90 2024-04-25T15:57:45.617 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.616+0000 7f6bcd7fa700 1 -- 172.21.15.110:0/3436406816 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6bd0101c20 con 0x7f6bc0076b90 2024-04-25T15:57:45.619 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6bc0076b90 msgr2=0x7f6bc0079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6bc0076b90 0x7f6bc0079040 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0067350 tx=0x7f6bbc0091a0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 msgr2=0x7f6bd010da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 0x7f6bd010da20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6bc4009fd0 tx=0x7f6bc4009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 shutdown_connections 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6bc0076b90 0x7f6bc0079040 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6bd0066e90 0x7f6bd010da20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 --2- 172.21.15.110:0/3436406816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bd00fffb0 0x7f6bd010df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 >> 172.21.15.110:0/3436406816 conn(0x7f6bd00f7440 msgr2=0x7f6bd00f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 shutdown_connections 2024-04-25T15:57:45.620 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.618+0000 7f6bd6b5f700 1 -- 172.21.15.110:0/3436406816 wait complete. 2024-04-25T15:57:45.629 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:57:45.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.722+0000 7f72d5814700 1 -- 172.21.15.110:0/1688653176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fe420 msgr2=0x7f72d00fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:45.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.722+0000 7f72d5814700 1 --2- 172.21.15.110:0/1688653176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fe420 0x7f72d00fe830 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f72c0004940 tx=0x7f72c0009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:45.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.722+0000 7f72d5814700 1 -- 172.21.15.110:0/1688653176 shutdown_connections 2024-04-25T15:57:45.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.722+0000 7f72d5814700 1 --2- 172.21.15.110:0/1688653176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72d00fed70 0x7f72d00faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.722+0000 7f72d5814700 1 --2- 172.21.15.110:0/1688653176 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fe420 0x7f72d00fe830 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.724 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.722+0000 7f72d5814700 1 -- 172.21.15.110:0/1688653176 >> 172.21.15.110:0/1688653176 conn(0x7f72d00f6be0 msgr2=0x7f72d00f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:45.724 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:45 smithi110 ceph-mon[90991]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:57:45.724 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:45 smithi110 ceph-mon[90991]: mds.? [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] up:active 2024-04-25T15:57:45.725 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:45 smithi110 ceph-mon[90991]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:57:45.725 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:45 smithi110 ceph-mon[90991]: pgmap v216: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-25T15:57:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.723+0000 7f72d5814700 1 -- 172.21.15.110:0/1688653176 shutdown_connections 2024-04-25T15:57:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.723+0000 7f72d5814700 1 -- 172.21.15.110:0/1688653176 wait complete. 2024-04-25T15:57:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.723+0000 7f72d5814700 1 Processor -- start 2024-04-25T15:57:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.723+0000 7f72d5814700 1 -- start start 2024-04-25T15:57:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.723+0000 7f72d5814700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72d00fe420 0x7f72d010d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:45.726 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.723+0000 7f72d5814700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fed70 0x7f72d010de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.723+0000 7f72d5814700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d010e430 con 0x7f72d00fed70 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.723+0000 7f72d5814700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d010e5a0 con 0x7f72d00fe420 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72ce7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fed70 0x7f72d010de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72ce7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fed70 0x7f72d010de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60252/0 (socket says 172.21.15.110:60252) 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72ce7fc700 1 -- 172.21.15.110:0/1178263377 learned_addr learned my addr 172.21.15.110:0/1178263377 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72ce7fc700 1 -- 172.21.15.110:0/1178263377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72d00fe420 msgr2=0x7f72d010d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72ce7fc700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72d00fe420 0x7f72d010d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72ce7fc700 1 -- 172.21.15.110:0/1178263377 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0005040 con 0x7f72d00fed70 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72ce7fc700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fed70 0x7f72d010de30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f72c4002a50 tx=0x7f72c40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72b77fe700 1 -- 172.21.15.110:0/1178263377 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c4007cc0 con 0x7f72d00fed70 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d0113000 con 0x7f72d00fed70 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72b77fe700 1 -- 172.21.15.110:0/1178263377 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72c400f040 con 0x7f72d00fed70 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72b77fe700 1 -- 172.21.15.110:0/1178263377 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c400db70 con 0x7f72d00fed70 2024-04-25T15:57:45.727 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.724+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d0113590 con 0x7f72d00fed70 2024-04-25T15:57:45.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.725+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d004c710 con 0x7f72d00fed70 2024-04-25T15:57:45.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.726+0000 7f72b77fe700 1 -- 172.21.15.110:0/1178263377 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f72c400dd10 con 0x7f72d00fed70 2024-04-25T15:57:45.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.726+0000 7f72b77fe700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72b8076b90 0x7f72b8079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:45.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.726+0000 7f72ceffd700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72b8076b90 0x7f72b8079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:45.728 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.726+0000 7f72b77fe700 1 -- 172.21.15.110:0/1178263377 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f72c4097b90 con 0x7f72d00fed70 2024-04-25T15:57:45.729 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.727+0000 7f72ceffd700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72b8076b90 0x7f72b8079040 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72c0004f90 tx=0x7f72c0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:45.732 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.731+0000 7f72b77fe700 1 -- 172.21.15.110:0/1178263377 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f72c40605c0 con 0x7f72d00fed70 2024-04-25T15:57:45.832 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:45 smithi193 ceph-mon[80071]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T15:57:45.832 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:45 smithi193 ceph-mon[80071]: mds.? [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] up:active 2024-04-25T15:57:45.832 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:45 smithi193 ceph-mon[80071]: fsmap cephfs:1 {0=cephfs.smithi110.mvvbtv=up:active} 3 up:standby 2024-04-25T15:57:45.832 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:45 smithi193 ceph-mon[80071]: pgmap v216: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 6 op/s 2024-04-25T15:57:45.951 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.950+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72d01024f0 con 0x7f72b8076b90 2024-04-25T15:57:45.952 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.952+0000 7f72b77fe700 1 -- 172.21.15.110:0/1178263377 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f72d01024f0 con 0x7f72b8076b90 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72b8076b90 msgr2=0x7f72b8079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72b8076b90 0x7f72b8079040 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f72c0004f90 tx=0x7f72c0009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fed70 msgr2=0x7f72d010de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fed70 0x7f72d010de30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f72c4002a50 tx=0x7f72c40046b0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 shutdown_connections 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72d00fe420 0x7f72d010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72b8076b90 0x7f72b8079040 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 --2- 172.21.15.110:0/1178263377 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72d00fed70 0x7f72d010de30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 >> 172.21.15.110:0/1178263377 conn(0x7f72d00f6be0 msgr2=0x7f72d00f8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 shutdown_connections 2024-04-25T15:57:45.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:45.954+0000 7f72d5814700 1 -- 172.21.15.110:0/1178263377 wait complete. 2024-04-25T15:57:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 -- 172.21.15.110:0/3038090524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fdfe0 msgr2=0x7fa57c0fe450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 --2- 172.21.15.110:0/3038090524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fdfe0 0x7fa57c0fe450 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa56c004940 tx=0x7fa56c009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 -- 172.21.15.110:0/3038090524 shutdown_connections 2024-04-25T15:57:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 --2- 172.21.15.110:0/3038090524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fdfe0 0x7fa57c0fe450 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 --2- 172.21.15.110:0/3038090524 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c0fce70 0x7fa57c0fd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 -- 172.21.15.110:0/3038090524 >> 172.21.15.110:0/3038090524 conn(0x7fa57c0f8550 msgr2=0x7fa57c0fa960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 -- 172.21.15.110:0/3038090524 shutdown_connections 2024-04-25T15:57:46.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 -- 172.21.15.110:0/3038090524 wait complete. 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.055+0000 7fa581e25700 1 Processor -- start 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa581e25700 1 -- start start 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa581e25700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fce70 0x7fa57c10d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa581e25700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c0fdfe0 0x7fa57c10ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa581e25700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c10e3b0 con 0x7fa57c0fce70 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa581e25700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57c10e520 con 0x7fa57c0fdfe0 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa57b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fce70 0x7fa57c10d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa57b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fce70 0x7fa57c10d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60264/0 (socket says 172.21.15.110:60264) 2024-04-25T15:57:46.058 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa57b7fe700 1 -- 172.21.15.110:0/2099984246 learned_addr learned my addr 172.21.15.110:0/2099984246 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa57affd700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c0fdfe0 0x7fa57c10ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa57b7fe700 1 -- 172.21.15.110:0/2099984246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c0fdfe0 msgr2=0x7fa57c10ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa57b7fe700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c0fdfe0 0x7fa57c10ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.056+0000 7fa57b7fe700 1 -- 172.21.15.110:0/2099984246 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c005040 con 0x7fa57c0fce70 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.057+0000 7fa57b7fe700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fce70 0x7fa57c10d870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa57000eb40 tx=0x7fa57000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.057+0000 7fa563fff700 1 -- 172.21.15.110:0/2099984246 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa570007cc0 con 0x7fa57c0fce70 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.057+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57c112f20 con 0x7fa57c0fce70 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.057+0000 7fa563fff700 1 -- 172.21.15.110:0/2099984246 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa57000f040 con 0x7fa57c0fce70 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.057+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57c1133d0 con 0x7fa57c0fce70 2024-04-25T15:57:46.059 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.057+0000 7fa563fff700 1 -- 172.21.15.110:0/2099984246 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57000c9b0 con 0x7fa57c0fce70 2024-04-25T15:57:46.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.058+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57c04c710 con 0x7fa57c0fce70 2024-04-25T15:57:46.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.058+0000 7fa563fff700 1 -- 172.21.15.110:0/2099984246 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fa57000cbb0 con 0x7fa57c0fce70 2024-04-25T15:57:46.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.059+0000 7fa563fff700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa564076b90 0x7fa564079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.059+0000 7fa563fff700 1 -- 172.21.15.110:0/2099984246 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa570098040 con 0x7fa57c0fce70 2024-04-25T15:57:46.060 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.059+0000 7fa57affd700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa564076b90 0x7fa564079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.061 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.059+0000 7fa57affd700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa564076b90 0x7fa564079040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009b30 tx=0x7fa56c009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:46.065 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.064+0000 7fa563fff700 1 -- 172.21.15.110:0/2099984246 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa570060b20 con 0x7fa57c0fce70 2024-04-25T15:57:46.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.269+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa57c113610 con 0x7fa564076b90 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.279+0000 7fa563fff700 1 -- 172.21.15.110:0/2099984246 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa57c113610 con 0x7fa564076b90 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (8m) 47s ago 16m 25.1M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (5m) 47s ago 16m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (4m) 3s ago 15m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (7m) 47s ago 16m 85.4M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (52s) 47s ago 11m 82.5M - 19.0.0-3291-g994e2f20 d53c96779a29 051262111b6b 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (80s) 47s ago 11m 85.0M - 19.0.0-3291-g994e2f20 d53c96779a29 763d1dda41b7 2024-04-25T15:57:46.280 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (30s) 3s ago 11m 82.7M - 19.0.0-3291-g994e2f20 d53c96779a29 36e42c64ae3a 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (6s) 3s ago 11m 79.5M - 19.0.0-3291-g994e2f20 d53c96779a29 3dee50406dde 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (7m) 47s ago 18m 578M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (6m) 3s ago 15m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (6m) 47s ago 18m 119M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (5m) 3s ago 15m 104M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (8m) 47s ago 16m 16.0M - 1.5.0 0da6a335fe13 8aeab89546ac 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (7m) 3s ago 15m 15.5M - 1.5.0 0da6a335fe13 ef1c93d82a76 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (4m) 47s ago 14m 166M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (4m) 47s ago 14m 169M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (3m) 47s ago 14m 155M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (2m) 3s ago 13m 176M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (2m) 3s ago 13m 159M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (102s) 3s ago 12m 160M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 31c4aca48886 2024-04-25T15:57:46.281 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (6m) 47s ago 16m 63.0M - 2.43.0 a07b618ecd1d 216b0dda526a 2024-04-25T15:57:46.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa564076b90 msgr2=0x7fa564079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa564076b90 0x7fa564079040 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009b30 tx=0x7fa56c009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:46.282 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fce70 msgr2=0x7fa57c10d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fce70 0x7fa57c10d870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa57000eb40 tx=0x7fa57000b040 comp rx=0 tx=0).stop 2024-04-25T15:57:46.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 shutdown_connections 2024-04-25T15:57:46.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa564076b90 0x7fa564079040 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa57c0fce70 0x7fa57c10d870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 --2- 172.21.15.110:0/2099984246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57c0fdfe0 0x7fa57c10ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 >> 172.21.15.110:0/2099984246 conn(0x7fa57c0f8550 msgr2=0x7fa57c0f9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:46.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 shutdown_connections 2024-04-25T15:57:46.283 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.281+0000 7fa581e25700 1 -- 172.21.15.110:0/2099984246 wait complete. 2024-04-25T15:57:46.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.380+0000 7f7cfabf0700 1 -- 172.21.15.110:0/888047866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fcf60 msgr2=0x7f7cf40fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.380+0000 7f7cfabf0700 1 --2- 172.21.15.110:0/888047866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fcf60 0x7f7cf40fd3d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7ce8004940 tx=0x7f7ce8009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.380+0000 7f7cfabf0700 1 -- 172.21.15.110:0/888047866 shutdown_connections 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.380+0000 7f7cfabf0700 1 --2- 172.21.15.110:0/888047866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fcf60 0x7f7cf40fd3d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.380+0000 7f7cfabf0700 1 --2- 172.21.15.110:0/888047866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cf40fbcb0 0x7f7cf40fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.380+0000 7f7cfabf0700 1 -- 172.21.15.110:0/888047866 >> 172.21.15.110:0/888047866 conn(0x7f7cf40f7430 msgr2=0x7f7cf40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.380+0000 7f7cfabf0700 1 -- 172.21.15.110:0/888047866 shutdown_connections 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.380+0000 7f7cfabf0700 1 -- 172.21.15.110:0/888047866 wait complete. 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cfabf0700 1 Processor -- start 2024-04-25T15:57:46.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cfabf0700 1 -- start start 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cfabf0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fbcb0 0x7f7cf41106f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cfabf0700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cf40fcf60 0x7f7cf4112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cfabf0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf40663e0 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cfabf0700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf4066520 con 0x7f7cf40fcf60 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cf898c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fbcb0 0x7f7cf41106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cf898c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fbcb0 0x7f7cf41106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60278/0 (socket says 172.21.15.110:60278) 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cf898c700 1 -- 172.21.15.110:0/2915184307 learned_addr learned my addr 172.21.15.110:0/2915184307 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cf3fff700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cf40fcf60 0x7f7cf4112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cf898c700 1 -- 172.21.15.110:0/2915184307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cf40fcf60 msgr2=0x7f7cf4112c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cf898c700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cf40fcf60 0x7f7cf4112c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.381+0000 7f7cf898c700 1 -- 172.21.15.110:0/2915184307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce8005040 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.384 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.382+0000 7f7cf898c700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fbcb0 0x7f7cf41106f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00bbe0 tx=0x7f7cec009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:46.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.382+0000 7f7cf17fa700 1 -- 172.21.15.110:0/2915184307 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cec0092e0 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.382+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf41132a0 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.382+0000 7f7cf17fa700 1 -- 172.21.15.110:0/2915184307 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7cec00cb20 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.382+0000 7f7cf17fa700 1 -- 172.21.15.110:0/2915184307 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cec015db0 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.382+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf4113860 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.383+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf405f5c0 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.383+0000 7f7cf17fa700 1 -- 172.21.15.110:0/2915184307 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f7cec012070 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.384+0000 7f7cf17fa700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7cdc076b90 0x7f7cdc079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.384+0000 7f7cf17fa700 1 -- 172.21.15.110:0/2915184307 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7cec0981a0 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.384+0000 7f7cf3fff700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7cdc076b90 0x7f7cdc079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.386 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.384+0000 7f7cf3fff700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7cdc076b90 0x7f7cdc079040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800b970 tx=0x7f7ce8009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:46.389 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.388+0000 7f7cf17fa700 1 -- 172.21.15.110:0/2915184307 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7cec060c50 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.670 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.669+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7cf4113b80 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.670 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:46 smithi110 ceph-mon[90991]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:46.671 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.670+0000 7f7cf17fa700 1 -- 172.21.15.110:0/2915184307 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f7cec0605f0 con 0x7f7cf40fbcb0 2024-04-25T15:57:46.671 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:57:46.671 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:57:46.671 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:57:46.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:57:46.671 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:57:46.671 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:57:46.671 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 6 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 4 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 14 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:57:46.672 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:57:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7cdc076b90 msgr2=0x7f7cdc079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7cdc076b90 0x7f7cdc079040 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800b970 tx=0x7f7ce8009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:46.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fbcb0 msgr2=0x7f7cf41106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fbcb0 0x7f7cf41106f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7cec00bbe0 tx=0x7f7cec009840 comp rx=0 tx=0).stop 2024-04-25T15:57:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 shutdown_connections 2024-04-25T15:57:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7cdc076b90 0x7f7cdc079040 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cf40fbcb0 0x7f7cf41106f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 --2- 172.21.15.110:0/2915184307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7cf40fcf60 0x7f7cf4112c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 >> 172.21.15.110:0/2915184307 conn(0x7f7cf40f7430 msgr2=0x7f7cf40f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 shutdown_connections 2024-04-25T15:57:46.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.672+0000 7f7cfabf0700 1 -- 172.21.15.110:0/2915184307 wait complete. 2024-04-25T15:57:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 -- 172.21.15.110:0/3508941303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 msgr2=0x7f59100ffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 --2- 172.21.15.110:0/3508941303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 0x7f59100ffa20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5900004940 tx=0x7f5900009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 -- 172.21.15.110:0/3508941303 shutdown_connections 2024-04-25T15:57:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 --2- 172.21.15.110:0/3508941303 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59100fff60 0x7f5910102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 --2- 172.21.15.110:0/3508941303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 0x7f59100ffa20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 -- 172.21.15.110:0/3508941303 >> 172.21.15.110:0/3508941303 conn(0x7f59100f7430 msgr2=0x7f59100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:46.776 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 -- 172.21.15.110:0/3508941303 shutdown_connections 2024-04-25T15:57:46.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 -- 172.21.15.110:0/3508941303 wait complete. 2024-04-25T15:57:46.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 Processor -- start 2024-04-25T15:57:46.777 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 -- start start 2024-04-25T15:57:46.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 0x7f591010da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59100fff60 0x7f591010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.775+0000 7f5916208700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591010e570 con 0x7f5910066240 2024-04-25T15:57:46.778 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f5916208700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591010ed70 con 0x7f59100fff60 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f590f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 0x7f591010da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f590f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 0x7f591010da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60310/0 (socket says 172.21.15.110:60310) 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f590f7fe700 1 -- 172.21.15.110:0/1181027423 learned_addr learned my addr 172.21.15.110:0/1181027423 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f590effd700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59100fff60 0x7f591010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f590f7fe700 1 -- 172.21.15.110:0/1181027423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59100fff60 msgr2=0x7f591010df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f590f7fe700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59100fff60 0x7f591010df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f590f7fe700 1 -- 172.21.15.110:0/1181027423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900005040 con 0x7f5910066240 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f590f7fe700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 0x7f591010da30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5900000c00 tx=0x7f5900009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f5907fff700 1 -- 172.21.15.110:0/1181027423 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590000e030 con 0x7f5910066240 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f5907fff700 1 -- 172.21.15.110:0/1181027423 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5900013de0 con 0x7f5910066240 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591010eff0 con 0x7f5910066240 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f5907fff700 1 -- 172.21.15.110:0/1181027423 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590001ce50 con 0x7f5910066240 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.776+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591010f530 con 0x7f5910066240 2024-04-25T15:57:46.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.777+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591005f5c0 con 0x7f5910066240 2024-04-25T15:57:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.778+0000 7f5907fff700 1 -- 172.21.15.110:0/1181027423 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f590001c400 con 0x7f5910066240 2024-04-25T15:57:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.779+0000 7f5907fff700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f58fc076b10 0x7f58fc078fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.779+0000 7f590effd700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f58fc076b10 0x7f58fc078fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.779+0000 7f5907fff700 1 -- 172.21.15.110:0/1181027423 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f590009a150 con 0x7f5910066240 2024-04-25T15:57:46.780 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.780+0000 7f590effd700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f58fc076b10 0x7f58fc078fc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009b80 tx=0x7f58f800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:46.784 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:46.783+0000 7f5907fff700 1 -- 172.21.15.110:0/1181027423 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5900062c00 con 0x7f5910066240 2024-04-25T15:57:46.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:46 smithi193 ceph-mon[80071]: from='client.34474 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:47.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.033+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f59101058a0 con 0x7f5910066240 2024-04-25T15:57:47.034 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.033+0000 7f5907fff700 1 -- 172.21.15.110:0/1181027423 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1943 (secure 0 0 0) 0x7f59000625a0 con 0x7f5910066240 2024-04-25T15:57:47.034 INFO:teuthology.orchestra.run.smithi110.stdout:e35 2024-04-25T15:57:47.034 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 35 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:57:44.601048+0000 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:57:47.035 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 83 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:57:47.036 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 34404 members: 34404 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:34404} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/1518950103,v1:172.21.15.110:6829/1518950103] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/1783080009,v1:172.21.15.193:6825/1783080009] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:57:47.037 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/3103986931,v1:172.21.15.193:6827/3103986931] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:57:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.035+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f58fc076b10 msgr2=0x7f58fc078fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.035+0000 7f5916208700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f58fc076b10 0x7f58fc078fc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f58f8009b80 tx=0x7f58f800b040 comp rx=0 tx=0).stop 2024-04-25T15:57:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.035+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 msgr2=0x7f591010da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.036+0000 7f5916208700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 0x7f591010da30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5900000c00 tx=0x7f5900009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.036+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 shutdown_connections 2024-04-25T15:57:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.036+0000 7f5916208700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f58fc076b10 0x7f58fc078fc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.036+0000 7f5916208700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5910066240 0x7f591010da30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.036+0000 7f5916208700 1 --2- 172.21.15.110:0/1181027423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59100fff60 0x7f591010df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.036+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 >> 172.21.15.110:0/1181027423 conn(0x7f59100f7430 msgr2=0x7f59100f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.036+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 shutdown_connections 2024-04-25T15:57:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.036+0000 7f5916208700 1 -- 172.21.15.110:0/1181027423 wait complete. 2024-04-25T15:57:47.039 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.137+0000 7fa9a3070700 1 -- 172.21.15.110:0/4005728237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 msgr2=0x7fa99c0fe830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.137+0000 7fa9a3070700 1 --2- 172.21.15.110:0/4005728237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 0x7fa99c0fe830 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa990004940 tx=0x7fa990009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.137+0000 7fa9a3070700 1 -- 172.21.15.110:0/4005728237 shutdown_connections 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.137+0000 7fa9a3070700 1 --2- 172.21.15.110:0/4005728237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa99c0fed70 0x7fa99c0faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.137+0000 7fa9a3070700 1 --2- 172.21.15.110:0/4005728237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 0x7fa99c0fe830 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.137+0000 7fa9a3070700 1 -- 172.21.15.110:0/4005728237 >> 172.21.15.110:0/4005728237 conn(0x7fa99c0f6be0 msgr2=0x7fa99c0f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.137+0000 7fa9a3070700 1 -- 172.21.15.110:0/4005728237 shutdown_connections 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.137+0000 7fa9a3070700 1 -- 172.21.15.110:0/4005728237 wait complete. 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a3070700 1 Processor -- start 2024-04-25T15:57:47.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a3070700 1 -- start start 2024-04-25T15:57:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a3070700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 0x7fa99c10d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a3070700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa99c0fed70 0x7fa99c10de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a3070700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c10e440 con 0x7fa99c0fe420 2024-04-25T15:57:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a3070700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa99c10e5b0 con 0x7fa99c0fed70 2024-04-25T15:57:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a0e0c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 0x7fa99c10d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a0e0c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 0x7fa99c10d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60332/0 (socket says 172.21.15.110:60332) 2024-04-25T15:57:47.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa9a0e0c700 1 -- 172.21.15.110:0/116913873 learned_addr learned my addr 172.21.15.110:0/116913873 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.138+0000 7fa99bfff700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa99c0fed70 0x7fa99c10de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9a0e0c700 1 -- 172.21.15.110:0/116913873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa99c0fed70 msgr2=0x7fa99c10de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9a0e0c700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa99c0fed70 0x7fa99c10de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9a0e0c700 1 -- 172.21.15.110:0/116913873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa990005040 con 0x7fa99c0fe420 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9a0e0c700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 0x7fa99c10d900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b3b0 tx=0x7fa990009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9997fa700 1 -- 172.21.15.110:0/116913873 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa99000e040 con 0x7fa99c0fe420 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa99c112fb0 con 0x7fa99c0fe420 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9997fa700 1 -- 172.21.15.110:0/116913873 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa99000bbf0 con 0x7fa99c0fe420 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9997fa700 1 -- 172.21.15.110:0/116913873 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa99001c630 con 0x7fa99c0fe420 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.139+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa99c1134e0 con 0x7fa99c0fe420 2024-04-25T15:57:47.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.140+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa99c04c710 con 0x7fa99c0fe420 2024-04-25T15:57:47.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.141+0000 7fa9997fa700 1 -- 172.21.15.110:0/116913873 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fa99000bd60 con 0x7fa99c0fe420 2024-04-25T15:57:47.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.141+0000 7fa9997fa700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa98c076b90 0x7fa98c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:47.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.141+0000 7fa99bfff700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa98c076b90 0x7fa98c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:47.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.141+0000 7fa9997fa700 1 -- 172.21.15.110:0/116913873 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa990099020 con 0x7fa99c0fe420 2024-04-25T15:57:47.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.142+0000 7fa99bfff700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa98c076b90 0x7fa98c079040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa988004f40 tx=0x7fa98800b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:47.147 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.146+0000 7fa9997fa700 1 -- 172.21.15.110:0/116913873 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa990061ad0 con 0x7fa99c0fe420 2024-04-25T15:57:47.363 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.362+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa99c102540 con 0x7fa98c076b90 2024-04-25T15:57:47.364 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.363+0000 7fa9997fa700 1 -- 172.21.15.110:0/116913873 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa99c102540 con 0x7fa98c076b90 2024-04-25T15:57:47.364 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:57:47.364 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:57:47.364 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:57:47.364 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "mds", 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr", 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "crash" 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading mds daemons", 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:57:47.365 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.365+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa98c076b90 msgr2=0x7fa98c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa98c076b90 0x7fa98c079040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa988004f40 tx=0x7fa98800b040 comp rx=0 tx=0).stop 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 msgr2=0x7fa99c10d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 0x7fa99c10d900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa99000b3b0 tx=0x7fa990009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 shutdown_connections 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa98c076b90 0x7fa98c079040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa99c0fe420 0x7fa99c10d900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 --2- 172.21.15.110:0/116913873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa99c0fed70 0x7fa99c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 >> 172.21.15.110:0/116913873 conn(0x7fa99c0f6be0 msgr2=0x7fa99c0f8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:47.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 shutdown_connections 2024-04-25T15:57:47.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.366+0000 7fa9a3070700 1 -- 172.21.15.110:0/116913873 wait complete. 2024-04-25T15:57:47.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.469+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/179706955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4102360 msgr2=0x7fb7d4104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.469+0000 7fb7dbfeb700 1 --2- 172.21.15.110:0/179706955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4102360 0x7fb7d4104740 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4004940 tx=0x7fb7c4009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:47.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/179706955 shutdown_connections 2024-04-25T15:57:47.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 --2- 172.21.15.110:0/179706955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4102360 0x7fb7d4104740 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 --2- 172.21.15.110:0/179706955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d4066240 0x7fb7d4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.471 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/179706955 >> 172.21.15.110:0/179706955 conn(0x7fb7d40f7430 msgr2=0x7fb7d40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:47.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/179706955 shutdown_connections 2024-04-25T15:57:47.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/179706955 wait complete. 2024-04-25T15:57:47.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 Processor -- start 2024-04-25T15:57:47.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 -- start start 2024-04-25T15:57:47.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4066240 0x7fb7d410fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:47.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d4102360 0x7fb7d4110180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:47.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4110780 con 0x7fb7d4066240 2024-04-25T15:57:47.472 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.470+0000 7fb7dbfeb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7d4110f80 con 0x7fb7d4102360 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7d9d87700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4066240 0x7fb7d410fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7d9d87700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4066240 0x7fb7d410fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:60350/0 (socket says 172.21.15.110:60350) 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7d9d87700 1 -- 172.21.15.110:0/2313720614 learned_addr learned my addr 172.21.15.110:0/2313720614 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7d9586700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d4102360 0x7fb7d4110180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7d9d87700 1 -- 172.21.15.110:0/2313720614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d4102360 msgr2=0x7fb7d4110180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7d9d87700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d4102360 0x7fb7d4110180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7d9d87700 1 -- 172.21.15.110:0/2313720614 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c4005040 con 0x7fb7d4066240 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7d9d87700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4066240 0x7fb7d410fc40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000eb70 tx=0x7fb7d000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7ca7fc700 1 -- 172.21.15.110:0/2313720614 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d0007cc0 con 0x7fb7d4066240 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d4111200 con 0x7fb7d4066240 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7ca7fc700 1 -- 172.21.15.110:0/2313720614 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7d000f040 con 0x7fb7d4066240 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7ca7fc700 1 -- 172.21.15.110:0/2313720614 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d000cb20 con 0x7fb7d4066240 2024-04-25T15:57:47.474 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.471+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d4111740 con 0x7fb7d4066240 2024-04-25T15:57:47.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.472+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7d405f5c0 con 0x7fb7d4066240 2024-04-25T15:57:47.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.473+0000 7fb7ca7fc700 1 -- 172.21.15.110:0/2313720614 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fb7d000cc80 con 0x7fb7d4066240 2024-04-25T15:57:47.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.473+0000 7fb7ca7fc700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb7c0076b80 0x7fb7c0079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:57:47.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.474+0000 7fb7d9586700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb7c0076b80 0x7fb7c0079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:57:47.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.474+0000 7fb7ca7fc700 1 -- 172.21.15.110:0/2313720614 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fb7d0097e50 con 0x7fb7d4066240 2024-04-25T15:57:47.475 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.474+0000 7fb7d9586700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb7c0076b80 0x7fb7c0079030 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4013740 tx=0x7fb7c4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:57:47.480 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.479+0000 7fb7ca7fc700 1 -- 172.21.15.110:0/2313720614 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb7d0060930 con 0x7fb7d4066240 2024-04-25T15:57:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2915184307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:57:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: pgmap v217: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T15:57:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1181027423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:57:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:47.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.kdhjqx"}]': finished 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.mvvbtv"}]': finished 2024-04-25T15:57:47.741 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:57:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.ewxwfw"}]': finished 2024-04-25T15:57:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:57:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.jtyzhl"}]': finished 2024-04-25T15:57:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:47.742 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:47 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.778+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb7d4111ac0 con 0x7fb7d4066240 2024-04-25T15:57:47.779 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.778+0000 7fb7ca7fc700 1 -- 172.21.15.110:0/2313720614 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fb7d00602d0 con 0x7fb7d4066240 2024-04-25T15:57:47.779 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:57:47.779 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:57:47.780 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb7c0076b80 msgr2=0x7fb7c0079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb7c0076b80 0x7fb7c0079030 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4013740 tx=0x7fb7c4009e30 comp rx=0 tx=0).stop 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4066240 msgr2=0x7fb7d410fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4066240 0x7fb7d410fc40 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000eb70 tx=0x7fb7d000b040 comp rx=0 tx=0).stop 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 shutdown_connections 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb7c0076b80 0x7fb7c0079030 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7d4066240 0x7fb7d410fc40 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 --2- 172.21.15.110:0/2313720614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7d4102360 0x7fb7d4110180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.780+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 >> 172.21.15.110:0/2313720614 conn(0x7fb7d40f7430 msgr2=0x7fb7d40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:57:47.782 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.781+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 shutdown_connections 2024-04-25T15:57:47.783 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:57:47.781+0000 7fb7dbfeb700 1 -- 172.21.15.110:0/2313720614 wait complete. 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='client.34478 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='client.34482 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2915184307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: pgmap v217: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1181027423' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.kdhjqx"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.kdhjqx"}]': finished 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.mvvbtv"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi110.mvvbtv"}]': finished 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.ewxwfw"}]: dispatch 2024-04-25T15:57:48.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.ewxwfw"}]': finished 2024-04-25T15:57:48.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.jtyzhl"}]: dispatch 2024-04-25T15:57:48.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi193.jtyzhl"}]': finished 2024-04-25T15:57:48.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:47 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.824 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:48.824 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all mds 2024-04-25T15:57:48.824 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all rgw 2024-04-25T15:57:48.824 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all rbd-mirror 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all ceph-exporter 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all iscsi 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all nfs 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: Upgrade: Setting container_image for all nvmeof 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.825 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:48 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2313720614' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='client.34494 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all mds 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all rgw 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all rbd-mirror 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all ceph-exporter 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all iscsi 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all nfs 2024-04-25T15:57:48.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:57:48.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: Upgrade: Setting container_image for all nvmeof 2024-04-25T15:57:48.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:48.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:48 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2313720614' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:57:49.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:49 smithi110 ceph-mon[90991]: pgmap v218: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T15:57:49.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:49 smithi193 ceph-mon[80071]: pgmap v218: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T15:57:50.721 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:50 smithi110 ceph-mon[90991]: Upgrade: Updating node-exporter.smithi110 (1/2) 2024-04-25T15:57:50.721 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:50 smithi110 ceph-mon[90991]: Deploying daemon node-exporter.smithi110 on smithi110 2024-04-25T15:57:50.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:50 smithi193 ceph-mon[80071]: Upgrade: Updating node-exporter.smithi110 (1/2) 2024-04-25T15:57:50.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:50 smithi193 ceph-mon[80071]: Deploying daemon node-exporter.smithi110 on smithi110 2024-04-25T15:57:51.786 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:51 smithi110 ceph-mon[90991]: pgmap v219: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 9.3 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T15:57:51.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:51 smithi193 ceph-mon[80071]: pgmap v219: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 9.3 MiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T15:57:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:53 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:53.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:53 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:53.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:53 smithi193 ceph-mon[80071]: pgmap v220: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-25T15:57:53.303 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:53 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:53.303 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:53 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:53.303 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:53 smithi110 ceph-mon[90991]: pgmap v220: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.3 MiB/s rd, 4.2 KiB/s wr, 6 op/s 2024-04-25T15:57:54.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:54 smithi110 ceph-mon[90991]: Upgrade: Updating node-exporter.smithi193 (2/2) 2024-04-25T15:57:54.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:54 smithi110 ceph-mon[90991]: Deploying daemon node-exporter.smithi193 on smithi193 2024-04-25T15:57:54.699 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:54 smithi193 ceph-mon[80071]: Upgrade: Updating node-exporter.smithi193 (2/2) 2024-04-25T15:57:54.699 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:54 smithi193 ceph-mon[80071]: Deploying daemon node-exporter.smithi193 on smithi193 2024-04-25T15:57:55.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:55 smithi193 ceph-mon[80071]: pgmap v221: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:55.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:55 smithi110 ceph-mon[90991]: pgmap v221: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:57 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:57 smithi110 ceph-mon[90991]: pgmap v222: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 332 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:57.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:57.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:57.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:57 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:57:57.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:57 smithi193 ceph-mon[80071]: pgmap v222: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 332 KiB/s rd, 4.2 KiB/s wr, 5 op/s 2024-04-25T15:57:59.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:59.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:59.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:57:59 smithi110 ceph-mon[90991]: pgmap v223: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-04-25T15:57:59.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:59.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:57:59.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:57:59 smithi193 ceph-mon[80071]: pgmap v223: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 4.1 KiB/s wr, 3 op/s 2024-04-25T15:58:00.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:00 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:00.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:00 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:01.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:00 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:01.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:00 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:01.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:01.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:01.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:01 smithi193 ceph-mon[80071]: pgmap v224: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s wr, 1 op/s 2024-04-25T15:58:01.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:58:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:01 smithi110 ceph-mon[90991]: pgmap v224: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail; 4.1 KiB/s wr, 1 op/s 2024-04-25T15:58:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:58:02.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:02.942 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:03.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:03.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:03.978 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:03 smithi110 ceph-mon[90991]: pgmap v225: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:03.978 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:03.978 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:03 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:04.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:03 smithi193 ceph-mon[80071]: pgmap v225: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:04.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:04.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:03 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: pgmap v226: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:05 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: pgmap v226: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:06.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:05 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:07 smithi110 ceph-mon[90991]: pgmap v227: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:08.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:07 smithi193 ceph-mon[80071]: pgmap v227: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:08 smithi110 ceph-mon[90991]: Upgrade: Updating prometheus.smithi110 2024-04-25T15:58:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:08 smithi110 ceph-mon[90991]: Deploying daemon prometheus.smithi110 on smithi110 2024-04-25T15:58:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:08 smithi193 ceph-mon[80071]: Upgrade: Updating prometheus.smithi110 2024-04-25T15:58:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:08 smithi193 ceph-mon[80071]: Deploying daemon prometheus.smithi110 on smithi110 2024-04-25T15:58:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:09 smithi110 ceph-mon[90991]: pgmap v228: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:09 smithi193 ceph-mon[80071]: pgmap v228: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:10.985 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:10 smithi110 ceph-mon[90991]: pgmap v229: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:10.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:10 smithi193 ceph-mon[80071]: pgmap v229: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:14.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:13 smithi110 ceph-mon[90991]: pgmap v230: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:14.075 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:14.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:14.076 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:13 smithi193 ceph-mon[80071]: pgmap v230: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:15.855 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:15 smithi110 ceph-mon[90991]: pgmap v231: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:15 smithi193 ceph-mon[80071]: pgmap v231: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:17.112 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:58:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:58:17.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.889+0000 7f78a9e94700 1 -- 172.21.15.110:0/583072780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 msgr2=0x7f78a40ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:17.892 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.889+0000 7f78a9e94700 1 --2- 172.21.15.110:0/583072780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 0x7f78a40ffa50 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7894004940 tx=0x7f7894009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.889+0000 7f78a9e94700 1 -- 172.21.15.110:0/583072780 shutdown_connections 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.889+0000 7f78a9e94700 1 --2- 172.21.15.110:0/583072780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a40fff90 0x7f78a4102370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.889+0000 7f78a9e94700 1 --2- 172.21.15.110:0/583072780 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 0x7f78a40ffa50 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.889+0000 7f78a9e94700 1 -- 172.21.15.110:0/583072780 >> 172.21.15.110:0/583072780 conn(0x7f78a40f7440 msgr2=0x7f78a40f9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.889+0000 7f78a9e94700 1 -- 172.21.15.110:0/583072780 shutdown_connections 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.889+0000 7f78a9e94700 1 -- 172.21.15.110:0/583072780 wait complete. 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.890+0000 7f78a9e94700 1 Processor -- start 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.890+0000 7f78a9e94700 1 -- start start 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.890+0000 7f78a9e94700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 0x7f78a410da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.890+0000 7f78a9e94700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a40fff90 0x7f78a410df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.890+0000 7f78a9e94700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a410e580 con 0x7f78a4066ad0 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.890+0000 7f78a9e94700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a410edd0 con 0x7f78a40fff90 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 0x7f78a410da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 0x7f78a410da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41122/0 (socket says 172.21.15.110:41122) 2024-04-25T15:58:17.893 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a37fe700 1 -- 172.21.15.110:0/2660733332 learned_addr learned my addr 172.21.15.110:0/2660733332 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a2ffd700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a40fff90 0x7f78a410df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a37fe700 1 -- 172.21.15.110:0/2660733332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a40fff90 msgr2=0x7f78a410df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a37fe700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a40fff90 0x7f78a410df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a37fe700 1 -- 172.21.15.110:0/2660733332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7894005040 con 0x7f78a4066ad0 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a37fe700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 0x7f78a410da40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f789400b3b0 tx=0x7f7894009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f788bfff700 1 -- 172.21.15.110:0/2660733332 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f789400e040 con 0x7f78a4066ad0 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a410f050 con 0x7f78a4066ad0 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f788bfff700 1 -- 172.21.15.110:0/2660733332 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f789400bcc0 con 0x7f78a4066ad0 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a410f590 con 0x7f78a4066ad0 2024-04-25T15:58:17.894 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.891+0000 7f788bfff700 1 -- 172.21.15.110:0/2660733332 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f789401c630 con 0x7f78a4066ad0 2024-04-25T15:58:17.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.892+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a405f5c0 con 0x7f78a4066ad0 2024-04-25T15:58:17.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.894+0000 7f788bfff700 1 -- 172.21.15.110:0/2660733332 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f78940139f0 con 0x7f78a4066ad0 2024-04-25T15:58:17.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.894+0000 7f788bfff700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f788c076b90 0x7f788c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:17.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.894+0000 7f788bfff700 1 -- 172.21.15.110:0/2660733332 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f78940997c0 con 0x7f78a4066ad0 2024-04-25T15:58:17.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.895+0000 7f78a2ffd700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f788c076b90 0x7f788c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:17.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.895+0000 7f78a2ffd700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f788c076b90 0x7f788c079040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7898004990 tx=0x7f7898009ee0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:17.899 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:17.898+0000 7f788bfff700 1 -- 172.21.15.110:0/2660733332 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7894062310 con 0x7f78a4066ad0 2024-04-25T15:58:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:17 smithi110 ceph-mon[90991]: pgmap v232: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:18.119 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.118+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78a410f9d0 con 0x7f788c076b90 2024-04-25T15:58:18.120 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.120+0000 7f788bfff700 1 -- 172.21.15.110:0/2660733332 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f78a410f9d0 con 0x7f788c076b90 2024-04-25T15:58:18.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f788c076b90 msgr2=0x7f788c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f788c076b90 0x7f788c079040 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f7898004990 tx=0x7f7898009ee0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.123 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 msgr2=0x7f78a410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 0x7f78a410da40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f789400b3b0 tx=0x7f7894009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:18.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 shutdown_connections 2024-04-25T15:58:18.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f788c076b90 0x7f788c079040 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78a4066ad0 0x7f78a410da40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 --2- 172.21.15.110:0/2660733332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78a40fff90 0x7f78a410df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 >> 172.21.15.110:0/2660733332 conn(0x7f78a40f7440 msgr2=0x7f78a40f7860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:18.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 shutdown_connections 2024-04-25T15:58:18.124 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.122+0000 7f78a9e94700 1 -- 172.21.15.110:0/2660733332 wait complete. 2024-04-25T15:58:18.134 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.226+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/2016183378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f4066220 msgr2=0x7fb6f40ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.226+0000 7fb6f8d7b700 1 --2- 172.21.15.110:0/2016183378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f4066220 0x7fb6f40ffa40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4004940 tx=0x7fb6e4009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/2016183378 shutdown_connections 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 --2- 172.21.15.110:0/2016183378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f40fff80 0x7fb6f4102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 --2- 172.21.15.110:0/2016183378 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f4066220 0x7fb6f40ffa40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/2016183378 >> 172.21.15.110:0/2016183378 conn(0x7fb6f40f7430 msgr2=0x7fb6f40f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/2016183378 shutdown_connections 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/2016183378 wait complete. 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 Processor -- start 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 -- start start 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f4066220 0x7fb6f410da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f40fff80 0x7fb6f410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.227+0000 7fb6f8d7b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f410e570 con 0x7fb6f40fff80 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f8d7b700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f410edc0 con 0x7fb6f4066220 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f1d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f40fff80 0x7fb6f410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f1d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f40fff80 0x7fb6f410df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41132/0 (socket says 172.21.15.110:41132) 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f1d9b700 1 -- 172.21.15.110:0/295305995 learned_addr learned my addr 172.21.15.110:0/295305995 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f1d9b700 1 -- 172.21.15.110:0/295305995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f4066220 msgr2=0x7fb6f410da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f259c700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f4066220 0x7fb6f410da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f1d9b700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f4066220 0x7fb6f410da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f1d9b700 1 -- 172.21.15.110:0/295305995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e4005040 con 0x7fb6f40fff80 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f1d9b700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f40fff80 0x7fb6f410df70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c8c0 tx=0x7fb6e80044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6e2ffd700 1 -- 172.21.15.110:0/295305995 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6e800a980 con 0x7fb6f40fff80 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f410f0a0 con 0x7fb6f40fff80 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6e2ffd700 1 -- 172.21.15.110:0/295305995 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6e800ce60 con 0x7fb6f40fff80 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f410f5e0 con 0x7fb6f40fff80 2024-04-25T15:58:18.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.228+0000 7fb6e2ffd700 1 -- 172.21.15.110:0/295305995 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb6e80115e0 con 0x7fb6f40fff80 2024-04-25T15:58:18.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.229+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f405f5c0 con 0x7fb6f40fff80 2024-04-25T15:58:18.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.230+0000 7fb6e2ffd700 1 -- 172.21.15.110:0/295305995 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fb6e8012400 con 0x7fb6f40fff80 2024-04-25T15:58:18.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.230+0000 7fb6e2ffd700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb6dc076b90 0x7fb6dc079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.231+0000 7fb6e2ffd700 1 -- 172.21.15.110:0/295305995 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fb6e8098520 con 0x7fb6f40fff80 2024-04-25T15:58:18.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.231+0000 7fb6f259c700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb6dc076b90 0x7fb6dc079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.231+0000 7fb6f259c700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb6dc076b90 0x7fb6dc079040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b970 tx=0x7fb6e4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:17 smithi193 ceph-mon[80071]: pgmap v232: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:18.241 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.240+0000 7fb6e2ffd700 1 -- 172.21.15.110:0/295305995 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb6e8060fd0 con 0x7fb6f40fff80 2024-04-25T15:58:18.488 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.487+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6f410fa70 con 0x7fb6dc076b90 2024-04-25T15:58:18.489 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.488+0000 7fb6e2ffd700 1 -- 172.21.15.110:0/295305995 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7fb6f410fa70 con 0x7fb6dc076b90 2024-04-25T15:58:18.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb6dc076b90 msgr2=0x7fb6dc079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb6dc076b90 0x7fb6dc079040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400b970 tx=0x7fb6e4009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:18.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f40fff80 msgr2=0x7fb6f410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f40fff80 0x7fb6f410df70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800c8c0 tx=0x7fb6e80044c0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 shutdown_connections 2024-04-25T15:58:18.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6f4066220 0x7fb6f410da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb6dc076b90 0x7fb6dc079040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 --2- 172.21.15.110:0/295305995 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb6f40fff80 0x7fb6f410df70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 >> 172.21.15.110:0/295305995 conn(0x7fb6f40f7430 msgr2=0x7fb6f40f7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:18.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 shutdown_connections 2024-04-25T15:58:18.493 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.491+0000 7fb6f8d7b700 1 -- 172.21.15.110:0/295305995 wait complete. 2024-04-25T15:58:18.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 -- 172.21.15.110:0/2637906216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 msgr2=0x7f97f0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 --2- 172.21.15.110:0/2637906216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 0x7f97f0102310 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f97e4004940 tx=0x7f97e4009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:18.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 -- 172.21.15.110:0/2637906216 shutdown_connections 2024-04-25T15:58:18.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 --2- 172.21.15.110:0/2637906216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 0x7f97f0102310 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 --2- 172.21.15.110:0/2637906216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f0066f30 0x7f97f00ff9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 -- 172.21.15.110:0/2637906216 >> 172.21.15.110:0/2637906216 conn(0x7f97f00f7440 msgr2=0x7f97f00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:18.601 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 -- 172.21.15.110:0/2637906216 shutdown_connections 2024-04-25T15:58:18.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 -- 172.21.15.110:0/2637906216 wait complete. 2024-04-25T15:58:18.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 Processor -- start 2024-04-25T15:58:18.602 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.600+0000 7f97f4f51700 1 -- start start 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97f4f51700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f0066f30 0x7f97f010faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97f4f51700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 0x7f97f0110030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97f4f51700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f0110630 con 0x7f97f00ffef0 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97f4f51700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f01107a0 con 0x7f97f0066f30 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97edd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 0x7f97f0110030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97edd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 0x7f97f0110030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41164/0 (socket says 172.21.15.110:41164) 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97edd9b700 1 -- 172.21.15.110:0/4282393681 learned_addr learned my addr 172.21.15.110:0/4282393681 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97ee59c700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f0066f30 0x7f97f010faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.603 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97edd9b700 1 -- 172.21.15.110:0/4282393681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f0066f30 msgr2=0x7f97f010faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97edd9b700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f0066f30 0x7f97f010faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97edd9b700 1 -- 172.21.15.110:0/4282393681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4005040 con 0x7f97f00ffef0 2024-04-25T15:58:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97edd9b700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 0x7f97f0110030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97e4013870 tx=0x7f97e4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97deffd700 1 -- 172.21.15.110:0/4282393681 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e4021070 con 0x7f97f00ffef0 2024-04-25T15:58:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97deffd700 1 -- 172.21.15.110:0/4282393681 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f97e401c400 con 0x7f97f00ffef0 2024-04-25T15:58:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f01151a0 con 0x7f97f00ffef0 2024-04-25T15:58:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.601+0000 7f97deffd700 1 -- 172.21.15.110:0/4282393681 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97e400e070 con 0x7f97f00ffef0 2024-04-25T15:58:18.604 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.602+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f01156e0 con 0x7f97f00ffef0 2024-04-25T15:58:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.602+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f004c710 con 0x7f97f00ffef0 2024-04-25T15:58:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.603+0000 7f97deffd700 1 -- 172.21.15.110:0/4282393681 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f97e401ca20 con 0x7f97f00ffef0 2024-04-25T15:58:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.604+0000 7f97deffd700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f97d8076b90 0x7f97d8079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.604+0000 7f97deffd700 1 -- 172.21.15.110:0/4282393681 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f97e4099660 con 0x7f97f00ffef0 2024-04-25T15:58:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.604+0000 7f97ee59c700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f97d8076b90 0x7f97d8079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.605 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.604+0000 7f97ee59c700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f97d8076b90 0x7f97d8079040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f97e0004600 tx=0x7f97e000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:18.609 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.609+0000 7f97deffd700 1 -- 172.21.15.110:0/4282393681 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f97e4062110 con 0x7f97f00ffef0 2024-04-25T15:58:18.824 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.823+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f97f00fd170 con 0x7f97d8076b90 2024-04-25T15:58:18.834 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.833+0000 7f97deffd700 1 -- 172.21.15.110:0/4282393681 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f97f00fd170 con 0x7f97d8076b90 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (8m) 0s ago 17m 25.3M - 0.25.0 c8568f914cd2 348b3803c9d5 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (5m) 0s ago 17m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (5m) 20s ago 16m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (8m) 0s ago 16m 84.3M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (84s) 0s ago 11m 82.9M - 19.0.0-3291-g994e2f20 d53c96779a29 051262111b6b 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (112s) 0s ago 11m 160M - 19.0.0-3291-g994e2f20 d53c96779a29 763d1dda41b7 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (62s) 20s ago 11m 82.7M - 19.0.0-3291-g994e2f20 d53c96779a29 36e42c64ae3a 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (38s) 20s ago 11m 79.6M - 19.0.0-3291-g994e2f20 d53c96779a29 3dee50406dde 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (8m) 0s ago 18m 581M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (6m) 20s ago 16m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (6m) 0s ago 18m 121M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:58:18.835 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (6m) 20s ago 16m 105M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (27s) 0s ago 17m 7936k - 1.5.0 0da6a335fe13 c74eafffc54d 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (22s) 20s ago 16m 3556k - 1.5.0 0da6a335fe13 a766013ab03e 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (5m) 0s ago 15m 167M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (4m) 0s ago 15m 171M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (3m) 0s ago 14m 157M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (3m) 20s ago 14m 176M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (2m) 20s ago 13m 159M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (2m) 20s ago 13m 160M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 31c4aca48886 2024-04-25T15:58:18.836 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (6s) 0s ago 16m 42.0M - 2.43.0 a07b618ecd1d 01d861aba74b 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f97d8076b90 msgr2=0x7f97d8079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f97d8076b90 0x7f97d8079040 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f97e0004600 tx=0x7f97e000b040 comp rx=0 tx=0).stop 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 msgr2=0x7f97f0110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 0x7f97f0110030 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97e4013870 tx=0x7f97e4009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 shutdown_connections 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f0066f30 0x7f97f010faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f97d8076b90 0x7f97d8079040 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 --2- 172.21.15.110:0/4282393681 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97f00ffef0 0x7f97f0110030 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 >> 172.21.15.110:0/4282393681 conn(0x7f97f00f7440 msgr2=0x7f97f00fba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 shutdown_connections 2024-04-25T15:58:18.837 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.836+0000 7f97f4f51700 1 -- 172.21.15.110:0/4282393681 wait complete. 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.943+0000 7f7947abd700 1 -- 172.21.15.110:0/3086389107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 msgr2=0x7f79400fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.943+0000 7f7947abd700 1 --2- 172.21.15.110:0/3086389107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 0x7f79400fe500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7930004940 tx=0x7f7930009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.943+0000 7f7947abd700 1 -- 172.21.15.110:0/3086389107 shutdown_connections 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.943+0000 7f7947abd700 1 --2- 172.21.15.110:0/3086389107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 0x7f79400fe500 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.943+0000 7f7947abd700 1 --2- 172.21.15.110:0/3086389107 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79400fce90 0x7f79400fd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.943+0000 7f7947abd700 1 -- 172.21.15.110:0/3086389107 >> 172.21.15.110:0/3086389107 conn(0x7f79400f85c0 msgr2=0x7f79400fa9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7947abd700 1 -- 172.21.15.110:0/3086389107 shutdown_connections 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7947abd700 1 -- 172.21.15.110:0/3086389107 wait complete. 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7947abd700 1 Processor -- start 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7947abd700 1 -- start start 2024-04-25T15:58:18.946 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7947abd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79400fce90 0x7f7940071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7947abd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 0x7f79400702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7947abd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79400708f0 con 0x7f79400fe090 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7947abd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7940070a60 con 0x7f79400fce90 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7945058700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 0x7f79400702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7945058700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 0x7f79400702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41178/0 (socket says 172.21.15.110:41178) 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.944+0000 7f7945058700 1 -- 172.21.15.110:0/3369688122 learned_addr learned my addr 172.21.15.110:0/3369688122 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f7945859700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79400fce90 0x7f7940071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f7945058700 1 -- 172.21.15.110:0/3369688122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79400fce90 msgr2=0x7f7940071de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f7945058700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79400fce90 0x7f7940071de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f7945058700 1 -- 172.21.15.110:0/3369688122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7930005040 con 0x7f79400fe090 2024-04-25T15:58:18.947 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f7945058700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 0x7f79400702b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f793000b970 tx=0x7f7930009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:18.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f79367fc700 1 -- 172.21.15.110:0/3369688122 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f793000e040 con 0x7f79400fe090 2024-04-25T15:58:18.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f79367fc700 1 -- 172.21.15.110:0/3369688122 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7930004290 con 0x7f79400fe090 2024-04-25T15:58:18.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f7947abd700 1 -- 172.21.15.110:0/3369688122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7940070ce0 con 0x7f79400fe090 2024-04-25T15:58:18.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f79367fc700 1 -- 172.21.15.110:0/3369688122 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f793001c630 con 0x7f79400fe090 2024-04-25T15:58:18.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.945+0000 7f7947abd700 1 -- 172.21.15.110:0/3369688122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7940115a30 con 0x7f79400fe090 2024-04-25T15:58:18.948 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.947+0000 7f792bfff700 1 -- 172.21.15.110:0/3369688122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794005f5c0 con 0x7f79400fe090 2024-04-25T15:58:18.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.948+0000 7f79367fc700 1 -- 172.21.15.110:0/3369688122 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f793000bbf0 con 0x7f79400fe090 2024-04-25T15:58:18.949 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.948+0000 7f79367fc700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f792c076b80 0x7f792c079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:18.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.948+0000 7f79367fc700 1 -- 172.21.15.110:0/3369688122 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7930099ec0 con 0x7f79400fe090 2024-04-25T15:58:18.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.948+0000 7f7945859700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f792c076b80 0x7f792c079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:18.950 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.949+0000 7f7945859700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f792c076b80 0x7f792c079030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7940071b70 tx=0x7f793c0091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:18.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:18.953+0000 7f79367fc700 1 -- 172.21.15.110:0/3369688122 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7930062990 con 0x7f79400fe090 2024-04-25T15:58:19.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:18 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:19.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:18 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:18 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:18 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:19.256 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.255+0000 7f792bfff700 1 -- 172.21.15.110:0/3369688122 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f794004bed0 con 0x7f79400fe090 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.256+0000 7f79367fc700 1 -- 172.21.15.110:0/3369688122 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f793000bea0 con 0x7f79400fe090 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 6 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 4 2024-04-25T15:58:19.257 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:58:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:58:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 14 2024-04-25T15:58:19.258 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:58:19.258 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:58:19.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 -- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f792c076b80 msgr2=0x7f792c079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f792c076b80 0x7f792c079030 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7940071b70 tx=0x7f793c0091a0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 -- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 msgr2=0x7f79400702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 0x7f79400702b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f793000b970 tx=0x7f7930009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 -- 172.21.15.110:0/3369688122 shutdown_connections 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79400fce90 0x7f7940071de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f792c076b80 0x7f792c079030 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 --2- 172.21.15.110:0/3369688122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f79400fe090 0x7f79400702b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 -- 172.21.15.110:0/3369688122 >> 172.21.15.110:0/3369688122 conn(0x7f79400f85c0 msgr2=0x7f79400f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 -- 172.21.15.110:0/3369688122 shutdown_connections 2024-04-25T15:58:19.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.258+0000 7f792bfff700 1 -- 172.21.15.110:0/3369688122 wait complete. 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.362+0000 7fe2103cb700 1 -- 172.21.15.110:0/2139609160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 msgr2=0x7fe2080fc0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.362+0000 7fe2103cb700 1 --2- 172.21.15.110:0/2139609160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 0x7fe2080fc0c0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe204004940 tx=0x7fe204009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.362+0000 7fe2103cb700 1 -- 172.21.15.110:0/2139609160 shutdown_connections 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.362+0000 7fe2103cb700 1 --2- 172.21.15.110:0/2139609160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2080fcf60 0x7fe2080fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.362+0000 7fe2103cb700 1 --2- 172.21.15.110:0/2139609160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 0x7fe2080fc0c0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.362+0000 7fe2103cb700 1 -- 172.21.15.110:0/2139609160 >> 172.21.15.110:0/2139609160 conn(0x7fe2080f7430 msgr2=0x7fe2080f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.362+0000 7fe2103cb700 1 -- 172.21.15.110:0/2139609160 shutdown_connections 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe2103cb700 1 -- 172.21.15.110:0/2139609160 wait complete. 2024-04-25T15:58:19.365 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe2103cb700 1 Processor -- start 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe2103cb700 1 -- start start 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe2103cb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 0x7fe20810dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe2103cb700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2080fcf60 0x7fe20810e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe2103cb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20810e620 con 0x7fe2080fbcb0 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe2103cb700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe20810edd0 con 0x7fe2080fcf60 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe20e167700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 0x7fe20810dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe20e167700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 0x7fe20810dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:41186/0 (socket says 172.21.15.110:41186) 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.363+0000 7fe20e167700 1 -- 172.21.15.110:0/289416134 learned_addr learned my addr 172.21.15.110:0/289416134 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe20d966700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2080fcf60 0x7fe20810e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe20e167700 1 -- 172.21.15.110:0/289416134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2080fcf60 msgr2=0x7fe20810e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe20e167700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2080fcf60 0x7fe20810e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe20e167700 1 -- 172.21.15.110:0/289416134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204005040 con 0x7fe2080fbcb0 2024-04-25T15:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe20d966700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2080fcf60 0x7fe20810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe20e167700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 0x7fe20810dae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe204009150 tx=0x7fe204009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe1feffd700 1 -- 172.21.15.110:0/289416134 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20400e040 con 0x7fe2080fbcb0 2024-04-25T15:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20810f050 con 0x7fe2080fbcb0 2024-04-25T15:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe1feffd700 1 -- 172.21.15.110:0/289416134 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe20400bcc0 con 0x7fe2080fbcb0 2024-04-25T15:58:19.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe1feffd700 1 -- 172.21.15.110:0/289416134 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20401c630 con 0x7fe2080fbcb0 2024-04-25T15:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.364+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20810f590 con 0x7fe2080fbcb0 2024-04-25T15:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.365+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe208101af0 con 0x7fe2080fbcb0 2024-04-25T15:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.366+0000 7fe1feffd700 1 -- 172.21.15.110:0/289416134 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fe204013420 con 0x7fe2080fbcb0 2024-04-25T15:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.366+0000 7fe1feffd700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe1f4076b90 0x7fe1f4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.366+0000 7fe1feffd700 1 -- 172.21.15.110:0/289416134 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fe204099100 con 0x7fe2080fbcb0 2024-04-25T15:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.366+0000 7fe20d966700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe1f4076b90 0x7fe1f4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:19.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.367+0000 7fe20d966700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe1f4076b90 0x7fe1f4079040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8007fd0 tx=0x7fe1f8009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:19.376 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.376+0000 7fe1feffd700 1 -- 172.21.15.110:0/289416134 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fe204061bb0 con 0x7fe2080fbcb0 2024-04-25T15:58:19.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.638+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe208101510 con 0x7fe2080fbcb0 2024-04-25T15:58:19.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.638+0000 7fe1feffd700 1 -- 172.21.15.110:0/289416134 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1943 (secure 0 0 0) 0x7fe204061550 con 0x7fe2080fbcb0 2024-04-25T15:58:19.639 INFO:teuthology.orchestra.run.smithi110.stdout:e35 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 35 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:57:44.601048+0000 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:58:19.640 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 83 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:58:19.641 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 34404 members: 34404 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:34404} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/1518950103,v1:172.21.15.110:6829/1518950103] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:58:19.642 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/1783080009,v1:172.21.15.193:6825/1783080009] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:58:19.643 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/3103986931,v1:172.21.15.193:6827/3103986931] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:58:19.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe1f4076b90 msgr2=0x7fe1f4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe1f4076b90 0x7fe1f4079040 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8007fd0 tx=0x7fe1f8009040 comp rx=0 tx=0).stop 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 msgr2=0x7fe20810dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 0x7fe20810dae0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe204009150 tx=0x7fe204009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 shutdown_connections 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fe1f4076b90 0x7fe1f4079040 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2080fbcb0 0x7fe20810dae0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 --2- 172.21.15.110:0/289416134 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2080fcf60 0x7fe20810e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 >> 172.21.15.110:0/289416134 conn(0x7fe2080f7430 msgr2=0x7fe208065b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.640+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 shutdown_connections 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.641+0000 7fe2103cb700 1 -- 172.21.15.110:0/289416134 wait complete. 2024-04-25T15:58:19.644 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-04-25T15:58:19.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.744+0000 7f5c56f11700 1 -- 172.21.15.110:0/1499457416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fce60 msgr2=0x7f5c500fd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.744+0000 7f5c56f11700 1 --2- 172.21.15.110:0/1499457416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fce60 0x7f5c500fd270 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5c44004940 tx=0x7f5c44009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:19.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.744+0000 7f5c56f11700 1 -- 172.21.15.110:0/1499457416 shutdown_connections 2024-04-25T15:58:19.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.744+0000 7f5c56f11700 1 --2- 172.21.15.110:0/1499457416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c500fdfd0 0x7f5c500fe440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.744+0000 7f5c56f11700 1 --2- 172.21.15.110:0/1499457416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fce60 0x7f5c500fd270 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.744+0000 7f5c56f11700 1 -- 172.21.15.110:0/1499457416 >> 172.21.15.110:0/1499457416 conn(0x7f5c500f8520 msgr2=0x7f5c500fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.744+0000 7f5c56f11700 1 -- 172.21.15.110:0/1499457416 shutdown_connections 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.744+0000 7f5c56f11700 1 -- 172.21.15.110:0/1499457416 wait complete. 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c56f11700 1 Processor -- start 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c56f11700 1 -- start start 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c56f11700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c500fce60 0x7f5c5010d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c56f11700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fdfd0 0x7f5c5010dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c56f11700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5010e530 con 0x7f5c500fdfd0 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c56f11700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c50112ed0 con 0x7f5c500fce60 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fdfd0 0x7f5c5010dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c4ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fdfd0 0x7f5c5010dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35260/0 (socket says 172.21.15.110:35260) 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.745+0000 7f5c4ffff700 1 -- 172.21.15.110:0/1337352810 learned_addr learned my addr 172.21.15.110:0/1337352810 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c54cad700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c500fce60 0x7f5c5010d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:19.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c4ffff700 1 -- 172.21.15.110:0/1337352810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c500fce60 msgr2=0x7f5c5010d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c4ffff700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c500fce60 0x7f5c5010d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c4ffff700 1 -- 172.21.15.110:0/1337352810 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c44005040 con 0x7f5c500fdfd0 2024-04-25T15:58:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c4ffff700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fdfd0 0x7f5c5010dea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00b790 tx=0x7f5c3c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c4d7fa700 1 -- 172.21.15.110:0/1337352810 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c3c00f770 con 0x7f5c500fdfd0 2024-04-25T15:58:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c50113130 con 0x7f5c500fdfd0 2024-04-25T15:58:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c4d7fa700 1 -- 172.21.15.110:0/1337352810 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c3c00fda0 con 0x7f5c500fdfd0 2024-04-25T15:58:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c501136a0 con 0x7f5c500fdfd0 2024-04-25T15:58:19.749 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.746+0000 7f5c4d7fa700 1 -- 172.21.15.110:0/1337352810 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c3c01a640 con 0x7f5c500fdfd0 2024-04-25T15:58:19.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.747+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5004c710 con 0x7f5c500fdfd0 2024-04-25T15:58:19.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.748+0000 7f5c4d7fa700 1 -- 172.21.15.110:0/1337352810 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f5c3c010420 con 0x7f5c500fdfd0 2024-04-25T15:58:19.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.748+0000 7f5c4d7fa700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5c40076b90 0x7f5c40079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:19.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.748+0000 7f5c4d7fa700 1 -- 172.21.15.110:0/1337352810 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5c3c098ca0 con 0x7f5c500fdfd0 2024-04-25T15:58:19.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.748+0000 7f5c54cad700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5c40076b90 0x7f5c40079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:19.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.749+0000 7f5c54cad700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5c40076b90 0x7f5c40079040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400b970 tx=0x7f5c44009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:19.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.753+0000 7f5c4d7fa700 1 -- 172.21.15.110:0/1337352810 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5c3c0617f0 con 0x7f5c500fdfd0 2024-04-25T15:58:19.960 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:19 smithi110 ceph-mon[90991]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:19.960 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:19 smithi110 ceph-mon[90991]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:19.960 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:19 smithi110 ceph-mon[90991]: pgmap v233: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:19.960 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:19 smithi110 ceph-mon[90991]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:19.960 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:19 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:19.961 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:19 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:19.961 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:19 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3369688122' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:19.961 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:19 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/289416134' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:58:19.986 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.985+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c501022a0 con 0x7f5c40076b90 2024-04-25T15:58:19.987 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.986+0000 7f5c4d7fa700 1 -- 172.21.15.110:0/1337352810 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+438 (secure 0 0 0) 0x7f5c501022a0 con 0x7f5c40076b90 2024-04-25T15:58:19.987 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "mds", 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr", 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "crash" 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading prometheus daemons", 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:58:19.988 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.988+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5c40076b90 msgr2=0x7f5c40079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.988+0000 7f5c56f11700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5c40076b90 0x7f5c40079040 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5c4400b970 tx=0x7f5c44009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.988+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fdfd0 msgr2=0x7f5c5010dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.988+0000 7f5c56f11700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fdfd0 0x7f5c5010dea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00b790 tx=0x7f5c3c009250 comp rx=0 tx=0).stop 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.989+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 shutdown_connections 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.989+0000 7f5c56f11700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c500fce60 0x7f5c5010d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.989+0000 7f5c56f11700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5c40076b90 0x7f5c40079040 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.989+0000 7f5c56f11700 1 --2- 172.21.15.110:0/1337352810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5c500fdfd0 0x7f5c5010dea0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.989+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 >> 172.21.15.110:0/1337352810 conn(0x7f5c500f8520 msgr2=0x7f5c500f9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.989+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 shutdown_connections 2024-04-25T15:58:19.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:19.989+0000 7f5c56f11700 1 -- 172.21.15.110:0/1337352810 wait complete. 2024-04-25T15:58:20.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 -- 172.21.15.110:0/3186472517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fcf60 msgr2=0x7f90100fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:20.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 --2- 172.21.15.110:0/3186472517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fcf60 0x7f90100fd3d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9004004940 tx=0x7f9004009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:20.092 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 -- 172.21.15.110:0/3186472517 shutdown_connections 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 --2- 172.21.15.110:0/3186472517 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fcf60 0x7f90100fd3d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 --2- 172.21.15.110:0/3186472517 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90100fbcb0 0x7f90100fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 -- 172.21.15.110:0/3186472517 >> 172.21.15.110:0/3186472517 conn(0x7f90100f7430 msgr2=0x7f90100f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 -- 172.21.15.110:0/3186472517 shutdown_connections 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 -- 172.21.15.110:0/3186472517 wait complete. 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 Processor -- start 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 -- start start 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fbcb0 0x7f901010d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90100fcf60 0x7f901010df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901010e5c0 con 0x7f90100fbcb0 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.090+0000 7f9014f22700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901010edb0 con 0x7f90100fcf60 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f900e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fbcb0 0x7f901010d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:20.093 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f900e59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fbcb0 0x7f901010d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:35274/0 (socket says 172.21.15.110:35274) 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f900e59c700 1 -- 172.21.15.110:0/3401971103 learned_addr learned my addr 172.21.15.110:0/3401971103 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f900dd9b700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90100fcf60 0x7f901010df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f900e59c700 1 -- 172.21.15.110:0/3401971103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90100fcf60 msgr2=0x7f901010df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f900e59c700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90100fcf60 0x7f901010df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f900e59c700 1 -- 172.21.15.110:0/3401971103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004005040 con 0x7f90100fbcb0 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f900e59c700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fbcb0 0x7f901010d9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f900800bbb0 tx=0x7f9008009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f8ffeffd700 1 -- 172.21.15.110:0/3401971103 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90080092e0 con 0x7f90100fbcb0 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f901010f090 con 0x7f90100fbcb0 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f8ffeffd700 1 -- 172.21.15.110:0/3401971103 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f900800cb20 con 0x7f90100fbcb0 2024-04-25T15:58:20.094 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f8ffeffd700 1 -- 172.21.15.110:0/3401971103 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9008015d70 con 0x7f90100fbcb0 2024-04-25T15:58:20.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.091+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f901010f650 con 0x7f90100fbcb0 2024-04-25T15:58:20.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.092+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f901005f5c0 con 0x7f90100fbcb0 2024-04-25T15:58:20.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.093+0000 7f8ffeffd700 1 -- 172.21.15.110:0/3401971103 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f9008012070 con 0x7f90100fbcb0 2024-04-25T15:58:20.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.093+0000 7f8ffeffd700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8ff8076b90 0x7f8ff8079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:20.095 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.094+0000 7f8ffeffd700 1 -- 172.21.15.110:0/3401971103 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f9008098170 con 0x7f90100fbcb0 2024-04-25T15:58:20.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.094+0000 7f900dd9b700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8ff8076b90 0x7f8ff8079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:20.096 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.094+0000 7f900dd9b700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8ff8076b90 0x7f8ff8079040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f900400b970 tx=0x7f9004009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:20.099 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.098+0000 7f8ffeffd700 1 -- 172.21.15.110:0/3401971103 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f9008060ba0 con 0x7f90100fbcb0 2024-04-25T15:58:20.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:19 smithi193 ceph-mon[80071]: from='client.34502 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:20.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:19 smithi193 ceph-mon[80071]: from='client.34506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:20.241 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:19 smithi193 ceph-mon[80071]: pgmap v233: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:20.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:19 smithi193 ceph-mon[80071]: from='client.34510 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:20.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:19 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:20.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:19 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:20.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:19 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3369688122' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:20.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:19 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/289416134' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:58:20.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.383+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f901004bed0 con 0x7f90100fbcb0 2024-04-25T15:58:20.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.384+0000 7f8ffeffd700 1 -- 172.21.15.110:0/3401971103 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f9008060540 con 0x7f90100fbcb0 2024-04-25T15:58:20.386 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:58:20.386 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:58:20.386 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:58:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8ff8076b90 msgr2=0x7f8ff8079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8ff8076b90 0x7f8ff8079040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f900400b970 tx=0x7f9004009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fbcb0 msgr2=0x7f901010d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fbcb0 0x7f901010d9f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f900800bbb0 tx=0x7f9008009840 comp rx=0 tx=0).stop 2024-04-25T15:58:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 shutdown_connections 2024-04-25T15:58:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f8ff8076b90 0x7f8ff8079040 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:20.387 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f90100fbcb0 0x7f901010d9f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 --2- 172.21.15.110:0/3401971103 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90100fcf60 0x7f901010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 >> 172.21.15.110:0/3401971103 conn(0x7f90100f7430 msgr2=0x7f90100f8a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 shutdown_connections 2024-04-25T15:58:20.388 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:20.386+0000 7f9014f22700 1 -- 172.21.15.110:0/3401971103 wait complete. 2024-04-25T15:58:20.967 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:20 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3401971103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:58:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:20 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3401971103' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:58:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:21 smithi110 ceph-mon[90991]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:21 smithi110 ceph-mon[90991]: pgmap v234: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:21 smithi193 ceph-mon[80071]: from='client.34522 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:21 smithi193 ceph-mon[80071]: pgmap v234: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:58:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:58:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: pgmap v235: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:58:23.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:23.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:23 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:24.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:24.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:58:24.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: pgmap v235: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.116 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:23 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:24.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:24 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:58:25.071 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:24 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-25T15:58:25.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:25 smithi193 ceph-mon[80071]: Upgrade: Updating alertmanager.smithi110 2024-04-25T15:58:25.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:25 smithi193 ceph-mon[80071]: Deploying daemon alertmanager.smithi110 on smithi110 2024-04-25T15:58:25.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:25 smithi193 ceph-mon[80071]: pgmap v236: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:25 smithi110 ceph-mon[90991]: Upgrade: Updating alertmanager.smithi110 2024-04-25T15:58:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:25 smithi110 ceph-mon[90991]: Deploying daemon alertmanager.smithi110 on smithi110 2024-04-25T15:58:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:25 smithi110 ceph-mon[90991]: pgmap v236: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:27.989 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:27 smithi110 ceph-mon[90991]: pgmap v237: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:28.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:27 smithi193 ceph-mon[80071]: pgmap v237: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:29 smithi110 ceph-mon[90991]: pgmap v238: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:29 smithi193 ceph-mon[80071]: pgmap v238: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:31.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:31.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:31.770 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:31.771 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:31 smithi110 ceph-mon[90991]: pgmap v239: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:31.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:31.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:31.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:31.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:31 smithi193 ceph-mon[80071]: pgmap v239: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:32.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:32 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:58:32.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:32 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:58:33.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:33 smithi193 ceph-mon[80071]: pgmap v240: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:34.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:33 smithi110 ceph-mon[90991]: pgmap v240: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:35 smithi110 ceph-mon[90991]: pgmap v241: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:35 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:35 smithi193 ceph-mon[80071]: pgmap v241: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:35 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:37.613 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:37.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:37 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:37.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:37 smithi110 ceph-mon[90991]: pgmap v242: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:37.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:37.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:37 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:37.740 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:37 smithi193 ceph-mon[80071]: pgmap v242: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:39.916 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:39 smithi110 ceph-mon[90991]: pgmap v243: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:39.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:39.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:39.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:58:39.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:58:39.917 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:39 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:39 smithi193 ceph-mon[80071]: pgmap v243: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:58:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:58:40.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:39 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:58:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110:9093"}]: dispatch 2024-04-25T15:58:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:40.991 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:40 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110:9093"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:41.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:40 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:41 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:58:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:41 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110:9093"}]: dispatch 2024-04-25T15:58:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:41 smithi110 ceph-mon[90991]: pgmap v244: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:41 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-25T15:58:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:41 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi110:9093"}]: dispatch 2024-04-25T15:58:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:41 smithi193 ceph-mon[80071]: pgmap v244: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:42 smithi110 ceph-mon[90991]: Upgrade: Updating grafana.smithi110 2024-04-25T15:58:42.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:42 smithi110 ceph-mon[90991]: Deploying daemon grafana.smithi110 on smithi110 2024-04-25T15:58:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:42 smithi193 ceph-mon[80071]: Upgrade: Updating grafana.smithi110 2024-04-25T15:58:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:42 smithi193 ceph-mon[80071]: Deploying daemon grafana.smithi110 on smithi110 2024-04-25T15:58:44.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:43 smithi110 ceph-mon[90991]: pgmap v245: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:43 smithi193 ceph-mon[80071]: pgmap v245: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:45 smithi110 ceph-mon[90991]: pgmap v246: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:45 smithi193 ceph-mon[80071]: pgmap v246: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:58:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:46 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:58:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:47 smithi110 ceph-mon[90991]: pgmap v247: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:47 smithi193 ceph-mon[80071]: pgmap v247: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:49 smithi110 ceph-mon[90991]: pgmap v248: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:49 smithi193 ceph-mon[80071]: pgmap v248: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:50.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.498+0000 7fa60bb1a700 1 -- 172.21.15.110:0/2624001221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 msgr2=0x7fa6040fe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:50.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.498+0000 7fa60bb1a700 1 --2- 172.21.15.110:0/2624001221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 0x7fa6040fe440 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8004940 tx=0x7fa5f8009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:50.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 -- 172.21.15.110:0/2624001221 shutdown_connections 2024-04-25T15:58:50.501 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 --2- 172.21.15.110:0/2624001221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 0x7fa6040fe440 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 --2- 172.21.15.110:0/2624001221 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6040fce60 0x7fa6040fd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 -- 172.21.15.110:0/2624001221 >> 172.21.15.110:0/2624001221 conn(0x7fa6040f8520 msgr2=0x7fa6040fa950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 -- 172.21.15.110:0/2624001221 shutdown_connections 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 -- 172.21.15.110:0/2624001221 wait complete. 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 Processor -- start 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 -- start start 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6040fce60 0x7fa60410d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 0x7fa60410de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60410e510 con 0x7fa6040fdfd0 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa60bb1a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa604112eb0 con 0x7fa6040fce60 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa6090b5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 0x7fa60410de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.499+0000 7fa6090b5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 0x7fa60410de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58484/0 (socket says 172.21.15.110:58484) 2024-04-25T15:58:50.502 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6090b5700 1 -- 172.21.15.110:0/1599077174 learned_addr learned my addr 172.21.15.110:0/1599077174 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6098b6700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6040fce60 0x7fa60410d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6090b5700 1 -- 172.21.15.110:0/1599077174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6040fce60 msgr2=0x7fa60410d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6090b5700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6040fce60 0x7fa60410d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6090b5700 1 -- 172.21.15.110:0/1599077174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8005040 con 0x7fa6040fdfd0 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6090b5700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 0x7fa60410de80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b3b0 tx=0x7fa5f8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6027fc700 1 -- 172.21.15.110:0/1599077174 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f800e040 con 0x7fa6040fdfd0 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6027fc700 1 -- 172.21.15.110:0/1599077174 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5f800bbf0 con 0x7fa6040fdfd0 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6041130b0 con 0x7fa6040fdfd0 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa6027fc700 1 -- 172.21.15.110:0/1599077174 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f801c630 con 0x7fa6040fdfd0 2024-04-25T15:58:50.503 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.500+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa604113620 con 0x7fa6040fdfd0 2024-04-25T15:58:50.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.501+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60404c710 con 0x7fa6040fdfd0 2024-04-25T15:58:50.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.502+0000 7fa6027fc700 1 -- 172.21.15.110:0/1599077174 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fa5f800bd60 con 0x7fa6040fdfd0 2024-04-25T15:58:50.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.502+0000 7fa6027fc700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa5ec076b90 0x7fa5ec079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:50.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.503+0000 7fa6098b6700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa5ec076b90 0x7fa5ec079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:50.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.503+0000 7fa6027fc700 1 -- 172.21.15.110:0/1599077174 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa5f8099070 con 0x7fa6040fdfd0 2024-04-25T15:58:50.504 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.503+0000 7fa6098b6700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa5ec076b90 0x7fa5ec079040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009bf0 tx=0x7fa5fc0091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:50.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.507+0000 7fa6027fc700 1 -- 172.21.15.110:0/1599077174 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa5f8061b20 con 0x7fa6040fdfd0 2024-04-25T15:58:50.747 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.746+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6041022b0 con 0x7fa5ec076b90 2024-04-25T15:58:50.748 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.747+0000 7fa6027fc700 1 -- 172.21.15.110:0/1599077174 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7fa6041022b0 con 0x7fa5ec076b90 2024-04-25T15:58:50.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.749+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa5ec076b90 msgr2=0x7fa5ec079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:50.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa5ec076b90 0x7fa5ec079040 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa5fc009bf0 tx=0x7fa5fc0091a0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 msgr2=0x7fa60410de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:50.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 0x7fa60410de80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800b3b0 tx=0x7fa5f8009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:50.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 shutdown_connections 2024-04-25T15:58:50.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6040fce60 0x7fa60410d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa5ec076b90 0x7fa5ec079040 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 --2- 172.21.15.110:0/1599077174 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6040fdfd0 0x7fa60410de80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 >> 172.21.15.110:0/1599077174 conn(0x7fa6040f8520 msgr2=0x7fa6040f9b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:50.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 shutdown_connections 2024-04-25T15:58:50.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.750+0000 7fa60bb1a700 1 -- 172.21.15.110:0/1599077174 wait complete. 2024-04-25T15:58:50.761 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T15:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 -- 172.21.15.110:0/1698666770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 msgr2=0x7f5f780fd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 --2- 172.21.15.110:0/1698666770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 0x7f5f780fd3d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5f68004940 tx=0x7f5f68009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 -- 172.21.15.110:0/1698666770 shutdown_connections 2024-04-25T15:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 --2- 172.21.15.110:0/1698666770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 0x7f5f780fd3d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 --2- 172.21.15.110:0/1698666770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f780fbcb0 0x7f5f780fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 -- 172.21.15.110:0/1698666770 >> 172.21.15.110:0/1698666770 conn(0x7f5f780f7430 msgr2=0x7f5f780f9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:50.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 -- 172.21.15.110:0/1698666770 shutdown_connections 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 -- 172.21.15.110:0/1698666770 wait complete. 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 Processor -- start 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.859+0000 7f5f806bd700 1 -- start start 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f806bd700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f780fbcb0 0x7f5f78110680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f806bd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 0x7f5f78112bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f806bd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f78066340 con 0x7f5f780fcf60 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f806bd700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f78066480 con 0x7f5f780fbcb0 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f7dc58700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 0x7f5f78112bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f7dc58700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 0x7f5f78112bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58498/0 (socket says 172.21.15.110:58498) 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f7dc58700 1 -- 172.21.15.110:0/1056117817 learned_addr learned my addr 172.21.15.110:0/1056117817 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f7e459700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f780fbcb0 0x7f5f78110680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f7dc58700 1 -- 172.21.15.110:0/1056117817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f780fbcb0 msgr2=0x7f5f78110680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f7dc58700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f780fbcb0 0x7f5f78110680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:50.862 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f7dc58700 1 -- 172.21.15.110:0/1056117817 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f68005040 con 0x7f5f780fcf60 2024-04-25T15:58:50.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f7dc58700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 0x7f5f78112bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5f68013870 tx=0x7f5f68009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.860+0000 7f5f6effd700 1 -- 172.21.15.110:0/1056117817 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f68021070 con 0x7f5f780fcf60 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.861+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f78113170 con 0x7f5f780fcf60 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.861+0000 7f5f6effd700 1 -- 172.21.15.110:0/1056117817 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f68009430 con 0x7f5f780fcf60 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.861+0000 7f5f6effd700 1 -- 172.21.15.110:0/1056117817 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f6800e070 con 0x7f5f780fcf60 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.861+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f78113660 con 0x7f5f780fcf60 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.861+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7805f5c0 con 0x7f5f780fcf60 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.862+0000 7f5f6effd700 1 -- 172.21.15.110:0/1056117817 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f5f6801c750 con 0x7f5f780fcf60 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.863+0000 7f5f6effd700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5f64076b10 0x7f5f64078fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.863+0000 7f5f6effd700 1 -- 172.21.15.110:0/1056117817 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5f68099600 con 0x7f5f780fcf60 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.863+0000 7f5f7e459700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5f64076b10 0x7f5f64078fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:50.864 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.863+0000 7f5f7e459700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5f64076b10 0x7f5f64078fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f78065fc0 tx=0x7f5f7400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:50.868 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:50.867+0000 7f5f6effd700 1 -- 172.21.15.110:0/1056117817 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5f680620b0 con 0x7f5f780fcf60 2024-04-25T15:58:51.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.096+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f78101250 con 0x7f5f64076b10 2024-04-25T15:58:51.098 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.097+0000 7f5f6effd700 1 -- 172.21.15.110:0/1056117817 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f5f78101250 con 0x7f5f64076b10 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5f64076b10 msgr2=0x7f5f64078fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5f64076b10 0x7f5f64078fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5f78065fc0 tx=0x7f5f7400b040 comp rx=0 tx=0).stop 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 msgr2=0x7f5f78112bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 0x7f5f78112bd0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f5f68013870 tx=0x7f5f68009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 shutdown_connections 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f780fbcb0 0x7f5f78110680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5f64076b10 0x7f5f64078fc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 --2- 172.21.15.110:0/1056117817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5f780fcf60 0x7f5f78112bd0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.100 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 >> 172.21.15.110:0/1056117817 conn(0x7f5f780f7430 msgr2=0x7f5f780f8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:51.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 shutdown_connections 2024-04-25T15:58:51.101 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.099+0000 7f5f806bd700 1 -- 172.21.15.110:0/1056117817 wait complete. 2024-04-25T15:58:51.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 -- 172.21.15.110:0/1745955162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 msgr2=0x7f08540fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 --2- 172.21.15.110:0/1745955162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 0x7f08540fc0d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0850004940 tx=0x7f0850009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 -- 172.21.15.110:0/1745955162 shutdown_connections 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 --2- 172.21.15.110:0/1745955162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08540fcf70 0x7f08540fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 --2- 172.21.15.110:0/1745955162 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 0x7f08540fc0d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 -- 172.21.15.110:0/1745955162 >> 172.21.15.110:0/1745955162 conn(0x7f08540f7440 msgr2=0x7f08540f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 -- 172.21.15.110:0/1745955162 shutdown_connections 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 -- 172.21.15.110:0/1745955162 wait complete. 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 Processor -- start 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 -- start start 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 0x7f0854066cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08540fcf70 0x7f0854112760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0854112ca0 con 0x7f08540fbcc0 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f085b555700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0854112e10 con 0x7f08540fcf70 2024-04-25T15:58:51.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.200+0000 7f08592f1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 0x7f0854066cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f08592f1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 0x7f0854066cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58510/0 (socket says 172.21.15.110:58510) 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f08592f1700 1 -- 172.21.15.110:0/3123032788 learned_addr learned my addr 172.21.15.110:0/3123032788 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f0858af0700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08540fcf70 0x7f0854112760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f08592f1700 1 -- 172.21.15.110:0/3123032788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08540fcf70 msgr2=0x7f0854112760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f08592f1700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08540fcf70 0x7f0854112760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f08592f1700 1 -- 172.21.15.110:0/3123032788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850005040 con 0x7f08540fbcc0 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f0858af0700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08540fcf70 0x7f0854112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f08592f1700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 0x7f0854066cd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0850009fd0 tx=0x7f0850009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f0849ffb700 1 -- 172.21.15.110:0/3123032788 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085000e040 con 0x7f08540fbcc0 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0854113090 con 0x7f08540fbcc0 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f0849ffb700 1 -- 172.21.15.110:0/3123032788 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f085000bcc0 con 0x7f08540fbcc0 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f0849ffb700 1 -- 172.21.15.110:0/3123032788 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f085001c630 con 0x7f08540fbcc0 2024-04-25T15:58:51.204 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.201+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0854113650 con 0x7f08540fbcc0 2024-04-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.202+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f085404c710 con 0x7f08540fbcc0 2024-04-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.203+0000 7f0849ffb700 1 -- 172.21.15.110:0/3123032788 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f0850013980 con 0x7f08540fbcc0 2024-04-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.203+0000 7f0849ffb700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0840076b90 0x7f0840079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.203+0000 7f0849ffb700 1 -- 172.21.15.110:0/3123032788 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f0850099770 con 0x7f08540fbcc0 2024-04-25T15:58:51.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.203+0000 7f0858af0700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0840076b90 0x7f0840079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.204+0000 7f0858af0700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0840076b90 0x7f0840079040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0844006fd0 tx=0x7f0844008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:51.209 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.208+0000 7f0849ffb700 1 -- 172.21.15.110:0/3123032788 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f08500621a0 con 0x7f08540fbcc0 2024-04-25T15:58:51.423 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.422+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0854101290 con 0x7f0840076b90 2024-04-25T15:58:51.433 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.432+0000 7f0849ffb700 1 -- 172.21.15.110:0/3123032788 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f0854101290 con 0x7f0840076b90 2024-04-25T15:58:51.433 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:58:51.433 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (21s) 16s ago 17m 15.0M - 0.25.0 c8568f914cd2 e4c0cbbecaf4 2024-04-25T15:58:51.433 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (6m) 16s ago 18m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:58:51.433 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (6m) 52s ago 17m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (8m) 16s ago 17m 84.3M - 9.4.7 954c08fa6188 ea6f96abf201 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (117s) 16s ago 12m 83.1M - 19.0.0-3291-g994e2f20 d53c96779a29 051262111b6b 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (2m) 16s ago 12m 160M - 19.0.0-3291-g994e2f20 d53c96779a29 763d1dda41b7 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (95s) 52s ago 12m 82.7M - 19.0.0-3291-g994e2f20 d53c96779a29 36e42c64ae3a 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (71s) 52s ago 12m 79.6M - 19.0.0-3291-g994e2f20 d53c96779a29 3dee50406dde 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (8m) 16s ago 19m 581M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (7m) 52s ago 16m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (7m) 16s ago 19m 122M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (6m) 52s ago 16m 105M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (59s) 16s ago 17m 7684k - 1.5.0 0da6a335fe13 c74eafffc54d 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (55s) 52s ago 16m 3556k - 1.5.0 0da6a335fe13 a766013ab03e 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (5m) 16s ago 16m 167M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (5m) 16s ago 15m 171M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:58:51.434 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (4m) 16s ago 15m 157M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:58:51.435 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (3m) 52s ago 14m 176M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:58:51.435 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (3m) 52s ago 14m 159M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T15:58:51.435 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (2m) 52s ago 13m 160M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 31c4aca48886 2024-04-25T15:58:51.435 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (38s) 16s ago 17m 47.0M - 2.43.0 a07b618ecd1d 01d861aba74b 2024-04-25T15:58:51.435 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0840076b90 msgr2=0x7f0840079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0840076b90 0x7f0840079040 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0844006fd0 tx=0x7f0844008040 comp rx=0 tx=0).stop 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 msgr2=0x7f0854066cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 0x7f0854066cd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0850009fd0 tx=0x7f0850009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 shutdown_connections 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0840076b90 0x7f0840079040 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f08540fbcc0 0x7f0854066cd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 --2- 172.21.15.110:0/3123032788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08540fcf70 0x7f0854112760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 >> 172.21.15.110:0/3123032788 conn(0x7f08540f7440 msgr2=0x7f08540f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 shutdown_connections 2024-04-25T15:58:51.436 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.434+0000 7f085b555700 1 -- 172.21.15.110:0/3123032788 wait complete. 2024-04-25T15:58:51.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 -- 172.21.15.110:0/2714599827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 msgr2=0x7f688c0ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 --2- 172.21.15.110:0/2714599827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 0x7f688c0ffa30 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f687c004940 tx=0x7f687c009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 -- 172.21.15.110:0/2714599827 shutdown_connections 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 --2- 172.21.15.110:0/2714599827 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f688c0fff70 0x7f688c102350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 --2- 172.21.15.110:0/2714599827 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 0x7f688c0ffa30 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 -- 172.21.15.110:0/2714599827 >> 172.21.15.110:0/2714599827 conn(0x7f688c0f7440 msgr2=0x7f688c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 -- 172.21.15.110:0/2714599827 shutdown_connections 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 -- 172.21.15.110:0/2714599827 wait complete. 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.534+0000 7f6891c33700 1 Processor -- start 2024-04-25T15:58:51.536 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f6891c33700 1 -- start start 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f6891c33700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 0x7f688c10d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f6891c33700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f688c0fff70 0x7f688c10deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f6891c33700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c10e540 con 0x7f688c066ed0 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f6891c33700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c112e90 con 0x7f688c0fff70 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f688b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 0x7f688c10d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f688b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 0x7f688c10d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58530/0 (socket says 172.21.15.110:58530) 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f688b7fe700 1 -- 172.21.15.110:0/3261671048 learned_addr learned my addr 172.21.15.110:0/3261671048 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f688affd700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f688c0fff70 0x7f688c10deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f688b7fe700 1 -- 172.21.15.110:0/3261671048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f688c0fff70 msgr2=0x7f688c10deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.538 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f688b7fe700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f688c0fff70 0x7f688c10deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.535+0000 7f688b7fe700 1 -- 172.21.15.110:0/3261671048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c005040 con 0x7f688c066ed0 2024-04-25T15:58:51.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.536+0000 7f688b7fe700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 0x7f688c10d970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:51.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.536+0000 7f6873fff700 1 -- 172.21.15.110:0/3261671048 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f687c00e040 con 0x7f688c066ed0 2024-04-25T15:58:51.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.536+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f688c113090 con 0x7f688c066ed0 2024-04-25T15:58:51.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.536+0000 7f6873fff700 1 -- 172.21.15.110:0/3261671048 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f687c00bcc0 con 0x7f688c066ed0 2024-04-25T15:58:51.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.536+0000 7f6873fff700 1 -- 172.21.15.110:0/3261671048 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f687c01c630 con 0x7f688c066ed0 2024-04-25T15:58:51.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.536+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f688c113600 con 0x7f688c066ed0 2024-04-25T15:58:51.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.537+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f688c04c710 con 0x7f688c066ed0 2024-04-25T15:58:51.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.538+0000 7f6873fff700 1 -- 172.21.15.110:0/3261671048 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f687c013980 con 0x7f688c066ed0 2024-04-25T15:58:51.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.538+0000 7f6873fff700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6874076b90 0x7f6874079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.538+0000 7f6873fff700 1 -- 172.21.15.110:0/3261671048 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f687c0998f0 con 0x7f688c066ed0 2024-04-25T15:58:51.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.538+0000 7f688affd700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6874076b90 0x7f6874079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.539+0000 7f688affd700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6874076b90 0x7f6874079040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6880006fd0 tx=0x7f6880008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:51.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.543+0000 7f6873fff700 1 -- 172.21.15.110:0/3261671048 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f687c0623b0 con 0x7f688c066ed0 2024-04-25T15:58:51.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.834+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f688c113820 con 0x7f688c066ed0 2024-04-25T15:58:51.836 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.834+0000 7f6873fff700 1 -- 172.21.15.110:0/3261671048 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f687c061d50 con 0x7f688c066ed0 2024-04-25T15:58:51.836 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 6 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 4 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:58:51.837 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:58:51.838 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 14 2024-04-25T15:58:51.838 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:58:51.838 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:58:51.838 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6874076b90 msgr2=0x7f6874079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6874076b90 0x7f6874079040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6880006fd0 tx=0x7f6880008040 comp rx=0 tx=0).stop 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 msgr2=0x7f688c10d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 0x7f688c10d970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f687c000c00 tx=0x7f687c009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 shutdown_connections 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f6874076b90 0x7f6874079040 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f688c066ed0 0x7f688c10d970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 --2- 172.21.15.110:0/3261671048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f688c0fff70 0x7f688c10deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.836+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 >> 172.21.15.110:0/3261671048 conn(0x7f688c0f7440 msgr2=0x7f688c0f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.837+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 shutdown_connections 2024-04-25T15:58:51.839 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.837+0000 7f6891c33700 1 -- 172.21.15.110:0/3261671048 wait complete. 2024-04-25T15:58:51.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 -- 172.21.15.110:0/1934584031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fbd40 msgr2=0x7f31980fc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.953 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 --2- 172.21.15.110:0/1934584031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fbd40 0x7f31980fc150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3190004940 tx=0x7f3190009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 -- 172.21.15.110:0/1934584031 shutdown_connections 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 --2- 172.21.15.110:0/1934584031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31980fcff0 0x7f31980fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 --2- 172.21.15.110:0/1934584031 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fbd40 0x7f31980fc150 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 -- 172.21.15.110:0/1934584031 >> 172.21.15.110:0/1934584031 conn(0x7f31980f74c0 msgr2=0x7f31980f98d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 -- 172.21.15.110:0/1934584031 shutdown_connections 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 -- 172.21.15.110:0/1934584031 wait complete. 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 Processor -- start 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 -- start start 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31980fbd40 0x7f31980ffdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fcff0 0x7f31980fe330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980fe870 con 0x7f31980fcff0 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.950+0000 7f319cc15700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31980fe9b0 con 0x7f31980fbd40 2024-04-25T15:58:51.954 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3195d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fcff0 0x7f31980fe330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3195d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fcff0 0x7f31980fe330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58544/0 (socket says 172.21.15.110:58544) 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3195d9b700 1 -- 172.21.15.110:0/3700203053 learned_addr learned my addr 172.21.15.110:0/3700203053 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f319659c700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31980fbd40 0x7f31980ffdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3195d9b700 1 -- 172.21.15.110:0/3700203053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31980fbd40 msgr2=0x7f31980ffdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3195d9b700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31980fbd40 0x7f31980ffdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3195d9b700 1 -- 172.21.15.110:0/3700203053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3190005040 con 0x7f31980fcff0 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3195d9b700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fcff0 0x7f31980fe330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f318c00c8c0 tx=0x7f318c0044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3186ffd700 1 -- 172.21.15.110:0/3700203053 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f318c00a760 con 0x7f31980fcff0 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31980fec90 con 0x7f31980fcff0 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3186ffd700 1 -- 172.21.15.110:0/3700203053 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f318c00ad90 con 0x7f31980fcff0 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f3186ffd700 1 -- 172.21.15.110:0/3700203053 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f318c0113a0 con 0x7f31980fcff0 2024-04-25T15:58:51.955 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.951+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3198115ad0 con 0x7f31980fcff0 2024-04-25T15:58:51.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.953+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3198101b80 con 0x7f31980fcff0 2024-04-25T15:58:51.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.954+0000 7f3186ffd700 1 -- 172.21.15.110:0/3700203053 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f318c012400 con 0x7f31980fcff0 2024-04-25T15:58:51.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.954+0000 7f3186ffd700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3180076b80 0x7f3180079030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:51.956 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.954+0000 7f319659c700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3180076b80 0x7f3180079030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:51.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.954+0000 7f3186ffd700 1 -- 172.21.15.110:0/3700203053 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f318c098470 con 0x7f31980fcff0 2024-04-25T15:58:51.957 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.955+0000 7f319659c700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3180076b80 0x7f3180079030 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f319000b3b0 tx=0x7f3190009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:51.961 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:51.959+0000 7f3186ffd700 1 -- 172.21.15.110:0/3700203053 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f318c060f20 con 0x7f31980fcff0 2024-04-25T15:58:52.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:51 smithi110 ceph-mon[90991]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:52.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:51 smithi110 ceph-mon[90991]: pgmap v249: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:52.049 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:51 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3261671048' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:52.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.229+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f319805f5c0 con 0x7f31980fcff0 2024-04-25T15:58:52.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.229+0000 7f3186ffd700 1 -- 172.21.15.110:0/3700203053 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1943 (secure 0 0 0) 0x7f318c0608c0 con 0x7f31980fcff0 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:e35 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 35 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T15:58:52.233 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:57:44.601048+0000 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 83 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T15:58:52.234 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 34404 members: 34404 2024-04-25T15:58:52.235 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:34404} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:58:52.236 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:58:52.236 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:58:52.236 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T15:58:52.236 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T15:58:52.236 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/1518950103,v1:172.21.15.110:6829/1518950103] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:58:52.236 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/1783080009,v1:172.21.15.193:6825/1783080009] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:58:52.236 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/3103986931,v1:172.21.15.193:6827/3103986931] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T15:58:52.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.231+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3180076b80 msgr2=0x7f3180079030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:52.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.231+0000 7f319cc15700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3180076b80 0x7f3180079030 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f319000b3b0 tx=0x7f3190009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:52.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.231+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fcff0 msgr2=0x7f31980fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:52.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.231+0000 7f319cc15700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fcff0 0x7f31980fe330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f318c00c8c0 tx=0x7f318c0044c0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.231+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 shutdown_connections 2024-04-25T15:58:52.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.232+0000 7f319cc15700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31980fbd40 0x7f31980ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.232+0000 7f319cc15700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3180076b80 0x7f3180079030 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.237 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.232+0000 7f319cc15700 1 --2- 172.21.15.110:0/3700203053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f31980fcff0 0x7f31980fe330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.232+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 >> 172.21.15.110:0/3700203053 conn(0x7f31980f74c0 msgr2=0x7f3198066410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:52.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.232+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 shutdown_connections 2024-04-25T15:58:52.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.232+0000 7f319cc15700 1 -- 172.21.15.110:0/3700203053 wait complete. 2024-04-25T15:58:52.238 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-04-25T15:58:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:51 smithi193 ceph-mon[80071]: from='client.34530 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:51 smithi193 ceph-mon[80071]: pgmap v249: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:51 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3261671048' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.339+0000 7f4bda16d700 1 -- 172.21.15.110:0/988847721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 msgr2=0x7f4bd40fc0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.339+0000 7f4bda16d700 1 --2- 172.21.15.110:0/988847721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 0x7f4bd40fc0d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4004940 tx=0x7f4bc4009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.339+0000 7f4bda16d700 1 -- 172.21.15.110:0/988847721 shutdown_connections 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.339+0000 7f4bda16d700 1 --2- 172.21.15.110:0/988847721 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bd40fcf70 0x7f4bd40fd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.339+0000 7f4bda16d700 1 --2- 172.21.15.110:0/988847721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 0x7f4bd40fc0d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.339+0000 7f4bda16d700 1 -- 172.21.15.110:0/988847721 >> 172.21.15.110:0/988847721 conn(0x7f4bd40f7420 msgr2=0x7f4bd40f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.339+0000 7f4bda16d700 1 -- 172.21.15.110:0/988847721 shutdown_connections 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.339+0000 7f4bda16d700 1 -- 172.21.15.110:0/988847721 wait complete. 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bda16d700 1 Processor -- start 2024-04-25T15:58:52.343 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bda16d700 1 -- start start 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bda16d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 0x7f4bd410c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bda16d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bd40fcf70 0x7f4bd410ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bda16d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd410d430 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bda16d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd410dc80 con 0x7f4bd40fcf70 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bd37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 0x7f4bd410c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bd37fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 0x7f4bd410c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58548/0 (socket says 172.21.15.110:58548) 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bd37fe700 1 -- 172.21.15.110:0/2159772061 learned_addr learned my addr 172.21.15.110:0/2159772061 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bd37fe700 1 -- 172.21.15.110:0/2159772061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bd40fcf70 msgr2=0x7f4bd410ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bd37fe700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bd40fcf70 0x7f4bd410ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.340+0000 7f4bd37fe700 1 -- 172.21.15.110:0/2159772061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc4005040 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.341+0000 7f4bd37fe700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 0x7f4bd410c8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009150 tx=0x7f4bc4009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:52.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.341+0000 7f4bbbfff700 1 -- 172.21.15.110:0/2159772061 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc400e040 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.341+0000 7f4bbbfff700 1 -- 172.21.15.110:0/2159772061 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bc400bcc0 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.341+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd410df00 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.341+0000 7f4bbbfff700 1 -- 172.21.15.110:0/2159772061 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bc401c630 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.341+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd410e440 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.342+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd405f5c0 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.342+0000 7f4bbbfff700 1 -- 172.21.15.110:0/2159772061 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f4bc4026440 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.343+0000 7f4bbbfff700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bbc076b90 0x7f4bbc079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:52.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.343+0000 7f4bbbfff700 1 -- 172.21.15.110:0/2159772061 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4bc40990f0 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.343+0000 7f4bd2ffd700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bbc076b90 0x7f4bbc079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:52.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.343+0000 7f4bd2ffd700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bbc076b90 0x7f4bbc079040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8004ad0 tx=0x7f4bc8007d40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:52.349 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.348+0000 7f4bbbfff700 1 -- 172.21.15.110:0/2159772061 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4bc4061ba0 con 0x7f4bd40fbcc0 2024-04-25T15:58:52.571 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.569+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bd410e7d0 con 0x7f4bbc076b90 2024-04-25T15:58:52.572 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.571+0000 7f4bbbfff700 1 -- 172.21.15.110:0/2159772061 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+435 (secure 0 0 0) 0x7f4bd410e7d0 con 0x7f4bbc076b90 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385", 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": true, 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [ 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "mon", 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "mds", 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr", 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "osd", 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "crash" 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: ], 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": "16/21 daemons upgraded", 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "Currently upgrading grafana daemons", 2024-04-25T15:58:52.573 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:58:52.574 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bbc076b90 msgr2=0x7f4bbc079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bbc076b90 0x7f4bbc079040 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4bc8004ad0 tx=0x7f4bc8007d40 comp rx=0 tx=0).stop 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 msgr2=0x7f4bd410c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 0x7f4bd410c8f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4009150 tx=0x7f4bc4009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 shutdown_connections 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bbc076b90 0x7f4bbc079040 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bd40fbcc0 0x7f4bd410c8f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 --2- 172.21.15.110:0/2159772061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bd40fcf70 0x7f4bd410ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 >> 172.21.15.110:0/2159772061 conn(0x7f4bd40f7420 msgr2=0x7f4bd40f8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 shutdown_connections 2024-04-25T15:58:52.575 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.573+0000 7f4bda16d700 1 -- 172.21.15.110:0/2159772061 wait complete. 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.676+0000 7f33079f2700 1 -- 172.21.15.110:0/1909340835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fdff0 msgr2=0x7f33000fe460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.676+0000 7f33079f2700 1 --2- 172.21.15.110:0/1909340835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fdff0 0x7f33000fe460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f32f0004940 tx=0x7f32f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.676+0000 7f33079f2700 1 -- 172.21.15.110:0/1909340835 shutdown_connections 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.676+0000 7f33079f2700 1 --2- 172.21.15.110:0/1909340835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fdff0 0x7f33000fe460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.676+0000 7f33079f2700 1 --2- 172.21.15.110:0/1909340835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33000fce80 0x7f33000fd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.676+0000 7f33079f2700 1 -- 172.21.15.110:0/1909340835 >> 172.21.15.110:0/1909340835 conn(0x7f33000f8540 msgr2=0x7f33000fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f33079f2700 1 -- 172.21.15.110:0/1909340835 shutdown_connections 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f33079f2700 1 -- 172.21.15.110:0/1909340835 wait complete. 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f33079f2700 1 Processor -- start 2024-04-25T15:58:52.680 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f33079f2700 1 -- start start 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f33079f2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fce80 0x7f3300071de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f33079f2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33000fdff0 0x7f33000702b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f33079f2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3300070860 con 0x7f33000fce80 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f33079f2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33000709d0 con 0x7f33000fdff0 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f330578e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fce80 0x7f3300071de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f330578e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fce80 0x7f3300071de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58578/0 (socket says 172.21.15.110:58578) 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.677+0000 7f330578e700 1 -- 172.21.15.110:0/2602371712 learned_addr learned my addr 172.21.15.110:0/2602371712 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f3304f8d700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33000fdff0 0x7f33000702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f330578e700 1 -- 172.21.15.110:0/2602371712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33000fdff0 msgr2=0x7f33000702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f330578e700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33000fdff0 0x7f33000702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f330578e700 1 -- 172.21.15.110:0/2602371712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f0005040 con 0x7f33000fce80 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f330578e700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fce80 0x7f3300071de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c830 tx=0x7f32fc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f32f5ffb700 1 -- 172.21.15.110:0/2602371712 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32fc00a760 con 0x7f33000fce80 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3300070cb0 con 0x7f33000fce80 2024-04-25T15:58:52.681 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f32f5ffb700 1 -- 172.21.15.110:0/2602371712 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f32fc00ad90 con 0x7f33000fce80 2024-04-25T15:58:52.682 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f32f5ffb700 1 -- 172.21.15.110:0/2602371712 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32fc0113a0 con 0x7f33000fce80 2024-04-25T15:58:52.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.678+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3300115a10 con 0x7f33000fce80 2024-04-25T15:58:52.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.679+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330005f5c0 con 0x7f33000fce80 2024-04-25T15:58:52.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.680+0000 7f32f5ffb700 1 -- 172.21.15.110:0/2602371712 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f32fc012400 con 0x7f33000fce80 2024-04-25T15:58:52.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.680+0000 7f32f5ffb700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f32ec076b90 0x7f32ec079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:58:52.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.680+0000 7f32f5ffb700 1 -- 172.21.15.110:0/2602371712 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f32fc098490 con 0x7f33000fce80 2024-04-25T15:58:52.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.680+0000 7f3304f8d700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f32ec076b90 0x7f32ec079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:58:52.683 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.681+0000 7f3304f8d700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f32ec076b90 0x7f32ec079040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b3b0 tx=0x7f32f0009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:58:52.687 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:52.685+0000 7f32f5ffb700 1 -- 172.21.15.110:0/2602371712 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f32fc060f40 con 0x7f33000fce80 2024-04-25T15:58:52.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:52 smithi110 ceph-mon[90991]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:52.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:52 smithi110 ceph-mon[90991]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:52.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:52 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3700203053' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:58:53.013 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.011+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f330004bed0 con 0x7f33000fce80 2024-04-25T15:58:53.014 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.011+0000 7f32f5ffb700 1 -- 172.21.15.110:0/2602371712 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f32fc0608e0 con 0x7f33000fce80 2024-04-25T15:58:53.014 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:58:53.014 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:58:53.014 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f32ec076b90 msgr2=0x7f32ec079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f32ec076b90 0x7f32ec079040 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f32f000b3b0 tx=0x7f32f0009e30 comp rx=0 tx=0).stop 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fce80 msgr2=0x7f3300071de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fce80 0x7f3300071de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00c830 tx=0x7f32fc004470 comp rx=0 tx=0).stop 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 shutdown_connections 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f32ec076b90 0x7f32ec079040 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33000fce80 0x7f3300071de0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 --2- 172.21.15.110:0/2602371712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33000fdff0 0x7f33000702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.013+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 >> 172.21.15.110:0/2602371712 conn(0x7f33000f8540 msgr2=0x7f33000f9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.014+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 shutdown_connections 2024-04-25T15:58:53.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:58:53.014+0000 7f33079f2700 1 -- 172.21.15.110:0/2602371712 wait complete. 2024-04-25T15:58:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:52 smithi193 ceph-mon[80071]: from='client.34534 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:52 smithi193 ceph-mon[80071]: from='client.34538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:52 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3700203053' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T15:58:53.966 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:53 smithi110 ceph-mon[90991]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:53.966 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:53 smithi110 ceph-mon[90991]: pgmap v250: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:53.966 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:53 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2602371712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:58:54.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:53 smithi193 ceph-mon[80071]: from='client.34550 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:58:54.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:53 smithi193 ceph-mon[80071]: pgmap v250: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:54.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:53 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2602371712' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:58:55.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:55 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:55 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:55 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:55.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:55 smithi193 ceph-mon[80071]: pgmap v251: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:55.491 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:55.491 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:58:55.492 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:55 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:58:55.492 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:55 smithi110 ceph-mon[90991]: pgmap v251: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:58.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:57 smithi110 ceph-mon[90991]: pgmap v252: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:58:58.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:57 smithi193 ceph-mon[80071]: pgmap v252: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:59 smithi110 ceph-mon[90991]: pgmap v253: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:00.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:58:59 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:59 smithi193 ceph-mon[80071]: pgmap v253: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:00.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:58:59 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:00 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:00 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:00 smithi110 ceph-mon[90991]: pgmap v254: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:01.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:00 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:01.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:00 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:01.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:00 smithi193 ceph-mon[80071]: pgmap v254: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:02 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:59:02.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:02 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:59:03.366 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:03 smithi110 ceph-mon[90991]: pgmap v255: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:03 smithi193 ceph-mon[80071]: pgmap v255: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:05 smithi110 ceph-mon[90991]: pgmap v256: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:05 smithi193 ceph-mon[80071]: pgmap v256: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:07.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:59:07.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:59:07.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110:3000"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110:3000"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.491 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: Upgrade: Finalizing container_image settings 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-25T15:59:07.492 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-25T15:59:07.493 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: Upgrade: Complete! 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: pgmap v257: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:59:07.494 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:59:07.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:59:07.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:59:07.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:59:07.495 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110:3000"}]: dispatch 2024-04-25T15:59:07.614 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi110:3000"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.615 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: Upgrade: Finalizing container_image settings 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-25T15:59:07.616 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-25T15:59:07.617 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: Upgrade: Complete! 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: pgmap v257: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.618 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:59:07.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:59:07.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:07.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T15:59:07.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T15:59:07.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T15:59:07.619 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T15:59:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:09 smithi110 ceph-mon[90991]: pgmap v258: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:09 smithi193 ceph-mon[80071]: pgmap v258: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:11 smithi110 ceph-mon[90991]: pgmap v259: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:11 smithi193 ceph-mon[80071]: pgmap v259: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:13 smithi110 ceph-mon[90991]: pgmap v260: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:13 smithi193 ceph-mon[80071]: pgmap v260: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:16.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:15 smithi110 ceph-mon[90991]: pgmap v261: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:15 smithi193 ceph-mon[80071]: pgmap v261: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:59:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:59:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:17 smithi110 ceph-mon[90991]: pgmap v262: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:17 smithi193 ceph-mon[80071]: pgmap v262: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:20.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:19 smithi110 ceph-mon[90991]: pgmap v263: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:19 smithi193 ceph-mon[80071]: pgmap v263: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:22.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:21 smithi110 ceph-mon[90991]: pgmap v264: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:21 smithi193 ceph-mon[80071]: pgmap v264: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 -- 172.21.15.110:0/1685950944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fc710 msgr2=0x7f78f80fcb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 --2- 172.21.15.110:0/1685950944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fc710 0x7f78f80fcb80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f78e8004a50 tx=0x7f78e800af40 comp rx=0 tx=0).stop 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 -- 172.21.15.110:0/1685950944 shutdown_connections 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 --2- 172.21.15.110:0/1685950944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fc710 0x7f78f80fcb80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 --2- 172.21.15.110:0/1685950944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fb460 0x7f78f80fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 -- 172.21.15.110:0/1685950944 >> 172.21.15.110:0/1685950944 conn(0x7f78f80f6be0 msgr2=0x7f78f80f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 -- 172.21.15.110:0/1685950944 shutdown_connections 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 -- 172.21.15.110:0/1685950944 wait complete. 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 Processor -- start 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.129+0000 7f78ffcff700 1 -- start start 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78ffcff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fb460 0x7f78f810cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:23.132 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78ffcff700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fc710 0x7f78f810b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78ffcff700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f810d3d0 con 0x7f78f80fb460 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78ffcff700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f810b9c0 con 0x7f78f80fc710 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78fda9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fb460 0x7f78f810cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78fda9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fb460 0x7f78f810cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45296/0 (socket says 172.21.15.110:45296) 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78fda9b700 1 -- 172.21.15.110:0/4250566903 learned_addr learned my addr 172.21.15.110:0/4250566903 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78fd29a700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fc710 0x7f78f810b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78fda9b700 1 -- 172.21.15.110:0/4250566903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fc710 msgr2=0x7f78f810b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78fda9b700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fc710 0x7f78f810b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78fda9b700 1 -- 172.21.15.110:0/4250566903 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8005040 con 0x7f78f80fb460 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78fda9b700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fb460 0x7f78f810cdd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f78f4002a80 tx=0x7f78f40046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78ee7fc700 1 -- 172.21.15.110:0/4250566903 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78f4007cc0 con 0x7f78f80fb460 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.130+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f810bc70 con 0x7f78f80fb460 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.131+0000 7f78ee7fc700 1 -- 172.21.15.110:0/4250566903 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78f400f040 con 0x7f78f80fb460 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.131+0000 7f78ee7fc700 1 -- 172.21.15.110:0/4250566903 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78f40096f0 con 0x7f78f80fb460 2024-04-25T15:59:23.133 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.131+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f810c230 con 0x7f78f80fb460 2024-04-25T15:59:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.132+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f804c710 con 0x7f78f80fb460 2024-04-25T15:59:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.132+0000 7f78ee7fc700 1 -- 172.21.15.110:0/4250566903 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f78f400d690 con 0x7f78f80fb460 2024-04-25T15:59:23.134 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.133+0000 7f78ee7fc700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b90 0x7f78e4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.133+0000 7f78fd29a700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b90 0x7f78e4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.133+0000 7f78ee7fc700 1 -- 172.21.15.110:0/4250566903 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f78f4097bc0 con 0x7f78f80fb460 2024-04-25T15:59:23.135 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.133+0000 7f78fd29a700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b90 0x7f78e4079040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c3b0 tx=0x7f78e800af40 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:23.139 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.138+0000 7f78ee7fc700 1 -- 172.21.15.110:0/4250566903 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f78f40605f0 con 0x7f78f80fb460 2024-04-25T15:59:23.358 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.357+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78f81009b0 con 0x7f78e4076b90 2024-04-25T15:59:23.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.358+0000 7f78ee7fc700 1 -- 172.21.15.110:0/4250566903 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f78f81009b0 con 0x7f78e4076b90 2024-04-25T15:59:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b90 msgr2=0x7f78e4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b90 0x7f78e4079040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78e800c3b0 tx=0x7f78e800af40 comp rx=0 tx=0).stop 2024-04-25T15:59:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fb460 msgr2=0x7f78f810cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:23.361 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fb460 0x7f78f810cdd0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f78f4002a80 tx=0x7f78f40046b0 comp rx=0 tx=0).stop 2024-04-25T15:59:23.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 shutdown_connections 2024-04-25T15:59:23.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b90 0x7f78e4079040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:23.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f80fb460 0x7f78f810cdd0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:23.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 --2- 172.21.15.110:0/4250566903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fc710 0x7f78f810b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:23.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 >> 172.21.15.110:0/4250566903 conn(0x7f78f80f6be0 msgr2=0x7f78f80f8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:23.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 shutdown_connections 2024-04-25T15:59:23.362 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:23.360+0000 7f78ffcff700 1 -- 172.21.15.110:0/4250566903 wait complete. 2024-04-25T15:59:24.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:23 smithi110 ceph-mon[90991]: pgmap v265: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:23 smithi193 ceph-mon[80071]: pgmap v265: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:24.287 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph orch ps' 2024-04-25T15:59:24.586 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:59:25.041 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:24 smithi110 ceph-mon[90991]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:59:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:24 smithi193 ceph-mon[80071]: from='client.34558 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:59:26.015 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:25 smithi110 ceph-mon[90991]: pgmap v266: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:25 smithi193 ceph-mon[80071]: pgmap v266: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:26.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.256+0000 7f5629ab2700 1 -- 172.21.15.110:0/651657944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624102680 msgr2=0x7f56240fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:26.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.256+0000 7f5629ab2700 1 --2- 172.21.15.110:0/651657944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624102680 0x7f56240fbcf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f5618004940 tx=0x7f5618009e30 comp rx=0 tx=0).stop 2024-04-25T15:59:26.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.256+0000 7f5629ab2700 1 -- 172.21.15.110:0/651657944 shutdown_connections 2024-04-25T15:59:26.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.256+0000 7f5629ab2700 1 --2- 172.21.15.110:0/651657944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624102680 0x7f56240fbcf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:26.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.256+0000 7f5629ab2700 1 --2- 172.21.15.110:0/651657944 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5624101ce0 0x7f56241020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:26.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.256+0000 7f5629ab2700 1 -- 172.21.15.110:0/651657944 >> 172.21.15.110:0/651657944 conn(0x7f56240f7850 msgr2=0x7f56240f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:26.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.256+0000 7f5629ab2700 1 -- 172.21.15.110:0/651657944 shutdown_connections 2024-04-25T15:59:26.258 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.256+0000 7f5629ab2700 1 -- 172.21.15.110:0/651657944 wait complete. 2024-04-25T15:59:26.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.257+0000 7f5629ab2700 1 Processor -- start 2024-04-25T15:59:26.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.257+0000 7f5629ab2700 1 -- start start 2024-04-25T15:59:26.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.257+0000 7f5629ab2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624101ce0 0x7f5624112b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:26.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.257+0000 7f5629ab2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5624102680 0x7f5624113050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:26.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.257+0000 7f5629ab2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5624113730 con 0x7f5624101ce0 2024-04-25T15:59:26.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.257+0000 7f5629ab2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562410ca90 con 0x7f5624102680 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f56237fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624101ce0 0x7f5624112b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f56237fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624101ce0 0x7f5624112b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45314/0 (socket says 172.21.15.110:45314) 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f56237fe700 1 -- 172.21.15.110:0/436886205 learned_addr learned my addr 172.21.15.110:0/436886205 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f5622ffd700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5624102680 0x7f5624113050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f56237fe700 1 -- 172.21.15.110:0/436886205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5624102680 msgr2=0x7f5624113050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f56237fe700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5624102680 0x7f5624113050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f56237fe700 1 -- 172.21.15.110:0/436886205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5618005040 con 0x7f5624101ce0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f56237fe700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624101ce0 0x7f5624112b10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f561400b790 tx=0x7f5614009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f560bfff700 1 -- 172.21.15.110:0/436886205 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f561400f6f0 con 0x7f5624101ce0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562410cd70 con 0x7f5624101ce0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f560bfff700 1 -- 172.21.15.110:0/436886205 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f561400fd20 con 0x7f5624101ce0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.258+0000 7f560bfff700 1 -- 172.21.15.110:0/436886205 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56140103a0 con 0x7f5624101ce0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.259+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562410d280 con 0x7f5624101ce0 2024-04-25T15:59:26.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.260+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56240fd430 con 0x7f5624101ce0 2024-04-25T15:59:26.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.260+0000 7f560bfff700 1 -- 172.21.15.110:0/436886205 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f5614011400 con 0x7f5624101ce0 2024-04-25T15:59:26.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.260+0000 7f560bfff700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f560c076b90 0x7f560c079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:26.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.261+0000 7f560bfff700 1 -- 172.21.15.110:0/436886205 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5614098230 con 0x7f5624101ce0 2024-04-25T15:59:26.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.261+0000 7f5622ffd700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f560c076b90 0x7f560c079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:26.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.261+0000 7f5622ffd700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f560c076b90 0x7f560c079040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f562410dd70 tx=0x7f5618009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:26.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.266+0000 7f560bfff700 1 -- 172.21.15.110:0/436886205 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5614060ce0 con 0x7f5624101ce0 2024-04-25T15:59:26.482 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.481+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5624108ea0 con 0x7f560c076b90 2024-04-25T15:59:26.492 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.491+0000 7f560bfff700 1 -- 172.21.15.110:0/436886205 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f5624108ea0 con 0x7f560c076b90 2024-04-25T15:59:26.492 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (56s) 27s ago 18m 15.0M - 0.25.0 c8568f914cd2 e4c0cbbecaf4 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (6m) 27s ago 18m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (6m) 87s ago 17m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (32s) 27s ago 18m 52.0M - 9.4.12 2bacad6d85d8 640190cc1123 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (2m) 27s ago 12m 83.3M - 19.0.0-3291-g994e2f20 d53c96779a29 051262111b6b 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (3m) 27s ago 13m 160M - 19.0.0-3291-g994e2f20 d53c96779a29 763d1dda41b7 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (2m) 87s ago 13m 82.7M - 19.0.0-3291-g994e2f20 d53c96779a29 36e42c64ae3a 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (106s) 87s ago 12m 79.6M - 19.0.0-3291-g994e2f20 d53c96779a29 3dee50406dde 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (9m) 27s ago 19m 589M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (8m) 87s ago 17m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (7m) 27s ago 19m 123M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (7m) 87s ago 17m 105M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (94s) 27s ago 18m 7810k - 1.5.0 0da6a335fe13 c74eafffc54d 2024-04-25T15:59:26.493 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (90s) 87s ago 17m 3556k - 1.5.0 0da6a335fe13 a766013ab03e 2024-04-25T15:59:26.494 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (6m) 27s ago 16m 168M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T15:59:26.494 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (5m) 27s ago 16m 171M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T15:59:26.494 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (5m) 27s ago 15m 157M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T15:59:26.494 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (4m) 87s ago 15m 176M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T15:59:26.494 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (3m) 87s ago 14m 159M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T15:59:26.494 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (3m) 87s ago 14m 160M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 31c4aca48886 2024-04-25T15:59:26.494 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (73s) 27s ago 17m 49.4M - 2.43.0 a07b618ecd1d 01d861aba74b 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.493+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f560c076b90 msgr2=0x7f560c079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.493+0000 7f5629ab2700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f560c076b90 0x7f560c079040 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f562410dd70 tx=0x7f5618009e30 comp rx=0 tx=0).stop 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.493+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624101ce0 msgr2=0x7f5624112b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.493+0000 7f5629ab2700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624101ce0 0x7f5624112b10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f561400b790 tx=0x7f5614009250 comp rx=0 tx=0).stop 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.494+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 shutdown_connections 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.494+0000 7f5629ab2700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f560c076b90 0x7f560c079040 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.494+0000 7f5629ab2700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5624101ce0 0x7f5624112b10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.494+0000 7f5629ab2700 1 --2- 172.21.15.110:0/436886205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5624102680 0x7f5624113050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.494+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 >> 172.21.15.110:0/436886205 conn(0x7f56240f7850 msgr2=0x7f56240f8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.494+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 shutdown_connections 2024-04-25T15:59:26.495 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:26.494+0000 7f5629ab2700 1 -- 172.21.15.110:0/436886205 wait complete. 2024-04-25T15:59:27.211 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph orch upgrade status' 2024-04-25T15:59:27.467 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:59:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:27 smithi110 ceph-mon[90991]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:59:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:27 smithi110 ceph-mon[90991]: pgmap v267: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:28.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:27 smithi193 ceph-mon[80071]: from='client.34562 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:59:28.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:27 smithi193 ceph-mon[80071]: pgmap v267: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:29.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 -- 172.21.15.110:0/1141387406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc102de0 msgr2=0x7f4bdc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:29.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 --2- 172.21.15.110:0/1141387406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc102de0 0x7f4bdc1031b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc004940 tx=0x7f4bcc009e30 comp rx=0 tx=0).stop 2024-04-25T15:59:29.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 -- 172.21.15.110:0/1141387406 shutdown_connections 2024-04-25T15:59:29.140 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 --2- 172.21.15.110:0/1141387406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bdc103780 0x7f4bdc0fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:29.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 --2- 172.21.15.110:0/1141387406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc102de0 0x7f4bdc1031b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:29.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 -- 172.21.15.110:0/1141387406 >> 172.21.15.110:0/1141387406 conn(0x7f4bdc0f8990 msgr2=0x7f4bdc0fada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:29.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 -- 172.21.15.110:0/1141387406 shutdown_connections 2024-04-25T15:59:29.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 -- 172.21.15.110:0/1141387406 wait complete. 2024-04-25T15:59:29.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 Processor -- start 2024-04-25T15:59:29.141 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.139+0000 7f4be0eb2700 1 -- start start 2024-04-25T15:59:29.142 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4be0eb2700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bdc102de0 0x7f4bdc115ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4be0eb2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc103780 0x7f4bdc116020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4be0eb2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc116700 con 0x7f4bdc103780 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4be0eb2700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc10fc50 con 0x7f4bdc102de0 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bd9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc103780 0x7f4bdc116020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bd9d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc103780 0x7f4bdc116020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:45320/0 (socket says 172.21.15.110:45320) 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bd9d9b700 1 -- 172.21.15.110:0/2991803857 learned_addr learned my addr 172.21.15.110:0/2991803857 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bda59c700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bdc102de0 0x7f4bdc115ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bd9d9b700 1 -- 172.21.15.110:0/2991803857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bdc102de0 msgr2=0x7f4bdc115ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bd9d9b700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bdc102de0 0x7f4bdc115ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bd9d9b700 1 -- 172.21.15.110:0/2991803857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bcc005040 con 0x7f4bdc103780 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bd9d9b700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc103780 0x7f4bdc116020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b790 tx=0x7f4bd0009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bcaffd700 1 -- 172.21.15.110:0/2991803857 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bd000f6f0 con 0x7f4bdc103780 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.140+0000 7f4bcaffd700 1 -- 172.21.15.110:0/2991803857 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bd000fd20 con 0x7f4bdc103780 2024-04-25T15:59:29.143 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.141+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc10fed0 con 0x7f4bdc103780 2024-04-25T15:59:29.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.141+0000 7f4bcaffd700 1 -- 172.21.15.110:0/2991803857 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bd00103a0 con 0x7f4bdc103780 2024-04-25T15:59:29.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.141+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc110490 con 0x7f4bdc103780 2024-04-25T15:59:29.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.141+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc106930 con 0x7f4bdc103780 2024-04-25T15:59:29.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.142+0000 7f4bcaffd700 1 -- 172.21.15.110:0/2991803857 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f4bd0011400 con 0x7f4bdc103780 2024-04-25T15:59:29.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.143+0000 7f4bcaffd700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bc4076b90 0x7f4bc4079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:29.144 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.143+0000 7f4bda59c700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bc4076b90 0x7f4bc4079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:29.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.143+0000 7f4bcaffd700 1 -- 172.21.15.110:0/2991803857 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4bd0098270 con 0x7f4bdc103780 2024-04-25T15:59:29.145 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.143+0000 7f4bda59c700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bc4076b90 0x7f4bc4079040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc004f90 tx=0x7f4bcc009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:29.148 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.148+0000 7f4bcaffd700 1 -- 172.21.15.110:0/2991803857 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4bd0060ca0 con 0x7f4bdc103780 2024-04-25T15:59:29.366 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.365+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4bdc109f50 con 0x7f4bc4076b90 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.366+0000 7f4bcaffd700 1 -- 172.21.15.110:0/2991803857 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f4bdc109f50 con 0x7f4bc4076b90 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "target_image": null, 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "in_progress": false, 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "which": "", 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "services_complete": [], 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "progress": null, 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "message": "", 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout: "is_paused": false 2024-04-25T15:59:29.367 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:59:29.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bc4076b90 msgr2=0x7f4bc4079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:29.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bc4076b90 0x7f4bc4079040 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc004f90 tx=0x7f4bcc009e30 comp rx=0 tx=0).stop 2024-04-25T15:59:29.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc103780 msgr2=0x7f4bdc116020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:29.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc103780 0x7f4bdc116020 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000b790 tx=0x7f4bd0009220 comp rx=0 tx=0).stop 2024-04-25T15:59:29.369 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 shutdown_connections 2024-04-25T15:59:29.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4bdc102de0 0x7f4bdc115ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:29.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4bc4076b90 0x7f4bc4079040 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:29.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 --2- 172.21.15.110:0/2991803857 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4bdc103780 0x7f4bdc116020 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:29.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 >> 172.21.15.110:0/2991803857 conn(0x7f4bdc0f8990 msgr2=0x7f4bdc0fa4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:29.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 shutdown_connections 2024-04-25T15:59:29.370 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:29.368+0000 7f4be0eb2700 1 -- 172.21.15.110:0/2991803857 wait complete. 2024-04-25T15:59:30.112 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:29 smithi110 ceph-mon[90991]: pgmap v268: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:30.137 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph health detail' 2024-04-25T15:59:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:29 smithi193 ceph-mon[80071]: pgmap v268: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:30.391 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:59:31.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:30 smithi110 ceph-mon[90991]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:59:31.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:30 smithi110 ceph-mon[90991]: pgmap v269: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:30 smithi193 ceph-mon[80071]: from='client.34566 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T15:59:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:30 smithi193 ceph-mon[80071]: pgmap v269: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:32.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 -- 172.21.15.110:0/3307307979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc102e50 msgr2=0x7f14dc103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:32.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 --2- 172.21.15.110:0/3307307979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc102e50 0x7f14dc103220 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f14d8004940 tx=0x7f14d8009e30 comp rx=0 tx=0).stop 2024-04-25T15:59:32.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 -- 172.21.15.110:0/3307307979 shutdown_connections 2024-04-25T15:59:32.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 --2- 172.21.15.110:0/3307307979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14dc065640 0x7f14dc065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:32.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 --2- 172.21.15.110:0/3307307979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc102e50 0x7f14dc103220 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 -- 172.21.15.110:0/3307307979 >> 172.21.15.110:0/3307307979 conn(0x7f14dc071d00 msgr2=0x7f14dc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 -- 172.21.15.110:0/3307307979 shutdown_connections 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 -- 172.21.15.110:0/3307307979 wait complete. 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 Processor -- start 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 -- start start 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc065640 0x7f14dc0fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14dc102e50 0x7f14dc0fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0fc910 con 0x7f14dc065640 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.080+0000 7f14e441e700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14dc0fca50 con 0x7f14dc102e50 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e21ba700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc065640 0x7f14dc0fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e21ba700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc065640 0x7f14dc0fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54406/0 (socket says 172.21.15.110:54406) 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e21ba700 1 -- 172.21.15.110:0/1725896578 learned_addr learned my addr 172.21.15.110:0/1725896578 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:59:32.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e21ba700 1 -- 172.21.15.110:0/1725896578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14dc102e50 msgr2=0x7f14dc0fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e19b9700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14dc102e50 0x7f14dc0fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e21ba700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14dc102e50 0x7f14dc0fc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e21ba700 1 -- 172.21.15.110:0/1725896578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d8005040 con 0x7f14dc065640 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e21ba700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc065640 0x7f14dc0fbdb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b8b0 tx=0x7f14d8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14d2ffd700 1 -- 172.21.15.110:0/1725896578 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14d800e040 con 0x7f14dc065640 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14d2ffd700 1 -- 172.21.15.110:0/1725896578 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f14d8004290 con 0x7f14dc065640 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14dc119ea0 con 0x7f14dc065640 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14d2ffd700 1 -- 172.21.15.110:0/1725896578 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14d801c3f0 con 0x7f14dc065640 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.081+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14dc11a380 con 0x7f14dc065640 2024-04-25T15:59:32.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.082+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14dc0fe540 con 0x7f14dc065640 2024-04-25T15:59:32.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.083+0000 7f14d2ffd700 1 -- 172.21.15.110:0/1725896578 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f14d80137b0 con 0x7f14dc065640 2024-04-25T15:59:32.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.084+0000 7f14d2ffd700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f14c8076b90 0x7f14c8079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:32.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.084+0000 7f14d2ffd700 1 -- 172.21.15.110:0/1725896578 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f14d8099740 con 0x7f14dc065640 2024-04-25T15:59:32.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.084+0000 7f14e19b9700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f14c8076b90 0x7f14c8079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:32.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.084+0000 7f14e19b9700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f14c8076b90 0x7f14c8079040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006fd0 tx=0x7f14cc008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:32.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.088+0000 7f14d2ffd700 1 -- 172.21.15.110:0/1725896578 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f14d80621f0 con 0x7f14dc065640 2024-04-25T15:59:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:59:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:59:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.367+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f14dc05f5c0 con 0x7f14dc065640 2024-04-25T15:59:32.368 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.367+0000 7f14d2ffd700 1 -- 172.21.15.110:0/1725896578 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f14d8061b90 con 0x7f14dc065640 2024-04-25T15:59:32.368 INFO:teuthology.orchestra.run.smithi110.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T15:59:32.369 INFO:teuthology.orchestra.run.smithi110.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-04-25T15:59:32.369 INFO:teuthology.orchestra.run.smithi110.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f14c8076b90 msgr2=0x7f14c8079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f14c8076b90 0x7f14c8079040 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006fd0 tx=0x7f14cc008040 comp rx=0 tx=0).stop 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc065640 msgr2=0x7f14dc0fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc065640 0x7f14dc0fbdb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f14d800b8b0 tx=0x7f14d8009e30 comp rx=0 tx=0).stop 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 shutdown_connections 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f14c8076b90 0x7f14c8079040 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f14dc065640 0x7f14dc0fbdb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 --2- 172.21.15.110:0/1725896578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14dc102e50 0x7f14dc0fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.369+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 >> 172.21.15.110:0/1725896578 conn(0x7f14dc071d00 msgr2=0x7f14dc0fa0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.370+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 shutdown_connections 2024-04-25T15:59:32.371 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:32.370+0000 7f14e441e700 1 -- 172.21.15.110:0/1725896578 wait complete. 2024-04-25T15:59:32.884 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:32 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1725896578' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:59:32.884 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:32 smithi110 ceph-mon[90991]: pgmap v270: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:33.054 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph versions' 2024-04-25T15:59:33.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:32 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1725896578' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-25T15:59:33.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:32 smithi193 ceph-mon[80071]: pgmap v270: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:33.310 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:59:34.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.939+0000 7fa10a045700 1 -- 172.21.15.110:0/2824527251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1041037a0 msgr2=0x7fa1040fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:34.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.939+0000 7fa10a045700 1 --2- 172.21.15.110:0/2824527251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1041037a0 0x7fa1040fce10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec004940 tx=0x7fa0ec009e30 comp rx=0 tx=0).stop 2024-04-25T15:59:34.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.939+0000 7fa10a045700 1 -- 172.21.15.110:0/2824527251 shutdown_connections 2024-04-25T15:59:34.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.939+0000 7fa10a045700 1 --2- 172.21.15.110:0/2824527251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1041037a0 0x7fa1040fce10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:34.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.939+0000 7fa10a045700 1 --2- 172.21.15.110:0/2824527251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa104102e00 0x7fa1041031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:34.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.939+0000 7fa10a045700 1 -- 172.21.15.110:0/2824527251 >> 172.21.15.110:0/2824527251 conn(0x7fa1040f89b0 msgr2=0x7fa1040fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:34.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.939+0000 7fa10a045700 1 -- 172.21.15.110:0/2824527251 shutdown_connections 2024-04-25T15:59:34.958 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.940+0000 7fa10a045700 1 -- 172.21.15.110:0/2824527251 wait complete. 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.940+0000 7fa10a045700 1 Processor -- start 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.940+0000 7fa10a045700 1 -- start start 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.940+0000 7fa10a045700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa104102e00 0x7fa104070260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.940+0000 7fa10a045700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1041037a0 0x7fa1040707a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.940+0000 7fa10a045700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040721f0 con 0x7fa104102e00 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.940+0000 7fa10a045700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104070ce0 con 0x7fa1041037a0 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.940+0000 7fa1037fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa104102e00 0x7fa104070260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa1037fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa104102e00 0x7fa104070260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:54422/0 (socket says 172.21.15.110:54422) 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa1037fe700 1 -- 172.21.15.110:0/1064430147 learned_addr learned my addr 172.21.15.110:0/1064430147 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa102ffd700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1041037a0 0x7fa1040707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa1037fe700 1 -- 172.21.15.110:0/1064430147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1041037a0 msgr2=0x7fa1040707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa1037fe700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1041037a0 0x7fa1040707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa1037fe700 1 -- 172.21.15.110:0/1064430147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec005040 con 0x7fa104102e00 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa102ffd700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1041037a0 0x7fa1040707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa1037fe700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa104102e00 0x7fa104070260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00b790 tx=0x7fa0fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:34.959 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa0fbfff700 1 -- 172.21.15.110:0/1064430147 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0fc00f770 con 0x7fa104102e00 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa104070fc0 con 0x7fa104102e00 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa0fbfff700 1 -- 172.21.15.110:0/1064430147 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0fc00fda0 con 0x7fa104102e00 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa0fbfff700 1 -- 172.21.15.110:0/1064430147 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0fc00d310 con 0x7fa104102e00 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.941+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10411a1b0 con 0x7fa104102e00 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.942+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa104100110 con 0x7fa104102e00 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.943+0000 7fa0fbfff700 1 -- 172.21.15.110:0/1064430147 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fa0fc010420 con 0x7fa104102e00 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.943+0000 7fa0fbfff700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0f0076b90 0x7fa0f0079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.943+0000 7fa0fbfff700 1 -- 172.21.15.110:0/1064430147 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa0fc0984a0 con 0x7fa104102e00 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.943+0000 7fa102ffd700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0f0076b90 0x7fa0f0079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.944+0000 7fa102ffd700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0f0076b90 0x7fa0f0079040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009fd0 tx=0x7fa0ec009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T15:59:34.960 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:34.948+0000 7fa0fbfff700 1 -- 172.21.15.110:0/1064430147 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa0fc060f50 con 0x7fa104102e00 2024-04-25T15:59:35.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.234+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa1040716f0 con 0x7fa104102e00 2024-04-25T15:59:35.235 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.234+0000 7fa0fbfff700 1 -- 172.21.15.110:0/1064430147 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fa0fc0608f0 con 0x7fa104102e00 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 6 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 4 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 14 2024-04-25T15:59:35.236 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T15:59:35.237 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T15:59:35.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0f0076b90 msgr2=0x7fa0f0079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:35.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0f0076b90 0x7fa0f0079040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009fd0 tx=0x7fa0ec009e30 comp rx=0 tx=0).stop 2024-04-25T15:59:35.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa104102e00 msgr2=0x7fa104070260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T15:59:35.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa104102e00 0x7fa104070260 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00b790 tx=0x7fa0fc009250 comp rx=0 tx=0).stop 2024-04-25T15:59:35.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 shutdown_connections 2024-04-25T15:59:35.238 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa0f0076b90 0x7fa0f0079040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:35.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa104102e00 0x7fa104070260 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:35.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 --2- 172.21.15.110:0/1064430147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1041037a0 0x7fa1040707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T15:59:35.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 >> 172.21.15.110:0/1064430147 conn(0x7fa1040f89b0 msgr2=0x7fa1041073a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T15:59:35.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 shutdown_connections 2024-04-25T15:59:35.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T15:59:35.237+0000 7fa10a045700 1 -- 172.21.15.110:0/1064430147 wait complete. 2024-04-25T15:59:35.921 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-25T15:59:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:35 smithi110 ceph-mon[90991]: pgmap v271: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:36.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:35 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1064430147' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:36.176 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:59:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:35 smithi193 ceph-mon[80071]: pgmap v271: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:35 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1064430147' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T15:59:37.774 INFO:teuthology.orchestra.run.smithi110.stdout:wait for servicemap items w/ changing names to refresh 2024-04-25T15:59:38.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:37 smithi110 ceph-mon[90991]: pgmap v272: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:37 smithi193 ceph-mon[80071]: pgmap v272: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:38.424 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'sleep 60' 2024-04-25T15:59:38.682 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T15:59:39.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:39 smithi193 ceph-mon[80071]: pgmap v273: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:39.788 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:39 smithi110 ceph-mon[90991]: pgmap v273: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:42.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:41 smithi110 ceph-mon[90991]: pgmap v274: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:41 smithi193 ceph-mon[80071]: pgmap v274: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:44.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:43 smithi110 ceph-mon[90991]: pgmap v275: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:43 smithi193 ceph-mon[80071]: pgmap v275: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:45 smithi110 ceph-mon[90991]: pgmap v276: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:45 smithi193 ceph-mon[80071]: pgmap v276: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:59:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:46 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T15:59:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:47 smithi110 ceph-mon[90991]: pgmap v277: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:47 smithi193 ceph-mon[80071]: pgmap v277: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:50.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:49 smithi110 ceph-mon[90991]: pgmap v278: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:49 smithi193 ceph-mon[80071]: pgmap v278: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:52.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:51 smithi110 ceph-mon[90991]: pgmap v279: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:51 smithi193 ceph-mon[80071]: pgmap v279: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:54.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:53 smithi193 ceph-mon[80071]: pgmap v280: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:54.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:53 smithi110 ceph-mon[90991]: pgmap v280: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:56.165 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:55 smithi110 ceph-mon[90991]: pgmap v281: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:55 smithi193 ceph-mon[80071]: pgmap v281: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:58.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:57 smithi193 ceph-mon[80071]: pgmap v282: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T15:59:58.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:57 smithi110 ceph-mon[90991]: pgmap v282: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 15:59:59 smithi193 ceph-mon[80071]: pgmap v283: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:00.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 15:59:59 smithi110 ceph-mon[90991]: pgmap v283: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:00 smithi193 ceph-mon[80071]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T16:00:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:00 smithi110 ceph-mon[90991]: overall HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-04-25T16:00:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:01 smithi193 ceph-mon[80071]: pgmap v284: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:00:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:01 smithi110 ceph-mon[90991]: pgmap v284: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:00:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:02 smithi193 ceph-mon[80071]: pgmap v285: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:03.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:02 smithi110 ceph-mon[90991]: pgmap v285: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:05 smithi110 ceph-mon[90991]: pgmap v286: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:05 smithi193 ceph-mon[80071]: pgmap v286: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:07 smithi110 ceph-mon[90991]: pgmap v287: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:08.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:07 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:00:08.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:07 smithi193 ceph-mon[80071]: pgmap v287: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:08.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:07 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:00:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:08 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:00:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:08 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:00:09.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:08 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:00:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:08 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:00:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:08 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:00:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:08 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:00:10.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:09 smithi110 ceph-mon[90991]: pgmap v288: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:09 smithi193 ceph-mon[80071]: pgmap v288: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:12.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:11 smithi110 ceph-mon[90991]: pgmap v289: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:11 smithi193 ceph-mon[80071]: pgmap v289: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:14.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:13 smithi110 ceph-mon[90991]: pgmap v290: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:13 smithi193 ceph-mon[80071]: pgmap v290: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:15 smithi193 ceph-mon[80071]: pgmap v291: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:16.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:15 smithi110 ceph-mon[90991]: pgmap v291: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:00:17.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:00:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:17 smithi193 ceph-mon[80071]: pgmap v292: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:18.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:17 smithi110 ceph-mon[90991]: pgmap v292: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:19 smithi193 ceph-mon[80071]: pgmap v293: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:20.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:19 smithi110 ceph-mon[90991]: pgmap v293: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:21 smithi193 ceph-mon[80071]: pgmap v294: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:22.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:21 smithi110 ceph-mon[90991]: pgmap v294: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:22 smithi193 ceph-mon[80071]: pgmap v295: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:22 smithi110 ceph-mon[90991]: pgmap v295: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:26.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:25 smithi110 ceph-mon[90991]: pgmap v296: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:25 smithi193 ceph-mon[80071]: pgmap v296: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:28.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:27 smithi110 ceph-mon[90991]: pgmap v297: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:28.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:27 smithi193 ceph-mon[80071]: pgmap v297: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:30.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:29 smithi110 ceph-mon[90991]: pgmap v298: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:29 smithi193 ceph-mon[80071]: pgmap v298: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:31 smithi193 ceph-mon[80071]: pgmap v299: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:00:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:31 smithi110 ceph-mon[90991]: pgmap v299: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:00:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:33 smithi193 ceph-mon[80071]: pgmap v300: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:34.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:33 smithi110 ceph-mon[90991]: pgmap v300: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:35 smithi193 ceph-mon[80071]: pgmap v301: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:36.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:35 smithi110 ceph-mon[90991]: pgmap v301: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:37 smithi193 ceph-mon[80071]: pgmap v302: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:38.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:37 smithi110 ceph-mon[90991]: pgmap v302: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:40.239 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:39 smithi110 ceph-mon[90991]: pgmap v303: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:39 smithi193 ceph-mon[80071]: pgmap v303: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:40.957 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph orch ps' 2024-04-25T16:00:41.214 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T16:00:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:40 smithi193 ceph-mon[80071]: pgmap v304: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:41.336 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:40 smithi110 ceph-mon[90991]: pgmap v304: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:43.037 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.034+0000 7f29f4b8a700 1 -- 172.21.15.110:0/979920988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 msgr2=0x7f29f00fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.034+0000 7f29f4b8a700 1 --2- 172.21.15.110:0/979920988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 0x7f29f00fc1b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f29e4004940 tx=0x7f29e4009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.034+0000 7f29f4b8a700 1 -- 172.21.15.110:0/979920988 shutdown_connections 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.034+0000 7f29f4b8a700 1 --2- 172.21.15.110:0/979920988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 0x7f29f00fc1b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.034+0000 7f29f4b8a700 1 --2- 172.21.15.110:0/979920988 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f0101d90 0x7f29f0102160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.034+0000 7f29f4b8a700 1 -- 172.21.15.110:0/979920988 >> 172.21.15.110:0/979920988 conn(0x7f29f0071d00 msgr2=0x7f29f0072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.034+0000 7f29f4b8a700 1 -- 172.21.15.110:0/979920988 shutdown_connections 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.034+0000 7f29f4b8a700 1 -- 172.21.15.110:0/979920988 wait complete. 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29f4b8a700 1 Processor -- start 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29f4b8a700 1 -- start start 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29f4b8a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 0x7f29f010eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29f4b8a700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f0101d90 0x7f29f010f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29f4b8a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f010f6c0 con 0x7f29f00fbd40 2024-04-25T16:00:43.038 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29f4b8a700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f010f800 con 0x7f29f0101d90 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29ee59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 0x7f29f010eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29ee59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 0x7f29f010eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42972/0 (socket says 172.21.15.110:42972) 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.035+0000 7f29ee59c700 1 -- 172.21.15.110:0/3332372240 learned_addr learned my addr 172.21.15.110:0/3332372240 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29edd9b700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f0101d90 0x7f29f010f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29ee59c700 1 -- 172.21.15.110:0/3332372240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f0101d90 msgr2=0x7f29f010f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29ee59c700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f0101d90 0x7f29f010f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29ee59c700 1 -- 172.21.15.110:0/3332372240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e4005040 con 0x7f29f00fbd40 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29ee59c700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 0x7f29f010eb30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f29e8001d80 tx=0x7f29e8009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29deffd700 1 -- 172.21.15.110:0/3332372240 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29e8007d10 con 0x7f29f00fbd40 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29deffd700 1 -- 172.21.15.110:0/3332372240 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29e8007e70 con 0x7f29f00fbd40 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f0117ba0 con 0x7f29f00fbd40 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29deffd700 1 -- 172.21.15.110:0/3332372240 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29e8014640 con 0x7f29f00fbd40 2024-04-25T16:00:43.039 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.036+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f0118100 con 0x7f29f00fbd40 2024-04-25T16:00:43.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.037+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f00fd430 con 0x7f29f00fbd40 2024-04-25T16:00:43.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.038+0000 7f29deffd700 1 -- 172.21.15.110:0/3332372240 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f29e800e960 con 0x7f29f00fbd40 2024-04-25T16:00:43.040 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.038+0000 7f29deffd700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f29d8076b90 0x7f29d8079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:43.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.038+0000 7f29deffd700 1 -- 172.21.15.110:0/3332372240 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f29e80981d0 con 0x7f29f00fbd40 2024-04-25T16:00:43.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.038+0000 7f29edd9b700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f29d8076b90 0x7f29d8079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:43.041 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.039+0000 7f29edd9b700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f29d8076b90 0x7f29d8079040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29e400b3b0 tx=0x7f29e4009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:43.044 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.043+0000 7f29deffd700 1 -- 172.21.15.110:0/3332372240 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f29e8060cb0 con 0x7f29f00fbd40 2024-04-25T16:00:43.259 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.258+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 --> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f29f0065d90 con 0x7f29d8076b90 2024-04-25T16:00:43.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.269+0000 7f29deffd700 1 -- 172.21.15.110:0/3332372240 <== mgr.34100 v2:172.21.15.110:6800/1904649981 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f29f0065d90 con 0x7f29d8076b90 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:alertmanager.smithi110 smithi110 *:9093,9094 running (2m) 104s ago 19m 15.0M - 0.25.0 c8568f914cd2 e4c0cbbecaf4 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi110 smithi110 running (8m) 104s ago 19m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 dff3c277e148 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:crash.smithi193 smithi193 running (7m) 2m ago 18m 6669k - 19.0.0-3291-g994e2f20 d53c96779a29 a68d3bf51f39 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:grafana.smithi110 smithi110 *:3000 running (109s) 104s ago 19m 52.0M - 9.4.12 2bacad6d85d8 640190cc1123 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.kdhjqx smithi110 running (3m) 104s ago 14m 83.3M - 19.0.0-3291-g994e2f20 d53c96779a29 051262111b6b 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi110.mvvbtv smithi110 running (4m) 104s ago 14m 160M - 19.0.0-3291-g994e2f20 d53c96779a29 763d1dda41b7 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.ewxwfw smithi193 running (3m) 2m ago 14m 82.7M - 19.0.0-3291-g994e2f20 d53c96779a29 36e42c64ae3a 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:mds.cephfs.smithi193.jtyzhl smithi193 running (3m) 2m ago 14m 79.6M - 19.0.0-3291-g994e2f20 d53c96779a29 3dee50406dde 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi110.lvgbkf smithi110 *:8443,9283,8765 running (10m) 104s ago 21m 589M - 19.0.0-3291-g994e2f20 d53c96779a29 07eb3edd742d 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:mgr.smithi193.mgoteh smithi193 *:8443,9283,8765 running (9m) 2m ago 18m 520M - 19.0.0-3291-g994e2f20 d53c96779a29 a6a4ae23ea31 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi110 smithi110 running (9m) 104s ago 21m 123M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 59d163e103ed 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:mon.smithi193 smithi193 running (8m) 2m ago 18m 105M 2048M 19.0.0-3291-g994e2f20 d53c96779a29 82fe48722b1a 2024-04-25T16:00:43.270 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi110 smithi110 *:9100 running (2m) 104s ago 19m 7810k - 1.5.0 0da6a335fe13 c74eafffc54d 2024-04-25T16:00:43.271 INFO:teuthology.orchestra.run.smithi110.stdout:node-exporter.smithi193 smithi193 *:9100 running (2m) 2m ago 18m 3556k - 1.5.0 0da6a335fe13 a766013ab03e 2024-04-25T16:00:43.271 INFO:teuthology.orchestra.run.smithi110.stdout:osd.0 smithi110 running (7m) 104s ago 17m 168M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 8e46029b0710 2024-04-25T16:00:43.271 INFO:teuthology.orchestra.run.smithi110.stdout:osd.1 smithi110 running (6m) 104s ago 17m 171M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 9b8b51eaafac 2024-04-25T16:00:43.271 INFO:teuthology.orchestra.run.smithi110.stdout:osd.2 smithi110 running (6m) 104s ago 16m 157M 1557M 19.0.0-3291-g994e2f20 d53c96779a29 4fba7cfd0663 2024-04-25T16:00:43.271 INFO:teuthology.orchestra.run.smithi110.stdout:osd.3 smithi193 running (5m) 2m ago 16m 176M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 daab482bd8fa 2024-04-25T16:00:43.271 INFO:teuthology.orchestra.run.smithi110.stdout:osd.4 smithi193 running (5m) 2m ago 16m 159M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 e67bdf50065e 2024-04-25T16:00:43.271 INFO:teuthology.orchestra.run.smithi110.stdout:osd.5 smithi193 running (4m) 2m ago 15m 160M 2581M 19.0.0-3291-g994e2f20 d53c96779a29 31c4aca48886 2024-04-25T16:00:43.271 INFO:teuthology.orchestra.run.smithi110.stdout:prometheus.smithi110 smithi110 *:9095 running (2m) 104s ago 19m 49.4M - 2.43.0 a07b618ecd1d 01d861aba74b 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f29d8076b90 msgr2=0x7f29d8079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f29d8076b90 0x7f29d8079040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f29e400b3b0 tx=0x7f29e4009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 msgr2=0x7f29f010eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 0x7f29f010eb30 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f29e8001d80 tx=0x7f29e8009ec0 comp rx=0 tx=0).stop 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 shutdown_connections 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f29d8076b90 0x7f29d8079040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29f00fbd40 0x7f29f010eb30 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 --2- 172.21.15.110:0/3332372240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29f0101d90 0x7f29f010f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 >> 172.21.15.110:0/3332372240 conn(0x7f29f0071d00 msgr2=0x7f29f0104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 shutdown_connections 2024-04-25T16:00:43.273 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:43.271+0000 7f29f4b8a700 1 -- 172.21.15.110:0/3332372240 wait complete. 2024-04-25T16:00:43.969 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:43 smithi110 ceph-mon[90991]: pgmap v305: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:43.993 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph versions' 2024-04-25T16:00:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:43 smithi193 ceph-mon[80071]: pgmap v305: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:44.262 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T16:00:45.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:44 smithi110 ceph-mon[90991]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:00:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:44 smithi193 ceph-mon[80071]: from='client.34578 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T16:00:45.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.932+0000 7f620803d700 1 -- 172.21.15.110:0/3172908788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 msgr2=0x7f62000fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:45.933 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.932+0000 7f620803d700 1 --2- 172.21.15.110:0/3172908788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 0x7f62000fbd00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f61fc004940 tx=0x7f61fc009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:45.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.932+0000 7f620803d700 1 -- 172.21.15.110:0/3172908788 shutdown_connections 2024-04-25T16:00:45.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.932+0000 7f620803d700 1 --2- 172.21.15.110:0/3172908788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 0x7f62000fbd00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:45.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.932+0000 7f620803d700 1 --2- 172.21.15.110:0/3172908788 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6200101cf0 0x7f62001020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:45.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.932+0000 7f620803d700 1 -- 172.21.15.110:0/3172908788 >> 172.21.15.110:0/3172908788 conn(0x7f62000f7860 msgr2=0x7f62000f9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:45.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.932+0000 7f620803d700 1 -- 172.21.15.110:0/3172908788 shutdown_connections 2024-04-25T16:00:45.934 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.932+0000 7f620803d700 1 -- 172.21.15.110:0/3172908788 wait complete. 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f620803d700 1 Processor -- start 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f620803d700 1 -- start start 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f620803d700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6200101cf0 0x7f6200115d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f620803d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 0x7f6200116280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f620803d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6200116960 con 0x7f6200102690 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f620803d700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620010fca0 con 0x7f6200101cf0 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f62055d8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 0x7f6200116280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f62055d8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 0x7f6200116280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42990/0 (socket says 172.21.15.110:42990) 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f62055d8700 1 -- 172.21.15.110:0/3261083722 learned_addr learned my addr 172.21.15.110:0/3261083722 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.933+0000 7f6205dd9700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6200101cf0 0x7f6200115d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f62055d8700 1 -- 172.21.15.110:0/3261083722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6200101cf0 msgr2=0x7f6200115d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:45.936 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f62055d8700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6200101cf0 0x7f6200115d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f62055d8700 1 -- 172.21.15.110:0/3261083722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc005040 con 0x7f6200102690 2024-04-25T16:00:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f62055d8700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 0x7f6200116280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009100 tx=0x7f61fc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f61f67fc700 1 -- 172.21.15.110:0/3261083722 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc00e040 con 0x7f6200102690 2024-04-25T16:00:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f620010fec0 con 0x7f6200102690 2024-04-25T16:00:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f61f67fc700 1 -- 172.21.15.110:0/3261083722 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61fc009af0 con 0x7f6200102690 2024-04-25T16:00:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f61f67fc700 1 -- 172.21.15.110:0/3261083722 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc01c3f0 con 0x7f6200102690 2024-04-25T16:00:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.934+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6200110480 con 0x7f6200102690 2024-04-25T16:00:45.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.935+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62000fd440 con 0x7f6200102690 2024-04-25T16:00:45.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.936+0000 7f61f67fc700 1 -- 172.21.15.110:0/3261083722 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f61fc00baf0 con 0x7f6200102690 2024-04-25T16:00:45.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.936+0000 7f61f67fc700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f61ec076b90 0x7f61ec079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:45.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.936+0000 7f6205dd9700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f61ec076b90 0x7f61ec079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:45.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.936+0000 7f61f67fc700 1 -- 172.21.15.110:0/3261083722 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f61fc099650 con 0x7f6200102690 2024-04-25T16:00:45.938 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.937+0000 7f6205dd9700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f61ec076b90 0x7f61ec079040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009b60 tx=0x7f61f00091a0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:45.942 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:45.941+0000 7f61f67fc700 1 -- 172.21.15.110:0/3261083722 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f61fc062100 con 0x7f6200102690 2024-04-25T16:00:46.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:45 smithi110 ceph-mon[90991]: pgmap v306: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:46.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.230+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6200108eb0 con 0x7f6200102690 2024-04-25T16:00:46.231 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.230+0000 7f61f67fc700 1 -- 172.21.15.110:0/3261083722 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f61fc061aa0 con 0x7f6200102690 2024-04-25T16:00:46.231 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "mon": { 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "mgr": { 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 2 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "osd": { 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 6 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 4 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "overall": { 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)": 14 2024-04-25T16:00:46.232 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-04-25T16:00:46.233 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-04-25T16:00:46.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f61ec076b90 msgr2=0x7f61ec079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f61ec076b90 0x7f61ec079040 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009b60 tx=0x7f61f00091a0 comp rx=0 tx=0).stop 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 msgr2=0x7f6200116280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 0x7f6200116280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f61fc009100 tx=0x7f61fc009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 shutdown_connections 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6200101cf0 0x7f6200115d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f61ec076b90 0x7f61ec079040 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 --2- 172.21.15.110:0/3261083722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6200102690 0x7f6200116280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.232+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 >> 172.21.15.110:0/3261083722 conn(0x7f62000f7860 msgr2=0x7f62000f8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.233+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 shutdown_connections 2024-04-25T16:00:46.234 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:46.233+0000 7f620803d700 1 -- 172.21.15.110:0/3261083722 wait complete. 2024-04-25T16:00:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:45 smithi193 ceph-mon[80071]: pgmap v306: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:46.929 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-25T16:00:47.184 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T16:00:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:46 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3261083722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T16:00:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:46 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:00:47.306 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:46 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3261083722' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T16:00:47.307 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:00:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:47 smithi193 ceph-mon[80071]: pgmap v307: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:48.269 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:47 smithi110 ceph-mon[90991]: pgmap v307: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.801+0000 7f223acf5700 1 -- 172.21.15.110:0/1171025087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 msgr2=0x7f22340fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.801+0000 7f223acf5700 1 --2- 172.21.15.110:0/1171025087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 0x7f22340fce10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2220004940 tx=0x7f2220009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.801+0000 7f223acf5700 1 -- 172.21.15.110:0/1171025087 shutdown_connections 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.801+0000 7f223acf5700 1 --2- 172.21.15.110:0/1171025087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 0x7f22340fce10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.801+0000 7f223acf5700 1 --2- 172.21.15.110:0/1171025087 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102e00 0x7f22341031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.801+0000 7f223acf5700 1 -- 172.21.15.110:0/1171025087 >> 172.21.15.110:0/1171025087 conn(0x7f22340f89b0 msgr2=0x7f22340fadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.802+0000 7f223acf5700 1 -- 172.21.15.110:0/1171025087 shutdown_connections 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.802+0000 7f223acf5700 1 -- 172.21.15.110:0/1171025087 wait complete. 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.802+0000 7f223acf5700 1 Processor -- start 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.802+0000 7f223acf5700 1 -- start start 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f223acf5700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102e00 0x7f2234113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:48.815 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f223acf5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 0x7f22341140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f223acf5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2234114710 con 0x7f22341037a0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f223acf5700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223410db70 con 0x7f2234102e00 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2233fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 0x7f22341140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2233fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 0x7f22341140c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:42996/0 (socket says 172.21.15.110:42996) 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2233fff700 1 -- 172.21.15.110:0/1920464244 learned_addr learned my addr 172.21.15.110:0/1920464244 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2238a91700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102e00 0x7f2234113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2233fff700 1 -- 172.21.15.110:0/1920464244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102e00 msgr2=0x7f2234113b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2233fff700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102e00 0x7f2234113b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2233fff700 1 -- 172.21.15.110:0/1920464244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2220005040 con 0x7f22341037a0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2238a91700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102e00 0x7f2234113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f2233fff700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 0x7f22341140c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2220009100 tx=0x7f2220009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f22317fa700 1 -- 172.21.15.110:0/1920464244 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f222000e040 con 0x7f22341037a0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223410ddf0 con 0x7f22341037a0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f22317fa700 1 -- 172.21.15.110:0/1920464244 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2220013420 con 0x7f22341037a0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.803+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223410e3b0 con 0x7f22341037a0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.804+0000 7f22317fa700 1 -- 172.21.15.110:0/1920464244 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22200255e0 con 0x7f22341037a0 2024-04-25T16:00:48.816 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.804+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22341052d0 con 0x7f22341037a0 2024-04-25T16:00:48.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.805+0000 7f22317fa700 1 -- 172.21.15.110:0/1920464244 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f2220013590 con 0x7f22341037a0 2024-04-25T16:00:48.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.805+0000 7f22317fa700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2224076b90 0x7f2224079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:48.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.806+0000 7f22317fa700 1 -- 172.21.15.110:0/1920464244 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f2220099650 con 0x7f22341037a0 2024-04-25T16:00:48.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.806+0000 7f2238a91700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2224076b90 0x7f2224079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:48.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.806+0000 7f2238a91700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2224076b90 0x7f2224079040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f22280041d0 tx=0x7f2228009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:48.817 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:48.811+0000 7f22317fa700 1 -- 172.21.15.110:0/1920464244 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2220062100 con 0x7f22341037a0 2024-04-25T16:00:49.107 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.106+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f223410f1d0 con 0x7f22341037a0 2024-04-25T16:00:49.108 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.107+0000 7f22317fa700 1 -- 172.21.15.110:0/1920464244 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f2220061aa0 con 0x7f22341037a0 2024-04-25T16:00:49.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2224076b90 msgr2=0x7f2224079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:49.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2224076b90 0x7f2224079040 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f22280041d0 tx=0x7f2228009040 comp rx=0 tx=0).stop 2024-04-25T16:00:49.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 msgr2=0x7f22341140c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:49.110 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 0x7f22341140c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2220009100 tx=0x7f2220009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:49.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 shutdown_connections 2024-04-25T16:00:49.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2234102e00 0x7f2234113b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:49.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2224076b90 0x7f2224079040 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:49.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 --2- 172.21.15.110:0/1920464244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22341037a0 0x7f22341140c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:49.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 >> 172.21.15.110:0/1920464244 conn(0x7f22340f89b0 msgr2=0x7f2234102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:49.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 shutdown_connections 2024-04-25T16:00:49.111 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:49.109+0000 7f223acf5700 1 -- 172.21.15.110:0/1920464244 wait complete. 2024-04-25T16:00:49.120 INFO:teuthology.orchestra.run.smithi110.stdout:true 2024-04-25T16:00:49.719 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -e sha1=994e2f20102a20fa9a27966c14b625cc01884df8 -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-25T16:00:49.974 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T16:00:50.096 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:49 smithi110 ceph-mon[90991]: pgmap v308: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:50.096 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:49 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1920464244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T16:00:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:49 smithi193 ceph-mon[80071]: pgmap v308: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:49 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1920464244' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T16:00:51.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 -- 172.21.15.110:0/4077295562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8102e50 msgr2=0x7f54c8103220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 --2- 172.21.15.110:0/4077295562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8102e50 0x7f54c8103220 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f54b8004940 tx=0x7f54b8009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 -- 172.21.15.110:0/4077295562 shutdown_connections 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 --2- 172.21.15.110:0/4077295562 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c8065640 0x7f54c8065ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 --2- 172.21.15.110:0/4077295562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8102e50 0x7f54c8103220 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 -- 172.21.15.110:0/4077295562 >> 172.21.15.110:0/4077295562 conn(0x7f54c8071d00 msgr2=0x7f54c8072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 -- 172.21.15.110:0/4077295562 shutdown_connections 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 -- 172.21.15.110:0/4077295562 wait complete. 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.619+0000 7f54cd087700 1 Processor -- start 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54cd087700 1 -- start start 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54cd087700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8065640 0x7f54c8113b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54cd087700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c8102e50 0x7f54c81140c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:51.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54cd087700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c8114710 con 0x7f54c8065640 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54cd087700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c810db00 con 0x7f54c8102e50 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54c6d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8065640 0x7f54c8113b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54c6d9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8065640 0x7f54c8113b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47692/0 (socket says 172.21.15.110:47692) 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54c6d9d700 1 -- 172.21.15.110:0/3312650541 learned_addr learned my addr 172.21.15.110:0/3312650541 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54c659c700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c8102e50 0x7f54c81140c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54c6d9d700 1 -- 172.21.15.110:0/3312650541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c8102e50 msgr2=0x7f54c81140c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54c6d9d700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c8102e50 0x7f54c81140c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54c6d9d700 1 -- 172.21.15.110:0/3312650541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b8005040 con 0x7f54c8065640 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54c6d9d700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8065640 0x7f54c8113b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f54b800b3b0 tx=0x7f54b8009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54af7fe700 1 -- 172.21.15.110:0/3312650541 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54b800e040 con 0x7f54c8065640 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c810dd80 con 0x7f54c8065640 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.620+0000 7f54af7fe700 1 -- 172.21.15.110:0/3312650541 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f54b800bbc0 con 0x7f54c8065640 2024-04-25T16:00:51.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.621+0000 7f54af7fe700 1 -- 172.21.15.110:0/3312650541 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54b801c3f0 con 0x7f54c8065640 2024-04-25T16:00:51.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.621+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c810e340 con 0x7f54c8065640 2024-04-25T16:00:51.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.622+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c8100100 con 0x7f54c8065640 2024-04-25T16:00:51.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.622+0000 7f54af7fe700 1 -- 172.21.15.110:0/3312650541 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f54b800bd30 con 0x7f54c8065640 2024-04-25T16:00:51.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.622+0000 7f54af7fe700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f54b0076b90 0x7f54b0079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:51.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.623+0000 7f54af7fe700 1 -- 172.21.15.110:0/3312650541 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f54b80996c0 con 0x7f54c8065640 2024-04-25T16:00:51.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.623+0000 7f54c659c700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f54b0076b90 0x7f54b0079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:51.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.623+0000 7f54c659c700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f54b0076b90 0x7f54b0079040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54bc007fd0 tx=0x7f54bc009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:51.628 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.627+0000 7f54af7fe700 1 -- 172.21.15.110:0/3312650541 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f54b80620f0 con 0x7f54c8065640 2024-04-25T16:00:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.915+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f54c810f1d0 con 0x7f54c8065640 2024-04-25T16:00:51.916 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.915+0000 7f54af7fe700 1 -- 172.21.15.110:0/3312650541 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f54b8061a90 con 0x7f54c8065640 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f54b0076b90 msgr2=0x7f54b0079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f54b0076b90 0x7f54b0079040 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f54bc007fd0 tx=0x7f54bc009040 comp rx=0 tx=0).stop 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8065640 msgr2=0x7f54c8113b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8065640 0x7f54c8113b80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f54b800b3b0 tx=0x7f54b8009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 shutdown_connections 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f54b0076b90 0x7f54b0079040 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54c8065640 0x7f54c8113b80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 --2- 172.21.15.110:0/3312650541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f54c8102e50 0x7f54c81140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.917+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 >> 172.21.15.110:0/3312650541 conn(0x7f54c8071d00 msgr2=0x7f54c80fb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.918+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 shutdown_connections 2024-04-25T16:00:51.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:51.918+0000 7f54cd087700 1 -- 172.21.15.110:0/3312650541 wait complete. 2024-04-25T16:00:51.928 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)" 2024-04-25T16:00:52.225 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:51 smithi110 ceph-mon[90991]: pgmap v309: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:51 smithi193 ceph-mon[80071]: pgmap v309: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:52.551 DEBUG:teuthology.parallel:result is None 2024-04-25T16:00:52.552 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T16:00:52.564 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T16:00:52.564 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T16:00:52.819 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/mon.smithi110/config 2024-04-25T16:00:53.182 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:52 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3312650541' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T16:00:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:52 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3312650541' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-25T16:00:54.062 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:53 smithi110 ceph-mon[90991]: pgmap v310: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:54.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:53 smithi193 ceph-mon[80071]: pgmap v310: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:54.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.432+0000 7f3480106700 1 -- 172.21.15.110:0/3586626571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3478101d00 msgr2=0x7f34781020d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:54.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.432+0000 7f3480106700 1 --2- 172.21.15.110:0/3586626571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3478101d00 0x7f34781020d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f3474004940 tx=0x7f3474009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:54.439 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.432+0000 7f3480106700 1 -- 172.21.15.110:0/3586626571 shutdown_connections 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.432+0000 7f3480106700 1 --2- 172.21.15.110:0/3586626571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34781026a0 0x7f34780fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.432+0000 7f3480106700 1 --2- 172.21.15.110:0/3586626571 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3478101d00 0x7f34781020d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.432+0000 7f3480106700 1 -- 172.21.15.110:0/3586626571 >> 172.21.15.110:0/3586626571 conn(0x7f34780f7870 msgr2=0x7f34780f9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.432+0000 7f3480106700 1 -- 172.21.15.110:0/3586626571 shutdown_connections 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.432+0000 7f3480106700 1 -- 172.21.15.110:0/3586626571 wait complete. 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f3480106700 1 Processor -- start 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f3480106700 1 -- start start 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f3480106700 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3478101d00 0x7f3478113910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f3480106700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34781026a0 0x7f3478113e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f3480106700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34781144a0 con 0x7f34781026a0 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f3480106700 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347810daa0 con 0x7f3478101d00 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f347d6a1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34781026a0 0x7f3478113e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f347d6a1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34781026a0 0x7f3478113e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47714/0 (socket says 172.21.15.110:47714) 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f347d6a1700 1 -- 172.21.15.110:0/515589960 learned_addr learned my addr 172.21.15.110:0/515589960 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:54.440 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f347d6a1700 1 -- 172.21.15.110:0/515589960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3478101d00 msgr2=0x7f3478113910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f347dea2700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3478101d00 0x7f3478113910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f347d6a1700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3478101d00 0x7f3478113910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f347d6a1700 1 -- 172.21.15.110:0/515589960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3474005040 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.433+0000 7f347dea2700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3478101d00 0x7f3478113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.434+0000 7f347d6a1700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34781026a0 0x7f3478113e50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3468002a50 tx=0x7f3468004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.434+0000 7f346e7fc700 1 -- 172.21.15.110:0/515589960 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3468007cc0 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.434+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347810dd80 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.434+0000 7f346e7fc700 1 -- 172.21.15.110:0/515589960 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f346800f040 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.434+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347810e340 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.434+0000 7f346e7fc700 1 -- 172.21.15.110:0/515589960 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f346800da40 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.435+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34780ff010 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.435+0000 7f346e7fc700 1 -- 172.21.15.110:0/515589960 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f34680097c0 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.436+0000 7f346e7fc700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3464076b90 0x7f3464079040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.436+0000 7f346e7fc700 1 -- 172.21.15.110:0/515589960 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f3468098230 con 0x7f34781026a0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.436+0000 7f347dea2700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3464076b90 0x7f3464079040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:54.441 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.436+0000 7f347dea2700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3464076b90 0x7f3464079040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3474004560 tx=0x7f3474009e30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:54.443 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.441+0000 7f346e7fc700 1 -- 172.21.15.110:0/515589960 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f3468060c60 con 0x7f34781026a0 2024-04-25T16:00:54.685 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.683+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f347810ef60 con 0x7f34781026a0 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.684+0000 7f346e7fc700 1 -- 172.21.15.110:0/515589960 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 35 v35) v1 ==== 76+0+1943 (secure 0 0 0) 0x7f3468060600 con 0x7f34781026a0 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stdout:e35 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stdout:legacy client fscid: 1 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stdout:Filesystem 'cephfs' (1) 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stdout:fs_name cephfs 2024-04-25T16:00:54.686 INFO:teuthology.orchestra.run.smithi110.stdout:epoch 35 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:created 2024-04-25T15:46:19.573081+0000 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:modified 2024-04-25T15:57:44.601048+0000 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:tableserver 0 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:root 0 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:session_timeout 60 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:session_autoclose 300 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:max_file_size 1099511627776 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:max_xattr_size 65536 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:required_client_features {} 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure 0 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:last_failure_osd_epoch 83 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:max_mds 1 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:in 0 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:up {0=34404} 2024-04-25T16:00:54.687 INFO:teuthology.orchestra.run.smithi110.stdout:failed 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:damaged 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:stopped 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:data_pools [3] 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:metadata_pool 2 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:inline_data enabled 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:balancer 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:bal_rank_mask -1 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:standby_count_wanted 1 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:qdb_cluster leader: 34404 members: 34404 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.mvvbtv{0:34404} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.110:6826/1865691008,v1:172.21.15.110:6827/1865691008] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:Standby daemons: 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi110.kdhjqx{-1:34436} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.110:6828/1518950103,v1:172.21.15.110:6829/1518950103] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.ewxwfw{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6824/1783080009,v1:172.21.15.193:6825/1783080009] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:00:54.688 INFO:teuthology.orchestra.run.smithi110.stdout:[mds.cephfs.smithi193.jtyzhl{-1:44359} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.193:6826/3103986931,v1:172.21.15.193:6827/3103986931] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T16:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3464076b90 msgr2=0x7f3464079040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3464076b90 0x7f3464079040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3474004560 tx=0x7f3474009e30 comp rx=0 tx=0).stop 2024-04-25T16:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34781026a0 msgr2=0x7f3478113e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34781026a0 0x7f3478113e50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3468002a50 tx=0x7f3468004600 comp rx=0 tx=0).stop 2024-04-25T16:00:54.689 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 shutdown_connections 2024-04-25T16:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3478101d00 0x7f3478113910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f3464076b90 0x7f3464079040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 --2- 172.21.15.110:0/515589960 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34781026a0 0x7f3478113e50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 >> 172.21.15.110:0/515589960 conn(0x7f34780f7870 msgr2=0x7f34781062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 shutdown_connections 2024-04-25T16:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:54.686+0000 7f3480106700 1 -- 172.21.15.110:0/515589960 wait complete. 2024-04-25T16:00:54.690 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-04-25T16:00:55.043 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:54 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/515589960' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T16:00:55.044 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:54 smithi110 ceph-mon[90991]: pgmap v311: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:55.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:54 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/515589960' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T16:00:55.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:54 smithi193 ceph-mon[80071]: pgmap v311: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:55.336 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-25T16:00:55.348 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:00:55.446 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 -- 172.21.15.110:0/2500767305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 msgr2=0x7fb4780facf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 --2- 172.21.15.110:0/2500767305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 0x7fb4780facf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fb4680048d0 tx=0x7fb468009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 -- 172.21.15.110:0/2500767305 shutdown_connections 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 --2- 172.21.15.110:0/2500767305 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 0x7fb4780facf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 --2- 172.21.15.110:0/2500767305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb478102210 0x7fb4781025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 -- 172.21.15.110:0/2500767305 >> 172.21.15.110:0/2500767305 conn(0x7fb478070900 msgr2=0x7fb478070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 -- 172.21.15.110:0/2500767305 shutdown_connections 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 -- 172.21.15.110:0/2500767305 wait complete. 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 Processor -- start 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.443+0000 7fb48006e640 1 -- start start 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb48006e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 0x7fb478112d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb48006e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb478102210 0x7fb478113280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb48006e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb478113910 con 0x7fb4780fa880 2024-04-25T16:00:55.447 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb48006e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47810ced0 con 0x7fb478102210 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb47de0a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 0x7fb478112d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb47de0a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 0x7fb478112d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47730/0 (socket says 172.21.15.110:47730) 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb47de0a640 1 -- 172.21.15.110:0/2789414128 learned_addr learned my addr 172.21.15.110:0/2789414128 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb47d609640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb478102210 0x7fb478113280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb47de0a640 1 -- 172.21.15.110:0/2789414128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb478102210 msgr2=0x7fb478113280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb47de0a640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb478102210 0x7fb478113280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb47de0a640 1 -- 172.21.15.110:0/2789414128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468005040 con 0x7fb4780fa880 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb47de0a640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 0x7fb478112d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb47000bbb0 tx=0x7fb470009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb46e7fc640 1 -- 172.21.15.110:0/2789414128 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb47000c9c0 con 0x7fb4780fa880 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47810d1b0 con 0x7fb4780fa880 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb46e7fc640 1 -- 172.21.15.110:0/2789414128 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb4700092e0 con 0x7fb4780fa880 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb46e7fc640 1 -- 172.21.15.110:0/2789414128 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb470007640 con 0x7fb4780fa880 2024-04-25T16:00:55.448 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.444+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47810d6a0 con 0x7fb4780fa880 2024-04-25T16:00:55.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.445+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4780652b0 con 0x7fb4780fa880 2024-04-25T16:00:55.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.446+0000 7fb46e7fc640 1 -- 172.21.15.110:0/2789414128 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fb470012070 con 0x7fb4780fa880 2024-04-25T16:00:55.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.446+0000 7fb46e7fc640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb464076b00 0x7fb464078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:55.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.447+0000 7fb47d609640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb464076b00 0x7fb464078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:55.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.447+0000 7fb46e7fc640 1 -- 172.21.15.110:0/2789414128 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fb470098400 con 0x7fb4780fa880 2024-04-25T16:00:55.449 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.447+0000 7fb47d609640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb464076b00 0x7fb464078fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4781067d0 tx=0x7fb468009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:55.453 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.452+0000 7fb46e7fc640 1 -- 172.21.15.110:0/2789414128 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fb470060e30 con 0x7fb4780fa880 2024-04-25T16:00:55.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.632+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb47810e3e0 con 0x7fb4780fa880 2024-04-25T16:00:55.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.634+0000 7fb46e7fc640 1 -- 172.21.15.110:0/2789414128 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16981 (secure 0 0 0) 0x7fb4700607d0 con 0x7fb4780fa880 2024-04-25T16:00:55.636 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:55.636 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":83,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","created":"2024-04-25T15:39:31.460681+0000","modified":"2024-04-25T15:57:32.545877+0000","last_up_change":"2024-04-25T15:56:11.821632+0000","last_in_change":"2024-04-25T15:44:42.991633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T15:43:50.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-25T15:46:18.514536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-25T15:46:18.805544+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a0a2ad-2c02-4108-85a8-3ef716509956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6803","nonce":1940504504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6805","nonce":1940504504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6809","nonce":1940504504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6807","nonce":1940504504}]},"public_addr":"172.21.15.110:6803/1940504504","cluster_addr":"172.21.15.110:6805/1940504504","heartbeat_back_addr":"172.21.15.110:6809/1940504504","heartbeat_front_addr":"172.21.15.110:6807/1940504504","state":["exists","up"]},{"osd":1,"uuid":"90511b8f-4444-44ff-b30c-789349b354f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6811","nonce":4003194194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6813","nonce":4003194194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6817","nonce":4003194194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6815","nonce":4003194194}]},"public_addr":"172.21.15.110:6811/4003194194","cluster_addr":"172.21.15.110:6813/4003194194","heartbeat_back_addr":"172.21.15.110:6817/4003194194","heartbeat_front_addr":"172.21.15.110:6815/4003194194","state":["exists","up"]},{"osd":2,"uuid":"c74ff664-988d-4d4b-a248-176d11f44010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6819","nonce":3018305145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6821","nonce":3018305145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6825","nonce":3018305145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6823","nonce":3018305145}]},"public_addr":"172.21.15.110:6819/3018305145","cluster_addr":"172.21.15.110:6821/3018305145","heartbeat_back_addr":"172.21.15.110:6825/3018305145","heartbeat_front_addr":"172.21.15.110:6823/3018305145","state":["exists","up"]},{"osd":3,"uuid":"dd48aeea-c692-4258-a8df-848bf440b6ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6801","nonce":1773058821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6803","nonce":1773058821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6807","nonce":1773058821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6805","nonce":1773058821}]},"public_addr":"172.21.15.193:6801/1773058821","cluster_addr":"172.21.15.193:6803/1773058821","heartbeat_back_addr":"172.21.15.193:6807/1773058821","heartbeat_front_addr":"172.21.15.193:6805/1773058821","state":["exists","up"]},{"osd":4,"uuid":"3b0a0e9a-5308-421d-8091-fac8f7649509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6809","nonce":2972122544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6811","nonce":2972122544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6815","nonce":2972122544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6813","nonce":2972122544}]},"public_addr":"172.21.15.193:6809/2972122544","cluster_addr":"172.21.15.193:6811/2972122544","heartbeat_back_addr":"172.21.15.193:6815/2972122544","heartbeat_front_addr":"172.21.15.193:6813/2972122544","state":["exists","up"]},{"osd":5,"uuid":"c6cf2f8b-8105-475c-8a79-ce9bbcdf8644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6817","nonce":479474987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6819","nonce":479474987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6823","nonce":479474987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6821","nonce":479474987}]},"public_addr":"172.21.15.193:6817/479474987","cluster_addr":"172.21.15.193:6819/479474987","heartbeat_back_addr":"172.21.15.193:6823/479474987","heartbeat_front_addr":"172.21.15.193:6821/479474987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:42:54.075221+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:21.305107+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:48.660473+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:11.495485+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:35.758085+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:45:00.087588+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6826/1539252929":"2024-04-26T15:57:32.545837+0000","172.21.15.193:6825/746795019":"2024-04-26T15:57:07.402179+0000","172.21.15.193:6824/746795019":"2024-04-26T15:57:07.402179+0000","172.21.15.110:6828/164855346":"2024-04-26T15:56:42.732075+0000","172.21.15.110:0/668880779":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6801/1619647096":"2024-04-26T15:48:32.266095+0000","172.21.15.110:0/4219932763":"2024-04-26T15:48:32.266095+0000","172.21.15.193:0/632067029":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6829/164855346":"2024-04-26T15:56:42.732075+0000","172.21.15.110:0/1729346603":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/1043733842":"2024-04-26T15:48:32.266095+0000","172.21.15.110:0/322959174":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.193:6829/2194935512":"2024-04-26T15:50:46.461397+0000","172.21.15.193:0/2544900458":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6827/123050770":"2024-04-26T15:56:15.605317+0000","172.21.15.110:6801/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/3611960738":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6800/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2164855910":"2024-04-26T15:51:46.657552+0000","172.21.15.193:6827/1539252929":"2024-04-26T15:57:32.545837+0000","172.21.15.110:6800/1619647096":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6801/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.110:6801/1601336217":"2024-04-26T15:51:46.657552+0000","172.21.15.110:6800/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.193:0/3753576575":"2024-04-26T15:50:46.461397+0000","172.21.15.110:0/920532485":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2245327398":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/952937745":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/3260352407":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6801/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2365399538":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/2468496169":"2024-04-26T15:41:40.227647+0000","172.21.15.193:0/1470428832":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6800/1601336217":"2024-04-26T15:51:46.657552+0000","172.21.15.193:6828/2194935512":"2024-04-26T15:50:46.461397+0000","172.21.15.110:0/2068930777":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/3933829355":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/2390441078":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/1601321214":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/271262503":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/3391538140":"2024-04-26T15:51:46.657552+0000","172.21.15.110:6826/123050770":"2024-04-26T15:56:15.605317+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:00:55.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb464076b00 msgr2=0x7fb464078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:55.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb464076b00 0x7fb464078fb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb4781067d0 tx=0x7fb468009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 msgr2=0x7fb478112d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:55.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 0x7fb478112d40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb47000bbb0 tx=0x7fb470009840 comp rx=0 tx=0).stop 2024-04-25T16:00:55.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 shutdown_connections 2024-04-25T16:00:55.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fb464076b00 0x7fb464078fb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb478102210 0x7fb478113280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 --2- 172.21.15.110:0/2789414128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4780fa880 0x7fb478112d40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 >> 172.21.15.110:0/2789414128 conn(0x7fb478070900 msgr2=0x7fb4780f9ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:55.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 shutdown_connections 2024-04-25T16:00:55.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.637+0000 7fb48006e640 1 -- 172.21.15.110:0/2789414128 wait complete. 2024-04-25T16:00:55.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T15:43:50.541493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-25T15:46:18.514536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-25T15:46:18.805544+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:00:55.653 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:00:55.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.748+0000 7f147086e640 1 -- 172.21.15.110:0/1059882387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 msgr2=0x7f146c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:55.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.748+0000 7f147086e640 1 --2- 172.21.15.110:0/1059882387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 0x7f146c101480 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f145c0048d0 tx=0x7f145c009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.748+0000 7f147086e640 1 -- 172.21.15.110:0/1059882387 shutdown_connections 2024-04-25T16:00:55.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.748+0000 7f147086e640 1 --2- 172.21.15.110:0/1059882387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146c101a50 0x7f146c0fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.748+0000 7f147086e640 1 --2- 172.21.15.110:0/1059882387 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 0x7f146c101480 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.748+0000 7f147086e640 1 -- 172.21.15.110:0/1059882387 >> 172.21.15.110:0/1059882387 conn(0x7f146c0f6c60 msgr2=0x7f146c0f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.748+0000 7f147086e640 1 -- 172.21.15.110:0/1059882387 shutdown_connections 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.748+0000 7f147086e640 1 -- 172.21.15.110:0/1059882387 wait complete. 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f147086e640 1 Processor -- start 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f147086e640 1 -- start start 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f147086e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 0x7f146c10e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f147086e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146c101a50 0x7f146c10e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f147086e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c10ec40 con 0x7f146c1010b0 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f147086e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146c10edb0 con 0x7f146c101a50 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f146a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 0x7f146c10e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f146a59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 0x7f146c10e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47740/0 (socket says 172.21.15.110:47740) 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f146a59c640 1 -- 172.21.15.110:0/230214967 learned_addr learned my addr 172.21.15.110:0/230214967 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:55.753 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f1469d9b640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146c101a50 0x7f146c10e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f146a59c640 1 -- 172.21.15.110:0/230214967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146c101a50 msgr2=0x7f146c10e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f146a59c640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146c101a50 0x7f146c10e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f146a59c640 1 -- 172.21.15.110:0/230214967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f145c005040 con 0x7f146c1010b0 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.749+0000 7f1469d9b640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146c101a50 0x7f146c10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.750+0000 7f146a59c640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 0x7f146c10e1c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f145c004930 tx=0x7f145c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.750+0000 7f145affd640 1 -- 172.21.15.110:0/230214967 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145c0094c0 con 0x7f146c1010b0 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.750+0000 7f145affd640 1 -- 172.21.15.110:0/230214967 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f145c032e50 con 0x7f146c1010b0 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.750+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f146c10f030 con 0x7f146c1010b0 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.750+0000 7f145affd640 1 -- 172.21.15.110:0/230214967 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f145c0433b0 con 0x7f146c1010b0 2024-04-25T16:00:55.754 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.750+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f146c117480 con 0x7f146c1010b0 2024-04-25T16:00:55.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.751+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f146c0fc820 con 0x7f146c1010b0 2024-04-25T16:00:55.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.751+0000 7f145affd640 1 -- 172.21.15.110:0/230214967 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f145c02f070 con 0x7f146c1010b0 2024-04-25T16:00:55.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.752+0000 7f145affd640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1454076b00 0x7f1454078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:55.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.752+0000 7f145affd640 1 -- 172.21.15.110:0/230214967 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f145c0ba110 con 0x7f146c1010b0 2024-04-25T16:00:55.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.752+0000 7f1469d9b640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1454076b00 0x7f1454078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:55.755 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.752+0000 7f1469d9b640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1454076b00 0x7f1454078fb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f146c065d00 tx=0x7f146000b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:55.758 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.757+0000 7f145affd640 1 -- 172.21.15.110:0/230214967 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f145c082b40 con 0x7f146c1010b0 2024-04-25T16:00:55.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.935+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f146c10f030 con 0x7f146c1010b0 2024-04-25T16:00:55.937 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.935+0000 7f145affd640 1 -- 172.21.15.110:0/230214967 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7f146c10f030 con 0x7f146c1010b0 2024-04-25T16:00:55.937 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T16:00:55.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1454076b00 msgr2=0x7f1454078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:55.939 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1454076b00 0x7f1454078fb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f146c065d00 tx=0x7f146000b040 comp rx=0 tx=0).stop 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 msgr2=0x7f146c10e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 0x7f146c10e1c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f145c004930 tx=0x7f145c009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 shutdown_connections 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1454076b00 0x7f1454078fb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146c101a50 0x7f146c10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 --2- 172.21.15.110:0/230214967 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f146c1010b0 0x7f146c10e1c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 >> 172.21.15.110:0/230214967 conn(0x7f146c0f6c60 msgr2=0x7f146c0f7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 shutdown_connections 2024-04-25T16:00:55.940 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:55.937+0000 7f147086e640 1 -- 172.21.15.110:0/230214967 wait complete. 2024-04-25T16:00:55.949 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:55 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2789414128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:00:55.950 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 -- 172.21.15.110:0/4191729592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541010b0 msgr2=0x7f2754101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 --2- 172.21.15.110:0/4191729592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541010b0 0x7f2754101480 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f27500048d0 tx=0x7f2750009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 -- 172.21.15.110:0/4191729592 shutdown_connections 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 --2- 172.21.15.110:0/4191729592 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2754101a50 0x7f27540fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 --2- 172.21.15.110:0/4191729592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27541010b0 0x7f2754101480 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 -- 172.21.15.110:0/4191729592 >> 172.21.15.110:0/4191729592 conn(0x7f27540f6c60 msgr2=0x7f27540f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 -- 172.21.15.110:0/4191729592 shutdown_connections 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 -- 172.21.15.110:0/4191729592 wait complete. 2024-04-25T16:00:56.048 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 Processor -- start 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.044+0000 7f275b71d640 1 -- start start 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f275b71d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27541010b0 0x7f2754111f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f275b71d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2754101a50 0x7f2754112480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f275b71d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2754112a80 con 0x7f2754101a50 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f275b71d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275410c0d0 con 0x7f27541010b0 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2758cb8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2754101a50 0x7f2754112480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2758cb8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2754101a50 0x7f2754112480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47764/0 (socket says 172.21.15.110:47764) 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2758cb8640 1 -- 172.21.15.110:0/2684221210 learned_addr learned my addr 172.21.15.110:0/2684221210 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f27594b9640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27541010b0 0x7f2754111f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2758cb8640 1 -- 172.21.15.110:0/2684221210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27541010b0 msgr2=0x7f2754111f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.049 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2758cb8640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27541010b0 0x7f2754111f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2758cb8640 1 -- 172.21.15.110:0/2684221210 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2750005040 con 0x7f2754101a50 2024-04-25T16:00:56.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2758cb8640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2754101a50 0x7f2754112480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2744002a50 tx=0x7f2744004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:56.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2749ffb640 1 -- 172.21.15.110:0/2684221210 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2744007c30 con 0x7f2754101a50 2024-04-25T16:00:56.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2749ffb640 1 -- 172.21.15.110:0/2684221210 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f274400f040 con 0x7f2754101a50 2024-04-25T16:00:56.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275410c3b0 con 0x7f2754101a50 2024-04-25T16:00:56.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.045+0000 7f2749ffb640 1 -- 172.21.15.110:0/2684221210 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f274400db50 con 0x7f2754101a50 2024-04-25T16:00:56.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.046+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275410c8d0 con 0x7f2754101a50 2024-04-25T16:00:56.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.047+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27540fc820 con 0x7f2754101a50 2024-04-25T16:00:56.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.047+0000 7f2749ffb640 1 -- 172.21.15.110:0/2684221210 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f274401c400 con 0x7f2754101a50 2024-04-25T16:00:56.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.047+0000 7f2749ffb640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2740076b00 0x7f2740078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.047+0000 7f2749ffb640 1 -- 172.21.15.110:0/2684221210 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f2744097c90 con 0x7f2754101a50 2024-04-25T16:00:56.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.048+0000 7f27594b9640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2740076b00 0x7f2740078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.048+0000 7f27594b9640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2740076b00 0x7f2740078fb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2750003080 tx=0x7f2750009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:56.054 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.053+0000 7f2749ffb640 1 -- 172.21.15.110:0/2684221210 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2744060770 con 0x7f2754101a50 2024-04-25T16:00:56.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.227+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f2754108210 con 0x7f2754101a50 2024-04-25T16:00:56.230 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.228+0000 7f2749ffb640 1 -- 172.21.15.110:0/2684221210 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2744060110 con 0x7f2754101a50 2024-04-25T16:00:56.230 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T16:00:56.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2740076b00 msgr2=0x7f2740078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2740076b00 0x7f2740078fb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2750003080 tx=0x7f2750009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2754101a50 msgr2=0x7f2754112480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2754101a50 0x7f2754112480 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2744002a50 tx=0x7f2744004600 comp rx=0 tx=0).stop 2024-04-25T16:00:56.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 shutdown_connections 2024-04-25T16:00:56.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2740076b00 0x7f2740078fb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.232 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2754101a50 0x7f2754112480 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 --2- 172.21.15.110:0/2684221210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f27541010b0 0x7f2754111f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 >> 172.21.15.110:0/2684221210 conn(0x7f27540f6c60 msgr2=0x7f27540f7c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:56.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 shutdown_connections 2024-04-25T16:00:56.233 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.230+0000 7f275b71d640 1 -- 172.21.15.110:0/2684221210 wait complete. 2024-04-25T16:00:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:55 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2789414128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:00:56.243 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-25T16:00:56.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.341+0000 7f15fc86e640 1 -- 172.21.15.110:0/2161154805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f80fb0e0 msgr2=0x7f15f80fb530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.341+0000 7f15fc86e640 1 --2- 172.21.15.110:0/2161154805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f80fb0e0 0x7f15f80fb530 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0048d0 tx=0x7f15ec009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.341+0000 7f15fc86e640 1 -- 172.21.15.110:0/2161154805 shutdown_connections 2024-04-25T16:00:56.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.341+0000 7f15fc86e640 1 --2- 172.21.15.110:0/2161154805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f80fb0e0 0x7f15f80fb530 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.341+0000 7f15fc86e640 1 --2- 172.21.15.110:0/2161154805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15f8101110 0x7f15f81014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.344 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.341+0000 7f15fc86e640 1 -- 172.21.15.110:0/2161154805 >> 172.21.15.110:0/2161154805 conn(0x7f15f80710c0 msgr2=0x7f15f80714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.341+0000 7f15fc86e640 1 -- 172.21.15.110:0/2161154805 shutdown_connections 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.341+0000 7f15fc86e640 1 -- 172.21.15.110:0/2161154805 wait complete. 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15fc86e640 1 Processor -- start 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15fc86e640 1 -- start start 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15fc86e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15f80fb0e0 0x7f15f8111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15fc86e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f8101110 0x7f15f8112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15fc86e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8112730 con 0x7f15f8101110 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15fc86e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f810bd80 con 0x7f15f80fb0e0 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15f5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f8101110 0x7f15f8112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15f5d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f8101110 0x7f15f8112130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47780/0 (socket says 172.21.15.110:47780) 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15f5d9b640 1 -- 172.21.15.110:0/3127019554 learned_addr learned my addr 172.21.15.110:0/3127019554 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15f5d9b640 1 -- 172.21.15.110:0/3127019554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15f80fb0e0 msgr2=0x7f15f8111bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15f659c640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15f80fb0e0 0x7f15f8111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.345 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15f5d9b640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15f80fb0e0 0x7f15f8111bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.342+0000 7f15f5d9b640 1 -- 172.21.15.110:0/3127019554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec005040 con 0x7f15f8101110 2024-04-25T16:00:56.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.343+0000 7f15f5d9b640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f8101110 0x7f15f8112130 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0041d0 tx=0x7f15ec009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:56.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.343+0000 7f15e6ffd640 1 -- 172.21.15.110:0/3127019554 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15ec0092e0 con 0x7f15f8101110 2024-04-25T16:00:56.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.343+0000 7f15e6ffd640 1 -- 172.21.15.110:0/3127019554 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f15ec009440 con 0x7f15f8101110 2024-04-25T16:00:56.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.343+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f810c000 con 0x7f15f8101110 2024-04-25T16:00:56.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.343+0000 7f15e6ffd640 1 -- 172.21.15.110:0/3127019554 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15ec043640 con 0x7f15f8101110 2024-04-25T16:00:56.346 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.343+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f810c520 con 0x7f15f8101110 2024-04-25T16:00:56.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.344+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f80fc7d0 con 0x7f15f8101110 2024-04-25T16:00:56.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.344+0000 7f15e6ffd640 1 -- 172.21.15.110:0/3127019554 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f15ec03b9f0 con 0x7f15f8101110 2024-04-25T16:00:56.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.345+0000 7f15e6ffd640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f15e0076b00 0x7f15e0078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.345+0000 7f15e6ffd640 1 -- 172.21.15.110:0/3127019554 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f15ec0bb240 con 0x7f15f8101110 2024-04-25T16:00:56.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.345+0000 7f15f659c640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f15e0076b00 0x7f15e0078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.347 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.345+0000 7f15f659c640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f15e0076b00 0x7f15e0078fb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15e8007fd0 tx=0x7f15e8009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:56.351 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.350+0000 7f15e6ffd640 1 -- 172.21.15.110:0/3127019554 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f15ec083cf0 con 0x7f15f8101110 2024-04-25T16:00:56.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.536+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f15f810c740 con 0x7f15f8101110 2024-04-25T16:00:56.537 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.536+0000 7f15e6ffd640 1 -- 172.21.15.110:0/3127019554 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f15ec083690 con 0x7f15f8101110 2024-04-25T16:00:56.537 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T16:00:56.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f15e0076b00 msgr2=0x7f15e0078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.539 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f15e0076b00 0x7f15e0078fb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f15e8007fd0 tx=0x7f15e8009040 comp rx=0 tx=0).stop 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f8101110 msgr2=0x7f15f8112130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f8101110 0x7f15f8112130 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0041d0 tx=0x7f15ec009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 shutdown_connections 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f15e0076b00 0x7f15e0078fb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f8101110 0x7f15f8112130 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 --2- 172.21.15.110:0/3127019554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15f80fb0e0 0x7f15f8111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 >> 172.21.15.110:0/3127019554 conn(0x7f15f80710c0 msgr2=0x7f15f80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 shutdown_connections 2024-04-25T16:00:56.540 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.538+0000 7f15fc86e640 1 -- 172.21.15.110:0/3127019554 wait complete. 2024-04-25T16:00:56.550 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.646+0000 7faf52e14640 1 -- 172.21.15.110:0/452409301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101100 msgr2=0x7faf4c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.646+0000 7faf52e14640 1 --2- 172.21.15.110:0/452409301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101100 0x7faf4c1014d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faf400048d0 tx=0x7faf40009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.646+0000 7faf52e14640 1 -- 172.21.15.110:0/452409301 shutdown_connections 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.646+0000 7faf52e14640 1 --2- 172.21.15.110:0/452409301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf4c101aa0 0x7faf4c0fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.646+0000 7faf52e14640 1 --2- 172.21.15.110:0/452409301 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101100 0x7faf4c1014d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.646+0000 7faf52e14640 1 -- 172.21.15.110:0/452409301 >> 172.21.15.110:0/452409301 conn(0x7faf4c0711c0 msgr2=0x7faf4c0715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.646+0000 7faf52e14640 1 -- 172.21.15.110:0/452409301 shutdown_connections 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.646+0000 7faf52e14640 1 -- 172.21.15.110:0/452409301 wait complete. 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.647+0000 7faf52e14640 1 Processor -- start 2024-04-25T16:00:56.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.647+0000 7faf52e14640 1 -- start start 2024-04-25T16:00:56.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.647+0000 7faf52e14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf4c101100 0x7faf4c111c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.647+0000 7faf52e14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101aa0 0x7faf4c1121a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.647+0000 7faf52e14640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c1127a0 con 0x7faf4c101aa0 2024-04-25T16:00:56.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.647+0000 7faf52e14640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c10bdf0 con 0x7faf4c101100 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf4bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101aa0 0x7faf4c1121a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf4bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101aa0 0x7faf4c1121a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47798/0 (socket says 172.21.15.110:47798) 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf4bfff640 1 -- 172.21.15.110:0/2292782945 learned_addr learned my addr 172.21.15.110:0/2292782945 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf50bb0640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf4c101100 0x7faf4c111c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf4bfff640 1 -- 172.21.15.110:0/2292782945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf4c101100 msgr2=0x7faf4c111c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf4bfff640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf4c101100 0x7faf4c111c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf4bfff640 1 -- 172.21.15.110:0/2292782945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf40005040 con 0x7faf4c101aa0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf4bfff640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101aa0 0x7faf4c1121a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf38002840 tx=0x7faf38004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf497fa640 1 -- 172.21.15.110:0/2292782945 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf38007c30 con 0x7faf4c101aa0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf497fa640 1 -- 172.21.15.110:0/2292782945 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf3800f040 con 0x7faf4c101aa0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c10c0d0 con 0x7faf4c101aa0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf497fa640 1 -- 172.21.15.110:0/2292782945 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf38009570 con 0x7faf4c101aa0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.648+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c10c5f0 con 0x7faf4c101aa0 2024-04-25T16:00:56.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.649+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c0fc880 con 0x7faf4c101aa0 2024-04-25T16:00:56.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.650+0000 7faf497fa640 1 -- 172.21.15.110:0/2292782945 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7faf38009750 con 0x7faf4c101aa0 2024-04-25T16:00:56.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.650+0000 7faf497fa640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf3c076b00 0x7faf3c078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.651+0000 7faf50bb0640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf3c076b00 0x7faf3c078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.651+0000 7faf497fa640 1 -- 172.21.15.110:0/2292782945 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7faf38097c00 con 0x7faf4c101aa0 2024-04-25T16:00:56.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.651+0000 7faf50bb0640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf3c076b00 0x7faf3c078fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faf40003080 tx=0x7faf40009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:56.656 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.655+0000 7faf497fa640 1 -- 172.21.15.110:0/2292782945 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7faf3809d020 con 0x7faf4c101aa0 2024-04-25T16:00:56.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.880+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf4c1082c0 con 0x7faf4c101aa0 2024-04-25T16:00:56.881 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.880+0000 7faf497fa640 1 -- 172.21.15.110:0/2292782945 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) v1 ==== 94+0+5219 (secure 0 0 0) 0x7faf380606e0 con 0x7faf4c101aa0 2024-04-25T16:00:56.882 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:56.882 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44359,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/3103986931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3103986931},{"type":"v1","addr":"172.21.15.193:6827","nonce":3103986931}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:44.601048+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":0,"incarnation":31,"state":"up:active","state_seq":22,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34404,"qdb_cluster":[34404]},"id":1}]} 2024-04-25T16:00:56.882 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 35 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.882+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf3c076b00 msgr2=0x7faf3c078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.882+0000 7faf52e14640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf3c076b00 0x7faf3c078fb0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7faf40003080 tx=0x7faf40009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.882+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101aa0 msgr2=0x7faf4c1121a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.882+0000 7faf52e14640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101aa0 0x7faf4c1121a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7faf38002840 tx=0x7faf38004570 comp rx=0 tx=0).stop 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.883+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 shutdown_connections 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.883+0000 7faf52e14640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7faf3c076b00 0x7faf3c078fb0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.883+0000 7faf52e14640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf4c101aa0 0x7faf4c1121a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.883+0000 7faf52e14640 1 --2- 172.21.15.110:0/2292782945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf4c101100 0x7faf4c111c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.883+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 >> 172.21.15.110:0/2292782945 conn(0x7faf4c0711c0 msgr2=0x7faf4c0faa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.883+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 shutdown_connections 2024-04-25T16:00:56.884 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.883+0000 7faf52e14640 1 -- 172.21.15.110:0/2292782945 wait complete. 2024-04-25T16:00:56.895 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 13, 'max_mds': 1, 'flags': 18} 2024-04-25T16:00:56.895 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 14 2024-04-25T16:00:56.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.990+0000 7fc86319d640 1 -- 172.21.15.110:0/2100867332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 msgr2=0x7fc85c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.990+0000 7fc86319d640 1 --2- 172.21.15.110:0/2100867332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 0x7fc85c101540 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc8500048d0 tx=0x7fc850009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.990+0000 7fc86319d640 1 -- 172.21.15.110:0/2100867332 shutdown_connections 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.990+0000 7fc86319d640 1 --2- 172.21.15.110:0/2100867332 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c0fb150 0x7fc85c0fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.990+0000 7fc86319d640 1 --2- 172.21.15.110:0/2100867332 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 0x7fc85c101540 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.990+0000 7fc86319d640 1 -- 172.21.15.110:0/2100867332 >> 172.21.15.110:0/2100867332 conn(0x7fc85c071fd0 msgr2=0x7fc85c0723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.990+0000 7fc86319d640 1 -- 172.21.15.110:0/2100867332 shutdown_connections 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.990+0000 7fc86319d640 1 -- 172.21.15.110:0/2100867332 wait complete. 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc86319d640 1 Processor -- start 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc86319d640 1 -- start start 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc86319d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c0fb150 0x7fc85c10de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc86319d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 0x7fc85c10e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.994 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc86319d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c0652f0 con 0x7fc85c101170 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc86319d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85c10e900 con 0x7fc85c0fb150 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc85bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 0x7fc85c10e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc85bfff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 0x7fc85c10e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47818/0 (socket says 172.21.15.110:47818) 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.991+0000 7fc85bfff640 1 -- 172.21.15.110:0/3760776257 learned_addr learned my addr 172.21.15.110:0/3760776257 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc860f39640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c0fb150 0x7fc85c10de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc85bfff640 1 -- 172.21.15.110:0/3760776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c0fb150 msgr2=0x7fc85c10de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc85bfff640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c0fb150 0x7fc85c10de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc85bfff640 1 -- 172.21.15.110:0/3760776257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc850005040 con 0x7fc85c101170 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc860f39640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c0fb150 0x7fc85c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc85bfff640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 0x7fc85c10e3c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc848002a50 tx=0x7fc848004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc8597fa640 1 -- 172.21.15.110:0/3760776257 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc848007c30 con 0x7fc85c101170 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc85c10ebe0 con 0x7fc85c101170 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc8597fa640 1 -- 172.21.15.110:0/3760776257 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc84800f040 con 0x7fc85c101170 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc8597fa640 1 -- 172.21.15.110:0/3760776257 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc84800db50 con 0x7fc85c101170 2024-04-25T16:00:56.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.992+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc85c1171a0 con 0x7fc85c101170 2024-04-25T16:00:56.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.993+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc85c0fc870 con 0x7fc85c101170 2024-04-25T16:00:56.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.994+0000 7fc8597fa640 1 -- 172.21.15.110:0/3760776257 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fc84801c400 con 0x7fc85c101170 2024-04-25T16:00:56.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.994+0000 7fc8597fa640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc84c076b00 0x7fc84c078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:56.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.994+0000 7fc8597fa640 1 -- 172.21.15.110:0/3760776257 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fc848097d30 con 0x7fc85c101170 2024-04-25T16:00:56.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.994+0000 7fc860f39640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc84c076b00 0x7fc84c078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:56.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.994+0000 7fc860f39640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc84c076b00 0x7fc84c078fb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc850000c00 tx=0x7fc850009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:56.999 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:56.999+0000 7fc8597fa640 1 -- 172.21.15.110:0/3760776257 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc848060760 con 0x7fc85c101170 2024-04-25T16:00:57.219 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.218+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 14, "format": "json"} v 0) v1 -- 0x7fc85c108300 con 0x7fc85c101170 2024-04-25T16:00:57.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.219+0000 7fc8597fa640 1 -- 172.21.15.110:0/3760776257 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 14, "format": "json"}]=0 dumped fsmap epoch 14 v35) v1 ==== 107+0+4070 (secure 0 0 0) 0x7fc848060100 con 0x7fc85c101170 2024-04-25T16:00:57.220 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:57.220 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":14,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14478,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.110:6829/164855346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":164855346},{"type":"v1","addr":"172.21.15.110:6829","nonce":164855346}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:15.605334+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:00:57.220 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 14 2024-04-25T16:00:57.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc84c076b00 msgr2=0x7fc84c078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc84c076b00 0x7fc84c078fb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc850000c00 tx=0x7fc850009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 msgr2=0x7fc85c10e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 0x7fc85c10e3c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fc848002a50 tx=0x7fc848004600 comp rx=0 tx=0).stop 2024-04-25T16:00:57.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 shutdown_connections 2024-04-25T16:00:57.222 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc84c076b00 0x7fc84c078fb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc85c101170 0x7fc85c10e3c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 --2- 172.21.15.110:0/3760776257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc85c0fb150 0x7fc85c10de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 >> 172.21.15.110:0/3760776257 conn(0x7fc85c071fd0 msgr2=0x7fc85c0fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:57.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 shutdown_connections 2024-04-25T16:00:57.223 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.221+0000 7fc86319d640 1 -- 172.21.15.110:0/3760776257 wait complete. 2024-04-25T16:00:57.232 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:56 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/230214967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:00:57.232 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:56 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2684221210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T16:00:57.232 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:56 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3127019554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T16:00:57.232 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:56 smithi110 ceph-mon[90991]: pgmap v312: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:57.232 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:56 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2292782945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:00:57.233 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 15 2024-04-25T16:00:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:56 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/230214967' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:00:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:56 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2684221210' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T16:00:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:56 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3127019554' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T16:00:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:56 smithi193 ceph-mon[80071]: pgmap v312: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:56 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2292782945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.328+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/4099465667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b281021a0 msgr2=0x7f0b28102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.328+0000 7f0b2cb9d640 1 --2- 172.21.15.110:0/4099465667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b281021a0 0x7f0b28102570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0048d0 tx=0x7f0b1c009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.328+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/4099465667 shutdown_connections 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.328+0000 7f0b2cb9d640 1 --2- 172.21.15.110:0/4099465667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b28102b40 0x7f0b280fc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.328+0000 7f0b2cb9d640 1 --2- 172.21.15.110:0/4099465667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b281021a0 0x7f0b28102570 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.328+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/4099465667 >> 172.21.15.110:0/4099465667 conn(0x7f0b280f7d50 msgr2=0x7f0b280fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.328+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/4099465667 shutdown_connections 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.328+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/4099465667 wait complete. 2024-04-25T16:00:57.331 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b2cb9d640 1 Processor -- start 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b2cb9d640 1 -- start start 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b2cb9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b281021a0 0x7f0b28112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b2cb9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28102b40 0x7f0b281131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b2cb9d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b281137f0 con 0x7f0b28102b40 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b2cb9d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2810ce40 con 0x7f0b281021a0 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b25d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28102b40 0x7f0b281131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b25d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28102b40 0x7f0b281131f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47822/0 (socket says 172.21.15.110:47822) 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b25d9b640 1 -- 172.21.15.110:0/3928506193 learned_addr learned my addr 172.21.15.110:0/3928506193 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b2659c640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b281021a0 0x7f0b28112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b25d9b640 1 -- 172.21.15.110:0/3928506193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b281021a0 msgr2=0x7f0b28112cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b25d9b640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b281021a0 0x7f0b28112cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.329+0000 7f0b25d9b640 1 -- 172.21.15.110:0/3928506193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c005040 con 0x7f0b28102b40 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.330+0000 7f0b25d9b640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28102b40 0x7f0b281131f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b18002a50 tx=0x7f0b18004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:57.332 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.330+0000 7f0b16ffd640 1 -- 172.21.15.110:0/3928506193 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b18007c30 con 0x7f0b28102b40 2024-04-25T16:00:57.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.330+0000 7f0b16ffd640 1 -- 172.21.15.110:0/3928506193 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b1800f040 con 0x7f0b28102b40 2024-04-25T16:00:57.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.330+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2810d120 con 0x7f0b28102b40 2024-04-25T16:00:57.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.330+0000 7f0b16ffd640 1 -- 172.21.15.110:0/3928506193 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b1800db50 con 0x7f0b28102b40 2024-04-25T16:00:57.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.330+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b2810d640 con 0x7f0b28102b40 2024-04-25T16:00:57.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.331+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b280fd910 con 0x7f0b28102b40 2024-04-25T16:00:57.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.331+0000 7f0b16ffd640 1 -- 172.21.15.110:0/3928506193 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f0b18009090 con 0x7f0b28102b40 2024-04-25T16:00:57.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.332+0000 7f0b16ffd640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0b10076b00 0x7f0b10078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.332+0000 7f0b2659c640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0b10076b00 0x7f0b10078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.332+0000 7f0b16ffd640 1 -- 172.21.15.110:0/3928506193 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f0b18097f90 con 0x7f0b28102b40 2024-04-25T16:00:57.334 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.333+0000 7f0b2659c640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0b10076b00 0x7f0b10078fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c003080 tx=0x7f0b1c009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:57.337 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.337+0000 7f0b16ffd640 1 -- 172.21.15.110:0/3928506193 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0b18060a40 con 0x7f0b28102b40 2024-04-25T16:00:57.556 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.555+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 15, "format": "json"} v 0) v1 -- 0x7f0b2810e3a0 con 0x7f0b28102b40 2024-04-25T16:00:57.557 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.556+0000 7f0b16ffd640 1 -- 172.21.15.110:0/3928506193 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 15, "format": "json"}]=0 dumped fsmap epoch 15 v35) v1 ==== 107+0+4081 (secure 0 0 0) 0x7f0b180603e0 con 0x7f0b28102b40 2024-04-25T16:00:57.557 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:57.557 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":15,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:15.610129+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi110.kdhjqx","rank":0,"incarnation":15,"state":"up:replay","state_seq":2,"addr":"172.21.15.110:6829/164855346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":164855346},{"type":"v1","addr":"172.21.15.110:6829","nonce":164855346}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:00:57.558 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 15 2024-04-25T16:00:57.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0b10076b00 msgr2=0x7f0b10078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.559 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0b10076b00 0x7f0b10078fb0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c003080 tx=0x7f0b1c009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28102b40 msgr2=0x7f0b281131f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28102b40 0x7f0b281131f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b18002a50 tx=0x7f0b18004600 comp rx=0 tx=0).stop 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 shutdown_connections 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0b10076b00 0x7f0b10078fb0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28102b40 0x7f0b281131f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 --2- 172.21.15.110:0/3928506193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0b281021a0 0x7f0b28112cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 >> 172.21.15.110:0/3928506193 conn(0x7f0b280f7d50 msgr2=0x7f0b280f8cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 shutdown_connections 2024-04-25T16:00:57.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.558+0000 7f0b2cb9d640 1 -- 172.21.15.110:0/3928506193 wait complete. 2024-04-25T16:00:57.570 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 16 2024-04-25T16:00:57.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.663+0000 7f1ad9f14640 1 -- 172.21.15.110:0/3358629705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 msgr2=0x7f1ad40fd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.663+0000 7f1ad9f14640 1 --2- 172.21.15.110:0/3358629705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 0x7f1ad40fd550 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4004900 tx=0x7f1ac4009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.663+0000 7f1ad9f14640 1 -- 172.21.15.110:0/3358629705 shutdown_connections 2024-04-25T16:00:57.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.663+0000 7f1ad9f14640 1 --2- 172.21.15.110:0/3358629705 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad40fdb20 0x7f1ad41061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.663+0000 7f1ad9f14640 1 --2- 172.21.15.110:0/3358629705 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 0x7f1ad40fd550 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.663+0000 7f1ad9f14640 1 -- 172.21.15.110:0/3358629705 >> 172.21.15.110:0/3358629705 conn(0x7f1ad40f6be0 msgr2=0x7f1ad40f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:57.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.663+0000 7f1ad9f14640 1 -- 172.21.15.110:0/3358629705 shutdown_connections 2024-04-25T16:00:57.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.663+0000 7f1ad9f14640 1 -- 172.21.15.110:0/3358629705 wait complete. 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad9f14640 1 Processor -- start 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad9f14640 1 -- start start 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad9f14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 0x7f1ad4112d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad9f14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad40fdb20 0x7f1ad4113270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad9f14640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad4113870 con 0x7f1ad40fd180 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad9f14640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad410cec0 con 0x7f1ad40fdb20 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 0x7f1ad4112d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 0x7f1ad4112d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47840/0 (socket says 172.21.15.110:47840) 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad37fe640 1 -- 172.21.15.110:0/2246814588 learned_addr learned my addr 172.21.15.110:0/2246814588 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad2ffd640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad40fdb20 0x7f1ad4113270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad37fe640 1 -- 172.21.15.110:0/2246814588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad40fdb20 msgr2=0x7f1ad4113270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad37fe640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad40fdb20 0x7f1ad4113270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.664+0000 7f1ad37fe640 1 -- 172.21.15.110:0/2246814588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac4005040 con 0x7f1ad40fd180 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.665+0000 7f1ad37fe640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 0x7f1ad4112d30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000c00 tx=0x7f1ac4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:57.667 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.665+0000 7f1abbfff640 1 -- 172.21.15.110:0/2246814588 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac40094c0 con 0x7f1ad40fd180 2024-04-25T16:00:57.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.665+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad410d140 con 0x7f1ad40fd180 2024-04-25T16:00:57.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.665+0000 7f1abbfff640 1 -- 172.21.15.110:0/2246814588 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ac403d920 con 0x7f1ad40fd180 2024-04-25T16:00:57.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.665+0000 7f1abbfff640 1 -- 172.21.15.110:0/2246814588 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac403b930 con 0x7f1ad40fd180 2024-04-25T16:00:57.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.665+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad410d660 con 0x7f1ad40fd180 2024-04-25T16:00:57.668 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.666+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad4065fc0 con 0x7f1ad40fd180 2024-04-25T16:00:57.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.666+0000 7f1abbfff640 1 -- 172.21.15.110:0/2246814588 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f1ac403da90 con 0x7f1ad40fd180 2024-04-25T16:00:57.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.667+0000 7f1abbfff640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1abc076b00 0x7f1abc078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.667+0000 7f1abbfff640 1 -- 172.21.15.110:0/2246814588 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f1ac40bacf0 con 0x7f1ad40fd180 2024-04-25T16:00:57.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.667+0000 7f1ad2ffd640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1abc076b00 0x7f1abc078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.669 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.667+0000 7f1ad2ffd640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1abc076b00 0x7f1abc078fb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ad410e5f0 tx=0x7f1ac8009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:57.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.672+0000 7f1abbfff640 1 -- 172.21.15.110:0/2246814588 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1ac4083720 con 0x7f1ad40fd180 2024-04-25T16:00:57.885 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.884+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 16, "format": "json"} v 0) v1 -- 0x7f1ad410d2d0 con 0x7f1ad40fd180 2024-04-25T16:00:57.886 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.885+0000 7f1abbfff640 1 -- 172.21.15.110:0/2246814588 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 16, "format": "json"}]=0 dumped fsmap epoch 16 v35) v1 ==== 107+0+4086 (secure 0 0 0) 0x7f1ad410d2d0 con 0x7f1ad40fd180 2024-04-25T16:00:57.886 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:57.886 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":16,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:25.000001+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi110.kdhjqx","rank":0,"incarnation":15,"state":"up:reconnect","state_seq":151,"addr":"172.21.15.110:6829/164855346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":164855346},{"type":"v1","addr":"172.21.15.110:6829","nonce":164855346}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:00:57.886 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 16 2024-04-25T16:00:57.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1abc076b00 msgr2=0x7f1abc078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1abc076b00 0x7f1abc078fb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ad410e5f0 tx=0x7f1ac8009040 comp rx=0 tx=0).stop 2024-04-25T16:00:57.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 msgr2=0x7f1ad4112d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 0x7f1ad4112d30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ac4000c00 tx=0x7f1ac4009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.888 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 shutdown_connections 2024-04-25T16:00:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1abc076b00 0x7f1abc078fb0 secure :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1ad410e5f0 tx=0x7f1ac8009040 comp rx=0 tx=0).stop 2024-04-25T16:00:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad40fdb20 0x7f1ad4113270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 --2- 172.21.15.110:0/2246814588 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1ad40fd180 0x7f1ad4112d30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 >> 172.21.15.110:0/2246814588 conn(0x7f1ad40f6be0 msgr2=0x7f1ad40f8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 shutdown_connections 2024-04-25T16:00:57.889 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.887+0000 7f1ad9f14640 1 -- 172.21.15.110:0/2246814588 wait complete. 2024-04-25T16:00:57.899 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 17 2024-04-25T16:00:57.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.992+0000 7f0f98261640 1 -- 172.21.15.110:0/872067468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 msgr2=0x7f0f900fb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.992+0000 7f0f98261640 1 --2- 172.21.15.110:0/872067468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 0x7f0f900fb5c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0f800048d0 tx=0x7f0f80009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.992+0000 7f0f98261640 1 -- 172.21.15.110:0/872067468 shutdown_connections 2024-04-25T16:00:57.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.992+0000 7f0f98261640 1 --2- 172.21.15.110:0/872067468 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 0x7f0f900fb5c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.992+0000 7f0f98261640 1 --2- 172.21.15.110:0/872067468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f90101170 0x7f0f90101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.995 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.992+0000 7f0f98261640 1 -- 172.21.15.110:0/872067468 >> 172.21.15.110:0/872067468 conn(0x7f0f90071fd0 msgr2=0x7f0f900723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f98261640 1 -- 172.21.15.110:0/872067468 shutdown_connections 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f98261640 1 -- 172.21.15.110:0/872067468 wait complete. 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f98261640 1 Processor -- start 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f98261640 1 -- start start 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f98261640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 0x7f0f90111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f98261640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f90101170 0x7f0f901120c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f98261640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f901126c0 con 0x7f0f900fb150 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f98261640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9010bd10 con 0x7f0f90101170 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f95ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 0x7f0f90111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f95ffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 0x7f0f90111b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47862/0 (socket says 172.21.15.110:47862) 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.993+0000 7f0f95ffd640 1 -- 172.21.15.110:0/1765850654 learned_addr learned my addr 172.21.15.110:0/1765850654 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f957fc640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f90101170 0x7f0f901120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f95ffd640 1 -- 172.21.15.110:0/1765850654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f90101170 msgr2=0x7f0f901120c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f95ffd640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f90101170 0x7f0f901120c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:57.996 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f95ffd640 1 -- 172.21.15.110:0/1765850654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80005040 con 0x7f0f900fb150 2024-04-25T16:00:57.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f957fc640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f90101170 0x7f0f901120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:00:57.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f95ffd640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 0x7f0f90111b80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00db00 tx=0x7f0f8c004e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:57.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f867fc640 1 -- 172.21.15.110:0/1765850654 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8c017070 con 0x7f0f900fb150 2024-04-25T16:00:57.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f867fc640 1 -- 172.21.15.110:0/1765850654 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f8c00f040 con 0x7f0f900fb150 2024-04-25T16:00:57.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f9010bff0 con 0x7f0f900fb150 2024-04-25T16:00:57.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f867fc640 1 -- 172.21.15.110:0/1765850654 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8c003ae0 con 0x7f0f900fb150 2024-04-25T16:00:57.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.994+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9010c510 con 0x7f0f900fb150 2024-04-25T16:00:57.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.995+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f900fc870 con 0x7f0f900fb150 2024-04-25T16:00:57.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.995+0000 7f0f867fc640 1 -- 172.21.15.110:0/1765850654 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f0f8c003c40 con 0x7f0f900fb150 2024-04-25T16:00:57.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.996+0000 7f0f867fc640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0f7c076b00 0x7f0f7c078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:57.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.996+0000 7f0f957fc640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0f7c076b00 0x7f0f7c078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:57.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.996+0000 7f0f867fc640 1 -- 172.21.15.110:0/1765850654 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f0f8c0985a0 con 0x7f0f900fb150 2024-04-25T16:00:57.998 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:57.996+0000 7f0f957fc640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0f7c076b00 0x7f0f7c078fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f80000c00 tx=0x7f0f80009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:58.001 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.001+0000 7f0f867fc640 1 -- 172.21.15.110:0/1765850654 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f0f8c060fd0 con 0x7f0f900fb150 2024-04-25T16:00:58.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.216+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 17, "format": "json"} v 0) v1 -- 0x7f0f90063f00 con 0x7f0f900fb150 2024-04-25T16:00:58.218 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.217+0000 7f0f867fc640 1 -- 172.21.15.110:0/1765850654 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 17, "format": "json"}]=0 dumped fsmap epoch 17 v35) v1 ==== 107+0+4083 (secure 0 0 0) 0x7f0f8c060970 con 0x7f0f900fb150 2024-04-25T16:00:58.218 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:58.218 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:25.764450+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi110.kdhjqx","rank":0,"incarnation":15,"state":"up:rejoin","state_seq":152,"addr":"172.21.15.110:6829/164855346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":164855346},{"type":"v1","addr":"172.21.15.110:6829","nonce":164855346}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:00:58.218 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 17 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0f7c076b00 msgr2=0x7f0f7c078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0f7c076b00 0x7f0f7c078fb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0f80000c00 tx=0x7f0f80009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 msgr2=0x7f0f90111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 0x7f0f90111b80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c00db00 tx=0x7f0f8c004e60 comp rx=0 tx=0).stop 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 shutdown_connections 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0f7c076b00 0x7f0f7c078fb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0f90101170 0x7f0f901120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 --2- 172.21.15.110:0/1765850654 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f900fb150 0x7f0f90111b80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 >> 172.21.15.110:0/1765850654 conn(0x7f0f90071fd0 msgr2=0x7f0f900fa300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:58.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 shutdown_connections 2024-04-25T16:00:58.221 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.219+0000 7f0f98261640 1 -- 172.21.15.110:0/1765850654 wait complete. 2024-04-25T16:00:58.229 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:57 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3760776257' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-25T16:00:58.230 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:57 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3928506193' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-25T16:00:58.230 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:57 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2246814588' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-25T16:00:58.231 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-25T16:00:58.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:57 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3760776257' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 14, "format": "json"}]: dispatch 2024-04-25T16:00:58.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:57 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3928506193' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 15, "format": "json"}]: dispatch 2024-04-25T16:00:58.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:57 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2246814588' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 16, "format": "json"}]: dispatch 2024-04-25T16:00:58.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.324+0000 7f491571d640 1 -- 172.21.15.110:0/126406993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910064390 msgr2=0x7f4910064800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.324+0000 7f491571d640 1 --2- 172.21.15.110:0/126406993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910064390 0x7f4910064800 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f4904009c50 tx=0x7f4904004d10 comp rx=0 tx=0).stop 2024-04-25T16:00:58.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.324+0000 7f491571d640 1 -- 172.21.15.110:0/126406993 shutdown_connections 2024-04-25T16:00:58.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.324+0000 7f491571d640 1 --2- 172.21.15.110:0/126406993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910064390 0x7f4910064800 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.324+0000 7f491571d640 1 --2- 172.21.15.110:0/126406993 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4910102270 0x7f4910102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.324+0000 7f491571d640 1 -- 172.21.15.110:0/126406993 >> 172.21.15.110:0/126406993 conn(0x7f4910071fd0 msgr2=0x7f49100723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.324+0000 7f491571d640 1 -- 172.21.15.110:0/126406993 shutdown_connections 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.324+0000 7f491571d640 1 -- 172.21.15.110:0/126406993 wait complete. 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f491571d640 1 Processor -- start 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f491571d640 1 -- start start 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f491571d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4910064390 0x7f4910112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f491571d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910102270 0x7f49101131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f491571d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49101137a0 con 0x7f4910102270 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f491571d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f491010cdf0 con 0x7f4910064390 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f490e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910102270 0x7f49101131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f490e7fc640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910102270 0x7f49101131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47884/0 (socket says 172.21.15.110:47884) 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f490e7fc640 1 -- 172.21.15.110:0/300300629 learned_addr learned my addr 172.21.15.110:0/300300629 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f490e7fc640 1 -- 172.21.15.110:0/300300629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4910064390 msgr2=0x7f4910112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f490e7fc640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4910064390 0x7f4910112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.325+0000 7f490e7fc640 1 -- 172.21.15.110:0/300300629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49040098b0 con 0x7f4910102270 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.326+0000 7f490e7fc640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910102270 0x7f49101131a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4904004a30 tx=0x7f4904004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:58.328 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.326+0000 7f48f77fe640 1 -- 172.21.15.110:0/300300629 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490403a070 con 0x7f4910102270 2024-04-25T16:00:58.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.326+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f491010d070 con 0x7f4910102270 2024-04-25T16:00:58.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.326+0000 7f48f77fe640 1 -- 172.21.15.110:0/300300629 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4904035b20 con 0x7f4910102270 2024-04-25T16:00:58.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.326+0000 7f48f77fe640 1 -- 172.21.15.110:0/300300629 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490403eaa0 con 0x7f4910102270 2024-04-25T16:00:58.329 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.326+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f491010d610 con 0x7f4910102270 2024-04-25T16:00:58.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.327+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49100fd970 con 0x7f4910102270 2024-04-25T16:00:58.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.327+0000 7f48f77fe640 1 -- 172.21.15.110:0/300300629 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f4904032070 con 0x7f4910102270 2024-04-25T16:00:58.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.328+0000 7f48f77fe640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f48f8076b00 0x7f48f8078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.328+0000 7f48f77fe640 1 -- 172.21.15.110:0/300300629 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f49040bacb0 con 0x7f4910102270 2024-04-25T16:00:58.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.328+0000 7f490effd640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f48f8076b00 0x7f48f8078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:58.330 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.328+0000 7f490effd640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f48f8076b00 0x7f48f8078fb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4908004af0 tx=0x7f4908007c30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:58.333 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.333+0000 7f48f77fe640 1 -- 172.21.15.110:0/300300629 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f49040836e0 con 0x7f4910102270 2024-04-25T16:00:58.546 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.545+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f4910109380 con 0x7f4910102270 2024-04-25T16:00:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.546+0000 7f48f77fe640 1 -- 172.21.15.110:0/300300629 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v35) v1 ==== 107+0+4092 (secure 0 0 0) 0x7f4904083080 con 0x7f4910102270 2024-04-25T16:00:58.547 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:58.547 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:27.808558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi110.kdhjqx","rank":0,"incarnation":15,"state":"up:active","state_seq":153,"addr":"172.21.15.110:6829/164855346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":164855346},{"type":"v1","addr":"172.21.15.110:6829","nonce":164855346}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14478,"qdb_cluster":[14478]},"id":1}]} 2024-04-25T16:00:58.547 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 18 2024-04-25T16:00:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f48f8076b00 msgr2=0x7f48f8078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f48f8076b00 0x7f48f8078fb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4908004af0 tx=0x7f4908007c30 comp rx=0 tx=0).stop 2024-04-25T16:00:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910102270 msgr2=0x7f49101131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910102270 0x7f49101131a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4904004a30 tx=0x7f4904004d10 comp rx=0 tx=0).stop 2024-04-25T16:00:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 shutdown_connections 2024-04-25T16:00:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f48f8076b00 0x7f48f8078fb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.549 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4910102270 0x7f49101131a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 --2- 172.21.15.110:0/300300629 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4910064390 0x7f4910112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 >> 172.21.15.110:0/300300629 conn(0x7f4910071fd0 msgr2=0x7f49100fa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:58.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 shutdown_connections 2024-04-25T16:00:58.550 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.548+0000 7f491571d640 1 -- 172.21.15.110:0/300300629 wait complete. 2024-04-25T16:00:58.560 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-25T16:00:58.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 -- 172.21.15.110:0/200150912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 msgr2=0x7efc40101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 --2- 172.21.15.110:0/200150912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 0x7efc40101540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7efc300048d0 tx=0x7efc30009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.660 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 -- 172.21.15.110:0/200150912 shutdown_connections 2024-04-25T16:00:58.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 --2- 172.21.15.110:0/200150912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc400fb150 0x7efc400fb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 --2- 172.21.15.110:0/200150912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 0x7efc40101540 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 -- 172.21.15.110:0/200150912 >> 172.21.15.110:0/200150912 conn(0x7efc40071fd0 msgr2=0x7efc400723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:58.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 -- 172.21.15.110:0/200150912 shutdown_connections 2024-04-25T16:00:58.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 -- 172.21.15.110:0/200150912 wait complete. 2024-04-25T16:00:58.661 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 Processor -- start 2024-04-25T16:00:58.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 -- start start 2024-04-25T16:00:58.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc400fb150 0x7efc40101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 0x7efc401024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc40103e60 con 0x7efc40101170 2024-04-25T16:00:58.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.654+0000 7efc45e6e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc40102a00 con 0x7efc400fb150 2024-04-25T16:00:58.662 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc37fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 0x7efc401024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:58.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc37fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 0x7efc401024c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47902/0 (socket says 172.21.15.110:47902) 2024-04-25T16:00:58.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc37fff640 1 -- 172.21.15.110:0/3899310205 learned_addr learned my addr 172.21.15.110:0/3899310205 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:58.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc3f7fe640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc400fb150 0x7efc40101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:58.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc37fff640 1 -- 172.21.15.110:0/3899310205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc400fb150 msgr2=0x7efc40101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc37fff640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc400fb150 0x7efc40101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.663 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc37fff640 1 -- 172.21.15.110:0/3899310205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc30005040 con 0x7efc40101170 2024-04-25T16:00:58.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc37fff640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 0x7efc401024c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc2800c810 tx=0x7efc280043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:58.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc3cff9640 1 -- 172.21.15.110:0/3899310205 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc2800a820 con 0x7efc40101170 2024-04-25T16:00:58.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc40102ce0 con 0x7efc40101170 2024-04-25T16:00:58.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc3cff9640 1 -- 172.21.15.110:0/3899310205 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efc2800ae50 con 0x7efc40101170 2024-04-25T16:00:58.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc3cff9640 1 -- 172.21.15.110:0/3899310205 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efc280115c0 con 0x7efc40101170 2024-04-25T16:00:58.664 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.655+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc401193c0 con 0x7efc40101170 2024-04-25T16:00:58.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.656+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc400fc870 con 0x7efc40101170 2024-04-25T16:00:58.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.657+0000 7efc3cff9640 1 -- 172.21.15.110:0/3899310205 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7efc2800a980 con 0x7efc40101170 2024-04-25T16:00:58.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.657+0000 7efc3cff9640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc2c076b00 0x7efc2c078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.657+0000 7efc3cff9640 1 -- 172.21.15.110:0/3899310205 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7efc280989d0 con 0x7efc40101170 2024-04-25T16:00:58.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.657+0000 7efc3f7fe640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc2c076b00 0x7efc2c078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:58.665 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.658+0000 7efc3f7fe640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc2c076b00 0x7efc2c078fb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc30003080 tx=0x7efc30009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:58.666 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.662+0000 7efc3cff9640 1 -- 172.21.15.110:0/3899310205 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7efc28061400 con 0x7efc40101170 2024-04-25T16:00:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.878+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7efc40063f00 con 0x7efc40101170 2024-04-25T16:00:58.879 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.878+0000 7efc3cff9640 1 -- 172.21.15.110:0/3899310205 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v35) v1 ==== 107+0+4942 (secure 0 0 0) 0x7efc28060da0 con 0x7efc40101170 2024-04-25T16:00:58.880 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:58.880 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:27.808558+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":14478},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14478":{"gid":14478,"name":"cephfs.smithi110.kdhjqx","rank":0,"incarnation":15,"state":"up:active","state_seq":153,"addr":"172.21.15.110:6829/164855346","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":164855346},{"type":"v1","addr":"172.21.15.110:6829","nonce":164855346}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14478,"qdb_cluster":[14478]},"id":1}]} 2024-04-25T16:00:58.880 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 19 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.880+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc2c076b00 msgr2=0x7efc2c078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.880+0000 7efc45e6e640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc2c076b00 0x7efc2c078fb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efc30003080 tx=0x7efc30009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.880+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 msgr2=0x7efc401024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.880+0000 7efc45e6e640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 0x7efc401024c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efc2800c810 tx=0x7efc280043c0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.881+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 shutdown_connections 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.881+0000 7efc45e6e640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7efc2c076b00 0x7efc2c078fb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.881+0000 7efc45e6e640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efc40101170 0x7efc401024c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.881+0000 7efc45e6e640 1 --2- 172.21.15.110:0/3899310205 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc400fb150 0x7efc40101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.881+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 >> 172.21.15.110:0/3899310205 conn(0x7efc40071fd0 msgr2=0x7efc40064690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.881+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 shutdown_connections 2024-04-25T16:00:58.882 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.881+0000 7efc45e6e640 1 -- 172.21.15.110:0/3899310205 wait complete. 2024-04-25T16:00:58.892 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.987+0000 7f25f4a6e640 1 -- 172.21.15.110:0/3251344011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 msgr2=0x7f25f0102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.987+0000 7f25f4a6e640 1 --2- 172.21.15.110:0/3251344011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 0x7f25f0102600 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f25e00048d0 tx=0x7f25e0009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 -- 172.21.15.110:0/3251344011 shutdown_connections 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 --2- 172.21.15.110:0/3251344011 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f0102bd0 0x7f25f00fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 --2- 172.21.15.110:0/3251344011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 0x7f25f0102600 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 -- 172.21.15.110:0/3251344011 >> 172.21.15.110:0/3251344011 conn(0x7f25f00717e0 msgr2=0x7f25f0071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 -- 172.21.15.110:0/3251344011 shutdown_connections 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 -- 172.21.15.110:0/3251344011 wait complete. 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 Processor -- start 2024-04-25T16:00:58.990 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 -- start start 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 0x7f25f0112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f0102bd0 0x7f25f0113250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f0113850 con 0x7f25f0102230 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25f4a6e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25f010cea0 con 0x7f25f0102bd0 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25ee59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 0x7f25f0112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25ee59c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 0x7f25f0112d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47924/0 (socket says 172.21.15.110:47924) 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.988+0000 7f25ee59c640 1 -- 172.21.15.110:0/902098076 learned_addr learned my addr 172.21.15.110:0/902098076 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25ee59c640 1 -- 172.21.15.110:0/902098076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f0102bd0 msgr2=0x7f25f0113250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25ee59c640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f0102bd0 0x7f25f0113250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25ee59c640 1 -- 172.21.15.110:0/902098076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0005040 con 0x7f25f0102230 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25ee59c640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 0x7f25f0112d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f25e0004930 tx=0x7f25e0009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25deffd640 1 -- 172.21.15.110:0/902098076 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25e00094c0 con 0x7f25f0102230 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25deffd640 1 -- 172.21.15.110:0/902098076 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25e0032e50 con 0x7f25f0102230 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25f010d120 con 0x7f25f0102230 2024-04-25T16:00:58.991 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25deffd640 1 -- 172.21.15.110:0/902098076 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25e00433b0 con 0x7f25f0102230 2024-04-25T16:00:58.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.989+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25f010d5e0 con 0x7f25f0102230 2024-04-25T16:00:58.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.990+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25f0064350 con 0x7f25f0102230 2024-04-25T16:00:58.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.990+0000 7f25deffd640 1 -- 172.21.15.110:0/902098076 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f25e002f070 con 0x7f25f0102230 2024-04-25T16:00:58.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.991+0000 7f25deffd640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f25d8076b00 0x7f25d8078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:58.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.991+0000 7f25deffd640 1 -- 172.21.15.110:0/902098076 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f25e00ba450 con 0x7f25f0102230 2024-04-25T16:00:58.992 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.991+0000 7f25edd9b640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f25d8076b00 0x7f25d8078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:58.993 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.991+0000 7f25edd9b640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f25d8076b00 0x7f25d8078fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f25f010e5d0 tx=0x7f25e4007c30 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:58.997 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:58.996+0000 7f25deffd640 1 -- 172.21.15.110:0/902098076 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f25e0082e80 con 0x7f25f0102230 2024-04-25T16:00:59.210 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.209+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f25f0065530 con 0x7f25f0102230 2024-04-25T16:00:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.210+0000 7f25deffd640 1 -- 172.21.15.110:0/902098076 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v35) v1 ==== 107+0+4141 (secure 0 0 0) 0x7f25e0082820 con 0x7f25f0102230 2024-04-25T16:00:59.211 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:59.211 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:42.732308+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:00:59.211 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 20 2024-04-25T16:00:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f25d8076b00 msgr2=0x7f25d8078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f25d8076b00 0x7f25d8078fb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f25f010e5d0 tx=0x7f25e4007c30 comp rx=0 tx=0).stop 2024-04-25T16:00:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 msgr2=0x7f25f0112d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.213 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 0x7f25f0112d10 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f25e0004930 tx=0x7f25e0009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 shutdown_connections 2024-04-25T16:00:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f25d8076b00 0x7f25d8078fb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25f0102bd0 0x7f25f0113250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 --2- 172.21.15.110:0/902098076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0102230 0x7f25f0112d10 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 >> 172.21.15.110:0/902098076 conn(0x7f25f00717e0 msgr2=0x7f25f00fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 shutdown_connections 2024-04-25T16:00:59.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.212+0000 7f25f4a6e640 1 -- 172.21.15.110:0/902098076 wait complete. 2024-04-25T16:00:59.222 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:58 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1765850654' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-25T16:00:59.222 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:58 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/300300629' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-25T16:00:59.222 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:58 smithi110 ceph-mon[90991]: pgmap v313: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:59.223 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:58 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3899310205' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-25T16:00:59.224 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-25T16:00:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:58 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1765850654' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 17, "format": "json"}]: dispatch 2024-04-25T16:00:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:58 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/300300629' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-25T16:00:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:58 smithi193 ceph-mon[80071]: pgmap v313: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:00:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:58 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3899310205' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-25T16:00:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.318+0000 7f2d5d99d640 1 -- 172.21.15.110:0/1804141293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102200 msgr2=0x7f2d581025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.318+0000 7f2d5d99d640 1 --2- 172.21.15.110:0/1804141293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102200 0x7f2d581025d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d480048d0 tx=0x7f2d48009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.318+0000 7f2d5d99d640 1 -- 172.21.15.110:0/1804141293 shutdown_connections 2024-04-25T16:00:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.318+0000 7f2d5d99d640 1 --2- 172.21.15.110:0/1804141293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d58102ba0 0x7f2d580fc230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.318+0000 7f2d5d99d640 1 --2- 172.21.15.110:0/1804141293 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102200 0x7f2d581025d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.318+0000 7f2d5d99d640 1 -- 172.21.15.110:0/1804141293 >> 172.21.15.110:0/1804141293 conn(0x7f2d580f7e10 msgr2=0x7f2d580fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:59.321 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.318+0000 7f2d5d99d640 1 -- 172.21.15.110:0/1804141293 shutdown_connections 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.318+0000 7f2d5d99d640 1 -- 172.21.15.110:0/1804141293 wait complete. 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d5d99d640 1 Processor -- start 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d5d99d640 1 -- start start 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d5d99d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d58102200 0x7f2d58112c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d5d99d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102ba0 0x7f2d581131c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d5d99d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d581137c0 con 0x7f2d58102ba0 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d5d99d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5810ce10 con 0x7f2d58102200 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d4ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102ba0 0x7f2d581131c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d4ffff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102ba0 0x7f2d581131c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:47950/0 (socket says 172.21.15.110:47950) 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d4ffff640 1 -- 172.21.15.110:0/3063547835 learned_addr learned my addr 172.21.15.110:0/3063547835 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d56ffd640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d58102200 0x7f2d58112c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d4ffff640 1 -- 172.21.15.110:0/3063547835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d58102200 msgr2=0x7f2d58112c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.322 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d4ffff640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d58102200 0x7f2d58112c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d4ffff640 1 -- 172.21.15.110:0/3063547835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d48005040 con 0x7f2d58102ba0 2024-04-25T16:00:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.319+0000 7f2d4ffff640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102ba0 0x7f2d581131c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c790 tx=0x7f2d400043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.320+0000 7f2d4f7fe640 1 -- 172.21.15.110:0/3063547835 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d4000a820 con 0x7f2d58102ba0 2024-04-25T16:00:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.320+0000 7f2d4f7fe640 1 -- 172.21.15.110:0/3063547835 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d4000ae50 con 0x7f2d58102ba0 2024-04-25T16:00:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.320+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5810d0f0 con 0x7f2d58102ba0 2024-04-25T16:00:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.320+0000 7f2d4f7fe640 1 -- 172.21.15.110:0/3063547835 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d400115e0 con 0x7f2d58102ba0 2024-04-25T16:00:59.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.320+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5810d610 con 0x7f2d58102ba0 2024-04-25T16:00:59.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.321+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d580fd970 con 0x7f2d58102ba0 2024-04-25T16:00:59.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.321+0000 7f2d4f7fe640 1 -- 172.21.15.110:0/3063547835 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f2d40012400 con 0x7f2d58102ba0 2024-04-25T16:00:59.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.322+0000 7f2d4f7fe640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d44076b00 0x7f2d44078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.322+0000 7f2d4f7fe640 1 -- 172.21.15.110:0/3063547835 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f2d40098620 con 0x7f2d58102ba0 2024-04-25T16:00:59.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.322+0000 7f2d56ffd640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d44076b00 0x7f2d44078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:59.324 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.322+0000 7f2d56ffd640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d44076b00 0x7f2d44078fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d48000c00 tx=0x7f2d48009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:59.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.326+0000 7f2d4f7fe640 1 -- 172.21.15.110:0/3063547835 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f2d400610d0 con 0x7f2d58102ba0 2024-04-25T16:00:59.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.540+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f2d58109360 con 0x7f2d58102ba0 2024-04-25T16:00:59.541 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.540+0000 7f2d4f7fe640 1 -- 172.21.15.110:0/3063547835 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v35) v1 ==== 107+0+4152 (secure 0 0 0) 0x7f2d40060a70 con 0x7f2d58102ba0 2024-04-25T16:00:59.542 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:59.542 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:42.737726+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":0,"incarnation":21,"state":"up:replay","state_seq":3,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:00:59.542 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 21 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d44076b00 msgr2=0x7f2d44078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d44076b00 0x7f2d44078fb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d48000c00 tx=0x7f2d48009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102ba0 msgr2=0x7f2d581131c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102ba0 0x7f2d581131c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000c790 tx=0x7f2d400043c0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 shutdown_connections 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f2d44076b00 0x7f2d44078fb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d58102ba0 0x7f2d581131c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 --2- 172.21.15.110:0/3063547835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2d58102200 0x7f2d58112c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.544 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.542+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 >> 172.21.15.110:0/3063547835 conn(0x7f2d580f7e10 msgr2=0x7f2d580f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:59.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.543+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 shutdown_connections 2024-04-25T16:00:59.545 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.543+0000 7f2d5d99d640 1 -- 172.21.15.110:0/3063547835 wait complete. 2024-04-25T16:00:59.554 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-25T16:00:59.650 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 -- 172.21.15.110:0/1640782003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102250 msgr2=0x7f5a3c102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 --2- 172.21.15.110:0/1640782003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102250 0x7f5a3c102620 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5a300048d0 tx=0x7f5a30009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 -- 172.21.15.110:0/1640782003 shutdown_connections 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 --2- 172.21.15.110:0/1640782003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a3c102b60 0x7f5a3c0fa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 --2- 172.21.15.110:0/1640782003 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102250 0x7f5a3c102620 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 -- 172.21.15.110:0/1640782003 >> 172.21.15.110:0/1640782003 conn(0x7f5a3c070900 msgr2=0x7f5a3c070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 -- 172.21.15.110:0/1640782003 shutdown_connections 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 -- 172.21.15.110:0/1640782003 wait complete. 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 Processor -- start 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 -- start start 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a3c102250 0x7f5a3c0fba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102b60 0x7f5a3c0fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0fd9e0 con 0x7f5a3c102b60 2024-04-25T16:00:59.651 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.648+0000 7f5a4466e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c0fc500 con 0x7f5a3c102250 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a41c09640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102b60 0x7f5a3c0fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a41c09640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102b60 0x7f5a3c0fbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50670/0 (socket says 172.21.15.110:50670) 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a41c09640 1 -- 172.21.15.110:0/3783460340 learned_addr learned my addr 172.21.15.110:0/3783460340 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a41c09640 1 -- 172.21.15.110:0/3783460340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a3c102250 msgr2=0x7f5a3c0fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a41c09640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a3c102250 0x7f5a3c0fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a41c09640 1 -- 172.21.15.110:0/3783460340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30005040 con 0x7f5a3c102b60 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a41c09640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102b60 0x7f5a3c0fbfc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c760 tx=0x7f5a380043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a2effd640 1 -- 172.21.15.110:0/3783460340 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a3800a790 con 0x7f5a3c102b60 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c0fc7e0 con 0x7f5a3c102b60 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a2effd640 1 -- 172.21.15.110:0/3783460340 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a3800adc0 con 0x7f5a3c102b60 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a2effd640 1 -- 172.21.15.110:0/3783460340 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a380115e0 con 0x7f5a3c102b60 2024-04-25T16:00:59.652 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.649+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c119510 con 0x7f5a3c102b60 2024-04-25T16:00:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.650+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c0654f0 con 0x7f5a3c102b60 2024-04-25T16:00:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.651+0000 7f5a2effd640 1 -- 172.21.15.110:0/3783460340 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f5a38012400 con 0x7f5a3c102b60 2024-04-25T16:00:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.651+0000 7f5a2effd640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5a28076b00 0x7f5a28078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.651+0000 7f5a2effd640 1 -- 172.21.15.110:0/3783460340 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f5a380985a0 con 0x7f5a3c102b60 2024-04-25T16:00:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.651+0000 7f5a4240a640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5a28076b00 0x7f5a28078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:59.653 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.651+0000 7f5a4240a640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5a28076b00 0x7f5a28078fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009a10 tx=0x7f5a30009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:59.657 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.656+0000 7f5a2effd640 1 -- 172.21.15.110:0/3783460340 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5a38061050 con 0x7f5a3c102b60 2024-04-25T16:00:59.871 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.870+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f5a3c109400 con 0x7f5a3c102b60 2024-04-25T16:00:59.872 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.871+0000 7f5a2effd640 1 -- 172.21.15.110:0/3783460340 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v35) v1 ==== 107+0+4157 (secure 0 0 0) 0x7f5a380609f0 con 0x7f5a3c102b60 2024-04-25T16:00:59.872 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:00:59.872 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:52.035572+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":0,"incarnation":21,"state":"up:reconnect","state_seq":158,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:00:59.872 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 22 2024-04-25T16:00:59.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5a28076b00 msgr2=0x7f5a28078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5a28076b00 0x7f5a28078fb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f5a30009a10 tx=0x7f5a30009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102b60 msgr2=0x7f5a3c0fbfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102b60 0x7f5a3c0fbfc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800c760 tx=0x7f5a380043c0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 shutdown_connections 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5a28076b00 0x7f5a28078fb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5a3c102b60 0x7f5a3c0fbfc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 --2- 172.21.15.110:0/3783460340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a3c102250 0x7f5a3c0fba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 >> 172.21.15.110:0/3783460340 conn(0x7f5a3c070900 msgr2=0x7f5a3c104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 shutdown_connections 2024-04-25T16:00:59.875 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.873+0000 7f5a4466e640 1 -- 172.21.15.110:0/3783460340 wait complete. 2024-04-25T16:00:59.885 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-25T16:00:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.979+0000 7f43b7e25640 1 -- 172.21.15.110:0/1772488206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101070 msgr2=0x7f43b0101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.979+0000 7f43b7e25640 1 --2- 172.21.15.110:0/1772488206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101070 0x7f43b0101440 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0048d0 tx=0x7f43ac009da0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.979+0000 7f43b7e25640 1 -- 172.21.15.110:0/1772488206 shutdown_connections 2024-04-25T16:00:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.979+0000 7f43b7e25640 1 --2- 172.21.15.110:0/1772488206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b0101a10 0x7f43b00fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.979+0000 7f43b7e25640 1 --2- 172.21.15.110:0/1772488206 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101070 0x7f43b0101440 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.979+0000 7f43b7e25640 1 -- 172.21.15.110:0/1772488206 >> 172.21.15.110:0/1772488206 conn(0x7f43b00f6be0 msgr2=0x7f43b00f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:00:59.982 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.979+0000 7f43b7e25640 1 -- 172.21.15.110:0/1772488206 shutdown_connections 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.979+0000 7f43b7e25640 1 -- 172.21.15.110:0/1772488206 wait complete. 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b7e25640 1 Processor -- start 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b7e25640 1 -- start start 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b7e25640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b0101070 0x7f43b010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b7e25640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101a10 0x7f43b010e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b7e25640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b0066740 con 0x7f43b0101a10 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b7e25640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b010e900 con 0x7f43b0101070 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b53c0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101a10 0x7f43b010e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b53c0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101a10 0x7f43b010e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50674/0 (socket says 172.21.15.110:50674) 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b53c0640 1 -- 172.21.15.110:0/3282642504 learned_addr learned my addr 172.21.15.110:0/3282642504 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.980+0000 7f43b5bc1640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b0101070 0x7f43b010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43b53c0640 1 -- 172.21.15.110:0/3282642504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b0101070 msgr2=0x7f43b010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43b53c0640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b0101070 0x7f43b010de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43b53c0640 1 -- 172.21.15.110:0/3282642504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac005040 con 0x7f43b0101a10 2024-04-25T16:00:59.983 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43b53c0640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101a10 0x7f43b010e390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43a0002a50 tx=0x7f43a0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43a67fc640 1 -- 172.21.15.110:0/3282642504 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43a0007c30 con 0x7f43b0101a10 2024-04-25T16:00:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43a67fc640 1 -- 172.21.15.110:0/3282642504 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43a000f040 con 0x7f43b0101a10 2024-04-25T16:00:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b010ebe0 con 0x7f43b0101a10 2024-04-25T16:00:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43a67fc640 1 -- 172.21.15.110:0/3282642504 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43a000db50 con 0x7f43b0101a10 2024-04-25T16:00:59.984 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.981+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b0117170 con 0x7f43b0101a10 2024-04-25T16:00:59.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.982+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b00fc7e0 con 0x7f43b0101a10 2024-04-25T16:00:59.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.982+0000 7f43a67fc640 1 -- 172.21.15.110:0/3282642504 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f43a001c400 con 0x7f43b0101a10 2024-04-25T16:00:59.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.983+0000 7f43a67fc640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f439c076b00 0x7f439c078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:00:59.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.983+0000 7f43a67fc640 1 -- 172.21.15.110:0/3282642504 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f43a0097c60 con 0x7f43b0101a10 2024-04-25T16:00:59.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.983+0000 7f43b5bc1640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f439c076b00 0x7f439c078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:00:59.985 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.983+0000 7f43b5bc1640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f439c076b00 0x7f439c078fb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f43ac003080 tx=0x7f43ac009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:00:59.989 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:00:59.988+0000 7f43a67fc640 1 -- 172.21.15.110:0/3282642504 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f43a0060740 con 0x7f43b0101a10 2024-04-25T16:01:00.202 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.201+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f43b0065d40 con 0x7f43b0101a10 2024-04-25T16:01:00.203 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.202+0000 7f43a67fc640 1 -- 172.21.15.110:0/3282642504 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v35) v1 ==== 107+0+4154 (secure 0 0 0) 0x7f43a00600e0 con 0x7f43b0101a10 2024-04-25T16:01:00.203 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:00.203 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:52.764205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":159,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:00.203 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 23 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f439c076b00 msgr2=0x7f439c078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f439c076b00 0x7f439c078fb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f43ac003080 tx=0x7f43ac009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101a10 msgr2=0x7f43b010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101a10 0x7f43b010e390 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f43a0002a50 tx=0x7f43a0004600 comp rx=0 tx=0).stop 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 shutdown_connections 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f439c076b00 0x7f439c078fb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43b0101a10 0x7f43b010e390 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 --2- 172.21.15.110:0/3282642504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b0101070 0x7f43b010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 >> 172.21.15.110:0/3282642504 conn(0x7f43b00f6be0 msgr2=0x7f43b00f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:00.205 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 shutdown_connections 2024-04-25T16:01:00.206 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.204+0000 7f43b7e25640 1 -- 172.21.15.110:0/3282642504 wait complete. 2024-04-25T16:01:00.214 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:59 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/902098076' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-25T16:01:00.214 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:59 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3063547835' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-25T16:01:00.214 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:00:59 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3783460340' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-25T16:01:00.216 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-25T16:01:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:59 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/902098076' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-25T16:01:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:59 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3063547835' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-25T16:01:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:00:59 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3783460340' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-25T16:01:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.308+0000 7fda876ae640 1 -- 172.21.15.110:0/890355439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 msgr2=0x7fda80106180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.308+0000 7fda876ae640 1 --2- 172.21.15.110:0/890355439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 0x7fda80106180 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fda740048d0 tx=0x7fda74009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.308+0000 7fda876ae640 1 -- 172.21.15.110:0/890355439 shutdown_connections 2024-04-25T16:01:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.308+0000 7fda876ae640 1 --2- 172.21.15.110:0/890355439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 0x7fda80106180 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.308+0000 7fda876ae640 1 --2- 172.21.15.110:0/890355439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda800fd1a0 0x7fda800fd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.311 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.308+0000 7fda876ae640 1 -- 172.21.15.110:0/890355439 >> 172.21.15.110:0/890355439 conn(0x7fda800f6c50 msgr2=0x7fda800f9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.308+0000 7fda876ae640 1 -- 172.21.15.110:0/890355439 shutdown_connections 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.308+0000 7fda876ae640 1 -- 172.21.15.110:0/890355439 wait complete. 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda876ae640 1 Processor -- start 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda876ae640 1 -- start start 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda876ae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda800fd1a0 0x7fda800fafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda876ae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 0x7fda800fb510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda876ae640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda800fcf40 con 0x7fda800fdb40 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda876ae640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda800fba50 con 0x7fda800fd1a0 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda84c49640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 0x7fda800fb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda84c49640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 0x7fda800fb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50696/0 (socket says 172.21.15.110:50696) 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda84c49640 1 -- 172.21.15.110:0/1809221925 learned_addr learned my addr 172.21.15.110:0/1809221925 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda84c49640 1 -- 172.21.15.110:0/1809221925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda800fd1a0 msgr2=0x7fda800fafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.309+0000 7fda8544a640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda800fd1a0 0x7fda800fafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.312 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda84c49640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda800fd1a0 0x7fda800fafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda84c49640 1 -- 172.21.15.110:0/1809221925 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74005040 con 0x7fda800fdb40 2024-04-25T16:01:00.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda8544a640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda800fd1a0 0x7fda800fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:01:00.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda84c49640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 0x7fda800fb510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fda74004930 tx=0x7fda74009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:00.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda71ffb640 1 -- 172.21.15.110:0/1809221925 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda740094c0 con 0x7fda800fdb40 2024-04-25T16:01:00.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda800fbc70 con 0x7fda800fdb40 2024-04-25T16:01:00.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda71ffb640 1 -- 172.21.15.110:0/1809221925 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda74032e50 con 0x7fda800fdb40 2024-04-25T16:01:00.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda71ffb640 1 -- 172.21.15.110:0/1809221925 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda740433b0 con 0x7fda800fdb40 2024-04-25T16:01:00.313 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.310+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda80119450 con 0x7fda800fdb40 2024-04-25T16:01:00.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.311+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda80066080 con 0x7fda800fdb40 2024-04-25T16:01:00.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.312+0000 7fda71ffb640 1 -- 172.21.15.110:0/1809221925 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fda7402f070 con 0x7fda800fdb40 2024-04-25T16:01:00.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.312+0000 7fda71ffb640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fda6c076b00 0x7fda6c078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.312+0000 7fda8544a640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fda6c076b00 0x7fda6c078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.312+0000 7fda71ffb640 1 -- 172.21.15.110:0/1809221925 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fda740ba3c0 con 0x7fda800fdb40 2024-04-25T16:01:00.314 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.313+0000 7fda8544a640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fda6c076b00 0x7fda6c078fb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda78007fd0 tx=0x7fda780074f0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:00.318 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.317+0000 7fda71ffb640 1 -- 172.21.15.110:0/1809221925 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fda74082df0 con 0x7fda800fdb40 2024-04-25T16:01:00.531 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.530+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fda80064350 con 0x7fda800fdb40 2024-04-25T16:01:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.531+0000 7fda71ffb640 1 -- 172.21.15.110:0/1809221925 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v35) v1 ==== 107+0+5013 (secure 0 0 0) 0x7fda74082790 con 0x7fda800fdb40 2024-04-25T16:01:00.532 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:00.532 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:56:54.768932+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24251},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24251":{"gid":24251,"name":"cephfs.smithi193.ewxwfw","rank":0,"incarnation":21,"state":"up:active","state_seq":160,"addr":"172.21.15.193:6825/746795019","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":746795019},{"type":"v1","addr":"172.21.15.193:6825","nonce":746795019}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24251,"qdb_cluster":[24251]},"id":1}]} 2024-04-25T16:01:00.532 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 24 2024-04-25T16:01:00.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fda6c076b00 msgr2=0x7fda6c078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fda6c076b00 0x7fda6c078fb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fda78007fd0 tx=0x7fda780074f0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 msgr2=0x7fda800fb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 0x7fda800fb510 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fda74004930 tx=0x7fda74009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 shutdown_connections 2024-04-25T16:01:00.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fda6c076b00 0x7fda6c078fb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.534 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fda800fdb40 0x7fda800fb510 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 --2- 172.21.15.110:0/1809221925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda800fd1a0 0x7fda800fafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 >> 172.21.15.110:0/1809221925 conn(0x7fda800f6c50 msgr2=0x7fda800f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:00.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 shutdown_connections 2024-04-25T16:01:00.535 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.533+0000 7fda876ae640 1 -- 172.21.15.110:0/1809221925 wait complete. 2024-04-25T16:01:00.545 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-25T16:01:00.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.638+0000 7f1015a6e640 1 -- 172.21.15.110:0/1229533120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064390 msgr2=0x7f1010064760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.638+0000 7f1015a6e640 1 --2- 172.21.15.110:0/1229533120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064390 0x7f1010064760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f10000048d0 tx=0x7f1000009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.638+0000 7f1015a6e640 1 -- 172.21.15.110:0/1229533120 shutdown_connections 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.638+0000 7f1015a6e640 1 --2- 172.21.15.110:0/1229533120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010064ca0 0x7f1010108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.638+0000 7f1015a6e640 1 --2- 172.21.15.110:0/1229533120 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064390 0x7f1010064760 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.638+0000 7f1015a6e640 1 -- 172.21.15.110:0/1229533120 >> 172.21.15.110:0/1229533120 conn(0x7f1010071fd0 msgr2=0x7f10100723d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f1015a6e640 1 -- 172.21.15.110:0/1229533120 shutdown_connections 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f1015a6e640 1 -- 172.21.15.110:0/1229533120 wait complete. 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f1015a6e640 1 Processor -- start 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f1015a6e640 1 -- start start 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f1015a6e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010064390 0x7f1010114e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f1015a6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064ca0 0x7f1010115390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f1015a6e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1010115990 con 0x7f1010064ca0 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f1015a6e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101010efe0 con 0x7f1010064390 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f100effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064ca0 0x7f1010115390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f100effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064ca0 0x7f1010115390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50706/0 (socket says 172.21.15.110:50706) 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.639+0000 7f100effd640 1 -- 172.21.15.110:0/1617984016 learned_addr learned my addr 172.21.15.110:0/1617984016 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f100f7fe640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010064390 0x7f1010114e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f100effd640 1 -- 172.21.15.110:0/1617984016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010064390 msgr2=0x7f1010114e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f100effd640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010064390 0x7f1010114e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f100effd640 1 -- 172.21.15.110:0/1617984016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000005040 con 0x7f1010064ca0 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f100effd640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064ca0 0x7f1010115390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f100400c810 tx=0x7f10040043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f0ff7fff640 1 -- 172.21.15.110:0/1617984016 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f100400a820 con 0x7f1010064ca0 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f0ff7fff640 1 -- 172.21.15.110:0/1617984016 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f100400ae50 con 0x7f1010064ca0 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101010f2c0 con 0x7f1010064ca0 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f0ff7fff640 1 -- 172.21.15.110:0/1617984016 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10040115c0 con 0x7f1010064ca0 2024-04-25T16:01:00.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.640+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101010f7e0 con 0x7f1010064ca0 2024-04-25T16:01:00.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.641+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10100fb760 con 0x7f1010064ca0 2024-04-25T16:01:00.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.642+0000 7f0ff7fff640 1 -- 172.21.15.110:0/1617984016 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f100400a980 con 0x7f1010064ca0 2024-04-25T16:01:00.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.642+0000 7f0ff7fff640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0ff8076b00 0x7f0ff8078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.642+0000 7f0ff7fff640 1 -- 172.21.15.110:0/1617984016 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f10040989d0 con 0x7f1010064ca0 2024-04-25T16:01:00.644 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.642+0000 7f100f7fe640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0ff8076b00 0x7f0ff8078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.643+0000 7f100f7fe640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0ff8076b00 0x7f0ff8078fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1000000c00 tx=0x7f1000009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:00.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.647+0000 7f0ff7fff640 1 -- 172.21.15.110:0/1617984016 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1004061400 con 0x7f1010064ca0 2024-04-25T16:01:00.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.862+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f101010b5e0 con 0x7f1010064ca0 2024-04-25T16:01:00.863 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.863+0000 7f0ff7fff640 1 -- 172.21.15.110:0/1617984016 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v35) v1 ==== 107+0+4212 (secure 0 0 0) 0x7f1004060da0 con 0x7f1010064ca0 2024-04-25T16:01:00.864 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:00.864 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13},{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:07.402469+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:00.864 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 25 2024-04-25T16:01:00.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.864+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0ff8076b00 msgr2=0x7f0ff8078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.864+0000 7f1015a6e640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0ff8076b00 0x7f0ff8078fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1000000c00 tx=0x7f1000009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064ca0 msgr2=0x7f1010115390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064ca0 0x7f1010115390 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f100400c810 tx=0x7f10040043c0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 shutdown_connections 2024-04-25T16:01:00.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f0ff8076b00 0x7f0ff8078fb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1010064ca0 0x7f1010115390 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.866 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 --2- 172.21.15.110:0/1617984016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1010064390 0x7f1010114e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 >> 172.21.15.110:0/1617984016 conn(0x7f1010071fd0 msgr2=0x7f1010104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:00.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 shutdown_connections 2024-04-25T16:01:00.867 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.865+0000 7f1015a6e640 1 -- 172.21.15.110:0/1617984016 wait complete. 2024-04-25T16:01:00.877 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 -- 172.21.15.110:0/4220802898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 msgr2=0x7f1774102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 --2- 172.21.15.110:0/4220802898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 0x7f1774102620 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f17680048d0 tx=0x7f1768009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 -- 172.21.15.110:0/4220802898 shutdown_connections 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 --2- 172.21.15.110:0/4220802898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1774102b60 0x7f17740fa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 --2- 172.21.15.110:0/4220802898 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 0x7f1774102620 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 -- 172.21.15.110:0/4220802898 >> 172.21.15.110:0/4220802898 conn(0x7f1774070900 msgr2=0x7f1774070d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 -- 172.21.15.110:0/4220802898 shutdown_connections 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 -- 172.21.15.110:0/4220802898 wait complete. 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 Processor -- start 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.971+0000 7f177aeae640 1 -- start start 2024-04-25T16:01:00.974 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f177aeae640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 0x7f17740fba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f177aeae640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1774102b60 0x7f17740fbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f177aeae640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17740fd9e0 con 0x7f1774102250 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f177aeae640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17740fc500 con 0x7f1774102b60 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f1778c4a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 0x7f17740fba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f1778c4a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 0x7f17740fba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50720/0 (socket says 172.21.15.110:50720) 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f1778c4a640 1 -- 172.21.15.110:0/2858172180 learned_addr learned my addr 172.21.15.110:0/2858172180 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f1773fff640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1774102b60 0x7f17740fbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f1778c4a640 1 -- 172.21.15.110:0/2858172180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1774102b60 msgr2=0x7f17740fbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f1778c4a640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1774102b60 0x7f17740fbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f1778c4a640 1 -- 172.21.15.110:0/2858172180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1768005040 con 0x7f1774102250 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f1778c4a640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 0x7f17740fba80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f17680041d0 tx=0x7f1768009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:00.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f17717fa640 1 -- 172.21.15.110:0/2858172180 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1768030870 con 0x7f1774102250 2024-04-25T16:01:00.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17740fc780 con 0x7f1774102250 2024-04-25T16:01:00.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f17717fa640 1 -- 172.21.15.110:0/2858172180 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17680092e0 con 0x7f1774102250 2024-04-25T16:01:00.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f17717fa640 1 -- 172.21.15.110:0/2858172180 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1768043400 con 0x7f1774102250 2024-04-25T16:01:00.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.972+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17741194b0 con 0x7f1774102250 2024-04-25T16:01:00.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.973+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17740654f0 con 0x7f1774102250 2024-04-25T16:01:00.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.974+0000 7f17717fa640 1 -- 172.21.15.110:0/2858172180 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f176803b420 con 0x7f1774102250 2024-04-25T16:01:00.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.974+0000 7f17717fa640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1764076b00 0x7f1764078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:00.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.974+0000 7f17717fa640 1 -- 172.21.15.110:0/2858172180 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f17680bb1b0 con 0x7f1774102250 2024-04-25T16:01:00.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.974+0000 7f1773fff640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1764076b00 0x7f1764078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:00.977 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.975+0000 7f1773fff640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1764076b00 0x7f1764078fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1760007fd0 tx=0x7f1760009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:00.980 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:00.979+0000 7f17717fa640 1 -- 172.21.15.110:0/2858172180 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f1768083c60 con 0x7f1774102250 2024-04-25T16:01:01.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.192+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f17740fcf60 con 0x7f1774102250 2024-04-25T16:01:01.193 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.192+0000 7f17717fa640 1 -- 172.21.15.110:0/2858172180 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v35) v1 ==== 107+0+4223 (secure 0 0 0) 0x7f1768083600 con 0x7f1774102250 2024-04-25T16:01:01.193 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:01.194 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:07.408098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":0,"incarnation":26,"state":"up:replay","state_seq":2,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:01.194 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 26 2024-04-25T16:01:01.195 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1764076b00 msgr2=0x7f1764078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1764076b00 0x7f1764078fb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1760007fd0 tx=0x7f1760009e90 comp rx=0 tx=0).stop 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 msgr2=0x7f17740fba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 0x7f17740fba80 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f17680041d0 tx=0x7f1768009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 shutdown_connections 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f1764076b00 0x7f1764078fb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1774102b60 0x7f17740fbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 --2- 172.21.15.110:0/2858172180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1774102250 0x7f17740fba80 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 >> 172.21.15.110:0/2858172180 conn(0x7f1774070900 msgr2=0x7f1774104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 shutdown_connections 2024-04-25T16:01:01.196 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.194+0000 7f177aeae640 1 -- 172.21.15.110:0/2858172180 wait complete. 2024-04-25T16:01:01.205 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:00 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3282642504' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-25T16:01:01.205 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:00 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1809221925' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-25T16:01:01.205 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:00 smithi110 ceph-mon[90991]: pgmap v314: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:01.205 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:00 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1617984016' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-25T16:01:01.206 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-25T16:01:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:00 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3282642504' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-25T16:01:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:00 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1809221925' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-25T16:01:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:00 smithi193 ceph-mon[80071]: pgmap v314: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:00 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1617984016' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 -- 172.21.15.110:0/1911104449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5828102b80 msgr2=0x7f58280fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 --2- 172.21.15.110:0/1911104449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5828102b80 0x7f58280fc210 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f58240048d0 tx=0x7f5824009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 -- 172.21.15.110:0/1911104449 shutdown_connections 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 --2- 172.21.15.110:0/1911104449 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5828102b80 0x7f58280fc210 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 --2- 172.21.15.110:0/1911104449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58281021e0 0x7f58281025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 -- 172.21.15.110:0/1911104449 >> 172.21.15.110:0/1911104449 conn(0x7f58280f7db0 msgr2=0x7f58280fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 -- 172.21.15.110:0/1911104449 shutdown_connections 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 -- 172.21.15.110:0/1911104449 wait complete. 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.300+0000 7f582f307640 1 Processor -- start 2024-04-25T16:01:01.303 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582f307640 1 -- start start 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582f307640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58281021e0 0x7f5828112ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582f307640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5828102b80 0x7f58281131e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582f307640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828113870 con 0x7f58281021e0 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582f307640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582810ce30 con 0x7f5828102b80 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582d0a3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58281021e0 0x7f5828112ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582d0a3640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58281021e0 0x7f5828112ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50734/0 (socket says 172.21.15.110:50734) 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582d0a3640 1 -- 172.21.15.110:0/1338629437 learned_addr learned my addr 172.21.15.110:0/1338629437 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582c8a2640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5828102b80 0x7f58281131e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582d0a3640 1 -- 172.21.15.110:0/1338629437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5828102b80 msgr2=0x7f58281131e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582d0a3640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5828102b80 0x7f58281131e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582d0a3640 1 -- 172.21.15.110:0/1338629437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5824005040 con 0x7f58281021e0 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582d0a3640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58281021e0 0x7f5828112ca0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f581800b790 tx=0x7f58180091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:01.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f581dffb640 1 -- 172.21.15.110:0/1338629437 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581800c700 con 0x7f58281021e0 2024-04-25T16:01:01.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582810d110 con 0x7f58281021e0 2024-04-25T16:01:01.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f581dffb640 1 -- 172.21.15.110:0/1338629437 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f581800cd30 con 0x7f58281021e0 2024-04-25T16:01:01.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.301+0000 7f581dffb640 1 -- 172.21.15.110:0/1338629437 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5818013370 con 0x7f58281021e0 2024-04-25T16:01:01.305 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.302+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582810d600 con 0x7f58281021e0 2024-04-25T16:01:01.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.303+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58280fd950 con 0x7f58281021e0 2024-04-25T16:01:01.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.303+0000 7f581dffb640 1 -- 172.21.15.110:0/1338629437 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f5818012070 con 0x7f58281021e0 2024-04-25T16:01:01.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.304+0000 7f581dffb640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5814076b00 0x7f5814078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.304+0000 7f581dffb640 1 -- 172.21.15.110:0/1338629437 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f58180980c0 con 0x7f58281021e0 2024-04-25T16:01:01.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.304+0000 7f582c8a2640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5814076b00 0x7f5814078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.306 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.304+0000 7f582c8a2640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5814076b00 0x7f5814078fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f58280f94c0 tx=0x7f5824009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:01.309 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.308+0000 7f581dffb640 1 -- 172.21.15.110:0/1338629437 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f5818060af0 con 0x7f58281021e0 2024-04-25T16:01:01.523 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.522+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f5828064350 con 0x7f58281021e0 2024-04-25T16:01:01.524 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.523+0000 7f581dffb640 1 -- 172.21.15.110:0/1338629437 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v35) v1 ==== 107+0+5078 (secure 0 0 0) 0x7f5818060490 con 0x7f58281021e0 2024-04-25T16:01:01.524 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:01.524 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:16.656656+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":0,"incarnation":26,"state":"up:reconnect","state_seq":163,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:01.524 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 27 2024-04-25T16:01:01.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5814076b00 msgr2=0x7f5814078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5814076b00 0x7f5814078fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f58280f94c0 tx=0x7f5824009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58281021e0 msgr2=0x7f5828112ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58281021e0 0x7f5828112ca0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f581800b790 tx=0x7f58180091d0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 shutdown_connections 2024-04-25T16:01:01.526 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f5814076b00 0x7f5814078fb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5828102b80 0x7f58281131e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 --2- 172.21.15.110:0/1338629437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58281021e0 0x7f5828112ca0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 >> 172.21.15.110:0/1338629437 conn(0x7f58280f7db0 msgr2=0x7f58280f9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:01.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 shutdown_connections 2024-04-25T16:01:01.527 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.525+0000 7f582f307640 1 -- 172.21.15.110:0/1338629437 wait complete. 2024-04-25T16:01:01.538 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-25T16:01:01.634 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 -- 172.21.15.110:0/187252345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 msgr2=0x7fef44101440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 --2- 172.21.15.110:0/187252345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 0x7fef44101440 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef340048d0 tx=0x7fef34009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 -- 172.21.15.110:0/187252345 shutdown_connections 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 --2- 172.21.15.110:0/187252345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef44101a10 0x7fef440fb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 --2- 172.21.15.110:0/187252345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 0x7fef44101440 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 -- 172.21.15.110:0/187252345 >> 172.21.15.110:0/187252345 conn(0x7fef440f6be0 msgr2=0x7fef440f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 -- 172.21.15.110:0/187252345 shutdown_connections 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 -- 172.21.15.110:0/187252345 wait complete. 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 Processor -- start 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 -- start start 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.632+0000 7fef4bb9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 0x7fef44101ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef4bb9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef44101a10 0x7fef44102430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef4bb9d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef44103e50 con 0x7fef44101070 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef4bb9d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef44102970 con 0x7fef44101a10 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef49939640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 0x7fef44101ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef49939640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 0x7fef44101ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50764/0 (socket says 172.21.15.110:50764) 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef49939640 1 -- 172.21.15.110:0/2022647389 learned_addr learned my addr 172.21.15.110:0/2022647389 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef49138640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef44101a10 0x7fef44102430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef49939640 1 -- 172.21.15.110:0/2022647389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef44101a10 msgr2=0x7fef44102430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef49939640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef44101a10 0x7fef44102430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef49939640 1 -- 172.21.15.110:0/2022647389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef34005040 con 0x7fef44101070 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef49939640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 0x7fef44101ef0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fef34004930 tx=0x7fef34009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef3a7fc640 1 -- 172.21.15.110:0/2022647389 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef340094c0 con 0x7fef44101070 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef3a7fc640 1 -- 172.21.15.110:0/2022647389 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fef34032e50 con 0x7fef44101070 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef44102bf0 con 0x7fef44101070 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef3a7fc640 1 -- 172.21.15.110:0/2022647389 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fef340433b0 con 0x7fef44101070 2024-04-25T16:01:01.636 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.633+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef441193d0 con 0x7fef44101070 2024-04-25T16:01:01.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.634+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef440fc7e0 con 0x7fef44101070 2024-04-25T16:01:01.637 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.635+0000 7fef3a7fc640 1 -- 172.21.15.110:0/2022647389 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fef3402f070 con 0x7fef44101070 2024-04-25T16:01:01.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.635+0000 7fef3a7fc640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef30076b00 0x7fef30078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.635+0000 7fef49138640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef30076b00 0x7fef30078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.636+0000 7fef3a7fc640 1 -- 172.21.15.110:0/2022647389 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fef340ba460 con 0x7fef44101070 2024-04-25T16:01:01.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.636+0000 7fef49138640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef30076b00 0x7fef30078fb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fef44103620 tx=0x7fef40009040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:01.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.640+0000 7fef3a7fc640 1 -- 172.21.15.110:0/2022647389 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fef34082e90 con 0x7fef44101070 2024-04-25T16:01:01.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.856+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fef44103400 con 0x7fef44101070 2024-04-25T16:01:01.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.857+0000 7fef3a7fc640 1 -- 172.21.15.110:0/2022647389 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v35) v1 ==== 107+0+5075 (secure 0 0 0) 0x7fef34082830 con 0x7fef44101070 2024-04-25T16:01:01.858 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:01.859 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:16.832894+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":0,"incarnation":26,"state":"up:rejoin","state_seq":164,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:01.859 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 28 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef30076b00 msgr2=0x7fef30078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef30076b00 0x7fef30078fb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fef44103620 tx=0x7fef40009040 comp rx=0 tx=0).stop 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 msgr2=0x7fef44101ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 0x7fef44101ef0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fef34004930 tx=0x7fef34009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 shutdown_connections 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fef30076b00 0x7fef30078fb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef44101a10 0x7fef44102430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 --2- 172.21.15.110:0/2022647389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fef44101070 0x7fef44101ef0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 >> 172.21.15.110:0/2022647389 conn(0x7fef440f6be0 msgr2=0x7fef44065780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 shutdown_connections 2024-04-25T16:01:01.861 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.859+0000 7fef4bb9d640 1 -- 172.21.15.110:0/2022647389 wait complete. 2024-04-25T16:01:01.871 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-25T16:01:01.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.967+0000 7f498166e640 1 -- 172.21.15.110:0/301510065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c102b50 msgr2=0x7f497c0fc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.967+0000 7f498166e640 1 --2- 172.21.15.110:0/301510065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c102b50 0x7f497c0fc1e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f49700048d0 tx=0x7f4970009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.967+0000 7f498166e640 1 -- 172.21.15.110:0/301510065 shutdown_connections 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.967+0000 7f498166e640 1 --2- 172.21.15.110:0/301510065 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c102b50 0x7f497c0fc1e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.967+0000 7f498166e640 1 --2- 172.21.15.110:0/301510065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f497c1021b0 0x7f497c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.967+0000 7f498166e640 1 -- 172.21.15.110:0/301510065 >> 172.21.15.110:0/301510065 conn(0x7f497c0f7da0 msgr2=0x7f497c0fa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.967+0000 7f498166e640 1 -- 172.21.15.110:0/301510065 shutdown_connections 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.967+0000 7f498166e640 1 -- 172.21.15.110:0/301510065 wait complete. 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f498166e640 1 Processor -- start 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f498166e640 1 -- start start 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f498166e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c1021b0 0x7f497c112cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f498166e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f497c102b50 0x7f497c113230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f498166e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c113830 con 0x7f497c1021b0 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f498166e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497c10ce80 con 0x7f497c102b50 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f497affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c1021b0 0x7f497c112cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f497affd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c1021b0 0x7f497c112cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50774/0 (socket says 172.21.15.110:50774) 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.968+0000 7f497affd640 1 -- 172.21.15.110:0/640293233 learned_addr learned my addr 172.21.15.110:0/640293233 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f497a7fc640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f497c102b50 0x7f497c113230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f497affd640 1 -- 172.21.15.110:0/640293233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f497c102b50 msgr2=0x7f497c113230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f497affd640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f497c102b50 0x7f497c113230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f497affd640 1 -- 172.21.15.110:0/640293233 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970005040 con 0x7f497c1021b0 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f497a7fc640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f497c102b50 0x7f497c113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f497affd640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c1021b0 0x7f497c112cf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4974002a50 tx=0x7f4974004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f49637fe640 1 -- 172.21.15.110:0/640293233 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4974007c30 con 0x7f497c1021b0 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f49637fe640 1 -- 172.21.15.110:0/640293233 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f497400f040 con 0x7f497c1021b0 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497c10d160 con 0x7f497c1021b0 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f49637fe640 1 -- 172.21.15.110:0/640293233 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f497400db50 con 0x7f497c1021b0 2024-04-25T16:01:01.972 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.969+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497c10d680 con 0x7f497c1021b0 2024-04-25T16:01:01.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.970+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497c0fd920 con 0x7f497c1021b0 2024-04-25T16:01:01.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.971+0000 7f49637fe640 1 -- 172.21.15.110:0/640293233 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f497401c400 con 0x7f497c1021b0 2024-04-25T16:01:01.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.971+0000 7f49637fe640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4964076b00 0x7f4964078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:01.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.971+0000 7f49637fe640 1 -- 172.21.15.110:0/640293233 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f4974097d40 con 0x7f497c1021b0 2024-04-25T16:01:01.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.971+0000 7f497a7fc640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4964076b00 0x7f4964078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:01.973 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.971+0000 7f497a7fc640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4964076b00 0x7f4964078fb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4970000c00 tx=0x7f4970009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:01.976 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:01.976+0000 7f49637fe640 1 -- 172.21.15.110:0/640293233 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f4974060770 con 0x7f497c1021b0 2024-04-25T16:01:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:01 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2858172180' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-25T16:01:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:01 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1338629437' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-25T16:01:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:01:02.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:01 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2022647389' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-25T16:01:02.188 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.187+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f497c063f00 con 0x7f497c1021b0 2024-04-25T16:01:02.189 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.188+0000 7f49637fe640 1 -- 172.21.15.110:0/640293233 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v35) v1 ==== 107+0+5084 (secure 0 0 0) 0x7f4974060110 con 0x7f497c1021b0 2024-04-25T16:01:02.189 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:02.189 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:18.836278+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":82,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24265},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24265":{"gid":24265,"name":"cephfs.smithi193.jtyzhl","rank":0,"incarnation":26,"state":"up:active","state_seq":165,"addr":"172.21.15.193:6827/1539252929","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1539252929},{"type":"v1","addr":"172.21.15.193:6827","nonce":1539252929}]},"join_fscid":1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24265,"qdb_cluster":[24265]},"id":1}]} 2024-04-25T16:01:02.189 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 29 2024-04-25T16:01:02.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4964076b00 msgr2=0x7f4964078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4964076b00 0x7f4964078fb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4970000c00 tx=0x7f4970009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c1021b0 msgr2=0x7f497c112cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.191 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c1021b0 0x7f497c112cf0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4974002a50 tx=0x7f4974004600 comp rx=0 tx=0).stop 2024-04-25T16:01:02.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 shutdown_connections 2024-04-25T16:01:02.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f4964076b00 0x7f4964078fb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f497c102b50 0x7f497c113230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 --2- 172.21.15.110:0/640293233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f497c1021b0 0x7f497c112cf0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 >> 172.21.15.110:0/640293233 conn(0x7f497c0f7da0 msgr2=0x7f497c0f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:02.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 shutdown_connections 2024-04-25T16:01:02.192 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.190+0000 7f498166e640 1 -- 172.21.15.110:0/640293233 wait complete. 2024-04-25T16:01:02.202 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-25T16:01:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:01 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2858172180' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-25T16:01:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:01 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1338629437' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-25T16:01:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:01:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:01 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2022647389' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-25T16:01:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.295+0000 7f211562e640 1 -- 172.21.15.110:0/3658696879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 msgr2=0x7f21100fb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.295+0000 7f211562e640 1 --2- 172.21.15.110:0/3658696879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 0x7f21100fb0d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f20f80048d0 tx=0x7f20f8009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.295+0000 7f211562e640 1 -- 172.21.15.110:0/3658696879 shutdown_connections 2024-04-25T16:01:02.298 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.295+0000 7f211562e640 1 --2- 172.21.15.110:0/3658696879 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 0x7f21100fb0d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.295+0000 7f211562e640 1 --2- 172.21.15.110:0/3658696879 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21101010a0 0x7f2110101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.295+0000 7f211562e640 1 -- 172.21.15.110:0/3658696879 >> 172.21.15.110:0/3658696879 conn(0x7f21100f6c10 msgr2=0x7f21100f9020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.295+0000 7f211562e640 1 -- 172.21.15.110:0/3658696879 shutdown_connections 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.295+0000 7f211562e640 1 -- 172.21.15.110:0/3658696879 wait complete. 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f211562e640 1 Processor -- start 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f211562e640 1 -- start start 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f211562e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21101010a0 0x7f211010de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f211562e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 0x7f211010e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f211562e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2110066720 con 0x7f2110101a40 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f211562e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211010e8d0 con 0x7f21101010a0 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f2107fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 0x7f211010e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:02.299 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f2107fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 0x7f211010e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50804/0 (socket says 172.21.15.110:50804) 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f2107fff640 1 -- 172.21.15.110:0/914147702 learned_addr learned my addr 172.21.15.110:0/914147702 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f2107fff640 1 -- 172.21.15.110:0/914147702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21101010a0 msgr2=0x7f211010de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f2107fff640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21101010a0 0x7f211010de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.296+0000 7f2107fff640 1 -- 172.21.15.110:0/914147702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8005040 con 0x7f2110101a40 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.297+0000 7f2107fff640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 0x7f211010e390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f20f8004930 tx=0x7f20f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.297+0000 7f21077fe640 1 -- 172.21.15.110:0/914147702 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20f80094c0 con 0x7f2110101a40 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.297+0000 7f21077fe640 1 -- 172.21.15.110:0/914147702 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20f8032e50 con 0x7f2110101a40 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.297+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211010eb50 con 0x7f2110101a40 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.297+0000 7f21077fe640 1 -- 172.21.15.110:0/914147702 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20f80433b0 con 0x7f2110101a40 2024-04-25T16:01:02.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.297+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21101170b0 con 0x7f2110101a40 2024-04-25T16:01:02.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.298+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21100fc810 con 0x7f2110101a40 2024-04-25T16:01:02.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.298+0000 7f21077fe640 1 -- 172.21.15.110:0/914147702 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f20f802f070 con 0x7f2110101a40 2024-04-25T16:01:02.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.299+0000 7f21077fe640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f20fc076b00 0x7f20fc078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.301 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.299+0000 7f21077fe640 1 -- 172.21.15.110:0/914147702 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f20f80ba4a0 con 0x7f2110101a40 2024-04-25T16:01:02.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.299+0000 7f210effd640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f20fc076b00 0x7f20fc078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:02.302 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.299+0000 7f210effd640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f20fc076b00 0x7f20fc078fb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2100004a50 tx=0x7f2100007cb0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:02.304 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.304+0000 7f21077fe640 1 -- 172.21.15.110:0/914147702 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f20f8082ed0 con 0x7f2110101a40 2024-04-25T16:01:02.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.517+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f2110064320 con 0x7f2110101a40 2024-04-25T16:01:02.518 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.517+0000 7f21077fe640 1 -- 172.21.15.110:0/914147702 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v35) v1 ==== 107+0+4280 (secure 0 0 0) 0x7f20f8082870 con 0x7f2110101a40 2024-04-25T16:01:02.519 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:02.519 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:32.546172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:02.519 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 30 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.519+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f20fc076b00 msgr2=0x7f20fc078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.519+0000 7f211562e640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f20fc076b00 0x7f20fc078fb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2100004a50 tx=0x7f2100007cb0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.519+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 msgr2=0x7f211010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.519+0000 7f211562e640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 0x7f211010e390 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f20f8004930 tx=0x7f20f8009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.520+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 shutdown_connections 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.520+0000 7f211562e640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f20fc076b00 0x7f20fc078fb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.520+0000 7f211562e640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2110101a40 0x7f211010e390 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.520+0000 7f211562e640 1 --2- 172.21.15.110:0/914147702 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21101010a0 0x7f211010de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.520+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 >> 172.21.15.110:0/914147702 conn(0x7f21100f6c10 msgr2=0x7f21100f7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:02.521 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.520+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 shutdown_connections 2024-04-25T16:01:02.522 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.520+0000 7f211562e640 1 -- 172.21.15.110:0/914147702 wait complete. 2024-04-25T16:01:02.531 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 -- 172.21.15.110:0/1474984926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 msgr2=0x7fcd800fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 --2- 172.21.15.110:0/1474984926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 0x7fcd800fb0a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd700048d0 tx=0x7fcd70009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 -- 172.21.15.110:0/1474984926 shutdown_connections 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 --2- 172.21.15.110:0/1474984926 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 0x7fcd800fb0a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 --2- 172.21.15.110:0/1474984926 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd80101070 0x7fcd80101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 -- 172.21.15.110:0/1474984926 >> 172.21.15.110:0/1474984926 conn(0x7fcd800f6be0 msgr2=0x7fcd800f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 -- 172.21.15.110:0/1474984926 shutdown_connections 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 -- 172.21.15.110:0/1474984926 wait complete. 2024-04-25T16:01:02.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.626+0000 7fcd8606e640 1 Processor -- start 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd8606e640 1 -- start start 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd8606e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd80101070 0x7fcd8010de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd8606e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 0x7fcd8010e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd8606e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd80066700 con 0x7fcd80101a10 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd8606e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8010e900 con 0x7fcd80101070 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd7effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 0x7fcd8010e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd7effd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 0x7fcd8010e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50818/0 (socket says 172.21.15.110:50818) 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd7effd640 1 -- 172.21.15.110:0/4099476306 learned_addr learned my addr 172.21.15.110:0/4099476306 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd7f7fe640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd80101070 0x7fcd8010de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd7effd640 1 -- 172.21.15.110:0/4099476306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd80101070 msgr2=0x7fcd8010de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd7effd640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd80101070 0x7fcd8010de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd7effd640 1 -- 172.21.15.110:0/4099476306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70005040 con 0x7fcd80101a10 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd7effd640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 0x7fcd8010e3c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcd70004930 tx=0x7fcd70009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:02.630 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.627+0000 7fcd67fff640 1 -- 172.21.15.110:0/4099476306 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd700094c0 con 0x7fcd80101a10 2024-04-25T16:01:02.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.628+0000 7fcd67fff640 1 -- 172.21.15.110:0/4099476306 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcd70032e50 con 0x7fcd80101a10 2024-04-25T16:01:02.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.628+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8010eb80 con 0x7fcd80101a10 2024-04-25T16:01:02.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.628+0000 7fcd67fff640 1 -- 172.21.15.110:0/4099476306 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd700433b0 con 0x7fcd80101a10 2024-04-25T16:01:02.631 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.628+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd801170e0 con 0x7fcd80101a10 2024-04-25T16:01:02.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.629+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd800fc7e0 con 0x7fcd80101a10 2024-04-25T16:01:02.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.629+0000 7fcd67fff640 1 -- 172.21.15.110:0/4099476306 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fcd7002f070 con 0x7fcd80101a10 2024-04-25T16:01:02.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.630+0000 7fcd67fff640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcd68076b00 0x7fcd68078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.630+0000 7fcd67fff640 1 -- 172.21.15.110:0/4099476306 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fcd700ba250 con 0x7fcd80101a10 2024-04-25T16:01:02.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.630+0000 7fcd7f7fe640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcd68076b00 0x7fcd68078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:02.632 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.630+0000 7fcd7f7fe640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcd68076b00 0x7fcd68078fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcd74004ed0 tx=0x7fcd7400b040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:02.635 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.634+0000 7fcd67fff640 1 -- 172.21.15.110:0/4099476306 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fcd70082d30 con 0x7fcd80101a10 2024-04-25T16:01:02.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.852+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7fcd80108230 con 0x7fcd80101a10 2024-04-25T16:01:02.853 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.853+0000 7fcd67fff640 1 -- 172.21.15.110:0/4099476306 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v35) v1 ==== 107+0+4359 (secure 0 0 0) 0x7fcd700826d0 con 0x7fcd80101a10 2024-04-25T16:01:02.854 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:02.854 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:32.551820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:02.854 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 31 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.854+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcd68076b00 msgr2=0x7fcd68078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.854+0000 7fcd8606e640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcd68076b00 0x7fcd68078fb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcd74004ed0 tx=0x7fcd7400b040 comp rx=0 tx=0).stop 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 msgr2=0x7fcd8010e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 0x7fcd8010e3c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fcd70004930 tx=0x7fcd70009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 shutdown_connections 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fcd68076b00 0x7fcd68078fb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcd80101a10 0x7fcd8010e3c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 --2- 172.21.15.110:0/4099476306 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd80101070 0x7fcd8010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.856 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 >> 172.21.15.110:0/4099476306 conn(0x7fcd800f6be0 msgr2=0x7fcd800f7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:02.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 shutdown_connections 2024-04-25T16:01:02.857 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.855+0000 7fcd8606e640 1 -- 172.21.15.110:0/4099476306 wait complete. 2024-04-25T16:01:02.866 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-25T16:01:02.964 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 -- 172.21.15.110:0/4009813314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 msgr2=0x7f34c00fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 --2- 172.21.15.110:0/4009813314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 0x7f34c00fb0a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0048d0 tx=0x7f34bc009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 -- 172.21.15.110:0/4009813314 shutdown_connections 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 --2- 172.21.15.110:0/4009813314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 0x7f34c00fb0a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 --2- 172.21.15.110:0/4009813314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34c0101070 0x7f34c0101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 -- 172.21.15.110:0/4009813314 >> 172.21.15.110:0/4009813314 conn(0x7f34c00f6be0 msgr2=0x7f34c00f8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 -- 172.21.15.110:0/4009813314 shutdown_connections 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 -- 172.21.15.110:0/4009813314 wait complete. 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 Processor -- start 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 -- start start 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34c0101070 0x7f34c010de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.965 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 0x7f34c010e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.962+0000 7f34c7e14640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c0066720 con 0x7f34c0101a10 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c7e14640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34c010e910 con 0x7f34c0101070 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c53af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 0x7f34c010e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c53af640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 0x7f34c010e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50844/0 (socket says 172.21.15.110:50844) 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c53af640 1 -- 172.21.15.110:0/1670026501 learned_addr learned my addr 172.21.15.110:0/1670026501 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c5bb0640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34c0101070 0x7f34c010de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c53af640 1 -- 172.21.15.110:0/1670026501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34c0101070 msgr2=0x7f34c010de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c53af640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34c0101070 0x7f34c010de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c53af640 1 -- 172.21.15.110:0/1670026501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34bc005040 con 0x7f34c0101a10 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c53af640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 0x7f34c010e3a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f34bc004930 tx=0x7f34bc009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34b67fc640 1 -- 172.21.15.110:0/1670026501 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34bc0094c0 con 0x7f34c0101a10 2024-04-25T16:01:02.966 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34b67fc640 1 -- 172.21.15.110:0/1670026501 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34bc032e50 con 0x7f34c0101a10 2024-04-25T16:01:02.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34c010eb90 con 0x7f34c0101a10 2024-04-25T16:01:02.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34b67fc640 1 -- 172.21.15.110:0/1670026501 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34bc0433b0 con 0x7f34c0101a10 2024-04-25T16:01:02.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.963+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34c01170c0 con 0x7f34c0101a10 2024-04-25T16:01:02.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.964+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34c00fc7e0 con 0x7f34c0101a10 2024-04-25T16:01:02.967 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.965+0000 7f34b67fc640 1 -- 172.21.15.110:0/1670026501 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f34bc02f070 con 0x7f34c0101a10 2024-04-25T16:01:02.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.965+0000 7f34b67fc640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f34ac076b00 0x7f34ac078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:02.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.965+0000 7f34b67fc640 1 -- 172.21.15.110:0/1670026501 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f34bc0ba300 con 0x7f34c0101a10 2024-04-25T16:01:02.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.965+0000 7f34c5bb0640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f34ac076b00 0x7f34ac078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:02.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.966+0000 7f34c5bb0640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f34ac076b00 0x7f34ac078fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f34b0007fd0 tx=0x7f34b0009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:02.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:02.970+0000 7f34b67fc640 1 -- 172.21.15.110:0/1670026501 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f34bc082d30 con 0x7f34c0101a10 2024-04-25T16:01:03.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.182+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f34c0064350 con 0x7f34c0101a10 2024-04-25T16:01:03.184 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.183+0000 7f34b67fc640 1 -- 172.21.15.110:0/1670026501 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v35) v1 ==== 107+0+5209 (secure 0 0 0) 0x7f34bc0826d0 con 0x7f34c0101a10 2024-04-25T16:01:03.184 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:03.184 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44359,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/3103986931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3103986931},{"type":"v1","addr":"172.21.15.193:6827","nonce":3103986931}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:32.551820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":0,"incarnation":31,"state":"up:replay","state_seq":1,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:03.184 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 32 2024-04-25T16:01:03.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f34ac076b00 msgr2=0x7f34ac078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f34ac076b00 0x7f34ac078fb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f34b0007fd0 tx=0x7f34b0009e90 comp rx=0 tx=0).stop 2024-04-25T16:01:03.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 msgr2=0x7f34c010e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 0x7f34c010e3a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f34bc004930 tx=0x7f34bc009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 shutdown_connections 2024-04-25T16:01:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f34ac076b00 0x7f34ac078fb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34c0101a10 0x7f34c010e3a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 --2- 172.21.15.110:0/1670026501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34c0101070 0x7f34c010de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 >> 172.21.15.110:0/1670026501 conn(0x7f34c00f6be0 msgr2=0x7f34c00f7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 shutdown_connections 2024-04-25T16:01:03.187 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.185+0000 7f34c7e14640 1 -- 172.21.15.110:0/1670026501 wait complete. 2024-04-25T16:01:03.195 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:02 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/640293233' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-25T16:01:03.196 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:02 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/914147702' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-25T16:01:03.196 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:02 smithi110 ceph-mon[90991]: pgmap v315: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:03.196 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:02 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/4099476306' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-25T16:01:03.197 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-25T16:01:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:02 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/640293233' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-25T16:01:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:02 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/914147702' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-25T16:01:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:02 smithi193 ceph-mon[80071]: pgmap v315: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:02 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/4099476306' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-25T16:01:03.293 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/1352339501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101a10 msgr2=0x7fc4c80fb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 --2- 172.21.15.110:0/1352339501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101a10 0x7fc4c80fb0a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0048d0 tx=0x7fc4bc009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/1352339501 shutdown_connections 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 --2- 172.21.15.110:0/1352339501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101a10 0x7fc4c80fb0a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 --2- 172.21.15.110:0/1352339501 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c8101070 0x7fc4c8101440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/1352339501 >> 172.21.15.110:0/1352339501 conn(0x7fc4c80f6c60 msgr2=0x7fc4c80f9070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/1352339501 shutdown_connections 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/1352339501 wait complete. 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 Processor -- start 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.291+0000 7fc4cfc6e640 1 -- start start 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cfc6e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101070 0x7fc4c810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cfc6e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c8101a10 0x7fc4c810e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cfc6e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c810e940 con 0x7fc4c8101070 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cfc6e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c810eab0 con 0x7fc4c8101a10 2024-04-25T16:01:03.294 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cda0a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101070 0x7fc4c810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cda0a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101070 0x7fc4c810dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50874/0 (socket says 172.21.15.110:50874) 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cda0a640 1 -- 172.21.15.110:0/421371340 learned_addr learned my addr 172.21.15.110:0/421371340 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cd209640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c8101a10 0x7fc4c810e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cda0a640 1 -- 172.21.15.110:0/421371340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c8101a10 msgr2=0x7fc4c810e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cda0a640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c8101a10 0x7fc4c810e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cda0a640 1 -- 172.21.15.110:0/421371340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4bc005040 con 0x7fc4c8101070 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cda0a640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101070 0x7fc4c810dec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000bbb0 tx=0x7fc4c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4c67fc640 1 -- 172.21.15.110:0/421371340 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4c000c9c0 con 0x7fc4c8101070 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c8116f30 con 0x7fc4c8101070 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4c67fc640 1 -- 172.21.15.110:0/421371340 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc4c00092e0 con 0x7fc4c8101070 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4c67fc640 1 -- 172.21.15.110:0/421371340 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4c0007640 con 0x7fc4c8101070 2024-04-25T16:01:03.295 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.292+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c8117260 con 0x7fc4c8101070 2024-04-25T16:01:03.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.293+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c80fc7e0 con 0x7fc4c8101070 2024-04-25T16:01:03.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.294+0000 7fc4c67fc640 1 -- 172.21.15.110:0/421371340 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fc4c0012070 con 0x7fc4c8101070 2024-04-25T16:01:03.296 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.294+0000 7fc4c67fc640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc4b0076b00 0x7fc4b0078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:03.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.294+0000 7fc4c67fc640 1 -- 172.21.15.110:0/421371340 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fc4c00984e0 con 0x7fc4c8101070 2024-04-25T16:01:03.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.294+0000 7fc4cd209640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc4b0076b00 0x7fc4b0078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:03.297 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.295+0000 7fc4cd209640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc4b0076b00 0x7fc4b0078fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8065fb0 tx=0x7fc4bc009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:03.300 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.299+0000 7fc4c67fc640 1 -- 172.21.15.110:0/421371340 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc4c0060f10 con 0x7fc4c8101070 2024-04-25T16:01:03.512 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.511+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7fc4c8064350 con 0x7fc4c8101070 2024-04-25T16:01:03.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.512+0000 7fc4c67fc640 1 -- 172.21.15.110:0/421371340 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v35) v1 ==== 107+0+5213 (secure 0 0 0) 0x7fc4c00608b0 con 0x7fc4c8101070 2024-04-25T16:01:03.513 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:03.513 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44359,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/3103986931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3103986931},{"type":"v1","addr":"172.21.15.193:6827","nonce":3103986931}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:42.115943+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":0,"incarnation":31,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:03.514 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 33 2024-04-25T16:01:03.515 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc4b0076b00 msgr2=0x7fc4b0078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc4b0076b00 0x7fc4b0078fb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8065fb0 tx=0x7fc4bc009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101070 msgr2=0x7fc4c810dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101070 0x7fc4c810dec0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc4c000bbb0 tx=0x7fc4c0009840 comp rx=0 tx=0).stop 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 shutdown_connections 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc4b0076b00 0x7fc4b0078fb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c8101a10 0x7fc4c810e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 --2- 172.21.15.110:0/421371340 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc4c8101070 0x7fc4c810dec0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 >> 172.21.15.110:0/421371340 conn(0x7fc4c80f6c60 msgr2=0x7fc4c80f7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 shutdown_connections 2024-04-25T16:01:03.516 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.514+0000 7fc4cfc6e640 1 -- 172.21.15.110:0/421371340 wait complete. 2024-04-25T16:01:03.526 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-25T16:01:03.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 -- 172.21.15.110:0/1804127545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc102b80 msgr2=0x7f72dc0fc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 --2- 172.21.15.110:0/1804127545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc102b80 0x7f72dc0fc210 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009c50 tx=0x7f72d0004d10 comp rx=0 tx=0).stop 2024-04-25T16:01:03.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 -- 172.21.15.110:0/1804127545 shutdown_connections 2024-04-25T16:01:03.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 --2- 172.21.15.110:0/1804127545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc102b80 0x7f72dc0fc210 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 --2- 172.21.15.110:0/1804127545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72dc1021e0 0x7f72dc1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 -- 172.21.15.110:0/1804127545 >> 172.21.15.110:0/1804127545 conn(0x7f72dc0f7db0 msgr2=0x7f72dc0fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:03.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 -- 172.21.15.110:0/1804127545 shutdown_connections 2024-04-25T16:01:03.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 -- 172.21.15.110:0/1804127545 wait complete. 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.620+0000 7f72e2261640 1 Processor -- start 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72e2261640 1 -- start start 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72e2261640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc1021e0 0x7f72dc112c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72e2261640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72dc102b80 0x7f72dc113180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72e2261640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc113780 con 0x7f72dc1021e0 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72e2261640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc10cdd0 con 0x7f72dc102b80 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc1021e0 0x7f72dc112c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72db7fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc1021e0 0x7f72dc112c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50896/0 (socket says 172.21.15.110:50896) 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72db7fe640 1 -- 172.21.15.110:0/209037030 learned_addr learned my addr 172.21.15.110:0/209037030 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72db7fe640 1 -- 172.21.15.110:0/209037030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72dc102b80 msgr2=0x7f72dc113180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72db7fe640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72dc102b80 0x7f72dc113180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72db7fe640 1 -- 172.21.15.110:0/209037030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d00098b0 con 0x7f72dc1021e0 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72db7fe640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc1021e0 0x7f72dc112c40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c790 tx=0x7f72cc0043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:03.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72c3fff640 1 -- 172.21.15.110:0/209037030 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72cc00a820 con 0x7f72dc1021e0 2024-04-25T16:01:03.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc10d0b0 con 0x7f72dc1021e0 2024-04-25T16:01:03.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72c3fff640 1 -- 172.21.15.110:0/209037030 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72cc00ae50 con 0x7f72dc1021e0 2024-04-25T16:01:03.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72c3fff640 1 -- 172.21.15.110:0/209037030 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72cc0115e0 con 0x7f72dc1021e0 2024-04-25T16:01:03.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.621+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc10d650 con 0x7f72dc1021e0 2024-04-25T16:01:03.625 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.623+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc0fd950 con 0x7f72dc1021e0 2024-04-25T16:01:03.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.623+0000 7f72c3fff640 1 -- 172.21.15.110:0/209037030 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f72cc012400 con 0x7f72dc1021e0 2024-04-25T16:01:03.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.623+0000 7f72c3fff640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72c4076b00 0x7f72c4078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:03.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.624+0000 7f72c3fff640 1 -- 172.21.15.110:0/209037030 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f72cc098620 con 0x7f72dc1021e0 2024-04-25T16:01:03.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.624+0000 7f72daffd640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72c4076b00 0x7f72c4078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:03.626 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.624+0000 7f72daffd640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72c4076b00 0x7f72c4078fb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72d002c120 tx=0x7f72d0004d10 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:03.629 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.629+0000 7f72c3fff640 1 -- 172.21.15.110:0/209037030 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f72cc0610d0 con 0x7f72dc1021e0 2024-04-25T16:01:03.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.844+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f72dc064350 con 0x7f72dc1021e0 2024-04-25T16:01:03.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.845+0000 7f72c3fff640 1 -- 172.21.15.110:0/209037030 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v35) v1 ==== 107+0+5210 (secure 0 0 0) 0x7f72cc060a70 con 0x7f72dc1021e0 2024-04-25T16:01:03.846 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:03.846 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34436,"name":"cephfs.smithi110.kdhjqx","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6829/1518950103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":1518950103},{"type":"v1","addr":"172.21.15.110:6829","nonce":1518950103}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":24},{"gid":44355,"name":"cephfs.smithi193.ewxwfw","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6825/1783080009","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":1783080009},{"type":"v1","addr":"172.21.15.193:6825","nonce":1783080009}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44359,"name":"cephfs.smithi193.jtyzhl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6827/3103986931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":3103986931},{"type":"v1","addr":"172.21.15.193:6827","nonce":3103986931}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T15:46:19.573081+0000","modified":"2024-04-25T15:57:42.598725+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":83,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":34404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34404":{"gid":34404,"name":"cephfs.smithi110.mvvbtv","rank":0,"incarnation":31,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.110:6827/1865691008","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":1865691008},{"type":"v1","addr":"172.21.15.110:6827","nonce":1865691008}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-25T16:01:03.846 INFO:teuthology.orchestra.run.smithi110.stderr:dumped fsmap epoch 34 2024-04-25T16:01:03.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72c4076b00 msgr2=0x7f72c4078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72c4076b00 0x7f72c4078fb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f72d002c120 tx=0x7f72d0004d10 comp rx=0 tx=0).stop 2024-04-25T16:01:03.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc1021e0 msgr2=0x7f72dc112c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:03.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc1021e0 0x7f72dc112c40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c790 tx=0x7f72cc0043c0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 shutdown_connections 2024-04-25T16:01:03.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f72c4076b00 0x7f72c4078fb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72dc102b80 0x7f72dc113180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 --2- 172.21.15.110:0/209037030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f72dc1021e0 0x7f72dc112c40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:03.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 >> 172.21.15.110:0/209037030 conn(0x7f72dc0f7db0 msgr2=0x7f72dc100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:03.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 shutdown_connections 2024-04-25T16:01:03.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:03.847+0000 7f72e2261640 1 -- 172.21.15.110:0/209037030 wait complete. 2024-04-25T16:01:03.859 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T16:01:03.870 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T16:01:03.871 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T16:01:03.871 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:01:03.888 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T16:01:03.888 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:01:03.944 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:01:04.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.079+0000 7fc3eb014640 1 -- 172.21.15.110:0/828702546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 msgr2=0x7fc3e40fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.082 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.079+0000 7fc3eb014640 1 --2- 172.21.15.110:0/828702546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 0x7fc3e40fb160 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc3d80048d0 tx=0x7fc3d8009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.079+0000 7fc3eb014640 1 -- 172.21.15.110:0/828702546 shutdown_connections 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.079+0000 7fc3eb014640 1 --2- 172.21.15.110:0/828702546 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 0x7fc3e40fb160 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.079+0000 7fc3eb014640 1 --2- 172.21.15.110:0/828702546 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4101100 0x7fc3e41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.079+0000 7fc3eb014640 1 -- 172.21.15.110:0/828702546 >> 172.21.15.110:0/828702546 conn(0x7fc3e40711c0 msgr2=0x7fc3e40715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.079+0000 7fc3eb014640 1 -- 172.21.15.110:0/828702546 shutdown_connections 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.079+0000 7fc3eb014640 1 -- 172.21.15.110:0/828702546 wait complete. 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3eb014640 1 Processor -- start 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3eb014640 1 -- start start 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3eb014640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4101100 0x7fc3e4111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3eb014640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 0x7fc3e41121e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3eb014640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e41127e0 con 0x7fc3e4101aa0 2024-04-25T16:01:04.083 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3eb014640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e410be30 con 0x7fc3e4101100 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3e3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 0x7fc3e41121e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3e3fff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 0x7fc3e41121e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50910/0 (socket says 172.21.15.110:50910) 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.080+0000 7fc3e3fff640 1 -- 172.21.15.110:0/3689637017 learned_addr learned my addr 172.21.15.110:0/3689637017 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3e8db0640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4101100 0x7fc3e4111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3e3fff640 1 -- 172.21.15.110:0/3689637017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4101100 msgr2=0x7fc3e4111ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3e3fff640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4101100 0x7fc3e4111ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3e3fff640 1 -- 172.21.15.110:0/3689637017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d8005040 con 0x7fc3e4101aa0 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3e3fff640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 0x7fc3e41121e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc3d802bda0 tx=0x7fc3d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3e17fa640 1 -- 172.21.15.110:0/3689637017 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3d80094c0 con 0x7fc3e4101aa0 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3e17fa640 1 -- 172.21.15.110:0/3689637017 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3d8032e50 con 0x7fc3e4101aa0 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3e17fa640 1 -- 172.21.15.110:0/3689637017 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3d803b400 con 0x7fc3e4101aa0 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e410c0b0 con 0x7fc3e4101aa0 2024-04-25T16:01:04.084 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.081+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e410c580 con 0x7fc3e4101aa0 2024-04-25T16:01:04.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.082+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e40fc880 con 0x7fc3e4101aa0 2024-04-25T16:01:04.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.082+0000 7fc3e17fa640 1 -- 172.21.15.110:0/3689637017 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fc3d802f070 con 0x7fc3e4101aa0 2024-04-25T16:01:04.085 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.083+0000 7fc3e17fa640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc3cc076b00 0x7fc3cc078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.083+0000 7fc3e8db0640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc3cc076b00 0x7fc3cc078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.083+0000 7fc3e17fa640 1 -- 172.21.15.110:0/3689637017 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fc3d80ba850 con 0x7fc3e4101aa0 2024-04-25T16:01:04.086 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.083+0000 7fc3e8db0640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc3cc076b00 0x7fc3cc078fb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4007fd0 tx=0x7fc3d4009e90 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:04.089 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.088+0000 7fc3e17fa640 1 -- 172.21.15.110:0/3689637017 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fc3d8083280 con 0x7fc3e4101aa0 2024-04-25T16:01:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:03 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/1670026501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-25T16:01:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:03 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/421371340' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-25T16:01:04.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:03 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/209037030' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-25T16:01:04.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:03 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/1670026501' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-25T16:01:04.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:03 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/421371340' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-25T16:01:04.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:03 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/209037030' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-25T16:01:04.266 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.265+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3e4108280 con 0x7fc3e4101aa0 2024-04-25T16:01:04.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.266+0000 7fc3e17fa640 1 -- 172.21.15.110:0/3689637017 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16981 (secure 0 0 0) 0x7fc3d8082c20 con 0x7fc3e4101aa0 2024-04-25T16:01:04.267 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:04.268 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":83,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","created":"2024-04-25T15:39:31.460681+0000","modified":"2024-04-25T15:57:32.545877+0000","last_up_change":"2024-04-25T15:56:11.821632+0000","last_in_change":"2024-04-25T15:44:42.991633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T15:43:50.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-25T15:46:18.514536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-25T15:46:18.805544+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a0a2ad-2c02-4108-85a8-3ef716509956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6803","nonce":1940504504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6805","nonce":1940504504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6809","nonce":1940504504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6807","nonce":1940504504}]},"public_addr":"172.21.15.110:6803/1940504504","cluster_addr":"172.21.15.110:6805/1940504504","heartbeat_back_addr":"172.21.15.110:6809/1940504504","heartbeat_front_addr":"172.21.15.110:6807/1940504504","state":["exists","up"]},{"osd":1,"uuid":"90511b8f-4444-44ff-b30c-789349b354f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6811","nonce":4003194194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6813","nonce":4003194194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6817","nonce":4003194194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6815","nonce":4003194194}]},"public_addr":"172.21.15.110:6811/4003194194","cluster_addr":"172.21.15.110:6813/4003194194","heartbeat_back_addr":"172.21.15.110:6817/4003194194","heartbeat_front_addr":"172.21.15.110:6815/4003194194","state":["exists","up"]},{"osd":2,"uuid":"c74ff664-988d-4d4b-a248-176d11f44010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6819","nonce":3018305145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6821","nonce":3018305145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6825","nonce":3018305145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6823","nonce":3018305145}]},"public_addr":"172.21.15.110:6819/3018305145","cluster_addr":"172.21.15.110:6821/3018305145","heartbeat_back_addr":"172.21.15.110:6825/3018305145","heartbeat_front_addr":"172.21.15.110:6823/3018305145","state":["exists","up"]},{"osd":3,"uuid":"dd48aeea-c692-4258-a8df-848bf440b6ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6801","nonce":1773058821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6803","nonce":1773058821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6807","nonce":1773058821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6805","nonce":1773058821}]},"public_addr":"172.21.15.193:6801/1773058821","cluster_addr":"172.21.15.193:6803/1773058821","heartbeat_back_addr":"172.21.15.193:6807/1773058821","heartbeat_front_addr":"172.21.15.193:6805/1773058821","state":["exists","up"]},{"osd":4,"uuid":"3b0a0e9a-5308-421d-8091-fac8f7649509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6809","nonce":2972122544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6811","nonce":2972122544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6815","nonce":2972122544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6813","nonce":2972122544}]},"public_addr":"172.21.15.193:6809/2972122544","cluster_addr":"172.21.15.193:6811/2972122544","heartbeat_back_addr":"172.21.15.193:6815/2972122544","heartbeat_front_addr":"172.21.15.193:6813/2972122544","state":["exists","up"]},{"osd":5,"uuid":"c6cf2f8b-8105-475c-8a79-ce9bbcdf8644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6817","nonce":479474987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6819","nonce":479474987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6823","nonce":479474987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6821","nonce":479474987}]},"public_addr":"172.21.15.193:6817/479474987","cluster_addr":"172.21.15.193:6819/479474987","heartbeat_back_addr":"172.21.15.193:6823/479474987","heartbeat_front_addr":"172.21.15.193:6821/479474987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:42:54.075221+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:21.305107+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:48.660473+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:11.495485+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:35.758085+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:45:00.087588+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6826/1539252929":"2024-04-26T15:57:32.545837+0000","172.21.15.193:6825/746795019":"2024-04-26T15:57:07.402179+0000","172.21.15.193:6824/746795019":"2024-04-26T15:57:07.402179+0000","172.21.15.110:6828/164855346":"2024-04-26T15:56:42.732075+0000","172.21.15.110:0/668880779":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6801/1619647096":"2024-04-26T15:48:32.266095+0000","172.21.15.110:0/4219932763":"2024-04-26T15:48:32.266095+0000","172.21.15.193:0/632067029":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6829/164855346":"2024-04-26T15:56:42.732075+0000","172.21.15.110:0/1729346603":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/1043733842":"2024-04-26T15:48:32.266095+0000","172.21.15.110:0/322959174":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.193:6829/2194935512":"2024-04-26T15:50:46.461397+0000","172.21.15.193:0/2544900458":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6827/123050770":"2024-04-26T15:56:15.605317+0000","172.21.15.110:6801/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/3611960738":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6800/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2164855910":"2024-04-26T15:51:46.657552+0000","172.21.15.193:6827/1539252929":"2024-04-26T15:57:32.545837+0000","172.21.15.110:6800/1619647096":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6801/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.110:6801/1601336217":"2024-04-26T15:51:46.657552+0000","172.21.15.110:6800/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.193:0/3753576575":"2024-04-26T15:50:46.461397+0000","172.21.15.110:0/920532485":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2245327398":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/952937745":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/3260352407":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6801/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2365399538":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/2468496169":"2024-04-26T15:41:40.227647+0000","172.21.15.193:0/1470428832":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6800/1601336217":"2024-04-26T15:51:46.657552+0000","172.21.15.193:6828/2194935512":"2024-04-26T15:50:46.461397+0000","172.21.15.110:0/2068930777":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/3933829355":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/2390441078":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/1601321214":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/271262503":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/3391538140":"2024-04-26T15:51:46.657552+0000","172.21.15.110:6826/123050770":"2024-04-26T15:56:15.605317+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:01:04.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc3cc076b00 msgr2=0x7fc3cc078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.269 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc3cc076b00 0x7fc3cc078fb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4007fd0 tx=0x7fc3d4009e90 comp rx=0 tx=0).stop 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 msgr2=0x7fc3e41121e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 0x7fc3e41121e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc3d802bda0 tx=0x7fc3d8009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 shutdown_connections 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fc3cc076b00 0x7fc3cc078fb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc3e4101aa0 0x7fc3e41121e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 --2- 172.21.15.110:0/3689637017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4101100 0x7fc3e4111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 >> 172.21.15.110:0/3689637017 conn(0x7fc3e40711c0 msgr2=0x7fc3e40faaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 shutdown_connections 2024-04-25T16:01:04.270 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.268+0000 7fc3eb014640 1 -- 172.21.15.110:0/3689637017 wait complete. 2024-04-25T16:01:04.280 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T15:43:50.541493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-25T15:46:18.514536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-25T15:46:18.805544+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:01:04.281 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:01:04.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.376+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/2943690364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 msgr2=0x7fd5b80fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.376+0000 7fd5bfb5d640 1 --2- 172.21.15.110:0/2943690364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 0x7fd5b80fc1b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fd5a80048d0 tx=0x7fd5a8009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.376+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/2943690364 shutdown_connections 2024-04-25T16:01:04.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.376+0000 7fd5bfb5d640 1 --2- 172.21.15.110:0/2943690364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 0x7fd5b80fc1b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.379 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.376+0000 7fd5bfb5d640 1 --2- 172.21.15.110:0/2943690364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b8102180 0x7fd5b8102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.376+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/2943690364 >> 172.21.15.110:0/2943690364 conn(0x7fd5b80f7db0 msgr2=0x7fd5b80fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.376+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/2943690364 shutdown_connections 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.376+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/2943690364 wait complete. 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bfb5d640 1 Processor -- start 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bfb5d640 1 -- start start 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bfb5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b8102180 0x7fd5b8112c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bfb5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 0x7fd5b8113170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bfb5d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b8113770 con 0x7fd5b8102b20 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bfb5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b81138e0 con 0x7fd5b8102180 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bd0f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 0x7fd5b8113170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bd0f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 0x7fd5b8113170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50934/0 (socket says 172.21.15.110:50934) 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bd0f8640 1 -- 172.21.15.110:0/808702438 learned_addr learned my addr 172.21.15.110:0/808702438 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:04.380 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bd0f8640 1 -- 172.21.15.110:0/808702438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b8102180 msgr2=0x7fd5b8112c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bd0f8640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b8102180 0x7fd5b8112c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bd0f8640 1 -- 172.21.15.110:0/808702438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a8005040 con 0x7fd5b8102b20 2024-04-25T16:01:04.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.377+0000 7fd5bd0f8640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 0x7fd5b8113170 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8004930 tx=0x7fd5a8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:04.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.378+0000 7fd5ae7fc640 1 -- 172.21.15.110:0/808702438 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a80094c0 con 0x7fd5b8102b20 2024-04-25T16:01:04.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.378+0000 7fd5ae7fc640 1 -- 172.21.15.110:0/808702438 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd5a8032e50 con 0x7fd5b8102b20 2024-04-25T16:01:04.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.378+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b810cf10 con 0x7fd5b8102b20 2024-04-25T16:01:04.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.378+0000 7fd5ae7fc640 1 -- 172.21.15.110:0/808702438 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a80433b0 con 0x7fd5b8102b20 2024-04-25T16:01:04.381 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.378+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b810d3d0 con 0x7fd5b8102b20 2024-04-25T16:01:04.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.379+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b80fd8f0 con 0x7fd5b8102b20 2024-04-25T16:01:04.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.379+0000 7fd5ae7fc640 1 -- 172.21.15.110:0/808702438 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fd5a802f070 con 0x7fd5b8102b20 2024-04-25T16:01:04.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.380+0000 7fd5ae7fc640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd5a4076b00 0x7fd5a4078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.380+0000 7fd5ae7fc640 1 -- 172.21.15.110:0/808702438 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd5a80ba880 con 0x7fd5b8102b20 2024-04-25T16:01:04.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.380+0000 7fd5bd8f9640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd5a4076b00 0x7fd5a4078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.382 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.380+0000 7fd5bd8f9640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd5a4076b00 0x7fd5a4078fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4004a50 tx=0x7fd5b4007cb0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:04.385 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.384+0000 7fd5ae7fc640 1 -- 172.21.15.110:0/808702438 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd5a8083330 con 0x7fd5b8102b20 2024-04-25T16:01:04.560 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.560+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd5b810e2e0 con 0x7fd5b8102b20 2024-04-25T16:01:04.561 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.560+0000 7fd5ae7fc640 1 -- 172.21.15.110:0/808702438 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd5a8082cd0 con 0x7fd5b8102b20 2024-04-25T16:01:04.561 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd5a4076b00 msgr2=0x7fd5a4078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd5a4076b00 0x7fd5a4078fb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4004a50 tx=0x7fd5b4007cb0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 msgr2=0x7fd5b8113170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 0x7fd5b8113170 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8004930 tx=0x7fd5a8009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 shutdown_connections 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd5a4076b00 0x7fd5a4078fb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd5b8102b20 0x7fd5b8113170 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 --2- 172.21.15.110:0/808702438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b8102180 0x7fd5b8112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 >> 172.21.15.110:0/808702438 conn(0x7fd5b80f7db0 msgr2=0x7fd5b80f8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 shutdown_connections 2024-04-25T16:01:04.564 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.562+0000 7fd5bfb5d640 1 -- 172.21.15.110:0/808702438 wait complete. 2024-04-25T16:01:04.574 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-25T16:01:04.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.669+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2210256295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe101012a0 msgr2=0x7fbe10101670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.669+0000 7fbe17f5d640 1 --2- 172.21.15.110:0/2210256295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe101012a0 0x7fbe10101670 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fbe040048d0 tx=0x7fbe04009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2210256295 shutdown_connections 2024-04-25T16:01:04.672 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 --2- 172.21.15.110:0/2210256295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe10101c40 0x7fbe100fb2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 --2- 172.21.15.110:0/2210256295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe101012a0 0x7fbe10101670 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2210256295 >> 172.21.15.110:0/2210256295 conn(0x7fbe100f6e30 msgr2=0x7fbe100f9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2210256295 shutdown_connections 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2210256295 wait complete. 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 Processor -- start 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 -- start start 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe101012a0 0x7fbe1010e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe10101c40 0x7fbe1010e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe100664d0 con 0x7fbe10101c40 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe17f5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1010eba0 con 0x7fbe101012a0 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe154f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe10101c40 0x7fbe1010e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe154f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe10101c40 0x7fbe1010e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50944/0 (socket says 172.21.15.110:50944) 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.670+0000 7fbe154f8640 1 -- 172.21.15.110:0/2922772900 learned_addr learned my addr 172.21.15.110:0/2922772900 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe15cf9640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe101012a0 0x7fbe1010e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe154f8640 1 -- 172.21.15.110:0/2922772900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe101012a0 msgr2=0x7fbe1010e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.673 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe154f8640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe101012a0 0x7fbe1010e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe154f8640 1 -- 172.21.15.110:0/2922772900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe04005040 con 0x7fbe10101c40 2024-04-25T16:01:04.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe154f8640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe10101c40 0x7fbe1010e630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00eb70 tx=0x7fbe0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:04.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe027fc640 1 -- 172.21.15.110:0/2922772900 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0c007c30 con 0x7fbe10101c40 2024-04-25T16:01:04.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1010ee20 con 0x7fbe10101c40 2024-04-25T16:01:04.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe027fc640 1 -- 172.21.15.110:0/2922772900 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbe0c00f040 con 0x7fbe10101c40 2024-04-25T16:01:04.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe027fc640 1 -- 172.21.15.110:0/2922772900 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe0c0095b0 con 0x7fbe10101c40 2024-04-25T16:01:04.674 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.671+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe101173b0 con 0x7fbe10101c40 2024-04-25T16:01:04.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.672+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe100fca10 con 0x7fbe10101c40 2024-04-25T16:01:04.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.673+0000 7fbe027fc640 1 -- 172.21.15.110:0/2922772900 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fbe0c009730 con 0x7fbe10101c40 2024-04-25T16:01:04.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.673+0000 7fbe027fc640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fbdfc076b00 0x7fbdfc078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.673+0000 7fbe027fc640 1 -- 172.21.15.110:0/2922772900 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fbe0c0981e0 con 0x7fbe10101c40 2024-04-25T16:01:04.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.673+0000 7fbe15cf9640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fbdfc076b00 0x7fbdfc078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.675 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.674+0000 7fbe15cf9640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fbdfc076b00 0x7fbdfc078fb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe04003080 tx=0x7fbe04009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:04.679 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.678+0000 7fbe027fc640 1 -- 172.21.15.110:0/2922772900 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fbe0c060c10 con 0x7fbe10101c40 2024-04-25T16:01:04.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.854+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fbe1004bcc0 con 0x7fbe10101c40 2024-04-25T16:01:04.855 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.854+0000 7fbe027fc640 1 -- 172.21.15.110:0/2922772900 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fbe0c0605b0 con 0x7fbe10101c40 2024-04-25T16:01:04.856 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T16:01:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fbdfc076b00 msgr2=0x7fbdfc078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fbdfc076b00 0x7fbdfc078fb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbe04003080 tx=0x7fbe04009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe10101c40 msgr2=0x7fbe1010e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe10101c40 0x7fbe1010e630 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c00eb70 tx=0x7fbe0c00b040 comp rx=0 tx=0).stop 2024-04-25T16:01:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 shutdown_connections 2024-04-25T16:01:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fbdfc076b00 0x7fbdfc078fb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbe10101c40 0x7fbe1010e630 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.858 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 --2- 172.21.15.110:0/2922772900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe101012a0 0x7fbe1010e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 >> 172.21.15.110:0/2922772900 conn(0x7fbe100f6e30 msgr2=0x7fbe100f7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:04.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 shutdown_connections 2024-04-25T16:01:04.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.857+0000 7fbe17f5d640 1 -- 172.21.15.110:0/2922772900 wait complete. 2024-04-25T16:01:04.868 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-25T16:01:04.968 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 -- 172.21.15.110:0/3198372323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 msgr2=0x7f83b41025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 --2- 172.21.15.110:0/3198372323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 0x7f83b41025c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f83a40048d0 tx=0x7f83a4009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 -- 172.21.15.110:0/3198372323 shutdown_connections 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 --2- 172.21.15.110:0/3198372323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b40fa840 0x7f83b40fac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 --2- 172.21.15.110:0/3198372323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 0x7f83b41025c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 -- 172.21.15.110:0/3198372323 >> 172.21.15.110:0/3198372323 conn(0x7f83b4071700 msgr2=0x7f83b4071b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 -- 172.21.15.110:0/3198372323 shutdown_connections 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 -- 172.21.15.110:0/3198372323 wait complete. 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.966+0000 7f83ba39d640 1 Processor -- start 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83ba39d640 1 -- start start 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83ba39d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b40fa840 0x7f83b4112ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83ba39d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 0x7f83b4113220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83ba39d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b4113820 con 0x7f83b41021f0 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83ba39d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83b410ce70 con 0x7f83b40fa840 2024-04-25T16:01:04.969 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 0x7f83b4113220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b37fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 0x7f83b4113220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50964/0 (socket says 172.21.15.110:50964) 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b37fe640 1 -- 172.21.15.110:0/3755398324 learned_addr learned my addr 172.21.15.110:0/3755398324 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b3fff640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b40fa840 0x7f83b4112ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b37fe640 1 -- 172.21.15.110:0/3755398324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b40fa840 msgr2=0x7f83b4112ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b37fe640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b40fa840 0x7f83b4112ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b37fe640 1 -- 172.21.15.110:0/3755398324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83a4005040 con 0x7f83b41021f0 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b37fe640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 0x7f83b4113220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f83a800c810 tx=0x7f83a80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b0ff9640 1 -- 172.21.15.110:0/3755398324 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83a800a820 con 0x7f83b41021f0 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b0ff9640 1 -- 172.21.15.110:0/3755398324 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f83a800ae50 con 0x7f83b41021f0 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83b410d150 con 0x7f83b41021f0 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83b0ff9640 1 -- 172.21.15.110:0/3755398324 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83a80115c0 con 0x7f83b41021f0 2024-04-25T16:01:04.970 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.967+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83b410d670 con 0x7f83b41021f0 2024-04-25T16:01:04.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.969+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83b40652b0 con 0x7f83b41021f0 2024-04-25T16:01:04.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.969+0000 7f83b0ff9640 1 -- 172.21.15.110:0/3755398324 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f83a800a980 con 0x7f83b41021f0 2024-04-25T16:01:04.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.969+0000 7f83b0ff9640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f839c076b00 0x7f839c078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:04.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.970+0000 7f83b3fff640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f839c076b00 0x7f839c078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:04.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.970+0000 7f83b0ff9640 1 -- 172.21.15.110:0/3755398324 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f83a80989d0 con 0x7f83b41021f0 2024-04-25T16:01:04.971 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.970+0000 7f83b3fff640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f839c076b00 0x7f839c078fb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f83a4004540 tx=0x7f83a4009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:04.975 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:04.974+0000 7f83b0ff9640 1 -- 172.21.15.110:0/3755398324 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f83a8061400 con 0x7f83b41021f0 2024-04-25T16:01:05.150 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.149+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f83b410e350 con 0x7f83b41021f0 2024-04-25T16:01:05.151 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.150+0000 7f83b0ff9640 1 -- 172.21.15.110:0/3755398324 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f83a8060da0 con 0x7f83b41021f0 2024-04-25T16:01:05.151 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T16:01:05.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f839c076b00 msgr2=0x7f839c078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f839c076b00 0x7f839c078fb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f83a4004540 tx=0x7f83a4009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 msgr2=0x7f83b4113220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 0x7f83b4113220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f83a800c810 tx=0x7f83a80043c0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 shutdown_connections 2024-04-25T16:01:05.153 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f839c076b00 0x7f839c078fb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b41021f0 0x7f83b4113220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 --2- 172.21.15.110:0/3755398324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83b40fa840 0x7f83b4112ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 >> 172.21.15.110:0/3755398324 conn(0x7f83b4071700 msgr2=0x7f83b40f9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:05.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 shutdown_connections 2024-04-25T16:01:05.154 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.152+0000 7f83ba39d640 1 -- 172.21.15.110:0/3755398324 wait complete. 2024-04-25T16:01:05.162 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:04 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3689637017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:01:05.162 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:04 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/808702438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:01:05.163 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:04 smithi110 ceph-mon[90991]: pgmap v316: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:05.163 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:04 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2922772900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T16:01:05.164 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:01:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:04 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3689637017' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:01:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:04 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/808702438' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:01:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:04 smithi193 ceph-mon[80071]: pgmap v316: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:04 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2922772900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T16:01:05.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.257+0000 7f78ff79d640 1 -- 172.21.15.110:0/502749971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 msgr2=0x7f78f81014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.257+0000 7f78ff79d640 1 --2- 172.21.15.110:0/502749971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 0x7f78f81014f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009c50 tx=0x7f78f4004d10 comp rx=0 tx=0).stop 2024-04-25T16:01:05.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 -- 172.21.15.110:0/502749971 shutdown_connections 2024-04-25T16:01:05.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 --2- 172.21.15.110:0/502749971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fb100 0x7f78f80fb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.260 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 --2- 172.21.15.110:0/502749971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 0x7f78f81014f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 -- 172.21.15.110:0/502749971 >> 172.21.15.110:0/502749971 conn(0x7f78f80710c0 msgr2=0x7f78f80714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 -- 172.21.15.110:0/502749971 shutdown_connections 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 -- 172.21.15.110:0/502749971 wait complete. 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 Processor -- start 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 -- start start 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fb100 0x7f78f810dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 0x7f78f810e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f8065310 con 0x7f78f8101120 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78ff79d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78f810e970 con 0x7f78f80fb100 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78fcd38640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 0x7f78f810e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78fcd38640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 0x7f78f810e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:50996/0 (socket says 172.21.15.110:50996) 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.258+0000 7f78fcd38640 1 -- 172.21.15.110:0/2716449545 learned_addr learned my addr 172.21.15.110:0/2716449545 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78fd539640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fb100 0x7f78f810dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78fcd38640 1 -- 172.21.15.110:0/2716449545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fb100 msgr2=0x7f78f810dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.261 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78fcd38640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fb100 0x7f78f810dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78fcd38640 1 -- 172.21.15.110:0/2716449545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f40098b0 con 0x7f78f8101120 2024-04-25T16:01:05.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78fcd38640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 0x7f78f810e400 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f78e8002a50 tx=0x7f78e8004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:05.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78edffb640 1 -- 172.21.15.110:0/2716449545 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e8007c30 con 0x7f78f8101120 2024-04-25T16:01:05.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78edffb640 1 -- 172.21.15.110:0/2716449545 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78e800f040 con 0x7f78f8101120 2024-04-25T16:01:05.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78f810ec50 con 0x7f78f8101120 2024-04-25T16:01:05.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78edffb640 1 -- 172.21.15.110:0/2716449545 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78e800db50 con 0x7f78f8101120 2024-04-25T16:01:05.262 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.259+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78f81171e0 con 0x7f78f8101120 2024-04-25T16:01:05.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.260+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78f80fc7e0 con 0x7f78f8101120 2024-04-25T16:01:05.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.261+0000 7f78edffb640 1 -- 172.21.15.110:0/2716449545 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f78e801c400 con 0x7f78f8101120 2024-04-25T16:01:05.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.261+0000 7f78edffb640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b00 0x7f78e4078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.261+0000 7f78fd539640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b00 0x7f78e4078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:05.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.261+0000 7f78edffb640 1 -- 172.21.15.110:0/2716449545 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f78e8097d10 con 0x7f78f8101120 2024-04-25T16:01:05.263 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.262+0000 7f78fd539640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b00 0x7f78e4078fb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f78f4004a30 tx=0x7f78f402e040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:05.267 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.266+0000 7f78edffb640 1 -- 172.21.15.110:0/2716449545 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f78e8060740 con 0x7f78f8101120 2024-04-25T16:01:05.456 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.456+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78f804bcc0 con 0x7f78f8101120 2024-04-25T16:01:05.457 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.456+0000 7f78edffb640 1 -- 172.21.15.110:0/2716449545 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v83) v1 ==== 81+0+2585 (secure 0 0 0) 0x7f78e80600e0 con 0x7f78f8101120 2024-04-25T16:01:05.457 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6826/1539252929 2024-04-26T15:57:32.545837+0000 2024-04-25T16:01:05.457 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6825/746795019 2024-04-26T15:57:07.402179+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6824/746795019 2024-04-26T15:57:07.402179+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6828/164855346 2024-04-26T15:56:42.732075+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/668880779 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1619647096 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/4219932763 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:0/632067029 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6829/164855346 2024-04-26T15:56:42.732075+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1729346603 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1043733842 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/322959174 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1682815503 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6829/2194935512 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:0/2544900458 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:05.458 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6827/123050770 2024-04-26T15:56:15.605317+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1682815503 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3611960738 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/3209979562 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2164855910 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6827/1539252929 2024-04-26T15:57:32.545837+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1619647096 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1116386591 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1601336217 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1116386591 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:0/3753576575 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/920532485 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2245327398 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/952937745 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3260352407 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:05.459 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/3209979562 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2365399538 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2468496169 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:0/1470428832 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1601336217 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6828/2194935512 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2068930777 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3933829355 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2390441078 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1601321214 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/271262503 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3391538140 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:05.460 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6826/123050770 2024-04-26T15:56:15.605317+0000 2024-04-25T16:01:05.461 INFO:teuthology.orchestra.run.smithi110.stderr:listed 43 entries 2024-04-25T16:01:05.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b00 msgr2=0x7f78e4078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b00 0x7f78e4078fb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f78f4004a30 tx=0x7f78f402e040 comp rx=0 tx=0).stop 2024-04-25T16:01:05.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 msgr2=0x7f78f810e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.461 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 0x7f78f810e400 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f78e8002a50 tx=0x7f78e8004600 comp rx=0 tx=0).stop 2024-04-25T16:01:05.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 shutdown_connections 2024-04-25T16:01:05.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f78e4076b00 0x7f78e4078fb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f78f8101120 0x7f78f810e400 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 --2- 172.21.15.110:0/2716449545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78f80fb100 0x7f78f810dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 >> 172.21.15.110:0/2716449545 conn(0x7f78f80710c0 msgr2=0x7f78f81041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:05.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 shutdown_connections 2024-04-25T16:01:05.462 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.458+0000 7f78ff79d640 1 -- 172.21.15.110:0/2716449545 wait complete. 2024-04-25T16:01:05.470 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-04-25T16:01:05.470 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T16:01:05.486 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.618+0000 7fdb88b1d640 1 -- 172.21.15.110:0/4095775987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 msgr2=0x7fdb841014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.618+0000 7fdb88b1d640 1 --2- 172.21.15.110:0/4095775987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 0x7fdb841014d0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fdb740048d0 tx=0x7fdb74009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.618+0000 7fdb88b1d640 1 -- 172.21.15.110:0/4095775987 shutdown_connections 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.618+0000 7fdb88b1d640 1 --2- 172.21.15.110:0/4095775987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84101aa0 0x7fdb840fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.618+0000 7fdb88b1d640 1 --2- 172.21.15.110:0/4095775987 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 0x7fdb841014d0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.618+0000 7fdb88b1d640 1 -- 172.21.15.110:0/4095775987 >> 172.21.15.110:0/4095775987 conn(0x7fdb840711c0 msgr2=0x7fdb840715c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.618+0000 7fdb88b1d640 1 -- 172.21.15.110:0/4095775987 shutdown_connections 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.618+0000 7fdb88b1d640 1 -- 172.21.15.110:0/4095775987 wait complete. 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb88b1d640 1 Processor -- start 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb88b1d640 1 -- start start 2024-04-25T16:01:05.621 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb88b1d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 0x7fdb8410df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb88b1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84101aa0 0x7fdb8410e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb88b1d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8410e9f0 con 0x7fdb84101100 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb88b1d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8410eb60 con 0x7fdb84101aa0 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb8259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 0x7fdb8410df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb8259c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 0x7fdb8410df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51018/0 (socket says 172.21.15.110:51018) 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb8259c640 1 -- 172.21.15.110:0/3687961098 learned_addr learned my addr 172.21.15.110:0/3687961098 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb81d9b640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84101aa0 0x7fdb8410e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb8259c640 1 -- 172.21.15.110:0/3687961098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84101aa0 msgr2=0x7fdb8410e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb8259c640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84101aa0 0x7fdb8410e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb8259c640 1 -- 172.21.15.110:0/3687961098 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb74005040 con 0x7fdb84101100 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.619+0000 7fdb8259c640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 0x7fdb8410df70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009fa0 tx=0x7fdb74009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.620+0000 7fdb72ffd640 1 -- 172.21.15.110:0/3687961098 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb740094c0 con 0x7fdb84101100 2024-04-25T16:01:05.622 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.620+0000 7fdb72ffd640 1 -- 172.21.15.110:0/3687961098 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb7403d920 con 0x7fdb84101100 2024-04-25T16:01:05.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.620+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8410ede0 con 0x7fdb84101100 2024-04-25T16:01:05.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.620+0000 7fdb72ffd640 1 -- 172.21.15.110:0/3687961098 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb7403b8c0 con 0x7fdb84101100 2024-04-25T16:01:05.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.620+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb84117290 con 0x7fdb84101100 2024-04-25T16:01:05.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.621+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb840fc880 con 0x7fdb84101100 2024-04-25T16:01:05.623 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.621+0000 7fdb72ffd640 1 -- 172.21.15.110:0/3687961098 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fdb7403da90 con 0x7fdb84101100 2024-04-25T16:01:05.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.621+0000 7fdb72ffd640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb6c076b00 0x7fdb6c078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.622+0000 7fdb72ffd640 1 -- 172.21.15.110:0/3687961098 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fdb7402d040 con 0x7fdb84101100 2024-04-25T16:01:05.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.622+0000 7fdb81d9b640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb6c076b00 0x7fdb6c078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:05.624 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.622+0000 7fdb81d9b640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb6c076b00 0x7fdb6c078fb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb84064b40 tx=0x7fdb78008040 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:05.627 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.626+0000 7fdb72ffd640 1 -- 172.21.15.110:0/3687961098 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fdb740efff0 con 0x7fdb84101100 2024-04-25T16:01:05.805 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.804+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb840648f0 con 0x7fdb84101100 2024-04-25T16:01:05.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.806+0000 7fdb72ffd640 1 -- 172.21.15.110:0/3687961098 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+16981 (secure 0 0 0) 0x7fdb74083bd0 con 0x7fdb84101100 2024-04-25T16:01:05.807 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-04-25T16:01:05.807 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":83,"fsid":"e79cc6fc-0319-11ef-bc93-c7b262605968","created":"2024-04-25T15:39:31.460681+0000","modified":"2024-04-25T15:57:32.545877+0000","last_up_change":"2024-04-25T15:56:11.821632+0000","last_in_change":"2024-04-25T15:44:42.991633+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T15:43:50.541493+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-25T15:46:18.514536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-25T15:46:18.805544+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"50a0a2ad-2c02-4108-85a8-3ef716509956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":49,"up_from":53,"up_thru":78,"down_at":50,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6803","nonce":1940504504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6805","nonce":1940504504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6809","nonce":1940504504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":1940504504},{"type":"v1","addr":"172.21.15.110:6807","nonce":1940504504}]},"public_addr":"172.21.15.110:6803/1940504504","cluster_addr":"172.21.15.110:6805/1940504504","heartbeat_back_addr":"172.21.15.110:6809/1940504504","heartbeat_front_addr":"172.21.15.110:6807/1940504504","state":["exists","up"]},{"osd":1,"uuid":"90511b8f-4444-44ff-b30c-789349b354f9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":54,"up_from":58,"up_thru":78,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6811","nonce":4003194194}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6813","nonce":4003194194}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6817","nonce":4003194194}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":4003194194},{"type":"v1","addr":"172.21.15.110:6815","nonce":4003194194}]},"public_addr":"172.21.15.110:6811/4003194194","cluster_addr":"172.21.15.110:6813/4003194194","heartbeat_back_addr":"172.21.15.110:6817/4003194194","heartbeat_front_addr":"172.21.15.110:6815/4003194194","state":["exists","up"]},{"osd":2,"uuid":"c74ff664-988d-4d4b-a248-176d11f44010","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":59,"up_from":63,"up_thru":78,"down_at":60,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6819","nonce":3018305145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6821","nonce":3018305145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6825","nonce":3018305145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":3018305145},{"type":"v1","addr":"172.21.15.110:6823","nonce":3018305145}]},"public_addr":"172.21.15.110:6819/3018305145","cluster_addr":"172.21.15.110:6821/3018305145","heartbeat_back_addr":"172.21.15.110:6825/3018305145","heartbeat_front_addr":"172.21.15.110:6823/3018305145","state":["exists","up"]},{"osd":3,"uuid":"dd48aeea-c692-4258-a8df-848bf440b6ab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":64,"up_from":68,"up_thru":78,"down_at":65,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6801","nonce":1773058821}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6803","nonce":1773058821}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6807","nonce":1773058821}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":1773058821},{"type":"v1","addr":"172.21.15.193:6805","nonce":1773058821}]},"public_addr":"172.21.15.193:6801/1773058821","cluster_addr":"172.21.15.193:6803/1773058821","heartbeat_back_addr":"172.21.15.193:6807/1773058821","heartbeat_front_addr":"172.21.15.193:6805/1773058821","state":["exists","up"]},{"osd":4,"uuid":"3b0a0e9a-5308-421d-8091-fac8f7649509","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":69,"up_from":73,"up_thru":78,"down_at":70,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6809","nonce":2972122544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6811","nonce":2972122544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6815","nonce":2972122544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":2972122544},{"type":"v1","addr":"172.21.15.193:6813","nonce":2972122544}]},"public_addr":"172.21.15.193:6809/2972122544","cluster_addr":"172.21.15.193:6811/2972122544","heartbeat_back_addr":"172.21.15.193:6815/2972122544","heartbeat_front_addr":"172.21.15.193:6813/2972122544","state":["exists","up"]},{"osd":5,"uuid":"c6cf2f8b-8105-475c-8a79-ce9bbcdf8644","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":74,"up_from":78,"up_thru":78,"down_at":75,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6817","nonce":479474987}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6819","nonce":479474987}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6822","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6823","nonce":479474987}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":479474987},{"type":"v1","addr":"172.21.15.193:6821","nonce":479474987}]},"public_addr":"172.21.15.193:6817/479474987","cluster_addr":"172.21.15.193:6819/479474987","heartbeat_back_addr":"172.21.15.193:6823/479474987","heartbeat_front_addr":"172.21.15.193:6821/479474987","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:42:54.075221+0000","dead_epoch":49},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:21.305107+0000","dead_epoch":54},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:43:48.660473+0000","dead_epoch":59},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:11.495485+0000","dead_epoch":64},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:44:35.758085+0000","dead_epoch":69},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T15:45:00.087588+0000","dead_epoch":74}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6826/1539252929":"2024-04-26T15:57:32.545837+0000","172.21.15.193:6825/746795019":"2024-04-26T15:57:07.402179+0000","172.21.15.193:6824/746795019":"2024-04-26T15:57:07.402179+0000","172.21.15.110:6828/164855346":"2024-04-26T15:56:42.732075+0000","172.21.15.110:0/668880779":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6801/1619647096":"2024-04-26T15:48:32.266095+0000","172.21.15.110:0/4219932763":"2024-04-26T15:48:32.266095+0000","172.21.15.193:0/632067029":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6829/164855346":"2024-04-26T15:56:42.732075+0000","172.21.15.110:0/1729346603":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/1043733842":"2024-04-26T15:48:32.266095+0000","172.21.15.110:0/322959174":"2024-04-26T15:40:02.513612+0000","172.21.15.110:6800/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.193:6829/2194935512":"2024-04-26T15:50:46.461397+0000","172.21.15.193:0/2544900458":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6827/123050770":"2024-04-26T15:56:15.605317+0000","172.21.15.110:6801/1682815503":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/3611960738":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6800/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2164855910":"2024-04-26T15:51:46.657552+0000","172.21.15.193:6827/1539252929":"2024-04-26T15:57:32.545837+0000","172.21.15.110:6800/1619647096":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6801/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.110:6801/1601336217":"2024-04-26T15:51:46.657552+0000","172.21.15.110:6800/1116386591":"2024-04-26T15:40:32.961057+0000","172.21.15.193:0/3753576575":"2024-04-26T15:50:46.461397+0000","172.21.15.110:0/920532485":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2245327398":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/952937745":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/3260352407":"2024-04-26T15:48:32.266095+0000","172.21.15.110:6801/3209979562":"2024-04-26T15:41:40.227647+0000","172.21.15.110:0/2365399538":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/2468496169":"2024-04-26T15:41:40.227647+0000","172.21.15.193:0/1470428832":"2024-04-26T15:50:46.461397+0000","172.21.15.110:6800/1601336217":"2024-04-26T15:51:46.657552+0000","172.21.15.193:6828/2194935512":"2024-04-26T15:50:46.461397+0000","172.21.15.110:0/2068930777":"2024-04-26T15:40:02.513612+0000","172.21.15.110:0/3933829355":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/2390441078":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/1601321214":"2024-04-26T15:40:32.961057+0000","172.21.15.110:0/271262503":"2024-04-26T15:51:46.657552+0000","172.21.15.110:0/3391538140":"2024-04-26T15:51:46.657552+0000","172.21.15.110:6826/123050770":"2024-04-26T15:56:15.605317+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T16:01:05.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb6c076b00 msgr2=0x7fdb6c078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb6c076b00 0x7fdb6c078fb0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb84064b40 tx=0x7fdb78008040 comp rx=0 tx=0).stop 2024-04-25T16:01:05.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 msgr2=0x7fdb8410df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 0x7fdb8410df70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdb74009fa0 tx=0x7fdb74009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.809 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 shutdown_connections 2024-04-25T16:01:05.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fdb6c076b00 0x7fdb6c078fb0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb84101aa0 0x7fdb8410e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 --2- 172.21.15.110:0/3687961098 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb84101100 0x7fdb8410df70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 >> 172.21.15.110:0/3687961098 conn(0x7fdb840711c0 msgr2=0x7fdb840faa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:05.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 shutdown_connections 2024-04-25T16:01:05.810 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.808+0000 7fdb88b1d640 1 -- 172.21.15.110:0/3687961098 wait complete. 2024-04-25T16:01:05.820 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T15:43:50.541493+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-25T15:46:18.514536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-25T15:46:18.805544+0000', 'flags': 131073, 'flags_names': 'hashpspool,bulk', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T16:01:05.820 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.915+0000 7fd16bb5d640 1 -- 172.21.15.110:0/3165508295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1641023a0 msgr2=0x7fd164102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.915+0000 7fd16bb5d640 1 --2- 172.21.15.110:0/3165508295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1641023a0 0x7fd164102770 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd1600048d0 tx=0x7fd160009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.915+0000 7fd16bb5d640 1 -- 172.21.15.110:0/3165508295 shutdown_connections 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.915+0000 7fd16bb5d640 1 --2- 172.21.15.110:0/3165508295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd164102d40 0x7fd1640fc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.915+0000 7fd16bb5d640 1 --2- 172.21.15.110:0/3165508295 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd1641023a0 0x7fd164102770 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.915+0000 7fd16bb5d640 1 -- 172.21.15.110:0/3165508295 >> 172.21.15.110:0/3165508295 conn(0x7fd1640f7f90 msgr2=0x7fd1640fa3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.915+0000 7fd16bb5d640 1 -- 172.21.15.110:0/3165508295 shutdown_connections 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.915+0000 7fd16bb5d640 1 -- 172.21.15.110:0/3165508295 wait complete. 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd16bb5d640 1 Processor -- start 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd16bb5d640 1 -- start start 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd16bb5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1641023a0 0x7fd164112ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.918 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd16bb5d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd164102d40 0x7fd164113430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd16bb5d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd164113a30 con 0x7fd164102d40 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd16bb5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16410d080 con 0x7fd1641023a0 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd1690f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd164102d40 0x7fd164113430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd1690f8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd164102d40 0x7fd164113430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51036/0 (socket says 172.21.15.110:51036) 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd1690f8640 1 -- 172.21.15.110:0/4005675555 learned_addr learned my addr 172.21.15.110:0/4005675555 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd1690f8640 1 -- 172.21.15.110:0/4005675555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1641023a0 msgr2=0x7fd164112ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd1690f8640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1641023a0 0x7fd164112ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd1690f8640 1 -- 172.21.15.110:0/4005675555 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd160005040 con 0x7fd164102d40 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd1690f8640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd164102d40 0x7fd164113430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd154002a50 tx=0x7fd154004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.916+0000 7fd15a7fc640 1 -- 172.21.15.110:0/4005675555 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd154007c30 con 0x7fd164102d40 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.917+0000 7fd15a7fc640 1 -- 172.21.15.110:0/4005675555 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd15400f040 con 0x7fd164102d40 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.917+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16410d360 con 0x7fd164102d40 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.917+0000 7fd15a7fc640 1 -- 172.21.15.110:0/4005675555 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd15400db50 con 0x7fd164102d40 2024-04-25T16:01:05.919 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.917+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16410d880 con 0x7fd164102d40 2024-04-25T16:01:05.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.917+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1640fdb10 con 0x7fd164102d40 2024-04-25T16:01:05.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.918+0000 7fd15a7fc640 1 -- 172.21.15.110:0/4005675555 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fd15401c400 con 0x7fd164102d40 2024-04-25T16:01:05.920 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.919+0000 7fd15a7fc640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd150076b00 0x7fd150078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:05.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.919+0000 7fd1698f9640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd150076b00 0x7fd150078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:05.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.919+0000 7fd15a7fc640 1 -- 172.21.15.110:0/4005675555 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fd154097d40 con 0x7fd164102d40 2024-04-25T16:01:05.921 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.919+0000 7fd1698f9640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd150076b00 0x7fd150078fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1600043d0 tx=0x7fd160009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:05.924 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:05.924+0000 7fd15a7fc640 1 -- 172.21.15.110:0/4005675555 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fd154060770 con 0x7fd164102d40 2024-04-25T16:01:06.102 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.101+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd164109590 con 0x7fd164102d40 2024-04-25T16:01:06.103 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.102+0000 7fd15a7fc640 1 -- 172.21.15.110:0/4005675555 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v83) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd154060110 con 0x7fd164102d40 2024-04-25T16:01:06.103 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 1 2024-04-25T16:01:06.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd150076b00 msgr2=0x7fd150078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd150076b00 0x7fd150078fb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd1600043d0 tx=0x7fd160009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd164102d40 msgr2=0x7fd164113430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd164102d40 0x7fd164113430 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd154002a50 tx=0x7fd154004600 comp rx=0 tx=0).stop 2024-04-25T16:01:06.105 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 shutdown_connections 2024-04-25T16:01:06.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fd150076b00 0x7fd150078fb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd164102d40 0x7fd164113430 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 --2- 172.21.15.110:0/4005675555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1641023a0 0x7fd164112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 >> 172.21.15.110:0/4005675555 conn(0x7fd1640f7f90 msgr2=0x7fd1640f8f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:06.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 shutdown_connections 2024-04-25T16:01:06.106 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.104+0000 7fd16bb5d640 1 -- 172.21.15.110:0/4005675555 wait complete. 2024-04-25T16:01:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:05 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3755398324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T16:01:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:05 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/2716449545' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T16:01:06.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:05 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3687961098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:01:06.116 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-25T16:01:06.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 -- 172.21.15.110:0/392090602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101a30 msgr2=0x7f7aac0fb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 --2- 172.21.15.110:0/392090602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101a30 0x7f7aac0fb0c0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0048d0 tx=0x7f7a9c009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 -- 172.21.15.110:0/392090602 shutdown_connections 2024-04-25T16:01:06.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 --2- 172.21.15.110:0/392090602 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101a30 0x7f7aac0fb0c0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 --2- 172.21.15.110:0/392090602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac101090 0x7f7aac101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 -- 172.21.15.110:0/392090602 >> 172.21.15.110:0/392090602 conn(0x7f7aac0f6c20 msgr2=0x7f7aac0f9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:06.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 -- 172.21.15.110:0/392090602 shutdown_connections 2024-04-25T16:01:06.214 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 -- 172.21.15.110:0/392090602 wait complete. 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 Processor -- start 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.211+0000 7f7ab419d640 1 -- start start 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab419d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101090 0x7f7aac111c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab419d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac101a30 0x7f7aac1121b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab419d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac112840 con 0x7f7aac101090 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab419d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac10be00 con 0x7f7aac101a30 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab1f39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101090 0x7f7aac111c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab1f39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101090 0x7f7aac111c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51054/0 (socket says 172.21.15.110:51054) 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab1f39640 1 -- 172.21.15.110:0/258176091 learned_addr learned my addr 172.21.15.110:0/258176091 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab1738640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac101a30 0x7f7aac1121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab1f39640 1 -- 172.21.15.110:0/258176091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac101a30 msgr2=0x7f7aac1121b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab1f39640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac101a30 0x7f7aac1121b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab1f39640 1 -- 172.21.15.110:0/258176091 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c005040 con 0x7f7aac101090 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab1f39640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101090 0x7f7aac111c70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800b790 tx=0x7f7aa80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:06.215 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7aa27fc640 1 -- 172.21.15.110:0/258176091 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aa800c700 con 0x7f7aac101090 2024-04-25T16:01:06.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7aa27fc640 1 -- 172.21.15.110:0/258176091 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7aa800cd30 con 0x7f7aac101090 2024-04-25T16:01:06.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac10c0e0 con 0x7f7aac101090 2024-04-25T16:01:06.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7aa27fc640 1 -- 172.21.15.110:0/258176091 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7aa8013370 con 0x7f7aac101090 2024-04-25T16:01:06.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.212+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac10c600 con 0x7f7aac101090 2024-04-25T16:01:06.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.213+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac0fc800 con 0x7f7aac101090 2024-04-25T16:01:06.216 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.214+0000 7f7aa27fc640 1 -- 172.21.15.110:0/258176091 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7f7aa8012070 con 0x7f7aac101090 2024-04-25T16:01:06.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.214+0000 7f7aa27fc640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7a98076b00 0x7f7a98078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.214+0000 7f7aa27fc640 1 -- 172.21.15.110:0/258176091 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7f7aa80980c0 con 0x7f7aac101090 2024-04-25T16:01:06.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.214+0000 7f7ab1738640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7a98076b00 0x7f7a98078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.217 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.215+0000 7f7ab1738640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7a98076b00 0x7f7a98078fb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c000c00 tx=0x7f7a9c009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:06.220 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.219+0000 7f7aa27fc640 1 -- 172.21.15.110:0/258176091 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7f7aa8060af0 con 0x7f7aac101090 2024-04-25T16:01:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:05 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3755398324' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T16:01:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:05 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/2716449545' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T16:01:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:05 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3687961098' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T16:01:06.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.396+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f7aac108250 con 0x7f7aac101090 2024-04-25T16:01:06.397 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.396+0000 7f7aa27fc640 1 -- 172.21.15.110:0/258176091 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7f7aa8060490 con 0x7f7aac101090 2024-04-25T16:01:06.397 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T16:01:06.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7a98076b00 msgr2=0x7f7a98078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7a98076b00 0x7f7a98078fb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c000c00 tx=0x7f7a9c009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101090 msgr2=0x7f7aac111c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.399 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101090 0x7f7aac111c70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800b790 tx=0x7f7aa80091d0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 shutdown_connections 2024-04-25T16:01:06.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7f7a98076b00 0x7f7a98078fb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7aac101a30 0x7f7aac1121b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 --2- 172.21.15.110:0/258176091 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7aac101090 0x7f7aac111c70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 >> 172.21.15.110:0/258176091 conn(0x7f7aac0f6c20 msgr2=0x7f7aac0f7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:06.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 shutdown_connections 2024-04-25T16:01:06.400 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.398+0000 7f7ab419d640 1 -- 172.21.15.110:0/258176091 wait complete. 2024-04-25T16:01:06.409 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-25T16:01:06.507 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 -- 172.21.15.110:0/4279995626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102b20 msgr2=0x7fa1ac0fc1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 --2- 172.21.15.110:0/4279995626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102b20 0x7fa1ac0fc1b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fa1940048d0 tx=0x7fa194009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 -- 172.21.15.110:0/4279995626 shutdown_connections 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 --2- 172.21.15.110:0/4279995626 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102b20 0x7fa1ac0fc1b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 --2- 172.21.15.110:0/4279995626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1ac102180 0x7fa1ac102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 -- 172.21.15.110:0/4279995626 >> 172.21.15.110:0/4279995626 conn(0x7fa1ac0f7db0 msgr2=0x7fa1ac0fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 -- 172.21.15.110:0/4279995626 shutdown_connections 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 -- 172.21.15.110:0/4279995626 wait complete. 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 Processor -- start 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 -- start start 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102180 0x7fa1ac112d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1ac102b20 0x7fa1ac113290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac113920 con 0x7fa1ac102180 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1b195d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac10cee0 con 0x7fa1ac102b20 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1aaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102180 0x7fa1ac112d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.508 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1aaffd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102180 0x7fa1ac112d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51076/0 (socket says 172.21.15.110:51076) 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.505+0000 7fa1aaffd640 1 -- 172.21.15.110:0/3302937064 learned_addr learned my addr 172.21.15.110:0/3302937064 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1a3fff640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1ac102b20 0x7fa1ac113290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1aaffd640 1 -- 172.21.15.110:0/3302937064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1ac102b20 msgr2=0x7fa1ac113290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1aaffd640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1ac102b20 0x7fa1ac113290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1aaffd640 1 -- 172.21.15.110:0/3302937064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa194005040 con 0x7fa1ac102180 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1aaffd640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102180 0x7fa1ac112d50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00b790 tx=0x7fa19c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1a37fe640 1 -- 172.21.15.110:0/3302937064 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19c00c700 con 0x7fa1ac102180 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac10d1c0 con 0x7fa1ac102180 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1a37fe640 1 -- 172.21.15.110:0/3302937064 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa19c00cd30 con 0x7fa1ac102180 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1a37fe640 1 -- 172.21.15.110:0/3302937064 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19c013370 con 0x7fa1ac102180 2024-04-25T16:01:06.509 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.506+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac10d6b0 con 0x7fa1ac102180 2024-04-25T16:01:06.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.507+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ac0fd8f0 con 0x7fa1ac102180 2024-04-25T16:01:06.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.508+0000 7fa1a37fe640 1 -- 172.21.15.110:0/3302937064 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7fa19c012070 con 0x7fa1ac102180 2024-04-25T16:01:06.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.508+0000 7fa1a37fe640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa198076b00 0x7fa198078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.510 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.508+0000 7fa1a37fe640 1 -- 172.21.15.110:0/3302937064 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7fa19c097ea0 con 0x7fa1ac102180 2024-04-25T16:01:06.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.508+0000 7fa1a3fff640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa198076b00 0x7fa198078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.511 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.508+0000 7fa1a3fff640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa198076b00 0x7fa198078fb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0fa870 tx=0x7fa194009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:06.513 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.513+0000 7fa1a37fe640 1 -- 172.21.15.110:0/3302937064 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7fa19c0608d0 con 0x7fa1ac102180 2024-04-25T16:01:06.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.690+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fa1ac10e3c0 con 0x7fa1ac102180 2024-04-25T16:01:06.691 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.690+0000 7fa1a37fe640 1 -- 172.21.15.110:0/3302937064 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v83) v1 ==== 107+0+11 (secure 0 0 0) 0x7fa19c060270 con 0x7fa1ac102180 2024-04-25T16:01:06.691 INFO:teuthology.orchestra.run.smithi110.stdout:pg_num: 32 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa198076b00 msgr2=0x7fa198078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa198076b00 0x7fa198078fb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac0fa870 tx=0x7fa194009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102180 msgr2=0x7fa1ac112d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102180 0x7fa1ac112d50 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa19c00b790 tx=0x7fa19c0091d0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 shutdown_connections 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fa198076b00 0x7fa198078fb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1ac102b20 0x7fa1ac113290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 --2- 172.21.15.110:0/3302937064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1ac102180 0x7fa1ac112d50 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.692+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 >> 172.21.15.110:0/3302937064 conn(0x7fa1ac0f7db0 msgr2=0x7fa1ac0f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:06.694 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.693+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 shutdown_connections 2024-04-25T16:01:06.695 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.693+0000 7fa1b195d640 1 -- 172.21.15.110:0/3302937064 wait complete. 2024-04-25T16:01:06.704 DEBUG:teuthology.orchestra.run.smithi110:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T16:01:06.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.797+0000 7feab0a21640 1 -- 172.21.15.110:0/410327818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fd1f0 msgr2=0x7feaac0fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.800 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.797+0000 7feab0a21640 1 --2- 172.21.15.110:0/410327818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fd1f0 0x7feaac0fd5c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fea9c004900 tx=0x7fea9c009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.797+0000 7feab0a21640 1 -- 172.21.15.110:0/410327818 shutdown_connections 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.797+0000 7feab0a21640 1 --2- 172.21.15.110:0/410327818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaac0fdb90 0x7feaac106260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.797+0000 7feab0a21640 1 --2- 172.21.15.110:0/410327818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fd1f0 0x7feaac0fd5c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.797+0000 7feab0a21640 1 -- 172.21.15.110:0/410327818 >> 172.21.15.110:0/410327818 conn(0x7feaac0f6cb0 msgr2=0x7feaac0f90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.797+0000 7feab0a21640 1 -- 172.21.15.110:0/410327818 shutdown_connections 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.797+0000 7feab0a21640 1 -- 172.21.15.110:0/410327818 wait complete. 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feab0a21640 1 Processor -- start 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feab0a21640 1 -- start start 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feab0a21640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaac0fd1f0 0x7feaac113070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feab0a21640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fdb90 0x7feaac1135b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feab0a21640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac113bb0 con 0x7feaac0fdb90 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feab0a21640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac10d200 con 0x7feaac0fd1f0 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feaa9d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fdb90 0x7feaac1135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.801 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feaa9d9b640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fdb90 0x7feaac1135b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51102/0 (socket says 172.21.15.110:51102) 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feaa9d9b640 1 -- 172.21.15.110:0/3213615061 learned_addr learned my addr 172.21.15.110:0/3213615061 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.798+0000 7feaaa59c640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaac0fd1f0 0x7feaac113070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7feaa9d9b640 1 -- 172.21.15.110:0/3213615061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaac0fd1f0 msgr2=0x7feaac113070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7feaa9d9b640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaac0fd1f0 0x7feaac113070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7feaa9d9b640 1 -- 172.21.15.110:0/3213615061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea9c005040 con 0x7feaac0fdb90 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7feaa9d9b640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fdb90 0x7feaac1135b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feaa000c810 tx=0x7feaa00043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7fea9affd640 1 -- 172.21.15.110:0/3213615061 <== mon.0 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa000a820 con 0x7feaac0fdb90 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7fea9affd640 1 -- 172.21.15.110:0/3213615061 <== mon.0 v2:172.21.15.110:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feaa000ae50 con 0x7feaac0fdb90 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7fea9affd640 1 -- 172.21.15.110:0/3213615061 <== mon.0 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaa00115c0 con 0x7feaac0fdb90 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac10d4e0 con 0x7feaac0fdb90 2024-04-25T16:01:06.802 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.799+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac10da00 con 0x7feaac0fdb90 2024-04-25T16:01:06.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.800+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaac065d90 con 0x7feaac0fdb90 2024-04-25T16:01:06.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.801+0000 7fea9affd640 1 -- 172.21.15.110:0/3213615061 <== mon.0 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 38) v1 ==== 98663+0+0 (secure 0 0 0) 0x7feaa000a980 con 0x7feaac0fdb90 2024-04-25T16:01:06.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.801+0000 7fea9affd640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fea94076b00 0x7fea94078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T16:01:06.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.801+0000 7feaaa59c640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fea94076b00 0x7fea94078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T16:01:06.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.801+0000 7fea9affd640 1 -- 172.21.15.110:0/3213615061 <== mon.0 v2:172.21.15.110:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6668+0+0 (secure 0 0 0) 0x7feaa0098920 con 0x7feaac0fdb90 2024-04-25T16:01:06.804 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.802+0000 7feaaa59c640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fea94076b00 0x7fea94078fb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009a30 tx=0x7fea9c009da0 comp rx=0 tx=0).ready entity=mgr.34100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T16:01:06.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:06.806+0000 7fea9affd640 1 -- 172.21.15.110:0/3213615061 <== mon.0 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188370 (secure 0 0 0) 0x7feaa0061350 con 0x7feaac0fdb90 2024-04-25T16:01:07.004 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.003+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feaac065210 con 0x7feaac0fdb90 2024-04-25T16:01:07.005 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.004+0000 7fea9affd640 1 -- 172.21.15.110:0/3213615061 <== mon.0 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 43 entries v83) v1 ==== 81+0+2585 (secure 0 0 0) 0x7feaa0060cf0 con 0x7feaac0fdb90 2024-04-25T16:01:07.005 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6826/1539252929 2024-04-26T15:57:32.545837+0000 2024-04-25T16:01:07.005 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6825/746795019 2024-04-26T15:57:07.402179+0000 2024-04-25T16:01:07.005 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6824/746795019 2024-04-26T15:57:07.402179+0000 2024-04-25T16:01:07.005 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6828/164855346 2024-04-26T15:56:42.732075+0000 2024-04-25T16:01:07.005 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/668880779 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1619647096 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/4219932763 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:0/632067029 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6829/164855346 2024-04-26T15:56:42.732075+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1729346603 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1043733842 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/322959174 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1682815503 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6829/2194935512 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:0/2544900458 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6827/123050770 2024-04-26T15:56:15.605317+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1682815503 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:07.006 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3611960738 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/3209979562 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2164855910 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6827/1539252929 2024-04-26T15:57:32.545837+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1619647096 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1116386591 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/1601336217 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1116386591 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:0/3753576575 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/920532485 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2245327398 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/952937745 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3260352407 2024-04-26T15:48:32.266095+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6801/3209979562 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2365399538 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2468496169 2024-04-26T15:41:40.227647+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:0/1470428832 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:07.007 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6800/1601336217 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:07.008 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.193:6828/2194935512 2024-04-26T15:50:46.461397+0000 2024-04-25T16:01:07.008 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2068930777 2024-04-26T15:40:02.513612+0000 2024-04-25T16:01:07.008 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3933829355 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:07.008 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/2390441078 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:07.008 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/1601321214 2024-04-26T15:40:32.961057+0000 2024-04-25T16:01:07.008 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/271262503 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:07.008 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:0/3391538140 2024-04-26T15:51:46.657552+0000 2024-04-25T16:01:07.008 INFO:teuthology.orchestra.run.smithi110.stdout:172.21.15.110:6826/123050770 2024-04-26T15:56:15.605317+0000 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:listed 43 entries 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fea94076b00 msgr2=0x7fea94078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fea94076b00 0x7fea94078fb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fea9c009a30 tx=0x7fea9c009da0 comp rx=0 tx=0).stop 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fdb90 msgr2=0x7feaac1135b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fdb90 0x7feaac1135b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7feaa000c810 tx=0x7feaa00043c0 comp rx=0 tx=0).stop 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 shutdown_connections 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:6800/1904649981,v1:172.21.15.110:6801/1904649981] conn(0x7fea94076b00 0x7fea94078fb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feaac0fdb90 0x7feaac1135b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 --2- 172.21.15.110:0/3213615061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feaac0fd1f0 0x7feaac113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T16:01:07.009 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 >> 172.21.15.110:0/3213615061 conn(0x7feaac0f6cb0 msgr2=0x7feaac105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T16:01:07.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 shutdown_connections 2024-04-25T16:01:07.010 INFO:teuthology.orchestra.run.smithi110.stderr:2024-04-25T16:01:07.006+0000 7feab0a21640 1 -- 172.21.15.110:0/3213615061 wait complete. 2024-04-25T16:01:07.016 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:06 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/4005675555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:01:07.017 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:06 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/258176091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T16:01:07.017 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:06 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3302937064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T16:01:07.017 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:06 smithi110 ceph-mon[90991]: pgmap v317: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:07.018 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi110.front.sepia.ceph.com... 2024-04-25T16:01:07.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T16:01:07.018 DEBUG:teuthology.orchestra.run.smithi110:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T16:01:07.041 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T16:01:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:06 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/4005675555' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T16:01:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:06 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/258176091' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-25T16:01:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:06 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3302937064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-25T16:01:07.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:06 smithi193 ceph-mon[80071]: pgmap v317: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:08.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:07 smithi193 ceph-mon[80071]: from='client.? 172.21.15.110:0/3213615061' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T16:01:08.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:07 smithi110 ceph-mon[90991]: from='client.? 172.21.15.110:0/3213615061' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T16:01:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:08 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:01:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:08 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:01:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:08 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:01:09.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:08 smithi193 ceph-mon[80071]: pgmap v318: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:09.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:08 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:01:09.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:08 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:01:09.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:08 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:01:09.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:08 smithi110 ceph-mon[90991]: pgmap v318: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:09 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:01:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:09 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:01:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:09 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:01:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:09 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:01:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:09 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:01:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:09 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:01:11.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:10 smithi193 ceph-mon[80071]: pgmap v319: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:10 smithi110 ceph-mon[90991]: pgmap v319: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:14.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:13 smithi193 ceph-mon[80071]: pgmap v320: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:14.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:13 smithi110 ceph-mon[90991]: pgmap v320: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:15 smithi193 ceph-mon[80071]: pgmap v321: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:16.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:15 smithi110 ceph-mon[90991]: pgmap v321: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:01:17.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:01:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:17 smithi193 ceph-mon[80071]: pgmap v322: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:18.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:17 smithi110 ceph-mon[90991]: pgmap v322: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:20.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:19 smithi193 ceph-mon[80071]: pgmap v323: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:20.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:19 smithi110 ceph-mon[90991]: pgmap v323: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:21 smithi193 ceph-mon[80071]: pgmap v324: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:22.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:21 smithi110 ceph-mon[90991]: pgmap v324: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:22 smithi193 ceph-mon[80071]: pgmap v325: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:22 smithi110 ceph-mon[90991]: pgmap v325: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:25 smithi193 ceph-mon[80071]: pgmap v326: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:26.338 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:25 smithi110 ceph-mon[90991]: pgmap v326: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:28.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:27 smithi193 ceph-mon[80071]: pgmap v327: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:28.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:27 smithi110 ceph-mon[90991]: pgmap v327: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:29 smithi193 ceph-mon[80071]: pgmap v328: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:30.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:29 smithi110 ceph-mon[90991]: pgmap v328: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:31 smithi193 ceph-mon[80071]: pgmap v329: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:32.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:01:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:31 smithi110 ceph-mon[90991]: pgmap v329: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:01:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:33 smithi193 ceph-mon[80071]: pgmap v330: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:34.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:33 smithi110 ceph-mon[90991]: pgmap v330: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:36.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:35 smithi193 ceph-mon[80071]: pgmap v331: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:36.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:35 smithi110 ceph-mon[90991]: pgmap v331: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:37 smithi193 ceph-mon[80071]: pgmap v332: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:38.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:37 smithi110 ceph-mon[90991]: pgmap v332: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:38 smithi193 ceph-mon[80071]: pgmap v333: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:39.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:38 smithi110 ceph-mon[90991]: pgmap v333: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:42.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:41 smithi193 ceph-mon[80071]: pgmap v334: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:42.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:41 smithi110 ceph-mon[90991]: pgmap v334: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:43 smithi193 ceph-mon[80071]: pgmap v335: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:44.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:43 smithi110 ceph-mon[90991]: pgmap v335: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:45 smithi193 ceph-mon[80071]: pgmap v336: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:46.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:45 smithi110 ceph-mon[90991]: pgmap v336: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:46 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:01:47.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:01:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:47 smithi193 ceph-mon[80071]: pgmap v337: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:48.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:47 smithi110 ceph-mon[90991]: pgmap v337: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:49.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:48 smithi193 ceph-mon[80071]: pgmap v338: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:49.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:48 smithi110 ceph-mon[90991]: pgmap v338: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:52.242 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:51 smithi193 ceph-mon[80071]: pgmap v339: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:52.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:51 smithi110 ceph-mon[90991]: pgmap v339: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:54.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:53 smithi193 ceph-mon[80071]: pgmap v340: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:54.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:53 smithi110 ceph-mon[90991]: pgmap v340: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:56.165 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:55 smithi110 ceph-mon[90991]: pgmap v341: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:55 smithi193 ceph-mon[80071]: pgmap v341: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:58.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:57 smithi193 ceph-mon[80071]: pgmap v342: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:01:58.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:57 smithi110 ceph-mon[90991]: pgmap v342: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:01:59 smithi193 ceph-mon[80071]: pgmap v343: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:00.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:01:59 smithi110 ceph-mon[90991]: pgmap v343: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:00 smithi193 ceph-mon[80071]: pgmap v344: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:00 smithi110 ceph-mon[90991]: pgmap v344: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:02:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:02:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:02 smithi193 ceph-mon[80071]: pgmap v345: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:03.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:02 smithi110 ceph-mon[90991]: pgmap v345: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:05 smithi193 ceph-mon[80071]: pgmap v346: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:06.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:05 smithi110 ceph-mon[90991]: pgmap v346: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:08.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:07 smithi193 ceph-mon[80071]: pgmap v347: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:08.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:07 smithi110 ceph-mon[90991]: pgmap v347: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:09 smithi193 ceph-mon[80071]: pgmap v348: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:09 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:02:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:09 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:02:10.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:09 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:02:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:09 smithi110 ceph-mon[90991]: pgmap v348: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:09 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:02:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:09 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:02:10.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:09 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:02:11.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:10 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:02:11.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:10 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:02:11.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:10 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:02:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:10 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:02:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:10 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:02:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:10 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:02:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:11 smithi193 ceph-mon[80071]: pgmap v349: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:12.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:11 smithi110 ceph-mon[90991]: pgmap v349: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:12 smithi193 ceph-mon[80071]: pgmap v350: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:12 smithi110 ceph-mon[90991]: pgmap v350: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:16.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:15 smithi193 ceph-mon[80071]: pgmap v351: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:16.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:15 smithi110 ceph-mon[90991]: pgmap v351: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:02:17.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:02:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:17 smithi193 ceph-mon[80071]: pgmap v352: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:18.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:17 smithi110 ceph-mon[90991]: pgmap v352: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:18 smithi193 ceph-mon[80071]: pgmap v353: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:19.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:18 smithi110 ceph-mon[90991]: pgmap v353: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:22.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:21 smithi193 ceph-mon[80071]: pgmap v354: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:22.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:21 smithi110 ceph-mon[90991]: pgmap v354: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:24.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:23 smithi193 ceph-mon[80071]: pgmap v355: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:24.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:23 smithi110 ceph-mon[90991]: pgmap v355: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:26.165 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:25 smithi110 ceph-mon[90991]: pgmap v356: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:26.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:25 smithi193 ceph-mon[80071]: pgmap v356: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:27.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:26 smithi193 ceph-mon[80071]: pgmap v357: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:26 smithi110 ceph-mon[90991]: pgmap v357: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:30.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:29 smithi193 ceph-mon[80071]: pgmap v358: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:30.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:29 smithi110 ceph-mon[90991]: pgmap v358: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:31 smithi193 ceph-mon[80071]: pgmap v359: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:02:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:31 smithi110 ceph-mon[90991]: pgmap v359: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:02:34.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:33 smithi193 ceph-mon[80071]: pgmap v360: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:34.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:33 smithi110 ceph-mon[90991]: pgmap v360: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:35.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:34 smithi193 ceph-mon[80071]: pgmap v361: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:34 smithi110 ceph-mon[90991]: pgmap v361: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:38.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:37 smithi193 ceph-mon[80071]: pgmap v362: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:38.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:37 smithi110 ceph-mon[90991]: pgmap v362: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:40.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:39 smithi193 ceph-mon[80071]: pgmap v363: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:40.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:39 smithi110 ceph-mon[90991]: pgmap v363: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:40 smithi193 ceph-mon[80071]: pgmap v364: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:41.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:40 smithi110 ceph-mon[90991]: pgmap v364: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:44.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:43 smithi193 ceph-mon[80071]: pgmap v365: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:44.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:43 smithi110 ceph-mon[90991]: pgmap v365: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:46.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:45 smithi193 ceph-mon[80071]: pgmap v366: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:46.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:45 smithi110 ceph-mon[90991]: pgmap v366: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:46 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:02:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:46 smithi193 ceph-mon[80071]: pgmap v367: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:47.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:02:47.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:46 smithi110 ceph-mon[90991]: pgmap v367: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:50.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:49 smithi193 ceph-mon[80071]: pgmap v368: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:50.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:49 smithi110 ceph-mon[90991]: pgmap v368: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:52.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:51 smithi193 ceph-mon[80071]: pgmap v369: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:52.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:51 smithi110 ceph-mon[90991]: pgmap v369: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:52 smithi193 ceph-mon[80071]: pgmap v370: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:53.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:52 smithi110 ceph-mon[90991]: pgmap v370: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:56.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:55 smithi193 ceph-mon[80071]: pgmap v371: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:56.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:55 smithi110 ceph-mon[90991]: pgmap v371: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:56 smithi193 ceph-mon[80071]: pgmap v372: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:02:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:56 smithi110 ceph-mon[90991]: pgmap v372: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:00.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:02:59 smithi193 ceph-mon[80071]: pgmap v373: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:00.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:02:59 smithi110 ceph-mon[90991]: pgmap v373: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:00 smithi193 ceph-mon[80071]: pgmap v374: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:00 smithi110 ceph-mon[90991]: pgmap v374: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:03:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:03:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:02 smithi193 ceph-mon[80071]: pgmap v375: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:03.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:02 smithi110 ceph-mon[90991]: pgmap v375: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:06.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:05 smithi193 ceph-mon[80071]: pgmap v376: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:06.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:05 smithi110 ceph-mon[90991]: pgmap v376: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:06 smithi193 ceph-mon[80071]: pgmap v377: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:07.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:06 smithi110 ceph-mon[90991]: pgmap v377: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:08 smithi193 ceph-mon[80071]: pgmap v378: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:09.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:08 smithi110 ceph-mon[90991]: pgmap v378: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:10.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:10 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:03:10.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:10 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:03:11.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:11 smithi110 ceph-mon[90991]: pgmap v379: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:11.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:03:11.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:03:11.864 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:03:11.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:11 smithi193 ceph-mon[80071]: pgmap v379: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:11.989 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:03:11.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:03:11.990 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:03:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:12 smithi193 ceph-mon[80071]: pgmap v380: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:12 smithi110 ceph-mon[90991]: pgmap v380: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:14 smithi193 ceph-mon[80071]: pgmap v381: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:15.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:14 smithi110 ceph-mon[90991]: pgmap v381: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:03:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:03:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:17 smithi110 ceph-mon[90991]: pgmap v382: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:17 smithi193 ceph-mon[80071]: pgmap v382: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:18 smithi193 ceph-mon[80071]: pgmap v383: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:19.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:18 smithi110 ceph-mon[90991]: pgmap v383: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:20 smithi193 ceph-mon[80071]: pgmap v384: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:21.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:20 smithi110 ceph-mon[90991]: pgmap v384: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:22 smithi193 ceph-mon[80071]: pgmap v385: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:22 smithi110 ceph-mon[90991]: pgmap v385: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:24 smithi193 ceph-mon[80071]: pgmap v386: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:24 smithi110 ceph-mon[90991]: pgmap v386: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:27.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:26 smithi193 ceph-mon[80071]: pgmap v387: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:26 smithi110 ceph-mon[90991]: pgmap v387: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:28 smithi193 ceph-mon[80071]: pgmap v388: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:29.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:28 smithi110 ceph-mon[90991]: pgmap v388: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:30 smithi193 ceph-mon[80071]: pgmap v389: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:30 smithi110 ceph-mon[90991]: pgmap v389: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:03:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:03:33.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:32 smithi193 ceph-mon[80071]: pgmap v390: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:33.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:32 smithi110 ceph-mon[90991]: pgmap v390: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:35.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:34 smithi193 ceph-mon[80071]: pgmap v391: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:34 smithi110 ceph-mon[90991]: pgmap v391: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:37.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:36 smithi193 ceph-mon[80071]: pgmap v392: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:37.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:36 smithi110 ceph-mon[90991]: pgmap v392: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:39.739 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:39 smithi193 ceph-mon[80071]: pgmap v393: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:39.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:39 smithi110 ceph-mon[90991]: pgmap v393: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:40 smithi193 ceph-mon[80071]: pgmap v394: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:41.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:40 smithi110 ceph-mon[90991]: pgmap v394: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:42 smithi193 ceph-mon[80071]: pgmap v395: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:43.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:42 smithi110 ceph-mon[90991]: pgmap v395: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:44 smithi193 ceph-mon[80071]: pgmap v396: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:45.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:44 smithi110 ceph-mon[90991]: pgmap v396: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:47.115 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:03:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:46 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:03:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:47 smithi110 ceph-mon[90991]: pgmap v397: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:48.456 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:47 smithi193 ceph-mon[80071]: pgmap v397: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:49.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:48 smithi193 ceph-mon[80071]: pgmap v398: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:49.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:48 smithi110 ceph-mon[90991]: pgmap v398: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:50 smithi193 ceph-mon[80071]: pgmap v399: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:50 smithi110 ceph-mon[90991]: pgmap v399: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:53.243 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:52 smithi193 ceph-mon[80071]: pgmap v400: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:53.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:52 smithi110 ceph-mon[90991]: pgmap v400: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:55.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:54 smithi193 ceph-mon[80071]: pgmap v401: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:54 smithi110 ceph-mon[90991]: pgmap v401: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:56 smithi193 ceph-mon[80071]: pgmap v402: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:56 smithi110 ceph-mon[90991]: pgmap v402: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:03:58 smithi193 ceph-mon[80071]: pgmap v403: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:03:59.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:03:58 smithi110 ceph-mon[90991]: pgmap v403: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:00 smithi193 ceph-mon[80071]: pgmap v404: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:00 smithi110 ceph-mon[90991]: pgmap v404: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:02.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:04:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:04:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:02 smithi193 ceph-mon[80071]: pgmap v405: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:03.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:02 smithi110 ceph-mon[90991]: pgmap v405: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:04 smithi193 ceph-mon[80071]: pgmap v406: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:05.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:04 smithi110 ceph-mon[90991]: pgmap v406: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:06 smithi193 ceph-mon[80071]: pgmap v407: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:07.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:06 smithi110 ceph-mon[90991]: pgmap v407: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:08 smithi193 ceph-mon[80071]: pgmap v408: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:09.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:08 smithi110 ceph-mon[90991]: pgmap v408: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:11.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:10 smithi193 ceph-mon[80071]: pgmap v409: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:10 smithi110 ceph-mon[90991]: pgmap v409: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:12.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:04:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:04:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:04:12.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:11 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:04:12.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:04:12.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:04:12.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:04:12.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:11 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:04:13.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:12 smithi193 ceph-mon[80071]: pgmap v410: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:12 smithi110 ceph-mon[90991]: pgmap v410: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:14 smithi193 ceph-mon[80071]: pgmap v411: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:15.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:14 smithi110 ceph-mon[90991]: pgmap v411: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:17.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:04:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:04:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:17 smithi110 ceph-mon[90991]: pgmap v412: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:17 smithi193 ceph-mon[80071]: pgmap v412: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:18 smithi193 ceph-mon[80071]: pgmap v413: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:19.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:18 smithi110 ceph-mon[90991]: pgmap v413: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:21.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:20 smithi193 ceph-mon[80071]: pgmap v414: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:21.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:20 smithi110 ceph-mon[90991]: pgmap v414: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:22 smithi193 ceph-mon[80071]: pgmap v415: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:22 smithi110 ceph-mon[90991]: pgmap v415: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:24 smithi193 ceph-mon[80071]: pgmap v416: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:24 smithi110 ceph-mon[90991]: pgmap v416: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:27.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:26 smithi193 ceph-mon[80071]: pgmap v417: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:26 smithi110 ceph-mon[90991]: pgmap v417: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:28 smithi193 ceph-mon[80071]: pgmap v418: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:29.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:28 smithi110 ceph-mon[90991]: pgmap v418: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:30 smithi193 ceph-mon[80071]: pgmap v419: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:30 smithi110 ceph-mon[90991]: pgmap v419: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:04:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:04:33.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:32 smithi193 ceph-mon[80071]: pgmap v420: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:33.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:32 smithi110 ceph-mon[90991]: pgmap v420: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:35.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:34 smithi193 ceph-mon[80071]: pgmap v421: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:34 smithi110 ceph-mon[90991]: pgmap v421: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:37.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:36 smithi193 ceph-mon[80071]: pgmap v422: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:37.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:36 smithi110 ceph-mon[90991]: pgmap v422: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:38 smithi193 ceph-mon[80071]: pgmap v423: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:39.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:38 smithi110 ceph-mon[90991]: pgmap v423: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:40 smithi193 ceph-mon[80071]: pgmap v424: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:41.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:40 smithi110 ceph-mon[90991]: pgmap v424: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:42 smithi193 ceph-mon[80071]: pgmap v425: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:43.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:42 smithi110 ceph-mon[90991]: pgmap v425: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:45.240 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:44 smithi193 ceph-mon[80071]: pgmap v426: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:45.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:44 smithi110 ceph-mon[90991]: pgmap v426: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:04:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:46 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:04:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:47 smithi110 ceph-mon[90991]: pgmap v427: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:47 smithi193 ceph-mon[80071]: pgmap v427: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:49.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:48 smithi193 ceph-mon[80071]: pgmap v428: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:49.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:48 smithi110 ceph-mon[90991]: pgmap v428: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:50 smithi193 ceph-mon[80071]: pgmap v429: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:50 smithi110 ceph-mon[90991]: pgmap v429: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:52 smithi193 ceph-mon[80071]: pgmap v430: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:53.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:52 smithi110 ceph-mon[90991]: pgmap v430: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:55.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:54 smithi193 ceph-mon[80071]: pgmap v431: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:54 smithi110 ceph-mon[90991]: pgmap v431: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:56 smithi193 ceph-mon[80071]: pgmap v432: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:56 smithi110 ceph-mon[90991]: pgmap v432: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:04:58 smithi193 ceph-mon[80071]: pgmap v433: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:04:59.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:04:58 smithi110 ceph-mon[90991]: pgmap v433: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:00 smithi193 ceph-mon[80071]: pgmap v434: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:00 smithi110 ceph-mon[90991]: pgmap v434: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:05:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:05:03.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:02 smithi193 ceph-mon[80071]: pgmap v435: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:03.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:02 smithi110 ceph-mon[90991]: pgmap v435: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:04 smithi193 ceph-mon[80071]: pgmap v436: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:05.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:04 smithi110 ceph-mon[90991]: pgmap v436: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:07.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:06 smithi193 ceph-mon[80071]: pgmap v437: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:07.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:06 smithi110 ceph-mon[90991]: pgmap v437: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:09.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:08 smithi193 ceph-mon[80071]: pgmap v438: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:09.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:08 smithi110 ceph-mon[90991]: pgmap v438: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:11.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:10 smithi193 ceph-mon[80071]: pgmap v439: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:11.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:10 smithi110 ceph-mon[90991]: pgmap v439: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:12.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:12 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:05:12.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:12 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T16:05:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:05:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:05:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:13 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:05:13.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:13 smithi110 ceph-mon[90991]: pgmap v440: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T16:05:13.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T16:05:13.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:13 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' 2024-04-25T16:05:13.490 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:13 smithi193 ceph-mon[80071]: pgmap v440: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:15.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:14 smithi193 ceph-mon[80071]: pgmap v441: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:15.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:14 smithi110 ceph-mon[90991]: pgmap v441: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:17.114 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:16 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:05:17.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:16 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:05:18.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:17 smithi110 ceph-mon[90991]: pgmap v442: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:18.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:17 smithi193 ceph-mon[80071]: pgmap v442: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:19.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:18 smithi193 ceph-mon[80071]: pgmap v443: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:19.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:18 smithi110 ceph-mon[90991]: pgmap v443: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:21.257 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:20 smithi193 ceph-mon[80071]: pgmap v444: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:21.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:20 smithi110 ceph-mon[90991]: pgmap v444: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:23.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:22 smithi193 ceph-mon[80071]: pgmap v445: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:23.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:22 smithi110 ceph-mon[90991]: pgmap v445: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:25.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:24 smithi193 ceph-mon[80071]: pgmap v446: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:25.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:24 smithi110 ceph-mon[90991]: pgmap v446: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:27.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:26 smithi193 ceph-mon[80071]: pgmap v447: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:27.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:26 smithi110 ceph-mon[90991]: pgmap v447: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:29.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:28 smithi193 ceph-mon[80071]: pgmap v448: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:29.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:28 smithi110 ceph-mon[90991]: pgmap v448: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:31.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:30 smithi193 ceph-mon[80071]: pgmap v449: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:31.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:30 smithi110 ceph-mon[90991]: pgmap v449: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:32.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:31 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:05:32.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:31 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:05:33.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:32 smithi193 ceph-mon[80071]: pgmap v450: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:33.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:32 smithi110 ceph-mon[90991]: pgmap v450: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:35.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:34 smithi193 ceph-mon[80071]: pgmap v451: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:35.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:34 smithi110 ceph-mon[90991]: pgmap v451: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:37.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:36 smithi193 ceph-mon[80071]: pgmap v452: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:37.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:36 smithi110 ceph-mon[90991]: pgmap v452: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:39.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:38 smithi193 ceph-mon[80071]: pgmap v453: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:39.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:38 smithi110 ceph-mon[90991]: pgmap v453: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:41.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:40 smithi193 ceph-mon[80071]: pgmap v454: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:41.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:40 smithi110 ceph-mon[90991]: pgmap v454: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:43.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:42 smithi193 ceph-mon[80071]: pgmap v455: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:43.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:42 smithi110 ceph-mon[90991]: pgmap v455: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:45.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:44 smithi193 ceph-mon[80071]: pgmap v456: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:45.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:44 smithi110 ceph-mon[90991]: pgmap v456: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:47.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:46 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:05:47.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:46 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:05:48.113 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:47 smithi110 ceph-mon[90991]: pgmap v457: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:48.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:47 smithi193 ceph-mon[80071]: pgmap v457: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:49.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:48 smithi193 ceph-mon[80071]: pgmap v458: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:49.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:48 smithi110 ceph-mon[90991]: pgmap v458: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:51.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:50 smithi193 ceph-mon[80071]: pgmap v459: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:51.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:50 smithi110 ceph-mon[90991]: pgmap v459: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:53.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:52 smithi193 ceph-mon[80071]: pgmap v460: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:53.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:52 smithi110 ceph-mon[90991]: pgmap v460: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:55.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:54 smithi193 ceph-mon[80071]: pgmap v461: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:55.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:54 smithi110 ceph-mon[90991]: pgmap v461: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:57.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:56 smithi193 ceph-mon[80071]: pgmap v462: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:57.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:56 smithi110 ceph-mon[90991]: pgmap v462: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:59.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:05:58 smithi193 ceph-mon[80071]: pgmap v463: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:05:59.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:05:58 smithi110 ceph-mon[90991]: pgmap v463: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:06:01.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:00 smithi193 ceph-mon[80071]: pgmap v464: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:06:01.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:00 smithi110 ceph-mon[90991]: pgmap v464: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:06:02.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:01 smithi193 ceph-mon[80071]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:06:02.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:01 smithi110 ceph-mon[90991]: from='mgr.34100 172.21.15.110:0/4089309670' entity='mgr.smithi110.lvgbkf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-25T16:06:03.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:02 smithi110 ceph-mon[90991]: pgmap v465: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:06:03.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:02 smithi193 ceph-mon[80071]: pgmap v465: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:06:05.239 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:04 smithi193 ceph-mon[80071]: pgmap v466: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:06:05.363 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:04 smithi110 ceph-mon[90991]: pgmap v466: 65 pgs: 65 active+clean; 259 MiB data, 1020 MiB used, 535 GiB / 536 GiB avail 2024-04-25T16:06:06.111 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-04-25T16:06:06.111 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-25T16:06:06.113 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T16:06:06.125 INFO:tasks.cephadm:Teardown begin 2024-04-25T16:06:06.125 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephadm.py", line 2246, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-25T16:06:06.126 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:06:06.158 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:06:06.191 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T16:06:06.191 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T16:06:06.216 DEBUG:teuthology.orchestra.run.smithi193:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T16:06:06.248 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T16:06:06.248 INFO:tasks.cephadm.mon.smithi110:Stopping mon.smithi110... 2024-04-25T16:06:06.248 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110 2024-04-25T16:06:06.563 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:06 smithi110 systemd[1]: Stopping Ceph mon.smithi110 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T16:06:06.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:06 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[90987]: 2024-04-25T16:06:06.561+0000 7effa58c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi110 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:06:06.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:06 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110[90987]: 2024-04-25T16:06:06.561+0000 7effa58c4700 -1 mon.smithi110@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T16:06:06.863 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:06 smithi110 podman[147240]: 2024-04-25 16:06:06.583313551 +0000 UTC m=+0.135698017 container died 59d163e103ed26d0770be306cb476e40b2f071b33a09091abd5d92aa58797603 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T16:06:07.306 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:07 smithi110 podman[147240]: 2024-04-25 16:06:07.054195209 +0000 UTC m=+0.606579671 container cleanup 59d163e103ed26d0770be306cb476e40b2f071b33a09091abd5d92aa58797603 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T16:06:07.306 INFO:journalctl@ceph.mon.smithi110.smithi110.stdout:Apr 25 16:06:07 smithi110 bash[147240]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi110 2024-04-25T16:06:07.442 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi110.service' 2024-04-25T16:06:07.495 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:06:07.495 INFO:tasks.cephadm.mon.smithi110:Stopped mon.smithi110 2024-04-25T16:06:07.495 INFO:tasks.cephadm.mon.smithi193:Stopping mon.smithi193... 2024-04-25T16:06:07.496 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi193 2024-04-25T16:06:07.891 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:07 smithi193 systemd[1]: Stopping Ceph mon.smithi193 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T16:06:08.224 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:07 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193[80067]: 2024-04-25T16:06:07.890+0000 7fe95929d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi193 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:06:08.224 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:07 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193[80067]: 2024-04-25T16:06:07.890+0000 7fe95929d700 -1 mon.smithi193@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T16:06:08.225 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:07 smithi193 podman[116600]: 2024-04-25 16:06:07.921626701 +0000 UTC m=+0.234743766 container died 82fe48722b1a7d2d91f9d08d07189afa0f9a8005f9912488334fb9c15d000956 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T16:06:08.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:08 smithi193 podman[116600]: 2024-04-25 16:06:08.449080287 +0000 UTC m=+0.762197393 container cleanup 82fe48722b1a7d2d91f9d08d07189afa0f9a8005f9912488334fb9c15d000956 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-04-25T16:06:08.489 INFO:journalctl@ceph.mon.smithi193.smithi193.stdout:Apr 25 16:06:08 smithi193 bash[116600]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-mon-smithi193 2024-04-25T16:06:08.872 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@mon.smithi193.service' 2024-04-25T16:06:08.912 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:06:08.913 INFO:tasks.cephadm.mon.smithi193:Stopped mon.smithi193 2024-04-25T16:06:08.913 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T16:06:08.913 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.0 2024-04-25T16:06:09.209 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:08 smithi110 systemd[1]: Stopping Ceph osd.0 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T16:06:09.613 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:09 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[99723]: 2024-04-25T16:06:09.208+0000 7fd1105da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:06:09.613 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:09 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[99723]: 2024-04-25T16:06:09.208+0000 7fd1105da700 -1 osd.0 83 *** Got signal Terminated *** 2024-04-25T16:06:09.613 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:09 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0[99723]: 2024-04-25T16:06:09.208+0000 7fd1105da700 -1 osd.0 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:06:14.860 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:14 smithi110 podman[147373]: 2024-04-25 16:06:14.571056105 +0000 UTC m=+5.474183957 container died 8e46029b07105c8d92c335fe8c871aa0a0fa0787e7db4e915c75472df614880b (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T16:06:15.113 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:14 smithi110 podman[147373]: 2024-04-25 16:06:14.951089978 +0000 UTC m=+5.854217831 container cleanup 8e46029b07105c8d92c335fe8c871aa0a0fa0787e7db4e915c75472df614880b (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:06:15.113 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:14 smithi110 bash[147373]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0 2024-04-25T16:06:15.525 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:15 smithi110 podman[147390]: 2024-04-25 16:06:15.202830924 +0000 UTC m=+0.634277842 container remove 8e46029b07105c8d92c335fe8c871aa0a0fa0787e7db4e915c75472df614880b (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T16:06:15.807 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:15 smithi110 podman[147485]: 2024-04-25 16:06:15.42871469 +0000 UTC m=+0.018036595 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:06:15.807 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:15 smithi110 podman[147485]: 2024-04-25 16:06:15.542914104 +0000 UTC m=+0.132235995 container create 751093aafc7d11f5ffdc250bd954a1365ca81e14d597629432d5abf495ab6264 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T16:06:15.807 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:15 smithi110 podman[147485]: 2024-04-25 16:06:15.806660367 +0000 UTC m=+0.395982262 container init 751093aafc7d11f5ffdc250bd954a1365ca81e14d597629432d5abf495ab6264 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T16:06:16.110 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:15 smithi110 podman[147485]: 2024-04-25 16:06:15.81003675 +0000 UTC m=+0.399358641 container start 751093aafc7d11f5ffdc250bd954a1365ca81e14d597629432d5abf495ab6264 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T16:06:16.110 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:15 smithi110 podman[147485]: 2024-04-25 16:06:15.923862864 +0000 UTC m=+0.513184765 container attach 751093aafc7d11f5ffdc250bd954a1365ca81e14d597629432d5abf495ab6264 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T16:06:16.363 INFO:journalctl@ceph.osd.0.smithi110.stdout:Apr 25 16:06:16 smithi110 podman[147485]: 2024-04-25 16:06:16.109804013 +0000 UTC m=+0.699125914 container died 751093aafc7d11f5ffdc250bd954a1365ca81e14d597629432d5abf495ab6264 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T16:06:16.779 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T16:06:16.816 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:06:16.817 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T16:06:16.817 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T16:06:16.817 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.1 2024-04-25T16:06:17.113 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:16 smithi110 systemd[1]: Stopping Ceph osd.1 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T16:06:17.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:17 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[105379]: 2024-04-25T16:06:17.164+0000 7fb2c789f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:06:17.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:17 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[105379]: 2024-04-25T16:06:17.164+0000 7fb2c789f700 -1 osd.1 83 *** Got signal Terminated *** 2024-04-25T16:06:17.613 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:17 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1[105379]: 2024-04-25T16:06:17.164+0000 7fb2c789f700 -1 osd.1 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:06:22.863 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:22 smithi110 podman[147604]: 2024-04-25 16:06:22.614879492 +0000 UTC m=+5.555571039 container died 9b8b51eaafaca75c507aba8a81ecf5f5b7f75cc5932cb01f3820c3542d8dd0a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T16:06:23.200 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:22 smithi110 podman[147604]: 2024-04-25 16:06:22.990544002 +0000 UTC m=+5.931235560 container cleanup 9b8b51eaafaca75c507aba8a81ecf5f5b7f75cc5932cb01f3820c3542d8dd0a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-25T16:06:23.200 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:22 smithi110 bash[147604]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1 2024-04-25T16:06:23.509 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:23 smithi110 podman[147622]: 2024-04-25 16:06:23.200174595 +0000 UTC m=+0.588149723 container remove 9b8b51eaafaca75c507aba8a81ecf5f5b7f75cc5932cb01f3820c3542d8dd0a6 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5) 2024-04-25T16:06:23.829 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:23 smithi110 podman[147718]: 2024-04-25 16:06:23.509602927 +0000 UTC m=+0.113080034 container create 913e7fe709becc5f17f02026c10a89bf1d0243d3a5992ad30eb1e879afabdf14 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T16:06:23.829 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:23 smithi110 podman[147718]: 2024-04-25 16:06:23.415026333 +0000 UTC m=+0.018503467 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:06:23.829 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:23 smithi110 podman[147718]: 2024-04-25 16:06:23.745927946 +0000 UTC m=+0.349405060 container init 913e7fe709becc5f17f02026c10a89bf1d0243d3a5992ad30eb1e879afabdf14 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T16:06:23.829 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:23 smithi110 podman[147718]: 2024-04-25 16:06:23.749317742 +0000 UTC m=+0.352794849 container start 913e7fe709becc5f17f02026c10a89bf1d0243d3a5992ad30eb1e879afabdf14 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T16:06:23.829 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:23 smithi110 podman[147718]: 2024-04-25 16:06:23.828885992 +0000 UTC m=+0.432363111 container attach 913e7fe709becc5f17f02026c10a89bf1d0243d3a5992ad30eb1e879afabdf14 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T16:06:24.113 INFO:journalctl@ceph.osd.1.smithi110.stdout:Apr 25 16:06:24 smithi110 podman[147744]: 2024-04-25 16:06:24.076587387 +0000 UTC m=+0.018751609 container died 913e7fe709becc5f17f02026c10a89bf1d0243d3a5992ad30eb1e879afabdf14 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T16:06:24.698 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T16:06:24.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:06:24.739 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T16:06:24.739 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T16:06:24.739 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.2 2024-04-25T16:06:25.087 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:24 smithi110 systemd[1]: Stopping Ceph osd.2 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T16:06:25.363 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:25 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[110895]: 2024-04-25T16:06:25.085+0000 7f4c4a3c8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:06:25.363 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:25 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[110895]: 2024-04-25T16:06:25.085+0000 7f4c4a3c8700 -1 osd.2 83 *** Got signal Terminated *** 2024-04-25T16:06:25.363 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:25 smithi110 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2[110895]: 2024-04-25T16:06:25.085+0000 7f4c4a3c8700 -1 osd.2 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:06:30.780 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:30 smithi110 podman[147823]: 2024-04-25 16:06:30.482881282 +0000 UTC m=+5.520594639 container died 4fba7cfd0663f27dca63e4b15f94fbc02dc82c7bed97b02aea8d3423cd27578c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T16:06:31.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:30 smithi110 podman[147823]: 2024-04-25 16:06:30.87804943 +0000 UTC m=+5.915762796 container cleanup 4fba7cfd0663f27dca63e4b15f94fbc02dc82c7bed97b02aea8d3423cd27578c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:06:31.113 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:30 smithi110 bash[147823]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2 2024-04-25T16:06:31.502 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:31 smithi110 podman[147846]: 2024-04-25 16:06:31.163157607 +0000 UTC m=+0.674638123 container remove 4fba7cfd0663f27dca63e4b15f94fbc02dc82c7bed97b02aea8d3423cd27578c (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T16:06:31.753 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:31 smithi110 podman[147938]: 2024-04-25 16:06:31.405434996 +0000 UTC m=+0.018227113 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:06:31.753 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:31 smithi110 podman[147938]: 2024-04-25 16:06:31.522591148 +0000 UTC m=+0.135383302 container create 78a490c9a36280678403a6159cb6e93094ff591c545926db16d436ed8a9aeddf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-25T16:06:31.753 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:31 smithi110 podman[147938]: 2024-04-25 16:06:31.750311311 +0000 UTC m=+0.363103421 container init 78a490c9a36280678403a6159cb6e93094ff591c545926db16d436ed8a9aeddf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True) 2024-04-25T16:06:31.754 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:31 smithi110 podman[147938]: 2024-04-25 16:06:31.753379786 +0000 UTC m=+0.366171897 container start 78a490c9a36280678403a6159cb6e93094ff591c545926db16d436ed8a9aeddf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T16:06:32.047 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:31 smithi110 podman[147938]: 2024-04-25 16:06:31.86707297 +0000 UTC m=+0.479865105 container attach 78a490c9a36280678403a6159cb6e93094ff591c545926db16d436ed8a9aeddf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T16:06:32.338 INFO:journalctl@ceph.osd.2.smithi110.stdout:Apr 25 16:06:32 smithi110 podman[147938]: 2024-04-25 16:06:32.046894603 +0000 UTC m=+0.659686704 container died 78a490c9a36280678403a6159cb6e93094ff591c545926db16d436ed8a9aeddf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T16:06:32.636 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T16:06:32.674 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:06:32.675 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T16:06:32.675 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T16:06:32.675 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.3 2024-04-25T16:06:32.951 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:32 smithi193 systemd[1]: Stopping Ceph osd.3 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T16:06:33.239 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:32 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[88999]: 2024-04-25T16:06:32.950+0000 7fc27f882700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:06:33.240 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:32 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[88999]: 2024-04-25T16:06:32.950+0000 7fc27f882700 -1 osd.3 83 *** Got signal Terminated *** 2024-04-25T16:06:33.240 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:32 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[88999]: 2024-04-25T16:06:32.950+0000 7fc27f882700 -1 osd.3 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:06:36.489 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:36 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:36.064+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:36.490 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:36 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:36.210+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:37.239 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:36 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[88999]: 2024-04-25T16:06:36.932+0000 7fc277e30700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:12.764935+0000 front 2024-04-25T16:06:12.764872+0000 (oldest deadline 2024-04-25T16:06:36.864760+0000) 2024-04-25T16:06:37.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:37 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:37.161+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:37.240 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:37 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:37.065+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:38.239 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:37 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3[88999]: 2024-04-25T16:06:37.923+0000 7fc277e30700 -1 osd.3 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:12.764935+0000 front 2024-04-25T16:06:12.764872+0000 (oldest deadline 2024-04-25T16:06:36.864760+0000) 2024-04-25T16:06:38.240 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:38 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:38.071+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:38.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:38 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:38.156+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:38.627 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:38 smithi193 podman[116725]: 2024-04-25 16:06:38.335591622 +0000 UTC m=+5.469469832 container died daab482bd8fad060f798eedc27b37e402e8f5d64f09d72d153f5e31e524bfd08 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T16:06:38.960 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:38 smithi193 podman[116725]: 2024-04-25 16:06:38.72564978 +0000 UTC m=+5.859527970 container cleanup daab482bd8fad060f798eedc27b37e402e8f5d64f09d72d153f5e31e524bfd08 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T16:06:38.960 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:38 smithi193 bash[116725]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3 2024-04-25T16:06:39.239 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:38 smithi193 podman[116755]: 2024-04-25 16:06:38.960366707 +0000 UTC m=+0.625333682 container remove daab482bd8fad060f798eedc27b37e402e8f5d64f09d72d153f5e31e524bfd08 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:06:39.240 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:39 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:39.052+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:39.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:39 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:39.199+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:39.611 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:39 smithi193 podman[116850]: 2024-04-25 16:06:39.199377947 +0000 UTC m=+0.017811091 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:06:39.612 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:39 smithi193 podman[116850]: 2024-04-25 16:06:39.303025657 +0000 UTC m=+0.121458787 container create e3249d9fb98b8a103ad4413a9b278e6485a292af2f06ec0bccbf6768b96cbcbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-25T16:06:39.612 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:39 smithi193 podman[116850]: 2024-04-25 16:06:39.522164015 +0000 UTC m=+0.340597145 container init e3249d9fb98b8a103ad4413a9b278e6485a292af2f06ec0bccbf6768b96cbcbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T16:06:39.612 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:39 smithi193 podman[116850]: 2024-04-25 16:06:39.525335027 +0000 UTC m=+0.343768158 container start e3249d9fb98b8a103ad4413a9b278e6485a292af2f06ec0bccbf6768b96cbcbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-04-25T16:06:39.612 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:39 smithi193 podman[116850]: 2024-04-25 16:06:39.610856581 +0000 UTC m=+0.429289732 container attach e3249d9fb98b8a103ad4413a9b278e6485a292af2f06ec0bccbf6768b96cbcbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T16:06:39.989 INFO:journalctl@ceph.osd.3.smithi193.stdout:Apr 25 16:06:39 smithi193 podman[116850]: 2024-04-25 16:06:39.829261769 +0000 UTC m=+0.647694905 container died e3249d9fb98b8a103ad4413a9b278e6485a292af2f06ec0bccbf6768b96cbcbf (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-25T16:06:40.384 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:40 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:40.014+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:40.384 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:40 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:40.217+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:40.400 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T16:06:40.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:06:40.441 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T16:06:40.441 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T16:06:40.441 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.4 2024-04-25T16:06:40.739 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:40 smithi193 systemd[1]: Stopping Ceph osd.4 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T16:06:41.195 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:40 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:40.778+0000 7f1474664700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:06:41.196 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:40 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:40.778+0000 7f1474664700 -1 osd.4 83 *** Got signal Terminated *** 2024-04-25T16:06:41.196 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:40 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:40.778+0000 7f1474664700 -1 osd.4 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:06:41.196 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:41 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:41.018+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:41.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:41 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:41.195+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:42.239 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:41 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:41.983+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:42.739 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:42 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:42.240+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:43.205 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:42 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:42.951+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:43.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:43 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:43.204+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:44.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:44 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:44.167+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:44.490 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:44 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:44.000+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:44.490 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:44 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:44.000+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:19.895032+0000 front 2024-04-25T16:06:19.895049+0000 (oldest deadline 2024-04-25T16:06:43.994836+0000) 2024-04-25T16:06:45.489 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:45 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:45.016+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:09.393320+0000 front 2024-04-25T16:06:09.393458+0000 (oldest deadline 2024-04-25T16:06:35.293312+0000) 2024-04-25T16:06:45.490 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:45 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4[94009]: 2024-04-25T16:06:45.016+0000 7f1470c1a700 -1 osd.4 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:19.895032+0000 front 2024-04-25T16:06:19.895049+0000 (oldest deadline 2024-04-25T16:06:43.994836+0000) 2024-04-25T16:06:45.490 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:45 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:45.171+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:45.490 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:45 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:45.171+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:46.464 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:46 smithi193 podman[116957]: 2024-04-25 16:06:46.181458554 +0000 UTC m=+5.518791465 container died e67bdf50065ed84beb37e24b7b5c430c22f92ee68328b1f1e7f03c48a3fb3680 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T16:06:46.465 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:46 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:46.167+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:46.465 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:46 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:46.167+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:46.739 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:46 smithi193 podman[116957]: 2024-04-25 16:06:46.553581415 +0000 UTC m=+5.890914326 container cleanup e67bdf50065ed84beb37e24b7b5c430c22f92ee68328b1f1e7f03c48a3fb3680 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T16:06:46.740 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:46 smithi193 bash[116957]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4 2024-04-25T16:06:47.172 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:46 smithi193 podman[116975]: 2024-04-25 16:06:46.821392917 +0000 UTC m=+0.634547846 container remove e67bdf50065ed84beb37e24b7b5c430c22f92ee68328b1f1e7f03c48a3fb3680 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T16:06:47.483 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:47 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:47.191+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:47.483 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:47 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:47.191+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:47.483 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:47 smithi193 podman[117070]: 2024-04-25 16:06:47.172298374 +0000 UTC m=+0.107619906 container create e3ee2e21b081c90fac93e02d4b10e9f998b0baee43a15a4d8202ddfc4a7170bd (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:06:47.483 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:47 smithi193 podman[117070]: 2024-04-25 16:06:47.082933078 +0000 UTC m=+0.018254617 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:06:47.483 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:47 smithi193 podman[117070]: 2024-04-25 16:06:47.399975754 +0000 UTC m=+0.335297287 container init e3ee2e21b081c90fac93e02d4b10e9f998b0baee43a15a4d8202ddfc4a7170bd (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-25T16:06:47.484 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:47 smithi193 podman[117070]: 2024-04-25 16:06:47.40304944 +0000 UTC m=+0.338370974 container start e3ee2e21b081c90fac93e02d4b10e9f998b0baee43a15a4d8202ddfc4a7170bd (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T16:06:47.484 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:47 smithi193 podman[117070]: 2024-04-25 16:06:47.483156873 +0000 UTC m=+0.418478405 container attach e3ee2e21b081c90fac93e02d4b10e9f998b0baee43a15a4d8202ddfc4a7170bd (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T16:06:47.740 INFO:journalctl@ceph.osd.4.smithi193.stdout:Apr 25 16:06:47 smithi193 podman[117070]: 2024-04-25 16:06:47.696431492 +0000 UTC m=+0.631753035 container died e3ee2e21b081c90fac93e02d4b10e9f998b0baee43a15a4d8202ddfc4a7170bd (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-25T16:06:48.239 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:48 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:48.169+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:48.239 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:48 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:48.169+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:48.341 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T16:06:48.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:06:48.379 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T16:06:48.379 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T16:06:48.379 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.5 2024-04-25T16:06:48.690 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:48 smithi193 systemd[1]: Stopping Ceph osd.5 for e79cc6fc-0319-11ef-bc93-c7b262605968... 2024-04-25T16:06:48.989 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:48 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:48.689+0000 7f8f32696700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T16:06:48.989 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:48 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:48.689+0000 7f8f32696700 -1 osd.5 83 *** Got signal Terminated *** 2024-04-25T16:06:48.990 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:48 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:48.689+0000 7f8f32696700 -1 osd.5 83 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T16:06:49.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:49 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:49.132+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:49.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:49 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:49.132+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:50.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:50 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:50.120+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:50.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:50 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:50.120+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:51.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:51 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:51.163+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:51.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:51 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:51.163+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:52.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:52 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:52.164+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:52.490 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:52 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:52.164+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:52.490 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:52 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:52.164+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-04-25T16:06:28.132986+0000 front 2024-04-25T16:06:28.132935+0000 (oldest deadline 2024-04-25T16:06:51.632851+0000) 2024-04-25T16:06:53.489 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:53 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:53.154+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6806 osd.0 since back 2024-04-25T16:06:10.730661+0000 front 2024-04-25T16:06:10.730701+0000 (oldest deadline 2024-04-25T16:06:35.430499+0000) 2024-04-25T16:06:53.490 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:53 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:53.154+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6814 osd.1 since back 2024-04-25T16:06:20.031908+0000 front 2024-04-25T16:06:20.031833+0000 (oldest deadline 2024-04-25T16:06:44.731712+0000) 2024-04-25T16:06:53.490 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:53 smithi193 ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5[99156]: 2024-04-25T16:06:53.154+0000 7f8f2ac44700 -1 osd.5 83 heartbeat_check: no reply from 172.21.15.110:6822 osd.2 since back 2024-04-25T16:06:28.132986+0000 front 2024-04-25T16:06:28.132935+0000 (oldest deadline 2024-04-25T16:06:51.632851+0000) 2024-04-25T16:06:54.298 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:54 smithi193 podman[117175]: 2024-04-25 16:06:54.027818378 +0000 UTC m=+5.424916561 container died 31c4aca488869588257725d278251a739e88eba2dae3c1a8c63a39b83f2f039f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:06:54.649 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:54 smithi193 podman[117175]: 2024-04-25 16:06:54.405882623 +0000 UTC m=+5.802980803 container cleanup 31c4aca488869588257725d278251a739e88eba2dae3c1a8c63a39b83f2f039f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2) 2024-04-25T16:06:54.649 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:54 smithi193 bash[117175]: ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5 2024-04-25T16:06:54.959 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:54 smithi193 podman[117192]: 2024-04-25 16:06:54.648797814 +0000 UTC m=+0.619046519 container remove 31c4aca488869588257725d278251a739e88eba2dae3c1a8c63a39b83f2f039f (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T16:06:55.239 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:54 smithi193 podman[117282]: 2024-04-25 16:06:54.862012077 +0000 UTC m=+0.017966380 image pull d53c96779a29b1ca3ec780b8b69a1863c25b0e59d8e476272e056726cdf012a4 quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385 2024-04-25T16:06:55.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:54 smithi193 podman[117282]: 2024-04-25 16:06:54.966252828 +0000 UTC m=+0.122207118 container create 7572284da1c932069d08bbce258ef422488aa96c4d087408a935a40ba02ed331 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T16:06:55.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:55 smithi193 podman[117282]: 2024-04-25 16:06:55.193581289 +0000 UTC m=+0.349535576 container init 7572284da1c932069d08bbce258ef422488aa96c4d087408a935a40ba02ed331 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-25T16:06:55.240 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:55 smithi193 podman[117282]: 2024-04-25 16:06:55.196657227 +0000 UTC m=+0.352611514 container start 7572284da1c932069d08bbce258ef422488aa96c4d087408a935a40ba02ed331 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-04-25T16:06:55.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:55 smithi193 podman[117282]: 2024-04-25 16:06:55.276937419 +0000 UTC m=+0.432891711 container attach 7572284da1c932069d08bbce258ef422488aa96c4d087408a935a40ba02ed331 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T16:06:55.740 INFO:journalctl@ceph.osd.5.smithi193.stdout:Apr 25 16:06:55 smithi193 podman[117282]: 2024-04-25 16:06:55.482682749 +0000 UTC m=+0.638637046 container died 7572284da1c932069d08bbce258ef422488aa96c4d087408a935a40ba02ed331 (image=quay.ceph.io/ceph-ci/ceph@sha256:c1dfc10be9702848c35bd11791796a771cb431046d56ece349f8a36761441385, name=ceph-e79cc6fc-0319-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240425.015853-debug-994e2f2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T16:06:56.071 DEBUG:teuthology.orchestra.run.smithi193:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e79cc6fc-0319-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T16:06:56.107 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T16:06:56.108 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T16:06:56.108 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T16:06:59.410 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi110.stderr:ceph-fuse[73738]: fuse finished with error 0 and tester_r 0 2024-04-25T16:07:15.913 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T16:07:29.892 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:07:29.919 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T16:07:29.945 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T16:07:29.947 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227/remote/smithi110/crash 2024-04-25T16:07:29.948 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/crash -- . 2024-04-25T16:07:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T16:07:29.982 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-04-25T16:07:29.984 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227/remote/smithi193/crash 2024-04-25T16:07:29.985 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/crash -- . 2024-04-25T16:07:30.008 INFO:teuthology.orchestra.run.smithi193.stderr:tar: /var/lib/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T16:07:30.008 INFO:teuthology.orchestra.run.smithi193.stderr:tar: Error is not recoverable: exiting now 2024-04-25T16:07:30.009 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T16:07:30.009 DEBUG:teuthology.orchestra.run.smithi110:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T16:07:30.065 INFO:tasks.cephadm:Compressing logs... 2024-04-25T16:07:30.065 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T16:07:30.107 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T16:07:30.128 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-04-25T16:07:30.128 INFO:teuthology.orchestra.run.smithi110.stderr:: No such file or directory 2024-04-25T16:07:30.129 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T16:07:30.129 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.log 2024-04-25T16:07:30.129 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T16:07:30.130 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T16:07:30.130 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mgr.smithi110.lvgbkf.log 2024-04-25T16:07:30.130 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T16:07:30.140 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mgr.smithi110.lvgbkf.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T16:07:30.140 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.1.log: 88.2% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T16:07:30.140 INFO:teuthology.orchestra.run.smithi110.stderr: 91.2% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T16:07:30.141 INFO:teuthology.orchestra.run.smithi193.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-04-25T16:07:30.141 INFO:teuthology.orchestra.run.smithi193.stderr:: No such file or directory 2024-04-25T16:07:30.141 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T16:07:30.141 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.1.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.log 2024-04-25T16:07:30.141 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T16:07:30.141 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi193.jtyzhl.log 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi193.jtyzhl.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi193.ewxwfw.log 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr: 92.4%/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.4.log: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi193.ewxwfw.log: 88.3% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mon.smithi193.log 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.4% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T16:07:30.142 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mgr.smithi193.mgoteh.log 2024-04-25T16:07:30.143 INFO:teuthology.orchestra.run.smithi193.stderr: 91.4%/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mon.smithi193.log: -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T16:07:30.144 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi110.mvvbtv.log 2024-04-25T16:07:30.144 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mon.smithi110.log 2024-04-25T16:07:30.149 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T16:07:30.163 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi110.mvvbtv.log: 92.0%gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T16:07:30.163 INFO:teuthology.orchestra.run.smithi110.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T16:07:30.172 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mon.smithi110.log: gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi110.kdhjqx.log 2024-04-25T16:07:30.174 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.cephadm.log: 85.5% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T16:07:30.174 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T16:07:30.203 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mgr.smithi193.mgoteh.log: /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T16:07:30.225 INFO:teuthology.orchestra.run.smithi193.stderr: 89.8% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mgr.smithi193.mgoteh.log.gz 2024-04-25T16:07:30.231 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi110.kdhjqx.log: /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T16:07:30.593 INFO:teuthology.orchestra.run.smithi193.stderr: 92.5% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mon.smithi193.log.gz 2024-04-25T16:07:30.601 INFO:teuthology.orchestra.run.smithi110.stderr: 88.7% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mgr.smithi110.lvgbkf.log.gz 2024-04-25T16:07:31.414 INFO:teuthology.orchestra.run.smithi110.stderr: 90.5% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mon.smithi110.log.gz 2024-04-25T16:07:33.543 INFO:teuthology.orchestra.run.smithi193.stderr: 93.5% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T16:07:34.493 INFO:teuthology.orchestra.run.smithi110.stderr: 93.6% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T16:07:35.348 INFO:teuthology.orchestra.run.smithi110.stderr: 93.7% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T16:07:35.847 INFO:teuthology.orchestra.run.smithi110.stderr: 93.7% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T16:07:36.268 INFO:teuthology.orchestra.run.smithi193.stderr: 93.7% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T16:07:36.428 INFO:teuthology.orchestra.run.smithi193.stderr: 93.7% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T16:07:37.351 INFO:teuthology.orchestra.run.smithi193.stderr: 94.8% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi193.jtyzhl.log.gz 2024-04-25T16:07:37.774 INFO:teuthology.orchestra.run.smithi110.stderr: 94.8% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi110.kdhjqx.log.gz 2024-04-25T16:07:38.290 INFO:teuthology.orchestra.run.smithi193.stderr: 94.8% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi193.ewxwfw.log.gz 2024-04-25T16:07:43.759 INFO:teuthology.orchestra.run.smithi193.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-04-25T16:07:43.759 INFO:teuthology.orchestra.run.smithi193.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-04-25T16:07:43.760 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-04-25T16:07:43.760 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m13.639s 2024-04-25T16:07:43.760 INFO:teuthology.orchestra.run.smithi193.stderr:user 0m44.073s 2024-04-25T16:07:43.760 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m1.250s 2024-04-25T16:07:44.562 INFO:teuthology.orchestra.run.smithi110.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-04-25T16:09:02.250 INFO:teuthology.orchestra.run.smithi110.stderr: 92.8% -- replaced with /var/log/ceph/e79cc6fc-0319-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi110.mvvbtv.log.gz 2024-04-25T16:09:02.252 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-04-25T16:09:02.252 INFO:teuthology.orchestra.run.smithi110.stderr:real 1m32.133s 2024-04-25T16:09:02.252 INFO:teuthology.orchestra.run.smithi110.stderr:user 2m6.562s 2024-04-25T16:09:02.252 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m3.983s 2024-04-25T16:09:02.253 INFO:tasks.cephadm:Archiving logs... 2024-04-25T16:09:02.253 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227/remote/smithi110/log 2024-04-25T16:09:02.254 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T16:09:21.654 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227/remote/smithi193/log 2024-04-25T16:09:21.655 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T16:09:27.283 INFO:tasks.cephadm:Removing cluster... 2024-04-25T16:09:27.283 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 --force 2024-04-25T16:09:27.894 DEBUG:teuthology.orchestra.run.smithi193:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid e79cc6fc-0319-11ef-bc93-c7b262605968 --force 2024-04-25T16:09:28.393 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T16:09:28.394 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T16:09:28.410 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T16:09:28.425 INFO:tasks.cephadm:Teardown complete 2024-04-25T16:09:28.426 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T16:09:28.437 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-25T16:09:28.438 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T16:09:28.438 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T16:09:28.454 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T16:09:28.532 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T16:09:28.543 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T16:09:28.543 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:09:28.546 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T16:09:28.558 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:09:28.559 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-04-25T16:09:28.562 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:09:28.562 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-04-25T16:09:28.562 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 17 +917us[ +906us] +/- 47ms 2024-04-25T16:09:28.562 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -1026us[-1037us] +/- 67ms 2024-04-25T16:09:28.562 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 19 -364us[ -375us] +/- 61ms 2024-04-25T16:09:28.562 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:09:28.563 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T16:09:28.563 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-04-25T16:09:28.563 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +882us[ +911us] +/- 47ms 2024-04-25T16:09:28.563 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -1045us[-1045us] +/- 67ms 2024-04-25T16:09:28.563 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -410us[ -410us] +/- 61ms 2024-04-25T16:09:28.563 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T16:09:28.564 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T16:09:28.576 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T16:09:28.578 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T16:09:28.588 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T16:09:28.681 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T16:09:28.751 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:09:28.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:09:28.779 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T16:09:28.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:09:28.806 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T16:09:28.817 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T16:09:28.828 INFO:teuthology.task.internal:Duration was 2175.548850 seconds 2024-04-25T16:09:28.828 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T16:09:28.841 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T16:09:28.841 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T16:09:28.844 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T16:09:28.878 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:09:28.882 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T16:09:29.278 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T16:09:29.278 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-04-25T16:09:29.278 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T16:09:29.298 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T16:09:29.298 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T16:09:29.340 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T16:09:29.340 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T16:09:29.344 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T16:09:29.405 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T16:09:29.405 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T16:09:29.409 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T16:09:30.056 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T16:09:30.070 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T16:09:30.070 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T16:09:30.094 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T16:09:30.119 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T16:09:30.134 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T16:09:30.137 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T16:09:30.156 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-04-25T16:09:30.183 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-04-25T16:09:30.209 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T16:09:30.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:09:30.246 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T16:09:30.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:09:30.263 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T16:09:30.275 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T16:09:30.276 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227/remote/smithi110 2024-04-25T16:09:30.276 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T16:09:30.347 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227/remote/smithi193 2024-04-25T16:09:30.348 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T16:09:30.385 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T16:09:30.386 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T16:09:30.388 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T16:09:30.434 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T16:09:30.449 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T16:09:30.449 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T16:09:30.462 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T16:09:30.463 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T16:09:30.465 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T16:09:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: 262213 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Apr 25 16:09 /home/ubuntu/cephtest 2024-04-25T16:09:30.477 INFO:teuthology.orchestra.run.smithi110.stdout: 554396 4 d--------- 2 ubuntu ubuntu 4096 Apr 25 15:46 /home/ubuntu/cephtest/mnt.0 2024-04-25T16:09:30.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T16:09:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-04-25T16:09:30.479 INFO:teuthology.orchestra.run.smithi110.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-04-25T16:09:30.479 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/ceph_fuse.py", line 183, in task mount.umount_wait() File "/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait run.wait([self.fuse_daemon], timeout) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi110 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-04-25T16:09:30.480 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T16:09:30.494 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T16:09:30.507 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (51) after waiting for 300 seconds 2024-04-25T16:09:30.507 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T16:09:30.533 DEBUG:teuthology.nuke:shortname: smithi110 2024-04-25T16:09:30.534 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T16:09:30.556 DEBUG:teuthology.nuke:shortname: smithi193 2024-04-25T16:09:30.557 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T16:09:30.560 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 15:27:54.038051', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAsE3bZBAWGFLdZIGuH4oDbcYzcmN2PuscP8jNOkAo9EXcakMElLjD28TLGKf0eGtjnNXCwLx+2o7oZxw1gPp/c='} 2024-04-25T16:09:30.584 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673227', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 15:27:54.036775', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBBXyEf7DAbFDAwRr+lEjRD7WZWBtpyYRpz0s6rqeS8/tDfH9zJH9K5P/M0XtiIvu200Damexr3gMvGGiQHIVd4='} 2024-04-25T16:09:30.614 INFO:teuthology.orchestra.console.smithi110:Power off 2024-04-25T16:09:30.614 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T16:09:30.637 DEBUG:teuthology.orchestra.console.smithi110:power off output: Chassis Power Control: Down/Off 2024-04-25T16:09:30.637 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T16:09:30.653 INFO:teuthology.orchestra.console.smithi193:Power off 2024-04-25T16:09:30.653 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T16:09:30.661 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on 2024-04-25T16:09:30.675 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-04-25T16:09:30.675 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T16:09:30.699 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-25T16:09:34.662 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T16:09:34.700 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T16:09:34.773 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is on 2024-04-25T16:09:34.812 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-25T16:09:38.779 DEBUG:teuthology.orchestra.console.smithi110:pexpect command: ipmitool -H smithi110.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T16:09:38.813 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T16:09:38.889 DEBUG:teuthology.orchestra.console.smithi110:check power output: Chassis Power is off 2024-04-25T16:09:38.921 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-25T16:09:38.990 INFO:teuthology.orchestra.console.smithi110:Power off completed 2024-04-25T16:09:42.926 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T16:09:43.036 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-04-25T16:09:43.136 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-04-25T16:09:43.241 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 2175.548850297928 failure_reason: reached maximum tries (51) after waiting for 300 seconds flavor: default owner: scheduled_pdonnell@teuthology status: fail success: false 2024-04-25T16:09:43.241 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T16:09:43.306 INFO:teuthology.run:FAIL